Amsterdam-based startup Reson8 has released Rumble-2-Flash, a 0.5B-parameter, roughly 1GB speech-to-text model that runs offline on a phone, covers 45 languages and ships free under Apache 2.0. It targets the audio that breaks most transcription tools, including thick accents, overlapping speakers, poor phone lines and street noise, and is claimed to beat models 20x its size.
On-Device Inference
端侧推理
Running model inference locally on user devices to avoid cloud latency and data egress.
Curated Intelligence
Tencent President Martin Lau said at an earnings call that WeChat's AI assistant Xiaowei has reached 30 million daily active users in its test pool, and its quarterly cost after full rollout will be lower than Yuanbao's earlier this year. Xiaowei runs on a custom WeLM model with 80B total and 3B activated parameters, with Tencent citing extremely low and falling inference costs and expecting some inference to move on-device.
WeChat announced that its personal AI assistant Xiaowei has expanded its user testing and reached 30 million daily active users, gradually integrating into chats, Moments, official accounts and Channels. Built on a customized WeLM model optimized for cost efficiency, Xiaowei may offload some inference tasks to user devices in the future.
AutoArk has open-sourced Edge0-145B and Edge0-120B alongside a unified inference framework and deployment tooling. The 145B variant is billed as the first open-source 145-billion-parameter model that runs locally on a personal computer, with reported peak memory of about 29GB and decoding above 80 tokens/s. The company claims coding, reasoning and agent capabilities competitive with select frontier cloud models, with core capabilities working offline and without accounts or usage quotas.
A Hugging Face team released BotPsy, a sub-billion-parameter agentic language model built to run as an on-device personal assistant, replacing costly and volatile cloud LLM calls with local inference. The post details evaluation protocols across ToolSandbox, GPQA Diamond, MMLU-Pro, IFEval and related benchmarks, arguing that tool-use and general capability can coexist at a very small parameter budget.
Apple has introduced a front-end module called Skimmer in the first developer beta of iOS/macOS 27.3, routing roughly 2% of English Siri AI requests to an on-device model for the first time, mainly covering high-frequency fixed commands such as timers, music playback and calls. The module matches preset tool flows via regular expressions and executes directly, bypassing full AI inference to cut latency and protect privacy.
Security researchers reveal that every Mac since macOS 13 ships a pair of undocumented daemons that continuously build a personal knowledge graph covering user identity, relationships, communications and behavior, feeding Apple Intelligence personalization. They also released a tool letting users inspect their own graph.
Underdog released Woof, a text-to-speech model just 31.7MB in size, demoed through a browser-playable game that runs entirely on-device as a jab at cloud-based models. The startup also teased a 27-billion-parameter model that runs on laptops and a waitlisted locally hosted app.
Alibaba's MNN team released MNN-Pocket-0.3B, an open-source ~0.3B-parameter model co-designed with its vocabulary, training data, post-training, quantization and inference engine for native on-device use. The team claims 350 tokens/s offline decoding on iPhone 18 Pro with peak memory under 300MB, targeting 8B-class task quality in business scenarios.
The Gradium team released Phonon on Hugging Face, a small open-source text-to-speech model that runs entirely on-device with no server or network dependency, keeping user input local. The team says it leads Hugging Face's Open TTS Leaderboard and has opened an online demo for testing.