Alibaba's MNN Team Open-Sources 0.3B On-Device Model Hitting 350 Tokens/s

1 reports archived1 independent sourcesupdated 11/5/2026, 16:09:00

Synthesis & Latest Updates

1 Sources Cross-Validated

Alibaba's MNN team released MNN-Pocket-0.3B, an open-source ~0.3B-parameter model co-designed with its vocabulary, training data, post-training, quantization and inference engine for native on-device use. The team claims 350 tokens/s offline decoding on iPhone 18 Pro with peak memory under 300MB, targeting 8B-class task quality in business scenarios.

LATEST/Alibaba's MNN team released MNN-Pocket-0.3B, an open-source ~0.3B-parameter model co-designed with its vocabulary, training data, post-training, quantization and inference engine for native on-device use. The team claims 350 tokens/s offline decoding on iPhone 18 Pro with peak memory under 300MB, targeting 8B-class task quality in business scenarios.

Hourly heat curve

13 fully observed hours

Now

9.0

Peak

11.817:00

24h change

–

The line compares only sources observed throughout; gap hours are interpolated to keep the trend continuous.

Coverage timeline

Total 1 reports · Latest first

量子位 (微信公众号)T2Nov 5, 16:09·78 pts
Alibaba's MNN Team Open-Sources 0.3B On-Device Model Hitting 350 Tokens/s
- The 0.3B model is co-optimized with quantization and the inference engine rather than merely compressed from a larger model
- Claimed metrics: >200 tokens/s decoding target, 350 tokens/s measured, peak memory under 300MB
- Power target: 100 inferences consume under 1% of battery, aimed at high-frequency invocation
View Related Reports & Analysis Original Source