Today for AI
ALL INTELLIGENCE
Xiaomi Open-Sources 0.8B OCR Model for Document Parsing and VQAIT之家 · 智能时代T2·Open Source

Xiaomi Open-Sources 0.8B OCR Model for Document Parsing and VQA

Original title: 小米开源 0.8B 参数 OCR 小模型：打通文档解析与视觉问答，支持 Markdown 与 JSON 输出
78AI Score

Xiaomi has open-sourced Xiaomi-OCR-0, a lightweight document understanding model built on the Qwen3.5-0.8B base. This single-model architecture covers document parsing, key information extraction, and OCR visual question answering, natively supporting Markdown, LaTeX, and JSON outputs. The training process employs a heterogeneous expert consensus mechanism and a 'predict-re-render-compare' closed-loop verification to significantly enhance accuracy in complex tables, formulas, and long-tail scenarios.

Full Report

IT Home October 30 News — Xiaomi officially announced today the open-sourcing of its lightweight document understanding model, Xiaomi-OCR-0. The model weights and inference code are now available on Hugging Face and GitHub.

The official parameter size is 0.8B, positioning it as a single model capable of covering three types of tasks: document parsing, key information extraction, and OCR-based visual question answering (VQA).

Xiaomi Open-Sources 0.8B Parameter OCR Model: Bridging Document Parsing and Visual QA, Supporting Markdown and JSON Output

In terms of document parsing, the model can identify body text, tables, and mathematical formulas, outputting structured results in reading order with native support for Markdown and LaTeX formats. For key information extraction, it targets invoices, receipts, cards, and business forms, locating specified fields and outputting structured data such as JSON to integrate into automated processing workflows.

OCR-based VQA allows cross-region retrieval, numerical calculation, and semantic reasoning around document and chart content, rather than merely transcribing text. Xiaomi stated that this combination covers the entire pipeline from low-level text perception to high-level content understanding.

Xiaomi Open-Sources 0.8B Parameter OCR Model: Bridging Document Parsing and Visual QA, Supporting Markdown and JSON Output

According to Xiaomi's official disclosure, Xiaomi-OCR-0 is built upon the Qwen3.5-0.8B-Base foundation, trained on approximately 170 million OCR and document understanding samples. On the data side, an heterogeneous expert consensus mechanism was adopted: multiple OCR expert models cross-validate different regions (body text, tables, formulas) and vote to filter high-confidence annotations.

Xiaomi Open-Sources 0.8B Parameter OCR Model: Bridging Document Parsing and Visual QA, Supporting Markdown and JSON Output

To handle complex tables and formulas, the team introduced a closed-loop verification process: "Prediction — Re-rendering — Visual Comparison." Candidate parsing results are re-rendered into images and compared pixel-wise and structurally against the original document regions to locate row/column misalignments or missing symbols for targeted correction. Additionally, difficult samples were synthesized based on layout complexity, font morphology, and physical degradation factors, supplementing long-tail data including multilingual texts, ancient handwritten scripts, screen moiré patterns, and distorted lighting.

Xiaomi Open-Sources 0.8B Parameter OCR Model: Bridging Document Parsing and Visual QA, Supporting Markdown and JSON Output

Training was divided into three stages:

  1. Q-Mask Text Anchoring: Enhances the spatial correspondence between local visual regions and text tokens.
  2. OCR Continued Pre-training: Establishes a baseline using multi-task mixed data comprising tens of billions of tokens.
  3. Mixed Task Reinforcement Learning: Uses verifiable reward signals such as structural edit distance, table matching accuracy, and QA correctness on difficult samples.

Xiaomi Open-Sources 0.8B Parameter OCR Model: Bridging Document Parsing and Visual QA, Supporting Markdown and JSON Output

Xiaomi's published evaluation results show that on Real5-OmniDocBench, which simulates five real-world capture methods (scanning, paper bending, screen photography, lighting changes, and tilted shooting), Xiaomi-OCR-0 scored 95.24, ranking first among the compared models. On Wild-OmniDocBench, it scored 87.94.

On the comprehensive benchmark OmniDocBench v1.6, the model scored 96.83, ranking second overall, just 0.04 points behind the leader, and first among models with fewer than 1B parameters. In specialized tests, it scored 97.84 on the mathematical formula recognition benchmark UniMER-Test and 92.27 on the table structure recognition benchmark PubTabNet.

In the ancient texts and calligraphy evaluation set Chronicles-OCR (Mature Scripts), Xiaomi-OCR-0 scored 73. Xiaomi noted that this performance is comparable to the closed-source model Seed 2.0 Pro, which scored 72.

For document understanding, Xiaomi averaged scores across five benchmarks: DocVQA, InfoVQA, ChartQA, OCRBench, and TextVQA. Xiaomi-OCR-0 achieved an average score of 83.2.

Official comparison data indicates this score is higher than HunyuanOCR-1.5 (1B, 76.5), TokenVL-2B (2B, 78.1), and TextHawk2 (7B, 78.5).

Xiaomi Open-Sources 0.8B Parameter OCR Model: Bridging Document Parsing and Visual QA, Supporting Markdown and JSON Output

In the same comparison group, the general-purpose multimodal model Qwen3.5-2B scored 80.9, the 8B-scale MiniCPM-V-4.5 scored 82.6, and Qwen3.5-4B scored 84.9.

Xiaomi Open-Sources 0.8B Parameter OCR Model: Bridging Document Parsing and Visual QA, Supporting Markdown and JSON Output

Regarding practical implementation, Xiaomi stated that related technologies have been integrated into the smartphone assistant "Super Xiao Ai". Users can short-press the AI button, swipe up with three fingers, or use voice commands to extract time-sensitive information from the screen—such as food delivery pickup codes, package collection codes, and queue numbers—and display them prominently.

In billing and card scenarios, screen recognition can extract transaction amounts, merchant names, payment times, and consumption categories for smart bookkeeping; paper documents like invoices, ID cards, and bank cards can undergo field extraction via photo capture. In scheduling and travel scenarios, the system can identify meeting notices in group chats or time/location details on posters to generate schedule cards.

Xiaomi Open-Sources 0.8B Parameter OCR Model: Bridging Document Parsing and Visual Question Answering, Supporting Markdown and JSON Output

IT Home noted that in addition to releasing the 0.8B model weights and inference code, Xiaomi also provides a browser-based interactive demo, an MCP service, and an Agent Skill toolkit, allowing developers to integrate these resources into their own business workflows or agent systems.

Xiaomi Open-Sources 0.8B Parameter OCR Model: Bridging Document Parsing and Visual Question Answering, Supporting Markdown and JSON Output

References: