Baidu Open-Sources Wenxin Unlimited OCR 2 with 96% KV Cache Reduction

Baidu’s OCR has leveled up again: Wenxin Unlimited OCR 2 is officially released and open-sourced!

This time, it directly tackles the challenge of reading an entire novel series in one go.

Seven Harry Potter books, from The Philosopher's Stone to The Deathly Hallows, are recognized and transcribed continuously across 3,600+ pages, all the way to the end.

Switching to the complete Three-Body Problem trilogy yields similar results: three books totaling 1,200+ pages and hundreds of thousands of words can be processed seamlessly without interruption.

However, processing such a massive volume of pages requires the model to first pass a critical hurdle: VRAM.

The longer the document, the more intermediate information (KV Cache) must be retained during inference—think of it as the model’s "cheat sheet" that grows thicker with every page read. A bulky cheat sheet not only consumes significant memory but also slows down retrieval.

In its technical report, Wenxin Unlimited OCR 2 presented a comparison from single-sequence tests involving hundred-page documents:

Peak KV Cache usage dropped from 47GB under standard decoding (which retains visual caches for all pages simultaneously in VRAM) to just 1.77GB, representing a reduction of approximately 96%.

To put 47GB into perspective, it exceeds the total RAM capacity of many laptops. And this figure accounts solely for the KV Cache; other overheads like model weights are calculated separately.

Its predecessor, Unlimited OCR, had already gained substantial popularity in the open-source community due to its ability to parse multi-page documents continuously, accumulating over 10 million downloads and roughly 28,600 stars on GitHub.

Community efforts have continued to expand around it, ranging from online demos to support for SGLang inference.

For the second generation, the upgrade focuses on reducing computational and caching burdens while maintaining the capability to process even longer documents continuously.

To achieve this, Wenxin Unlimited OCR 2 introduces the Sparse Reference Sliding Window Attention mechanism (SR-SWA), allowing the model to selectively retrieve relevant pages and visual information on demand, thereby minimizing redundant calculations.

Combined with a continuous decoding mechanism, the model can process ultra-long documents sequentially, ensuring that the cost per decoding step does not escalate as the document length increases.

Currently, Wenxin Unlimited OCR 2 is officially open-sourced on Hugging Face and GitHub.

Open Source Links: *GitHub: https://github.com/baidu/Unlimited-OCR *HuggingFace: https://huggingface.co/baidu/Unlimited-OCR-2

Deconstructing Core Technologies: How SR-SWA Reduces Computation and Chunked Decoding Minimizes Storage

How exactly does Wenxin Unlimited OCR 2 manage to read through millions of characters and 3,600+ pages of the Harry Potter series continuously?

It essentially borrows the logic of how humans copy text by hand.

What is the workflow when a human copies a book? You look at a section, copy it, keeping your eyes primarily focused on the page you are currently transcribing. Only when cross-referencing is needed do you flip back to check relevant pages.

Humans figured out the efficiency of copying long before models did: no one would try to keep every page of the Analects spread out in their mind simultaneously while copying just one section.

But the previous generation of Unlimited OCR didn't work this way. Although R-SWA significantly reduced KV Cache usage during inference, all input pages' visual information still participated in attention calculations during the review process.

As documents grew longer, the amount of computed information increased, yet much of it was irrelevant to the specific lines being recognized—a classic case of "wasted effort."

△ SR-SWA mimics the localized focus strategy used by humans when copying text.

Following this human-inspired approach, Wenxin Unlimited OCR 2 introduced the Sparse Reference Sliding Window Attention mechanism (SR-SWA).

Its filtering process operates in two layers: first, it identifies which pages need to be referenced; then, within those selected pages, it extracts truly relevant local information, blocking unnecessary computations.

From an architectural perspective, Wenxin Unlimited OCR 2 retains the foundational design of its predecessor.

△ Overall architecture of Wenxin Unlimited OCR 2.

Page images first enter the DeepEncoder, where they are converted into visual Tokens that the model can process—conceptually similar to data cards recording page content and layout features.

Next, the language model generates recognition results step-by-step based on these cards.

The key change occurs inside the language model: the Reference Sliding Window Attention mechanism (R-SWA) used in the previous generation has been upgraded to SR-SWA in this version.

The lower-left portion of the architecture diagram illustrates the two-layer filtering process of SR-SWA.

Layer 1: Page Gate – Determining "Which Pages to Look At".

Based on the model's current parsing position, it narrows candidate visual information to relevant page ranges, preventing irrelevant pages from interfering.

In the context of copying a book, this is akin to locating a few relevant pages when reaching a certain point, rather than flipping through the entire book from start to finish.

Layer 2: Token Sieve – Determining "Which Lines Within Those Pages to Focus On".

Selecting the right pages is only the first step: even within those pages, not all visual Tokens need to participate in computation. The Token Sieve performs a secondary filter, selecting visual information blocks most relevant to the current content to capture long-range visual dependencies.

The right side of the architecture diagram expands on this process: relevance scores are first calculated using concise representations of each information block, followed by a TopK selection to pick the highest-scoring, fixed number of blocks.

After selection, the model still examines the detailed visual information within these blocks, rather than relying solely on summaries.

By combining these two mechanisms, the model can locate relevant pages within long documents while reducing the volume of visual information processed at each step, thereby enhancing long-document parsing efficiency.

△ Token Sieve Details The filtered visual information also participates in parsing alongside the Local Window (local sliding window) shown in the diagram. This window has a fixed size, holding recent context and shifting forward as generation proceeds. It ensures the model continues writing based on the immediately preceding content. With a fixed number of information blocks and a fixed local window size, the amount of information referenced at each step is bounded—no matter how long the document is, the workload per step remains constant. This prevents scenarios where "the thicker the book, the harder it gets to write each word." SR-SWA is responsible for reducing computation per step; however, determining how much information to retain in VRAM is a separate issue. Each page image, once encoded, becomes a set of visual tokens. The model references these tokens when generating parsing results, so it stores the corresponding KV cache for reuse in subsequent calculations. If the entire document is loaded at once and all pages' visual caches are kept in VRAM, memory usage increases linearly with the number of pages. More critically: even if these pages are not involved in the current calculation, they continue to occupy VRAM as long as their cache is not released. This is akin to spreading every draft page from a book on your desk while copying text without discarding any—even a large desk would run out of space. Therefore, reducing computation must be paired with reducing storage. To address this, Wenxin Unlimited OCR 2 introduces the Unlimited-Horizon Decoding Paradigm, which controls cache occupancy through chunked processing.

△ Chunked Decoding Diagram This mechanism divides the document into manageable page chunks, parsing them sequentially. Adjacent chunks overlap by one page: the last page of the previous chunk serves as the starting point for the next—a ready-made bookmark. When entering the next chunk, the model directly reuses the visual KV cache of the overlapping page while retaining the text cache from the end of the previous output segment, ensuring continuity between sections. Visual caches for other pages that are no longer needed are promptly released to free up VRAM. As a result, the visual cache retained in VRAM is constrained within the scale of the currently processed chunk. After finishing a chunk, only the information necessary for bridging is kept before moving to the next—eliminating the need to hold the entire document's visual cache in memory from start to finish. *SR-SWA reduces computation per step, while the Unlimited-Horizon Decoding Paradigm controls cache occupancy. Their combination enables the model to continuously parse ultra-long documents. Reading an 884-page copy of The Lord of the Rings takes just 42 minutes, with cache usage reduced to 1.77GB. Having covered the technical details, we wanted to see the results firsthand: we fed it an 884-page English edition of The Lord of the Rings.

Approximately 42 minutes and 56 seconds later, parsing was complete. With 884 pages of input, there were 884 pages of output, averaging about 2.9 seconds per page. In roughly the time it takes to watch an episode of a TV show, the model traversed the entirety of "Middle-earth." Can recognition quality keep up as processing moves toward the latter parts of such a long document? We specifically spot-checked the body text on pages 600 and 750: the recognized text matched the original perfectly. Being able to reproduce page 750 verbatim indicates the model did not "lose focus" or degrade in performance earlier in the process.

Throughout the task, the system also identified 466 instances of cross-page merging. For example, sentences abruptly truncated by page breaks near the end of the book (pages 873–874) were correctly rejoined in the output.

While the live test demonstrated continuous parsing of the entire book, the technical report answers three additional questions: How much cache is required? How fast is it? And what is the recognition quality? On the hundred-page benchmark MarathonDocBench, the team compared standard decoding against Unlimited-Horizon Decoding. During single-sequence inference, using chunked processing and boundary cache reuse reduced peak KV cache usage from 47GB to 1.77GB, a decrease of approximately 96%.* Large amounts of page information that previously had to reside in VRAM simultaneously can now be processed in batches and released promptly. Note that the 1.77GB figure refers solely to KV cache; other VRAM overheads, such as model weights, are calculated separately. Beyond saving cache, the report also evaluated performance when processing multiple documents concurrently. In another 16-way concurrency test, adopting this decoding method increased end-to-end throughput from 360 to 1,315 tokens/second, approximately 3.65 times that of standard decoding.

△ Decoding Method Comparison: Single-Sequence Peak KV Cache vs. 16-Way Concurrency Throughput For services requiring batch processing of annual reports or technical manuals, both improvements have practical significance: lower VRAM usage allows more workloads to fit on the same GPU; higher concurrency yields more output per unit of time. Users do not need to worry about these details. They simply submit the entire document as usual, and the model handles chunking and stitching internally, eliminating the hassle of manually splitting files, submitting them in batches, and concatenating results. Beyond being faster and more efficient, parsing quality must also be maintained. MPDocBench evaluates the parsing quality of multi-page documents: facing documents tens of pages long, the model must accurately recognize content while reconstructing document structure. In this test, Wenxin Unlimited OCR 2 achieved a composite score of 84.59. To further test hundred-page-level documents, the team created MarathonDocBench. It includes 100 documents, each ranging from 103 to 300 pages. Many paragraphs and tables span across pages, specifically testing whether the model can correctly connect preceding and following content. On this longer and more challenging benchmark, Wenxin Unlimited OCR 2 achieved a composite score of 77.06. In both evaluations, its composite score was the highest among all models listed in the report.

△ From the technical report: Multi-page document parsing evaluation results

The improvement in long-document processing speed is equally significant. In the report's hundred-page-level tests, SR-SWA achieved a single-step decoding speed that was 7.0x faster than Full Attention and 5.6x faster than the R-SWA used in the previous generation.

The curves make this clearer: as documents get longer, the single-step decoding latency for the first two approaches climbs steadily, while SR-SWA’s latency remains nearly flat.

This means the time required to generate each token does not increase significantly with the number of input pages—the book gets thicker, but the writing speed stays constant.

△ Single-step decoding latency across different document lengths

Compared to the previous generation, Wenxin Unlimited OCR 2 maintains roughly the same overall score on single-page tasks.

The real difference emerges once documents become longer: it can process more consecutive pages, recognize cross-page content more accurately, and simultaneously reduce the computational and caching overhead for long-document parsing.

As for how people are using it...

You can input hundreds of pages of annual reports or thick technical manuals.

Especially for scanned documents: hand them over to the model for continuous parsing and text conversion, then retrieve information and verify content—this saves a lot of hassle involved in splitting pages and manually stitching results together.

But honestly, who uses such powerful OCR for work...

Of course, we use it to read novels!

The era of OCR 3.0 has arrived.

Naturally, as the scope expands from a few pages to entire books, the challenges OCR needs to solve also evolve.

In the technical report, the team summarizes the evolution of document parsing into three stages: OCR 1.0, 2.0, and 3.0.

Early OCR resembled a pipeline with clearly defined roles: first locate text positions, then recognize content, and finally organize layout and output results. For multi-page parsing, it typically processed one page at a time and stitched the results together—this was OCR 1.0.

Different modules had their specific duties, but errors in earlier steps could propagate downstream—if one stage sneezes, the whole system catches a cold.

With OCR 2.0, Vision-Language Models (VLMs) began generating parsing results directly from page images, reducing handoffs between multiple modules. The first-generation Unlimited OCR extended this end-to-end parsing capability to documents spanning dozens of pages.

△ AI-generated image

When document length increases from dozens to hundreds of pages, continuous parsing must ensure seamless content flow while controlling computational and caching overhead.

Wenxin Unlimited OCR 2 employs on-demand reference and chunked decoding, allowing the model to process more pages without expanding the computational scope per step with page count, nor retaining visual caches for all pages indefinitely.

*The team calls this step OCR 3.0: making continuous processing of entire documents the new technical goal—in one go, no pagination, infinite parsing, never crashing. *

And, this goal has an anime-inspired version.

The name "Unlimited OCR Works" pays homage to "Unlimited Blade Works" from Fate/stay night.

△ Homepage of the Fate/stay night anime official site

In the original setting, this is a type of magecraft called "Reality Marble," which, when activated, expands into a special space filled with countless swords.

After the previous generation was released, Hugging Face user Snarcy discovered this naming pun and adapted the classic incantation into an OCR version.

For the second generation, the team actually included this adaptation in the introduction of the technical report, crediting Snarcy.

So, opening this report discussing attention mechanisms and VRAM overhead, you will find the following incantation:

Pixels are my body, and tokens are my blood. 像素为身，而Token为血。

I have parsed over a thousand pages. 我已阅尽千余页文档。

Unknown to KV cache. 不为KV Cache所限，

Nor known to context limits. 不为上下文长度所缚。

So, as I pray： Unlimited OCR Works 故而，我在此祈愿：Unlimited OCR Works。

From 47GB down to 1.77GB, the line "Unknown to KV cache" wasn't just sung for show.

Truly a case of eternal chuunibyou.