Unlimited OCR Works

Youyang Yin; Huanhuan Liu; YY; Qunyi Xie; Chaorun Liu; Shiqi Yang; Shaohua Wang; Zhanlong Liu; Hao Zou; Jinyue Chen; Shu Wei; Jingjing Wu; Mingxin Huang; Zhen Wu; Guibin Wang; Tengyu Du; Lei Jia

Papers · arXiv 2606.23050

Unlimited OCR Works

Youyang Yin, Huanhuan Liu, YY, Qunyi Xie, Chaorun Liu, Shiqi Yang, Shaohua Wang, Zhanlong Liu, Hao Zou, Jinyue Chen, Shu Wei, Jingjing Wu et al.

Published 2026-06-22cs.CVcs.CL arXiv PDF

Read with translation and AI

Summary in three lines

🎯 Standard LLM-based OCR decoders suffer from linear KV cache growth and declining speed during long-horizon, multi-page document transcription.
🔧 Employs Reference Sliding Window Attention (R-SWA), where decoding tokens globally attend to all prefix/visual tokens while retaining only a constant causal window of recent outputs ( $n = 128$ ).
📈 Achieves 93.23% overall on OmniDocBench v1.5 (+6.22% over DeepSeek OCR) and maintains constant generation latency across long outputs.

Abstract

Recently, end-to-end OCR models, exemplified by DeepSeek OCR, have once again thrust OCR into the spotlight. A widely held view is that employing a large language model (LLM) as the decoder allows the model to leverage the prior distribution of language, leading to improved OCR performance. However, the downside is equally evident: as the output sequence lengthens, the accumulated KV cache drives up memory consumption and progressively slows down generation. This stands in stark contrast to humans, who exhibit no such decline in efficiency during long-horizon copying tasks. In this technical report, we propose Unlimited OCR, a model designed to emulate human parsing working memory. Taking DeepSeek OCR as the baseline, we replace all attention layers in the decoder with our proposed Reference Sliding Window Attention (R-SWA), which reduces attention computation costs while maintaining a constant KV cache throughout the entire decoding process. By combining the high compression rate of DeepSeek OCR's encoder with our constant KV cache design, Unlimited OCR can transcribe dozens of pages of documents in a single forward pass under a standard maximum length of 32K. More importantly, R-SWA is a general-purpose parsing attention mechanism - beyond OCR, it is equally applicable to tasks such as ASR, translation, etc. Codes and model weights are publicly available at http://github.com/baidu/Unlimited-OCR.

Reading guide

Problem and motivation

Humans can transcribe hundreds of pages or translate hours of audio continuously without losing speed, using working memory to track local context while original source material remains accessible. In contrast, current multimodal large language models (MLLMs) and dense OCR systems struggle with long-horizon parsing. Existing systems typically process multi-page documents in a page-by-page for-loop fashion that resets memory at every step, creating an engineering workaround rather than true continuous processing.

When end-to-end models attempt long decoding sequences, standard multi-head attention (MHA) accumulates key-value (KV) cache linearly with the output length $T$ . This causes progressively rising GPU memory consumption and latency degradation. Linear attention models with recurrent updates also fail because recurrent state updates progressively blur visual reference features.

Key ideas

Reference Sliding Window Attention (R-SWA): Generated tokens attend globally to all static prefix reference tokens (visual representations and instructions) while attending locally only to the preceding $n$ output tokens ( $n = 128$ by default).
Constant Decoding KV Cache: By evicting tokens older than $n$ in the generation buffer via a FIFO queue, the decoding cache footprint is strictly upper-bounded by $L_{m} + n$ , ensuring constant per-step latency and constant GPU memory usage.
High-Compression Visual Prefix: Paired with the DeepEncoder from DeepSeek OCR (which compresses a $1024 \times 1024$ image by $16 \times$ down to 256 tokens), dozens of pages can be prefilled in a single pass without exceeding standard context limits.

Method

Unlimited OCR builds on the DeepSeek OCR baseline, retaining its frozen DeepEncoder (cascading SAM-ViT and CLIP-ViT) paired with a 3B-parameter Mixture-of-Experts (MoE) LLM decoder (activating 500M parameters). All standard MHA layers in the decoder are replaced with R-SWA.

Mathematical Formulation

For a generated token at decoding step $t$ , the accessible attention set $N (t)$ comprises two segments:

$N (t) = P \cup D_{n} (t)$

where $P = {1, \dots, L_{m}}$ is the static prefix segment of length $L_{m}$ , and $D_{n} (t)$ is the causal sliding window over generated tokens:

$D_{n} (t) = {j ∣ max (L_{m} + 1, L_{m} + t - n) \leq j \leq L_{m} + t - 1}$

The attention weight $α_{t j}$ from token $t$ to position $j \in N (t)$ is:

$α_{t j} = \frac{exp ( \frac{q _{t}^{⊤} k _{j}}{d _{k}} )}{\sum _{i \in N (t)} exp ( \frac{q _{t}^{⊤} k _{i}}{d _{k}} )}, j \in N (t)$

The output representation is aggregated over the accessible set:

$o_{t} = j \in N (t) \sum α_{t j} v_{j}$

KV Cache Scaling

While standard MHA scales linearly with decode length $T$ ( $C_{MHA} (T) = L_{m} + T$ ), R-SWA bounds the cache capacity:

$C_{R-SWA} (T) = L_{m} + min (n, T) \leq L_{m} + n$

When decoding dominates ( $T ≫ n$ and $T ≫ L_{m}$ ), the relative cache ratio approaches zero:

$ρ (T) = \frac{L _{m} + n}{L _{m} + T} \approx \frac{L _{m} + n}{T} \to 0$

Results

OmniDocBench v1.5: Unlimited OCR achieves an overall score of 93.23% (a 6.22% absolute gain over DeepSeek OCR at 87.01%), lowering Text Edit Distance from 0.073 to 0.038 and raising Table TEDS from 84.97% to 90.93%.
OmniDocBench v1.6: Achieves 93.92% overall, matching or outperforming contemporary end-to-end models like Qianfan-OCR (93.90%) and FireRed-OCR (93.26%).
Inference Speed: In OmniDocBench tests, Unlimited OCR delivers 5,580 TPS compared to DeepSeek OCR's 4,951 TPS (a 12.7% speedup). Under theoretical maximum concurrency at 6,144 generated tokens, Unlimited OCR hits 7,847.71 TPS whereas DeepSeek OCR drops to 5,822.87 TPS (a 35% performance gap).
Multi-Page Long-Horizon OCR: Evaluated on multi-page books, Unlimited OCR achieves an Edit Distance of 0.0362 on 2 pages, 0.0572 on 20 pages, and 0.1069 on 40+ pages (retaining 96.90% Distinct-35).

Limitations and open questions

Prefill Bottleneck: The current model is still constrained by the maximum input context length (32K tokens). While DeepEncoder compresses a single page to 256 tokens, hundreds of pages still occupy significant prefill memory.
Small Text Degradation: In multi-page settings, using DeepEncoder's 1024×1024 Base mode causes recognition errors on tiny font sizes due to resolution limits rather than attention window loss.
External Prefill Pool: Future work explores training on 128K context and creating an autonomous dynamic prefill KV fetching mechanism that mimics flipping pages.

Key terms

Reference Sliding Window Attention (R-SWA): An attention mechanism where each decoding token attends globally to all static prefix/visual tokens and locally to the preceding $n$ generated tokens.
DeepEncoder: A visual encoder cascading SAM-ViT and CLIP-ViT that achieves a 16x token compression rate, compressing a 1024x1024 image into 256 visual tokens.
Constant KV Cache: A decoding cache structure capped at $L_{m} + n$ tokens using a FIFO eviction queue, keeping GPU memory and per-step latency flat.
Long-horizon Parsing: Continuously transcribing or parsing multi-page documents (e.g., 20 to 40+ pages) within a single uninterrupted forward pass.
OmniDocBench: A multi-faceted document parsing benchmark evaluating text edit distance, formula CDM, table structure (TEDS), and reading order.
Flash Attention v3 Kernel Latency: The per-step kernel duration during decoding, which increases linearly in standard MHA but remains constant under R-SWA.
Distinct-n: A diversity metric defined as the ratio of unique n-grams to total n-grams used to measure hallucination or repetitive loops during generation.

Questions

What is the default sliding window width $n$ in Unlimited OCR?

The causal decoding window $n$ defaults to 128 tokens, which is sufficient to track progress and intra-page continuity.

How does R-SWA differ from vanilla Sliding Window Attention (SWA)?

Vanilla SWA applies a local window across all tokens, which would discard or progressively blur visual reference tokens. R-SWA keeps all visual/prefix tokens globally accessible without state transitions while sliding only across generated text.

How many parameters are activated during inference?

Unlimited OCR uses a 3B total parameter MoE architecture with only 500M activated parameters during inference.

Why does DeepSeek OCR slow down as output length grows while Unlimited OCR does not?

DeepSeek OCR relies on standard MHA where the KV cache size grows with sequence length, increasing attention compute and memory overhead. Unlimited OCR maintains a constant KV cache of $L_{m} + n$ , preserving flat latency per step.

The overview on this page was written by AI from the paper and may contain mistakes. Read the paper for the details.

Read with translation and AI