Chandra 2.5 is available today in the Datalab API, and every mode (fast, balanced and accurate) runs on it. You don’t need to change anything to use it.

Here’s what changed, compared with Chandra OCR 2.4, the model it replaces:

Image descriptions in one fixed format. Every chart, diagram and photo comes back with the same six fields, including chart values and verbatim text from inside the figure. In json and chunks output they’re in a new image field.
38% fewer loops. Chandra 2.5 loops 38% less often than Chandra OCR 2.4 on our loop test set, and reads rotated pages better.
Faster in every mode. Median single-page latency is 40-43% lower in every mode. In balanced mode, it went from 12.9 s to 7.5 s.

Image descriptions in one format

Chandra OCR 2.4 described each figure in free text: an alt attribute and a block of prose, with no fixed fields. So the shape of the output changed from figure to figure.

Chandra 2.5 writes every figure in one structure, always in the same order:

A field is null when the figure doesn’t have it. Markdown and HTML output keep their current shape.

Here is a bar chart from a U.S. Department of Education report:

Figure 5 from “Characteristics of Public School Teachers,” The Condition of Education 2023, National Center for Education Statistics.

Chandra 2.5, balanced mode, the image field:

Every value matches the chart, including the small dip after 25 to 29 years. Code can depend on this shape: read data for chart values, mermaid for diagrams and text for labels, and skip the prose.

To get the field:

Accuracy

Chandra 2.5 is more reliable on hard pages, and it scores near the top of two public benchmarks.

Fewer loops. A loop is when an OCR model repeats the same line or table cell instead of moving on. On our loop test set, Chandra 2.5 loops 38% less often than Chandra OCR 2.4.

Rotated pages. Chandra 2.5 reads pages scanned sideways or upside down.

OmniParseBench. Last week we released OmniParseBench, an open parsing benchmark with 16,288 pass/fail tests on 2,937 pages in 92 languages. We’ll report releases on it from now on. With Chandra 2.5, Datalab accurate scores 92.7 and Datalab balanced scores 91.6:

* Used in creating and verifying at least some tests. Full results, and a tool to score any slice of the benchmark, are in the OmniParseBench post.

olmOCR-bench. Chandra 2.5 scores 87.4 on olmOCR-bench, up from 86.2 for Chandra OCR 2, our open-source model. Both use LightOn’s PP3, the benchmark postprocessing used in recent LightOnOCR and Infinity Parser results (it includes math rules from Infinity Parser’s evaluation code). We don’t think benchmark-specific postprocessing is good practice, so here are the scores raw, with each model’s own postprocessing, and with PP3:

Our runs: olmocr 0.4.27, olmOCR-bench revision 54a96a6. Headers and footers are left out. LightOnOCR’s own postprocessing is LightOn’s escaped-dollar fix, and its PP3 column includes it, as in LightOn’s own results. LightOnOCR-3-4B numbers are our reproduction of LightOn’s pipeline. LightOn reports 86.3.

Faster in every mode

Chandra 2.5 is faster in every mode. Here is what API users waited for, end to end (upload, queueing, the full pipeline and getting the result back), for every successful single-page request to the convert and marker endpoints:

Single-page /convert and /marker requests, end to end, one week before and after the release.

Median latency is 40-43% lower in every mode, and p90 latency is 34-51% lower.