How KIQP's technical team processed 30,000 pages of historical Arabic archives ~65% faster

The Iraqi Knowledge Portal (KIQP) is an Iraqi digital knowledge initiative launched by the Center for Cataloging and Information Systems within the Department of Intellectual and Cultural Affairs of the Al-Abbas Holy Shrine, in collaboration with Al-Kafeel University. Its mission is to preserve, organize, and expand access to Iraq’s historical, academic, cultural, and archival knowledge through a unified searchable digital environment.

It is the largest digital knowledge platform in Iraq. The platform currently works with materials such as newspapers, magazines, and books, with plans to expand further into manuscripts, theses, dissertations, and other archival collections. KIQP’s technical team worked with Datalab to test whether it could simplify how their materials are digitized.

The challenge: A fragmented document processing workflow

KIQP relies on document processing to make their historical knowledge base searchable. Before Datalab, they divided their document processing workflow across multiple OCR and image-processing tools. This was done for a few reasons:

• Page structure and formatting were often lost, especially in older scans.
• Historical documents required significant manual preparation, including adjustments to image quality, lighting, contrast, and readability.
• Multilingual accuracy was another challenge, particularly for Arabic and other less-represented languages that are important to their collections.

The complexity of a fragmented workflow led Mahdi, a software developer and programmer at KIQP, to explore other OCR solutions. His team tested and combined several OCR and preprocessing approaches, including Tesseract OCR and PaddleOCR, together with manual image adjustments and human review.

These tools performed well on languages such as English, German, and French, but the results were less consistent for Arabic, Javanese, and Swahili. Reaching an acceptable result often required moving between tools, preparing images manually, combining outputs, and correcting errors before the content could enter KIQP’s portal workflow.

The solution: Datalab’s API

His team learned about Datalab while researching high-performing OCR models and document-processing solutions. They first encountered the project through GitHub, which led them to evaluate further on the official Datalab platform.

The case for a deeper evaluation became clear during their early tests on older Arabic documents. The team found that Datalab could bring document understanding, OCR, and much of the preprocessing into a more unified workflow while preserving page structure more effectively than their previous process.

“The combination of stronger results on historical Arabic material and a substantial reduction in manual steps made Datalab a very promising option for our workflow.” — Mahdi, KIQP

“The combination of stronger results on historical Arabic material and a substantial reduction in manual steps made Datalab a very promising option for our workflow.” — Mahdi, KIQP

The pipeline the team tested is simple: a PDF or older scanned image is submitted through the Datalab API, Datalab processes the document and returns structured Markdown, and the output then goes through human review and validation before it would feed into KIQP’s indexing, search, and downstream knowledge workflows. In this setup, Datalab sits at the core document-conversion stage, replacing several separate preprocessing and OCR steps that were previously managed independently.

A printed two-column Arabic poem parsed by Datalab, side by side with the original scan

Some of KIQP’s most complex materials are older Arabic newspapers, magazines, books, and archival pages with degraded image quality, historical typography, dense layouts, or handwritten content and marginal notes. In testing, Datalab performed very well on older printed Arabic material and reduced the amount of manual preprocessing required. Handwritten Arabic remains more challenging than printed Arabic, so human review stays part of the workflow for difficult archival material.

A handwritten Arabic manuscript folio with marginal notes parsed by Datalab

The evaluation results

During the evaluation, Mahdi and the technical team processed approximately 30,000 pages through Datalab. Based on the team’s internal estimates, overall document-processing time in the testing period dropped by approximately 65% compared with their previous workflow. They also saw a material reduction in manual intervention and review burden over the course of testing.

“Without Datalab, we would have continued relying on a fragmented combination of OCR engines, image-preprocessing tools, manual corrections, and repeated human review.” — Mahdi, KIQP

“Without Datalab, we would have continued relying on a fragmented combination of OCR engines, image-preprocessing tools, manual corrections, and repeated human review.” — Mahdi, KIQP

The revised pipeline reduced manual processing, improved throughput, reduced exposure to human error, and simplified the technology stack. These results point to a promising path for KIQP’s broader digitization work, as the platform has an estimated 1–3M-page backlog of books, PDFs, scanned materials, and archives that will need processing as its collections grow.