Every flag needs evidence: how ReviewerZero AI traces integrity issues back to the page

Fighting bad science at scale

Daniel Acuna is an Associate Professor at the University of Colorado at Boulder. He is also the founder of ReviewerZero AI, a research-integrity platform for publishers, editors, and integrity teams.

ReviewerZero runs 65+ checks across 12 dimensions of integrity, including images, statistics, references, replicability, peer review, and traces every flag back to its source. When ReviewerZero tells an editor a paper has a problem, the editor needs to see the evidence on the page.

The team is deep into two of the hardest checks: statistical errors and hallucinated references. More recently, they’ve added tables. All three depend on one thing going right: turning a messy scientific PDF into verifiable, trusted text.

The challenge: a false positive is an accusation

Hallucinated references come in two forms. First, is there a citation that doesn’t exist anywhere in the document? The second type is more subtle: the title might be real, but the authors are invented. This happens when the LLM invents plausible names instead using the real ones. As an author, being caught with fabricated references has real consequences, such as being banned from preprint archives.

It’s a serious claim to make, which means that accuracy is non-negotiable.

“Accusing someone of a hallucinated reference is problematic,” Daniel explains. “We need zero false positives.”

“Accusing someone of a hallucinated reference is problematic,” Daniel explains. “We need zero false positives.”

What’s tricky is that most false positives start in the parser. If a PDF is misparsed, a real reference looks fake. Even frontier LLMs can misread a DOI, making a legitimate citation look hallucinated.

Scientific papers are especially difficult because they mix dense text, figures, tables, and long reference lists. Tables and references often break across pages. Figures often have multiple panels. Some submissions consist of hundreds of pages of physics stapled together from separate parts.

ReviewerZero’s first tools struggled with some of these hardest cases:

• Amazon Textract was slower and less accurate. A single long document could take ~30 minutes and still come back wrong.
• Open-source parsers they tested weren’t strong enough on tables and split references across pages or in footnotes.
• Custom code filled the gaps. Daniel had written logic to handle borderline cases, like a table or reference cut off mid-page, which was code that needed constant upkeep.

Finding Datalab

Daniel came across Datalab through LinkedIn. After reaching out, Datalab’s team directly worked through the problem with him.

“That’s the passion in a startup,” Daniel says.

“That’s the passion in a startup,” Daniel says.

What initially sold Daniel were bounding boxes that indicated positional data. Datalab returns the location of every element on the page, so ReviewerZero could show editors exactly where an issue lives.

A few other characteristics stood out:

• Self-serve setup. “It was easy to integrate. With other tools you had to reach out to someone, but I could do it the same day.” Signing up for Datalab’s self-serve platform and getting an API key took minutes.
• Outstanding support. “Some of the Datalab folks have actual deep experience with scientific documents, and they are always willing to help”. Responsive support through email or Slack was a plus.
• Open weights models. “I’m in science, so people like open stuff.” Datalab’s open-source roots built trust with a research audience before a single API call.
• Accurate mode. It handled content cut off mid-page, which was especially important for multi-page tables and references.

How it works: from PDF to evidence

Datalab sits at the front of ReviewerZero’s pipeline. Every check downstream depends on its output.

• Parse. The paper goes through the Datalab API in accurate mode. Out come structured blocks — paragraphs, tables, references, figure panels — each with a bounding box.
• Check. ReviewerZero’s integrity checks run on the clean text: reference lookups, statistical tests, table consistency.
• Match. An LLM pass maps each flagged issue back to the Datalab block where it appears.
• Show. The report highlights the exact paragraph, table, or figure panel, so reviewers see the evidence in context.

The block-level structure is especially important. Daniel tried matching issues at the word level first, but thousands of tiny boxes confused the LLM. Paragraph- and table-level blocks give it a clean target, and figures come back with a box per panel rather than one box for the whole figure.

The results: faster, more reliable flags

~30 minutes to 3–5 minutes: On the hardest documents, which could be up to a hundred pages, Textract could take half an hour and still get it wrong. Datalab processes the same document in 3–5 minutes.

Tables and references that survive page breaks. Multi-page tables and split reference lists now come through intact. The accuracy here enabled ReviewerZero’s newest checks on statistics inside tables.

Less code overhead. The previous borderline-case logic is largely retired. Each fix used to take about an hour and kept recurring; now, the team saves roughly 30 minutes of engineering time every week.

Fewer manual double-checks. The team used to verify conversions by hand before trusting a hallucination flag. With accurate parsing, a flag is far more likely to be real.

What’s next: moving earlier in the pipeline

ReviewerZero used to step in after a paper was published. Now it’s moving closer to authors, catching problems before submission. That widens the range of documents it needs to read:

• Conference presentations. Some conferences bar presenters from promoting products or embedding video. ReviewerZero hopes to expand their use case to include PPTX files to flag potential violations in slides, embeddings, and speaker notes.
• Word documents, comments, and redlines for customers working earlier in the drafting process.
• Accessibility. With the European Accessibility Act in force, ReviewerZero is exploring checks like flagging low-contrast figures against WCAG.