What a Sentence Forgets: Preserving Context in Document Ingestion

Key takeaways

Traditional document chunking strips vital context — headings, referents, and citations — turning valid sentences into unusable or misleading facts.

Page-based PDF parsing creates high fragment rates (up to 43%), whereas sentence-based division with heading-aware stack guards reduces errors to 1%.

Effective retrieval-augmented generation (RAG) ingestion requires parse-time context retention and explicit metadata tagging before vectors are indexed or passed to large language models (LLMs).

Retrieval systems depend on a step that receives comparatively little attention. Before a document can be searched, quoted, or used to answer a question, it must be divided into pieces small enough to index and small enough to pass to a model.

The division itself is trivial to implement. Preserving meaning across it is not.

A sentence removed from page 40 of a document no longer knows that it came from page 40. It does not know which section head it sat under, what its pronouns referred to, or what the statute it cites actually says. The document held all that information. The extracted sentence does not hold any of it, and nothing downstream can recover it.

In this blog post, I document five faults observed in a production document-ingest system — together with the measurements taken from it — present a summary of the relevant research, and provide five remediations of those faults.

The 5 faults in production ingestion pipelines

Parsing document text for downstream tasks like RAG often looks like a solved problem — until you examine the stored outputs. Most chunking strategies suffer not from complex algorithmic failures, but from structural oversights during extraction: Sentences lose their context, page splits bisect words mid-thought, and headings vanish or blur into adjacent prose.

These five distinct failure modes across production ingestion pipelines show how standard document processors unintentionally corrupt data before a model ever sees it:

• The sentence ceases to be interpretable

The sentence ceases to be interpretable

• A page boundary is not a sentence boundary

A page boundary is not a sentence boundary

• Heading structure is identified and then discarded

Heading structure is identified and then discarded

• On web content, heading text is worse than absent

On web content, heading text is worse than absent

• The sentence cites a reference that is never resolved

The sentence cites a reference that is never resolved

The sentence ceases to be interpretable

Consider a single extracted claim:

“It holds up to 128 keys.”

The referent for it appeared in a heading three lines above the sentence, and the heading was not included in the extracted piece. The claim is now unusable, though it remains grammatically well formed and will be returned by a search as though it were complete.

This is the general case. The four faults that follow are all variations of it.

This fault is addressed by remediations 1 and 2.

A page boundary is not a sentence boundary

Dividing a PDF page by page is an intuitive approach, because a page is a meaningful unit to a human reader. It is not a unit of meaning to a parser.

Measured across the 590 pages of a 686-page document that yielded any text at all, 73% of pages began in the middle of a sentence and 78% ended in the middle of one. In one instance, a page boundary divided the word standards, and two separate model invocations were each subsequently asked to derive a fact from one half of the word.

The consequence persists well beyond the ingestion run. When that document was processed using page-based division, 43% of the resulting stored claims did not end in a full stop. These are sentence fragments that were recorded as facts, and they are returned in answer to questions:

“total income at the rate as specified in any Central Act for relevant tax year.”

The same document, reprocessed four days after the division logic was corrected, produced a figure of 1%. It is worth noting that sentence-based division produced fewer pieces than page-based division, not more.

This fault is addressed by remediation 3.

Heading structure is identified and then discarded

Nearly every parser already locates headings, since heading detection is how it determines that one block has ended and another has begun. Having located them, most implementations then discard the heading text.

The structural information therefore passes through the parser and is deleted a line later.

Retaining it requires no additional reading of the document and no additional inference, because the information is already available at the point where it is dropped. In one production codebase, the entire remediation consisted of a stack maintained during a traversal that was already being performed.

This fault is addressed by remediations 1 and 2.

On web content, heading text is worse than absent

A heading does not terminate in a full stop. Consequently, text-extraction logic that segments on sentence-final punctuation will join the heading to the paragraph beneath it and store the result as a single unit. That unit then becomes the evidence quotation:

“EdgeKV Key limits A namespace holds up to 128 keys.”

No such sentence appears on the source page. Two headings and one paragraph were concatenated into a single assertion. On one measured page, 11 section headings were stored as stand-alone facts, each attributed to the document as something it had asserted.

This fault is addressed by remediation 4, and guarded at page boundaries by remediation 3.

The sentence cites a reference that is never resolved

Documents often refer to themselves and to one another, and statutory text does so more than most:

“Irrespective of anything contained in section 67, where the assets of a company are distributed …”

The sentence is accurate but cannot be used in isolation. The string “section 67” occupies the text in the same manner as any other words, so the claim presents as complete and carries no indication that it is contingent on material elsewhere.

Across 3,651 stored claims, 1,188 of them — approximately one in three — named some other part of the document or another document entirely. There were 1,665 references in total, of which 493 were distinct, comprising sections, subsections, schedules, chapters, and named statutes.

Three distinct operations are frequently conflated here, and the distinction matters.

• Recording the reference is free and carries no risk, because the reference is the document's own text.

Recording the reference is free and carries no risk, because the reference is the document's own text.

• Resolving the reference is a retrieval operation that requires its own evidence and can fail.

Resolving the reference is a retrieval operation that requires its own evidence and can fail.

• Substituting the reference with its content — replacing “section 67” with what section 67 states — constitutes a model that is editing a statute, and produces an artifact that nothing downstream can distinguish from the original.

Substituting the reference with its content — replacing “section 67” with what section 67 states — constitutes a model that is editing a statute, and produces an artifact that nothing downstream can distinguish from the original.

This fault is addressed by remediation 5.

How research addresses evidence extraction and origin tracking

Two questions in this area have been studied rigorously:

• How to make an extracted piece self-sufficient?

How to make an extracted piece self-sufficient?

• How to record where it originated?

How to record where it originated?

These questions have been addressed by research in the following areas:

Decontextualization

Decontextualization is defined as the task as rewriting a sentence such that it survives removal from its context without alteration of meaning. The most useful contribution of this research is the observation that certain sentences cannot be rewritten in this way, and that identifying them is itself part of a correct result: The authors report that roughly 40% of sentences were classified either as infeasible to decontextualize or as requiring no edit at all.

A citing study reports a relative improvement of up to 30% in mean reciprocal rank for decontextualized evidence over verbatim sentences. Mean reciprocal rank is the average of 1 divided by the position of the first correct result across all queries, so a score of 1.0 means the right answer always came back first, 0.5 means it typically sat second, and anything below about 0.1 means users are scrolling past 10 results before finding it.

Claimify

The LLM–based method Claimify extracts claims from a question-answer pair in four stages:

• Construction: The first stage splits the answer into sentences and constructs a context for each one, where that context is a configurable combination of surrounding sentences and document metadata, including the header hierarchy (remediation 1 arriving under another name).

Construction: The first stage splits the answer into sentences and constructs a context for each one, where that context is a configurable combination of surrounding sentences and document metadata, including the header hierarchy (remediation 1 arriving under another name).

• Selection: Selection then discards material that asserts nothing.

Selection: Selection then discards material that asserts nothing.

• Disambiguation: Disambiguation determines whether the sentence is ambiguous — and, when the surrounding context cannot resolve the ambiguity, declines rather than guessing.

Disambiguation: Disambiguation determines whether the sentence is ambiguous — and, when the surrounding context cannot resolve the ambiguity, declines rather than guessing.

• Decomposition: Decomposition produces the claims. The authors report that 99% of resulting claims are entailed by their source sentence.

Decomposition: Decomposition produces the claims. The authors report that 99% of resulting claims are entailed by their source sentence.

DnDScore

DnDScore serves as a caution against applying both decomposition and contextualization without care. Decomposition renders each fact simple; contextualization renders each fact larger. Applying both leaves the evaluator unable to determine which component is being verified.

HiChunk

HiChunk fine-tunes a language model to recover multilevel document structure. Its more significant contribution is the accompanying benchmark, HiCBench, which the authors introduced on the grounds that existing test sets could not distinguish good chunking from poor chunking, because the supporting evidence was too sparse for boundary placement to affect the score.

RAPTOR

RAPTOR clusters pieces, summarizes each cluster, then clusters the summaries recursively. Retrieval from the resulting tree, coupled with GPT-4, improved the best prior result on the QuALITY benchmark by 20 points of absolute accuracy.

Contextual Retrieval

Contextual Retrieval has a model that generates a short situating note for each piece and indexes the note alongside it, reporting that the top-20 retrieval failure rate fell by 49%, and by 67% in combination with re-ranking.

Late chunking

Late chunking processes the entire document through a long-context embedding model and applies the chunk boundaries afterward, at the mean-pooling step, so that each piece's vector reflects the whole document without any new text being generated.

Three systems address the underlying problem directly.

GROBID

GROBID restructures PDFs into TEI XML, though with a particular focus on technical and scientific publications rather than documents in general.

DocParser

DocParser parses the complete document structure, including nested figures, tables, and table cell structures.

Docling

Docling converts PDFs, Office files, HTML, and images into a single document model that preserves reading order, tables, and provenance. Its chunker emits a section path and page-level bounding boxes for every chunk, which makes it the closest thing available to a reference implementation of the remediations that we discuss next.

5 parse-time structural remediations

Parsing unstructured document text into clean, chunk-level context is rarely an AI problem — it is almost always an architectural one. Rather than relying on heavy LLM calls to repair fragmented documents or infer missing headings downstream, all five remediations outlined in the table execute directly at parse time without a single model dependency.

By shifting these fixes upstream into the parser's single-pass document traversal, you can systematically eliminate structural faults, retain cross-page context, and yield far more reliable chunking for downstream retrieval.

Retain heading text

1, 3

A stack maintained during parsing

Carry the stack across page boundaries

1, 3

Parser state scoped to the document

Divide on sentences rather than pages

Sentence splitter, with a guarded join

Keep headings out of paragraph text

Block typing prior to text extraction

Record references without resolving them

Pattern matching into a stored field

All five remediations occur at parse time; none requires a model

Retain heading text

Traverse the document in order, maintaining a single list of currently open headings. On encountering a heading of level N, pop all entries at level N or deeper, then push the new heading; every emitted block receives a copy of the list. Heading levels derive from h1 through h6 in HTML, or from clustered font size and weight in a PDF. No additional traversal is required, since the parser is already performing this walk.

Carry the stack across page boundaries

The heading stack is a property of the document rather than of the page. An implementation that iterates over pages and instantiates a fresh parser for each will discard the stack several hundred times over a long document; the iteration should occur inside a single traversal that owns it. Running headers must be suppressed first, otherwise the title repeated at the head of each page will be pushed onto the stack as a genuine heading.

Divide on sentences rather than pages

Join before splitting: Wherever the final block on a page does not terminate in a full stop and the following page does not open with a heading, concatenate them and repair any hyphenation introduced by the line break. Then apply a sentence splitter — pysbd, spaCy, and the ICU break iterator are all suitable and none is a model — and pack complete sentences up to the size limit.

The heading test is a stopping condition rather than a context mechanism. A page with a successor that opens with a heading has genuinely ended, notwithstanding the absence of punctuation, and an unguarded join would weld that heading to the fragment above it, reproducing fault 4 by a different route. This change reduced the fragment rate from 43% to 1%.

Keep headings out of paragraph text

The underlying defect is flattening: Markup is stripped and the enclosed text retained, so a heading and its following paragraph become a single string. Type blocks before extracting their text, so that a heading is written to the piece's metadata rather than its body. For data already stored, a serviceable detector is a sentence containing a capital letter mid-string with no preceding full stop.

Record references without resolving them

Match the citation forms the corpus actually uses — section 67, subsection (17), Schedule III, Chapter VI-A, and the names of other statutes — using a pattern list and a gazetteer. Store the matches in a field adjacent to the section path, so that each piece carries both its own text and an explicit list of its dependencies. Resolution belongs at query time, where it has access to its own evidence and can fail visibly.

When there is no context to capture

The five remediations recover the context that the document actually contains. Sometimes a document doesn’t contain any context. A page may carry no headings at all, a sentence may open with a pronoun whose referent was never written down, or a citation may point to a section held in some other file.

In those cases, the correct action is to record that the value is missing. An empty field means the document said nothing. It is not permission to supply a default.

Three substitutions are commonly made, and each is a guess presented as a fact:

• A piece with no heading above it is assigned the document title, or the last heading seen on an earlier page.

A piece with no heading above it is assigned the document title, or the last heading seen on an earlier page.

• A pronoun with no antecedent in the piece is resolved to the nearest preceding noun.

A pronoun with no antecedent in the piece is resolved to the nearest preceding noun.

• A citation is replaced with the text of the section it names.

A citation is replaced with the text of the section it names.

All three substitutions are inexpensive to implement, and all three produce output that looks more complete than what it replaced. That is precisely the difficulty. Once a guessed section is stored alongside a genuine one, no subsequent measurement can distinguish them, and the quality of the corpus can no longer be established by inspection.

How Akamai can help

Executing parse-time remediations at scale requires low-latency processing before data is stored in vector memory. By deploying custom parsing workers on Akamai Functions paired with Akamai Cloud for AI, platform engineering teams can process long-context document streams globally at the edge without incurring centralized cloud egress costs or ingestion bottlenecks.

About the author

Sachidananda Karkala

Sachidananda Karkala is a Singapore-based Enterprise Architect with more than 26 years of experience driving digital transformation. Progressing through roles ranging from Software Engineer to seasoned Solution Architect, he has designed and delivered scalable enterprise ecosystems, complex cloud migrations, and next-generation digital solutions for strategic customers across the APAC region. Sachi is passionate about artificial intelligence and using technology to elevate user experiences. He also enjoys music, reading, and spending quality time with his family outside of work.

Tags