Bring Your Own Schema: How We Built kniv v7

A family of open encoders, from 55M to 538M parameters, that read across sentences, parse every sentence, and extract the entities, relations and fact attributes you name at run time, with no LLM call and no retraining.

An LLM can extract anything you ask for, at one API call per message. A tagger is cheap and runs locally, but it only finds what it was trained to find.

Agent memory needs the first one's flexibility at the second one's cost.

In May we released kniv v5, a cascade that gives the linguistic layer in a single pass: part of speech, named entities, dependencies, semantic roles and dialogue acts. It did that job well. But the memory systems we built on it kept asking for the next thing, and the next thing was facts. Who lives where. Who works for whom. What someone plans, as opposed to what they did. They wanted those facts in their own schema, not ours, and they needed to know whether each one is asserted, negated, hoped for or just reported.

So today we're releasing kniv v7, a family of six models with the same outputs and the same runtime, from a 55M-parameter model for small devices to the two 500M-parameter models that set its quality:

All are Apache-2.0, with ONNX in three precisions included. Each pairs a fixed linguistic layer with a knowledge layer whose entity types and relations are inputs. You describe what you want in plain English, and the model reads for it.

This post follows the engineering behind that change: how we taught a shared encoder to read a schema, trained it on incomplete supervision, distilled it into smaller models, and made the resulting pipeline practical to run locally.

What v7 Does

One call returns two layers.

The linguistic layer is fixed. It covers part of speech, morphology, lemmas, named entities, dependency trees, semantic roles, dialogue acts and sentiment per sentence, and coreference.

The knowledge layer follows your schema:

• entities typed with your type names;
• facts for the relations you name;
• for each fact: its polarity (positive or negated); its modality (asserted, hypothetical, planned, desired, questioned or reported); its holder (who says so); its time (when it applies).
• its polarity (positive or negated);
• its modality (asserted, hypothetical, planned, desired, questioned or reported);
• its holder (who says so);
• its time (when it applies).

Think of it as a form whose blanks you write yourself. kniv fills them in from the text, and notes who said it and whether it holds.

Conversations are first-class: each speaker is an entity. So "I live in Berlin" becomes (user, lives in, Berlin), attributed to the user.

Change the relations list and you've changed what the model looks for. No training and no labelled examples are needed. The only dependencies are onnxruntime, tokenizers and numpy. Every model in the family takes the same call, and returns these two facts; only the repository name changes.

Documents, Not Sentences

Most open extraction models work one sentence, or one entity pair, at a time. Real text doesn't keep its facts in one sentence. In Re-DocRED, about half of the gold facts span sentences: the entity is named in one, and the fact about it arrives in the next, often through a pronoun.

kniv reads a whole window, up to a page of text, as one unit:

• coreference links mentions across the page;
• the marker pass sees every mention of every entity at once, and pools context from wherever it is in the window, so a pair can be related by evidence two sentences away;
• the evidence pass in training teaches the relation head which sentences support each fact.

The result is that the distance between two entities costs surprisingly little. Re-DocRED test, gold entities, by where the two entities are mentioned:

For DeBERTa, a fact whose entities are two or more sentences apart is extracted almost as well as one inside a single sentence. Given "Ada Lovelace was born in London in 1815. She later worked with Charles Babbage on his Analytical Engine.", it returns (Ada Lovelace, worked with, Charles Babbage), read through "She" in the second sentence.

Longer documents are split into windows automatically. Each window is read as a unit, and facts are not linked across windows (see "Where It Breaks").

Why a Fixed Schema Was a Dead End

Our previous research line (v6) reached 0.764 F1 on Re-DocRED, a document-level relation benchmark, given the gold entities. It got there by porting ATLOP's marker pass: mentions are wrapped in markers, and context is pooled locally around each entity pair. That number held for exactly one schema, Re-DocRED's 96 Wikipedia relations and 6 entity types. For a support desk, a CRM, a codebase or a personal memory, it was useless.

The first attempt was surgery. We transplanted the fixed relation head into a head that reads relation names. At step 0 the transplant was exact, reproducing the old logits, and then we trained on. It never regained parity: 0.755 against 0.764, and only 0.204 on unseen FewRel relations (8-way). The architecture could read labels. It just hadn't learned to.

The finding that changed the plan was that label variety beats architecture. Our own teacher-annotated data, however large, narrowed zero-shot behaviour to our own labelling style. Pretraining on thousands of knowledge-base relations, each worded in several ways, took unseen FewRel from 0.204 to 0.480. No architectural change we tried came close to that.

The second finding was that the encoder had to move. Keeping the encoder frozen and adding adapters left the linguistic layer bit-identical. But it capped relations at about 0.76: the representation simply didn't carry what document-level relations need. So v7 does it in two stages. Stage A pretrains the encoder itself for relations and entity types. Stage B then trains every head on that encoder, with the linguistic scores gated at every evaluation.

Two Teachers, Because No One Won

We ran a bake-off before committing compute: DeBERTa-v3-large, Ettin-400M and UltraBERT-380M. Each got the same budget, data and recipe, a short knowledge pretraining followed by a short linguistic stage. The selection rule was written down before any run started.

No single encoder won.

• DeBERTa led the linguistic tasks and the fixed-schema relations.
• UltraBERT led the zero-shot knowledge tasks by 4.2 points, and handles longer windows.
• Ettin-400M was behind both on knowledge and 2 points behind on parsing, so we dropped it.

So we trained both, each tuned for its own best result rather than for a matched comparison. They are the two xlarge models, xlarge-u (UltraBERT) and xlarge-d (DeBERTa), and the teachers the smaller models learn from:

One detail cost us a week. UltraBERT's tokenizer splits more words into pieces than DeBERTa's, and splits them with less of the word in the first piece (re|authorized). So reading each word from its first subword, as v6 did, cost UltraBERT SRL recall on long words: 0.652 against DeBERTa's 0.780. Pooling over all of a word's subwords fixed it.

The Architecture: Three Passes, One Encoder

kniv runs one encoder up to three times per window:

• Plain pass. The window as text. Every linguistic head reads it, along with coreference and the entity typer.
• Label pass. Your relation and type names, with their descriptions, are encoded into vectors. This happens once per schema, and the result is cached.
• Marker pass. The window again, with every entity mention marked and a speaker tag before each turn. Every entity pair is scored against every relation, then each kept fact gets its attributes.

Why passes instead of separate models? Small LoRA adapters give the label and marker passes their own capacity on one shared encoder. The plain pass uses the trained encoder itself.

Relations. We kept ATLOP's mention log-sum-exp and localised context pooling. On top sit head and tail extractors, a compact pair representation, cross-attention from labels to the document, and a learned threshold column. A relation is predicted when its score clears that pair's own threshold.

Entity types. Candidate spans are up to 12 words within a sentence. Each is scored against type vectors from the same label encoder that reads relations, so types and relations share one way of reading a schema.

Fact attributes. For every extracted (head, relation, tail), a small head reads the marker pass and predicts:

• polarity and modality, as fixed classes;
• the holder and the time, as pointers to entities in the window.

Training on Labels We Know Are Incomplete

Much of our relation supervision came from our own LLM annotation of a varied corpus: prose, conversations, technical and business text. We measured those teacher labels against gold. They were precise but incomplete: 0.71 precision, 0.31 recall with gold entities.

Train on that naively and you teach the model that every missing fact is false. Two-thirds of the true facts would become negative examples.

So v7 trains positive-unlabelled throughout:

• Every training item carries its own schema.
• Relation and type columns outside an item's schema are masked. They are never treated as negatives.
• Unlabelled pairs and spans are down-weighted rather than counted as "no relation".

Holdouts were fixed before any training. The 15 standard FewRel zero-shot relations were removed from every training source, so the zero-shot numbers below are real zero-shot.

Training then ran in stages, each gated on the linguistic scores:

• knowledge pretraining;
• every head jointly;
• gold linguistic data added to the mix;
• head fine-tunes with the encoder frozen: a predicate-aware SRL role head and a BiLSTM NER head;
• a knowledge-only pass;
• a coreference pass;
• an evidence pass that teaches the relation head which sentences support a fact.

Each stage ran on a single A100 40GB in bf16.

Calibration Was Worth More Than Architecture

The most surprising number of the project was zero. Early on, thresholded zero-shot F1 was 0. Yet the ranking was already right: the correct relation was usually the top-scoring one. The model knew the answer. Its scores were just on the wrong scale for an unseen relation to clear the threshold.

One offset, chosen on held-out data, fixed it. That pattern repeated so often that calibration is now part of the model: calibration.json ships with the weights and holds

• per-relation thresholds, chosen on dev;
• an offset for relations the model never trained on;
• an entity-span offset;
• the coreference link bias;
• the rule for grouping mentions into entities.

You can change every one of them at run time.

The span offset is the clearest case. Unseen entity types need a lower bar than trained ones. Moving the offset from 0 to about -2 was worth up to 8 points per CrossNER domain, with no retraining.

Calibration also told us where it can't help. On Re-DocRED, a threshold tuned per relation on the test set itself (a cheat, used only as an upper bound) adds at most 1 point. The gold relation is ranked first for 72-73% of facts and is in the top four for 98%. So what remains is choosing pairs and ranking, not thresholds.

What Mattered More Than a Bigger Head

The experiments that moved the numbers changed the supervision. The ones that changed the head, on the same supervision, mostly didn't:

• A learned entity linker. Grouping mentions into entities with a trained linker did not beat exact name matching plus coreference. The rule stays.
• Distilling one model's relations into the other. Averaging the two teachers' relation decisions is worth +2.7 F1, so we tried to teach UltraBERT DeBERTa's half. It gained 0.35 on Re-DocRED and lost 2 to 3 points of zero-shot. With the encoder frozen, a relation head can't reproduce decisions that come from another encoder's reading of the text. If you need the best relations, run both models and average them.
• Type descriptions. We rewrote entity-type descriptions in four styles. No style helped reliably: UltraBERT moved by at most a point, and DeBERTa swung from +7 to -11 depending on the domain.
• Bigger typing heads. An MLP span head and hierarchical type schemas changed CrossNER by under a point and cost 3 points on Few-NERD.
• Supervised heads on frozen vectors. Probes show that the information for entity typing is in the representations. But no head trained on frozen vectors beat the jointly trained one.

The lesson that runs through the whole project: the limit was usually the data, not the head. Broader, more varied supervision (thousands of relation types, many wordings, incomplete labels handled as incomplete) is what built the knowledge layer. A cleverer head on the same supervision rarely did.

Distilling the Family

The two xlarge models are accurate but large. The smaller models are students: each learns from both teachers at once, on 117,000 windows of varied text the teachers labelled with everything they produce, plus the gold linguistic data.

Two teachers are better than one. We mixed the teachers' outputs per task, with weights chosen on dev, before distilling anything. The mixture beats either teacher on every head: +2.2 points on SRL, +2.7 on relations and 1.3 to 2.2 on entity types. That mixture, not either teacher, is what the students learn to imitate.

What the students learn from. Copying the teachers' final decisions isn't enough for an open schema. The students also learn how the teachers rank every relation for each entity pair and every type for each span, and the vectors the teachers give to relation and type names. That is what lets a student handle a relation it never saw in training. Adding cross-attention layers to the student's relation head on top of that changed nothing.

Attributes have to be distilled too. Trained only on the teacher corpus's hard labels, where 1.7% of facts are negated, the first students never predicted negation at all: negation F1 0.00 for the smallest, 0.30 for large. Distilling the teachers' attribute predictions, and repeating the windows that contain a negated fact, brought large to 0.72, level with the teachers.

What longer training buys. Training the small students three times as long (30,000 steps instead of 10,000) improved relations and zero-shot clearly: tiny gained 4.7 points on Re-DocRED with gold entities and 6.8 on 10-way FewRel. It didn't move NER, SRL or coreference. Those are limited by how much a small encoder can hold, not by how long it trains.

Coreference is where size shows most. On tiny we tried teaching coreference from the teachers' antecedent choices, and then gave the coreference head an adapter of its own on the encoder. Neither moved LitBank (0.610, then 0.608 and 0.606). Below about 70M parameters, coreference is the clearest cost of the smaller size.

Calibration matters even more for students. A student's raw relation scores sit lower than its teacher's. With thresholds chosen on dev, tiny's end-to-end Re-DocRED went from 0.38 to 0.54, with no retraining.

Making It Small Enough to Ship

Each repository ships ONNX graphs and kniv_v7, a small host runtime that turns text into the full analysis with numpy, onnxruntime and tokenizers. No PyTorch is needed. The fp32 graphs decode identically to PyTorch on every check text.

Compression was the hardest engineering of the release.

Plain INT8 destroyed UltraBERT: Re-DocRED went from 0.754 to 0.168. The cause is massive activations. In UltraBERT's layer 14, the inputs to the MLP down-projection reach about 8,000, against a typical value near 4. With one INT8 scale per tensor, the outliers take all the range, and every ordinary value lands in the same few buckets. DeBERTa has the same pattern, milder, in its feed-forward outputs.

FP16 had its own trap. Converting the exported graphs to half precision produced graphs that wouldn't load. Exporting the encoders from PyTorch in half precision, with normalisation kept in fp32, matched fp32 within 0.001 on every task.

SmoothQuant fixed INT8. Per input channel of every feed-forward down-projection, we divide the activations by a scale and multiply the weights by the same scale. That is exact in fp32, and it moves the outliers into the weights, where per-channel INT8 handles them. DeBERTa's relation pass proved the most fragile, so it keeps per-channel INT8 with its down-projections in fp32.

(Dev subsets; DeBERTa INT8 runs at 0.58 s per 200 words, also within the gate.)

Small models put their weight elsewhere. In tiny, the transformer layers hold only 3.9M parameters. The word-embedding table holds 12.9M and the heads about 38M, and our INT8 recipe, built for the large encoders, had left both in fp32. Quantising the embedding table row by row and the head graphs weight-only took tiny's linguistic graph from 66 MB to 28 MB and halved its INT8 download, at a cost of at most 0.4 points on every task.

Agreement with fp32 is the wrong check. INT8 changes about 15% of the semantic-role arguments the model outputs, a number that looks alarming. Scored against gold, the cost is 0.2 points: the changes are borderline arguments, wrong about as often in either precision. The INT8 gate now scores NER, SRL and coreference against gold, not against fp32.

Which variant to use:

• FP16: a smaller download, and for GPUs.
• INT8: CPU speed, within 0.7 points of fp32 on every task, for every model.
• fp32: the reference.

Results

Test sets, the two xlarge models, against our previous versions and published specialist models:

• Coreference matches LingMess, a dedicated coreference model.
• Few-NERD is well ahead of GLiNER.
• End-to-end relations beat our fixed-schema v6 (0.627 against 0.566) while reading an open schema. With gold entities, the open schema gives up about 3 points.
• Where we trail: zero-shot relation classification against RE-Matching, and CrossNER against GLiNER-large.

On CrossNER: the text and its five domains were never trained on, and the types are given only as names at run time. Many of those type names ("politician", "political party") do appear in our label bank, which the models were trained on, so this measures entity typing in new domains rather than on never-seen type names. Scores are strict span F1: exact boundaries and type. The politics domain tunes the span offset and is left out of the mean.

The whole family. The same test sets for every size:

• large is the one to start with. It is within a point of the better teacher on tagging, parsing, NER and SRL, and within two on document-level relations, at about twice the speed.
• The students are better at zero-shot relations than their teachers. Every student we've scored beats both teachers at 5-way FewRel, and large leads at 15-way. Our guess is that learning the mixture's rankings over thousands of relation names generalises better than the teachers' own training did; we haven't tested why.
• What shrinks fastest is coreference and new entity types. For those, use large or an xlarge model.

Memory. We ran a memory probe on LongMemEval (150 questions), extracting from the sessions with one fixed, generic memory schema. No LLM answers any question: the probe only checks whether the answer reached the extracted store. For UltraBERT, the answer is in the extracted entities for 0.69 of questions and in the facts for 0.48. Temporal and multi-session questions are the weakest.

Better, Faster and Free: Quality and Latency Against Hosted LLM Extraction

When we built v6, we ran a bake-off to choose its annotators. Frontier LLMs were scored against the same gold test sets kniv is scored on. Here is GPT-6 against kniv v7 (the better of the two xlarge models on each task):

The SRL row uses v5's protocol, which scores each sentence's root predicate; that is the setting the LLMs were measured in. The results table above reports the harder setting, every predicate found end to end (0.810). The relations row supplies the reference entities to both systems; kniv's fully end-to-end score is 0.627.

On the reported scores, the more structural the task, the bigger the specialist's lead. The linguistic rows compare kniv's full test sets with GPT-6's 300-item samples, so read them as reported scores rather than matched runs. The relation row is the matched comparison: both systems on the full Re-DocRED test set, with the same reference entities.

• Coreference: level.
• The linguistic layer: kniv's reported scores are higher on every task, by up to 4 points.
• Document-level relations: kniv is 26 points ahead. Even handed every candidate entity pair explicitly, at 5.4 times the calls, GPT-6 reached only 0.545.

The relation gap has a clear cause. 75% of GPT-6's misses were pairs it never proposed, and 81% of its false facts were on pairs that hold no relation. Reading a whole document and deciding which pairs are related is exactly what the marker pass is built for.

Speed and cost. We timed both on the same job:

• 60 user messages from LongMemEval, each with the turn before it (a median of 141 words);
• a memory schema of 30 relations and 15 entity types;
• the same output from both: entities, plus facts with their attributes.

On this workload, kniv's median is about 90 times faster than the hosted call, on a laptop CPU, with no per-message API fees and nothing leaving the machine.

How the comparison was made: GPT-6's scores come from our v6 annotator bake-off, with zero-shot prompts. Its linguistic rows (POS, lemma, parsing, NER, SRL) were scored on 300-item samples of each test set, and coreference and relations on the full test sets. kniv's scores are on the full test sets throughout, taking the better of the two xlarge models per task. kniv's relation scores use thresholds tuned on dev. The metrics are task-specific: accuracy for POS and lemma, attachment scores for parsing, F1 elsewhere.

Where It Breaks

The guide in each repository covers these in detail, with workarounds.

• Document-level relations trail specialist systems. About 29% of gold facts are missed, mostly borderline decisions just under the threshold. Lowering the relation offsets trades precision for recall: you'll find more of the missed facts, and more false ones with them. It doesn't fix the underlying limit. As the calibration section showed, even thresholds tuned on the test set add at most a point of F1, because the remaining errors are in which pairs get related and how relations rank.
• Near-synonym relations get confused ("located in" against "headquarters location"). Don't put both in one schema. Merge them downstream if you need both.
• Facts are per window, up to a page of text. Nothing is linked across windows or documents; that's your memory layer's job.
• Conversation attribution isn't perfect. A relative's location can be attributed to the speaker as well ("my sister lives in Porto" can also give the user Porto). With UltraBERT, "you" in an assistant's turn can become an entity of its own.
• Negation and reported speech are the weakest attributes. DeBERTa separates plans from assertions well ("I work at Acme. Next month I'm joining Northstar." gives an asserted fact and a planned one); UltraBERT misses the plan in that example. But in our own spot checks, "I don't work there anymore" can still come back as a positive fact, and "Leon says I work at Northstar" is marked asserted, held by the speaker, rather than reported. Treat polarity and modality as signals to check, not guarantees.
• The smaller models miss more, and say more. The students return more facts per message than the teachers, many with low margins, so filter on margin for precision. tiny and small trail the teachers by 4 to 17 points on NER, SRL, coreference, relations and entity types, medium by 2 to 13, and each card lists its gaps.
• English only.

What Ships

• Six models, Apache-2.0: tiny, small, medium, large, xlarge-d and xlarge-u.
• Each repository holds: the PyTorch weights; calibration.json; a label bank of 1,423 relations and 1,559 entity types with descriptions, as starting points for your schema; the tokenizer; ONNX graphs in fp32, FP16 and INT8; the kniv_v7 runtime with a quick start; GUIDE.md, which covers choosing between the models, tuning thresholds and working around the known failure modes.
• the PyTorch weights;
• calibration.json;
• a label bank of 1,423 relations and 1,559 entity types with descriptions, as starting points for your schema;
• the tokenizer;

Try It, Break It