Dev48
Language
  • About
  • Services
  • Industries
  • Technologies
  • Articles
  • Contacts
Book a call
    Home/Articles/Does a code tuned embedding model pay off papr embed code 1 in qdrant
Dev48
info@dev48.org+375 29 320-40-96WhatsApp@isky_by

© 2026 · All rights reserved.

Does a Code-Tuned Embedding Model Pay Off? papr-embed-code-1 in Qdrant

Source: Qdrant•November 18, 2026 at 04:16 PM

Does a Code-Tuned Embedding Model Pay Off? papr-embed-code-1 in Qdrant

If you search code, you’ve probably asked this: is a code-specific embedding model worth it, or is a strong general one good enough? Papr’s papr-embed-code-1 is a good test case. It’s Qwen3-Embedding trained further on code retrieval, with open weights in two sizes: 0.6B and 4B…

If you search code, you’ve probably asked this: is a code-specific embedding model worth it, or is a strong general one good enough?

Papr’s papr-embed-code-1 is a good test case. It’s Qwen3-Embedding trained further on code retrieval, with open weights in two sizes: 0.6B and 4B. Papr reports top scores on CoIR, a code retrieval benchmark. But it trained on most of CoIR’s training data, and its model card says so. Papr also reports the 4B at #1 on MTEB’s two held-out code leaderboards, RTEB (Code) and CodeRAG.

We asked a narrower question. Does it beat the model it started from on code it never saw? And what does it cost to run in Qdrant?

How We Tested

We compared each papr-embed-code-1 model with the Qwen3-Embedding model it was trained from, with the same query instructions. The tasks come in three groups:

  • Code Papr didn’t train on: HumanEval and MBPP match a task description to the Python function that solves it. FreshStack matches a Stack Overflow question to the files of the library it’s about. WikiSQL matches a question about a table to its SQL query.
  • Code Papr trained on: three CoIR tasks, though the 4B skipped CodeTransOcean Contest. APPS matches a programming problem to its solution. CodeTransOcean Contest matches a program to the same program in another language. StackOverflowQA matches a question to its answer.
  • Prose: four BEIR datasets, NFCorpus, ArguAna, SCIDOCS, and FiQA.

Where Papr’s model cards score its own models, ours match to within 0.001 nDCG@10 on every task except the ones full of identical documents, marked † in the full results. There, the score depends on how ties break, which can change with batch size. On SyntheticText2SQL, we got 0.625 and 0.650 for the two sizes, against Papr’s 0.720 and 0.723.

Better on Code It Never Saw

On code Papr didn’t train on, both sizes beat their base model. The 4B gained 0.034 nDCG@10 on average, and the 0.6B 0.026. On the CoIR tasks, the gains were similar.

Code Search Gains, Prose LossesExact search, nDCG@10. Higher is better.nDCG@10Unseen codeCoIRProse0.00.20.40.60.81.0Qwen3 0.6BPapr 0.6BQwen3 4BPapr 4B

The gain held on every one of those tasks, but its size varied a lot. On HumanEval, the 4B base model already scored 0.98, so there wasn’t much left to gain. On WikiSQL, the 4B gained 0.070.

Prose went the other way. Both sizes scored a little lower than their base model, by 0.003 for the 0.6B and 0.016 for the 4B. On TREC-COVID, a set of COVID-19 research papers, the 0.6B lost 0.065. If most of what you search is prose, keep the base model.

Compressed to 1 Bit, the 4B Beats the 0.6B

Code search usually means a lot of vectors: a function, a file chunk, or a snippet each. The 4B’s vectors have 2,560 dimensions, which is 10 KB per vector in float32.

They compress very well. With 1-bit TurboQuant, each vector takes 328 bytes, 31x less. On our seven code tasks, 1-bit TurboQuant kept 99.7% of the 4B’s float32 score without rescoring, and all of it with rescoring at 2x oversampling.

That changes which model to pick. The 4B compressed to 1 bit scores higher than the 0.6B in float32, and takes 12x less RAM.

RAM per VectorLower is better.05,00010,0000.6Bfloat324096 B0.6B1-bit136 B4Bfloat3210240 B4B1-bit328 BBytes per vector

The 4B still costs more to run, because it embeds about four times slower than the 0.6B on the same GPU. And WikiSQL is the exception, where the 0.6B scored higher.

A Million Snippets in One Collection

Our other code tasks have at most 20,000 documents, which makes compression look easy. So we also loaded CodeSearchNet-CC into one collection: about a million snippets in six languages, from Python to Ruby. Each query is the start of a function, and the right document is the rest of it. A payload filter keeps the search within the query’s language.

Compression held up at this size. At hnsw_ef 128, 1-bit TurboQuant with rescoring kept 99.5% of exact search’s score, against 99.7% for float32. It was faster too, at 1.0 ms per query against 1.8 ms, with the original vectors for rescoring already in memory.

The bigger lever was hnsw_ef, the number of candidates the search explores. Raising it from 128 to 256 added 0.002 to 0.004 nDCG@10 to every setup. That’s enough for rescored 1-bit vectors at 256 to beat float32 at 128. At 256, float32, 2-bit, and rescored 1-bit vectors all kept 99.8% or more, and the 1-bit search took 1.4 ms against 2.9 ms for float32.

Dropping the filter and searching all six languages at once cost 0.005 to 0.007.

What About BM25?

Code is full of exact names, so keyword search seems like a natural partner. Qdrant 1.20 added a code tokenizer for full-text indexes. It splits getUserName and get_user_name into the same words. Qdrant can also score full-text matches with BM25, so a hybrid query can fuse BM25 with the dense search.

On our tasks, BM25 on its own was far behind dense search. On APPS, where the queries are problem statements and the documents are bare solutions, it scored 0.01 nDCG@10. The two share almost no words.

Fusing BM25 with dense search at equal weight made every task worse, by 0.02 to 0.17 for the 4B. Down-weighting BM25 helped, but only up to break-even. With BM25’s RRF weight at 0.1, the 4B averaged 0.0006 below dense search alone, and its best gain on any task was 0.002.

Our queries were questions, descriptions, and code, and for those, dense search alone was the better choice.

So, Does It Pay Off?

For code search, yes. On code it never saw, papr-embed-code-1 beat the model it started from at both sizes, by about 0.03 nDCG@10. On prose it did a little worse, so if most of your corpus is prose, stay with the base model.

The bigger choice is the size. Compressed to 1 bit, the 4B scored higher than the 0.6B in float32, in a twelfth of the RAM. If you can afford to embed with the 4B, we’d store it with 1-bit TurboQuant, keep rescoring on, and raise hnsw_ef before adding bits. Compare against an unquantized collection on your own queries, once with rescoring disabled and once with 2x oversampling. If you index several languages in one collection, filter by language.

We’d leave BM25 out to start. If your users search for exact identifiers, that’s the case to test it on, with a low weight.

The 0.6B is coming to FastEmbed in its next release.

Full Results

Every model got the same query instructions. † Many identical documents, so the score depends on how ties break. Left out of the averages. ‡ Papr trained on its train split. § The 4B didn’t train on it.

Mean over seven code tasks (HumanEval, MBPP, FreshStack, WikiSQL, APPS, CodeTransOcean Contest, and StackOverflowQA), searched with HNSW. Rescoring uses 2x oversampling. Bytes count only the vectors searched in RAM.

papr-embed-code-1-0.6b on CodeSearchNet-CC, 1,005,474 snippets, each query filtered to its language. Exact search scores 0.923. Latency is Qdrant’s server-side time.

papr-embed-code-1-4b. BM25 uses the code tokenizer. RRF uses Qdrant’s default k.

Thanks to Papr for releasing the code models with open weights, and for publishing exactly how they trained and evaluated them.

← All articles

What this article says

More in Software Development

All →
Why your software needs a CBOM
SonarSource

Why your software needs a CBOM

How to use AI prompts to create your online course
Teachable

How to use AI prompts to create your online course

What drives the cost of AI at work
Microsoft

What drives the cost of AI at work

Why all of you can build with us in Netlify’s next decade
Netlify

Why all of you can build with us in Netlify’s next decade

Inside the new Netlify
Netlify

Inside the new Netlify

AI sovereignty for your agents: Where should they live?
OutSystems

AI sovereignty for your agents: Where should they live?

More from Qdrant

How to Run an Embedding Model on Bare Metal
Qdrant

How to Run an Embedding Model on Bare Metal

Faster and Cheaper Search With Qdrant 1.20
Qdrant

Faster and Cheaper Search With Qdrant 1.20

Scalable PDF Retrieval with Cohere Embed 5 and Qdrant Asymmetric Quantization
Qdrant

Scalable PDF Retrieval with Cohere Embed 5 and Qdrant Asymmetric Quantization

Qdrant 1.20 - Full-Text Search Improvements
Qdrant

Qdrant 1.20 - Full-Text Search Improvements

UltraBERT Fits 32K Tokens in One Vector. RIP Chunking?
Qdrant

UltraBERT Fits 32K Tokens in One Vector. RIP Chunking?

How Poliscope Made All of Germany's Local Politics Searchable on One Server with Qdrant
Qdrant

How Poliscope Made All of Germany's Local Politics Searchable on One Server with Qdrant