65 Repositories, 190 Milliseconds, One Bit Per Dimension
Semantic search over every repository I own now answers in about 190 milliseconds, from an index small enough to ship as a release asset, using bekko-embedding-v1-a8m — a 4-layer mmBERT with 7.7M active parameters out of 106M, MIT-licensed. That is the point of the last three months: making dense retrieval cheap enough to put wherever you want it.
The compression is the interesting part. bekko emits 384 floats per chunk; other encoders will hand you 768 or 1536, and none of what follows cares which. Most of what you need for ranking survives at a bit or two per dimension, whatever you started with — 16 to 32× smaller, and the recall barely moves. Small enough that remax_kb packs a whole corpus into one file a browser searches with no server at all, and small enough that a 65-repo index loads once into a resident process and answers instantly after that.
Why one bit beat three
The reason it compresses that far is worth stating, because it is not obvious and it drove the design of both libraries.
Pulling a single bit per dimension out of a compressed embedding beat keeping two or three bits — by 13 and 4 points of recall on SPECTER2 embeddings of real papers. Less information, retrieved better.
The 2- and 3-bit methods place their boundaries to minimize reconstruction error, and reconstruction error is not the job. A search index has to preserve the order documents come back in. An interior bin can flip, swap which document lands first, and barely move the reconstruction number at all. Optimize the number you actually care about and you can throw away far more than you would expect.
That split the work in two. remex compresses embeddings at 2–16× and validates every setting against retrieval rather than reconstruction. remax goes to one bit and tunes specifically for rank. Both are general-purpose: any corpus, any encoder, nothing about code in either of them.
Where dense retrieval alone runs out
Applying it to code is where I found the boundary.
I benchmarked bekko against plain grep on source code. Over 59 queries, it
found the right file in its top 5 for 59.5% of them and grep managed 59.6% — a
tie. Reaching for a code-trained encoder did not change it: jina-code
scored 63.0% and lost to the larger general-purpose bekko-a25m at 65.6%, at six
times the encoding cost. (An earlier 6-query run had shown dense winning clearly.
It did not replicate.)
Merging the two rankings is what beat grep. Keyword list plus semantic list, fused, found the right file in its top 10 for 76.2% of queries against grep's 68.2%.
And this is where cheap vectors stop being a nice-to-have. Running two retrieval arms instead of one is only sensible if the second is close to free. At 190 milliseconds and 62 MB resident, the dense arm costs about nothing to keep running beside a keyword index — so the answer to "dense or keyword" gets to be "both" instead of a decision.
Two things I would not have guessed
A follow-up experiment over 24 queries, with answer keys that already existed for other purposes, pinned down the shape of the hybrid. Dense alone got 22 right, keyword alone 23, the two fused 24. The arms fail on different queries.
Stored BM25 beat ripgrep as the keyword arm, 23 to 17. Almost all of the gap was duplicate results. Ripgrep hands you a set of matches, and "find me a file like this one" is a ranking question — if you never produced a ranking, no amount of term counting afterwards gets you one.
A third arm made it worse, 24 down to 22. Dense, BM25 and ripgrep fused together lost to dense and BM25 alone. Rank fusion here is unweighted, so a weak arm votes exactly as loudly as a strong one, and ripgrep's duplicate-heavy ordering is close to noise. If you are adding a third source to a hybrid search, measure it against the pair, not against your best single arm.
What it adds up to
The compression is what makes the rest affordable. A dense arm that fits in 62 MB and answers in milliseconds is one you can run in a browser, or on a single CPU core, or next to a keyword index — and once it costs that little, you stop having to pick a single retrieval method and defend it.
On code, don't run it alone. Put BM25 beside it.