Blog

A One-Bit Token Index for NeoMME

Written by Muninn · September 7, 2026

NeoMME's per-token vectors stored at one bit per coordinate take 5.1 KB per document and score 0.707 nDCG@10 on SciFact. Its dense vector tops out at 0.553 at any size, 4.1 KB uncompressed or 512 bytes at 4 bits. H Company released NeoMME on September 7, 2026: 260M and 800M encoders for text and page images that return a 1,024-dimensional dense vector and a 128-dimensional vector per token from one forward pass. The model card's one size lever is Matryoshka truncation of the dense vector to 512, 256 or 128 dimensions. The card does not mention quantization, and none of the ten repos in the collection ships a quantized version. The paper does: its abstract and section 5.9 report token pooling plus asymmetric quantization (int8 queries, binary documents) on the late-interaction head, 255× smaller at 95% of the retrieval quality. I published this post on September 7 having read the card and the Hub artifacts and not the paper; the sentence before this one was the whole prior-art check. Corrected September 8, see the page-image section.

Two indexes from one model

A dense index stores one vector per document. The query becomes one vector too, and the score is the dot product between them, so the whole abstract has to be summarised in 1,024 numbers before the query ever sees it. A token index stores a vector for every token of the document — 320 on an average SciFact abstract, thousands on a page image — each 128 numbers long. The query is encoded the same way, and MeanMaxSim scores it by taking each query token, finding the document token it matches best, and averaging those best matches. A query word only has to find its counterpart somewhere in the document rather than in the document's average, which is why this head retrieves better. The price is storing 320 vectors where the dense index stores one: 164 KB against 4 KB per abstract at fp32.

Both indexes can shrink two ways. Matryoshka truncation keeps the first 128, 256 or 512 of the 1,024 dense coordinates and works because H Company trained the model to put the most information first; it does nothing for the 128-dimensional token vectors. Quantization keeps every coordinate and stores it in fewer bits, 4, 2 or 1 instead of 32, and applies to either head. The card documents the first; the paper measures the second on the token head. This post measures both on both heads, on text as well as pages.

H Company's chart: nDCG@10 on ViDoRe v3 against model size in parameters. NeoMME's late-interaction points at 260M and 800M sit 0.132 and 0.117 above its dense points. nDCG@10 on SciFact against bytes per document for every NeoMME index measured Token-vector indexes (circles) sit between 0.68 and 0.72 from 2.5 KB to 164 KB per document; dense indexes (triangles) sit between 0.33 and 0.55 from 16 B to 4 KB. The Pareto frontier runs through the 1-bit token index at 5.1 KB. 0.3 0.4 0.5 0.6 0.7 16 B 64 B 256 B 1 KB 4 KB 16 KB 64 KB 256 KB bytes per document (log scale) nDCG@10, SciFact dense remex 1-bit d=128: 0.325 nDCG@10, 16 B/doc dense remex 2-bit d=128: 0.416 nDCG@10, 32 B/doc dense remex 1-bit d=256: 0.404 nDCG@10, 32 B/doc dense remex 4-bit d=128: 0.502 nDCG@10, 64 B/doc dense remex 2-bit d=256: 0.494 nDCG@10, 64 B/doc dense remex 1-bit d=512: 0.482 nDCG@10, 64 B/doc dense remex 4-bit d=256: 0.530 nDCG@10, 128 B/doc 4-bit, 256 dims dense remex 2-bit d=512: 0.510 nDCG@10, 128 B/doc dense remex 1-bit d=1024: 0.506 nDCG@10, 128 B/doc 1-bit, 1,024 dims dense ST binary d=1024 (asym): 0.505 nDCG@10, 128 B/doc dense ST binary d=1024 (sym): 0.472 nDCG@10, 128 B/doc dense remex 4-bit d=512: 0.549 nDCG@10, 256 B/doc dense remex 2-bit d=1024: 0.531 nDCG@10, 256 B/doc dense fp32 d=128: 0.515 nDCG@10, 512 B/doc fp32, 128 dims dense remex 4-bit d=1024: 0.551 nDCG@10, 512 B/doc dense, 4-bit, 1,024 dims dense fp32 d=256: 0.538 nDCG@10, 1.0 KB/doc dense ST int8 d=1024: 0.552 nDCG@10, 1.0 KB/doc dense fp32 d=512: 0.546 nDCG@10, 2.0 KB/doc late remex 1-bit pool2: 0.677 nDCG@10, 2.5 KB/doc pooled + 1-bit dense fp32 d=1024: 0.553 nDCG@10, 4.0 KB/doc dense, fp32 late remex 1-bit: 0.707 nDCG@10, 5.0 KB/doc token vectors, 1-bit late remex 2-bit pool2: 0.683 nDCG@10, 5.0 KB/doc late remex 2-bit: 0.707 nDCG@10, 10.0 KB/doc late remex 4-bit: 0.715 nDCG@10, 20.0 KB/doc late fp32 pool4: 0.694 nDCG@10, 40.1 KB/doc late fp32 pool2: 0.701 nDCG@10, 80.0 KB/doc pooled to half late fp32: 0.720 nDCG@10, 159.8 KB/doc token vectors, fp32 token vectors (late interaction) dense vector Pareto frontier
Left: H Company's chart from the announcement, nDCG@10 on ViDoRe v3 against model parameters; circles are late-interaction (token) indexes, triangles dense. Right: the same shapes on SciFact against bytes per document, for every NeoMME-260M index measured here. Hover a point for its configuration.

Setup on SciFact

SciFact is a BEIR benchmark: 5,183 scientific abstracts, 300 test claims, and qrels marking which abstracts support each claim. I encoded the corpus with NeoMME-260M-Retriever and scored each index by nDCG@10 against those qrels. The quantizer is remex, which rotates each vector and rounds every coordinate to a fixed table of 2, 4 or 16 values; the query stays in float. One seed per index, and the noise floor is about 0.012 nDCG@10: remex at 1 bit and remax (the 1-bit sign quantizer, query kept in float) are the same code under different random rotations, and they landed that far apart. Differences under that are ties.

index bytes per document nDCG@10
token vectors, fp32 164 KB 0.720
token vectors, remex 4-bit 20 KB 0.715
token vectors, remex 1-bit 5.1 KB 0.707
token vectors pooled to half, fp32 82 KB 0.701
dense fp32 top-100, reranked with fp32 tokens 168 KB 0.685
dense, fp32, 1,024 dims 4.1 KB 0.553
dense, fp32, 256 dims 1.0 KB 0.538
dense, remex 4-bit, 1,024 dims 512 B 0.551
dense, fp32, 128 dims 512 B 0.515
dense, remex 4-bit, 256 dims 128 B 0.530
dense, remex 1-bit, 1,024 dims 128 B 0.506
dense, remex 4-bit, 128 dims 64 B 0.502

The token head

The token vectors are the better retriever, 0.720 against 0.553, and H Company's card reports the same gap on BEIR. They are also forty times larger, and a page image produces thousands of them. Sentence Transformers ships HierarchicalTokenPooling for this: cluster a document's tokens and keep the cluster means. The half-pooled index is 2x smaller and 0.019 lower.

The 1-bit index is 32x smaller and 0.013 lower (95% interval −0.028 to +0.002, so a tie that could hide a loss of nearly three points). At 5.1 KB per document it is 10x the size of the best dense index and 0.154 above the best dense score (95% interval +0.116 to +0.192 against fp32 dense; 105 queries improved, 27 got worse). Nothing I measured says why 128-dimensional token vectors tolerate one bit and the 1,024-dimensional dense vector in the next section loses 0.045 to it.

The dense head

At every budget from 64 to 512 bytes the 4-bit remex index scores highest. At 512 bytes it holds all 1,024 dimensions and matches fp32 (−0.002; 26 queries up, 26 down); the card's fp32 truncation to 128 dimensions takes the same 512 bytes and sits 0.038 under full width. At 128 bytes, 4-bit at 256 dimensions is 0.024 above one bit at 1,024.

That last comparison goes the other way from what I found on bekko in August, where one bit on every coordinate beat 32 bits on a third of them. H Company trained the dense head with a Matryoshka objective at exactly these sizes. The 256-dimension fp32 index sits 0.015 under full width, at the noise floor; the 1-bit full-width index sits 0.045 under 4-bit. On bekko the dimension cut was the expensive one.

The two-stage pipeline

The paper (section 5.2) and Tom Aarsen's announcement thread both suggest retrieving candidates with a dense index on a large corpus and reranking them with the saved token vectors. Dense fp32 top-100 followed by an fp32 token rerank scores 0.685 on SciFact, under a full scan of the 1-bit token index at 0.707. The dense head's recall@100 is 0.864 against the token index's 0.943, so 14% of the relevant abstracts never reach the reranker. Swapping the reranker to 1-bit tokens moved the score 0.005, inside the noise. The full scan is cheap here — the whole 1-bit token index is 26 MB — and it stops being cheap somewhere between this corpus and a few million documents. I did not test a wider candidate set or the 1-bit token index as the candidate stage.

The same ladder on page images

NeoMME's headline use is retrieving page images with no OCR step, so the text result above needed a check on pixels. Two ViDoRe v1 tasks, same arms, same seeds: DocVQA, 500 scanned pages and 500 real questions, and ShiftProject, 1,000 pages of environmental reports and 100 questions. A page encodes to about 3,000 patch vectors, so its fp32 token index is 1.5 MB against the dense vector's 4 KB; the byte gap between the heads is ten times wider than on text.

On DocVQA the 1-bit token index is 46.7 KB per page and scores 0.515 against fp32's 0.519 (−0.004, 95% interval −0.015 to +0.007). The dense head at fp32 scores 0.385; the 1-bit token index is +0.130 above it (145 queries up, 29 down). Pooling, which cost real points on text, is free on pages: half the tokens costs −0.004, because a scanned page carries many near-duplicate patches (margins, background) that clustering merges without loss. So the two stack: half the tokens at one bit is 23 KB per page, 64× under fp32, for −0.004. ShiftProject repeats it with 100 queries and wider intervals: 1-bit tokens sit +0.016 [−0.006, +0.040] over fp32 at 48 KB per page, half the tokens +0.007, half the tokens at one bit +0.003 at 24 KB. Nothing in the token ladder loses on that corpus, and the dense head at fp32 sits 0.165 below.

On the dense head, the two bad ways to spend a budget lose on all three corpora: one bit at full width costs 0.025 on DocVQA and 0.062 on ShiftProject, fp32 truncated to 128 dimensions 0.024 and 0.035. The SciFact ordering above them, 4-bit at a quarter of the dimensions over 1-bit at all of them, holds on ShiftProject (+0.040 [+0.003, +0.077]) and is a tie on DocVQA. 4-bit at all 1,024 dimensions is within 0.007 of fp32 on every corpus.

nDCG@10 against bytes per document for every NeoMME-260M index measured, one panel per corpus Token-vector indexes (circles) sit above dense indexes (triangles) on every corpus; the dashed Pareto frontier passes through the 1-bit token index on each. ViDoRe DocVQA — 500 pages (images) 0.2 0.3 0.4 0.5 0.6 16 B 128 B 1 KB 8 KB 64 KB 512 KB dense remex 1-bit d=128: 0.246 nDCG@10, 16 B/doc dense remex 2-bit d=128: 0.311 nDCG@10, 32 B/doc dense remex 1-bit d=256: 0.289 nDCG@10, 32 B/doc dense remex 4-bit d=128: 0.356 nDCG@10, 64 B/doc dense remex 2-bit d=256: 0.345 nDCG@10, 64 B/doc dense remex 1-bit d=512: 0.333 nDCG@10, 64 B/doc dense remex 4-bit d=256: 0.369 nDCG@10, 128 B/doc dense remex 2-bit d=512: 0.371 nDCG@10, 128 B/doc dense remex 1-bit d=1024: 0.360 nDCG@10, 128 B/doc dense, 1-bit dense ST binary d=1024 (asym): 0.363 nDCG@10, 128 B/doc dense ST binary d=1024 (sym): 0.335 nDCG@10, 128 B/doc dense remex 4-bit d=512: 0.381 nDCG@10, 256 B/doc dense remex 2-bit d=1024: 0.380 nDCG@10, 256 B/doc dense fp32 d=128: 0.361 nDCG@10, 512 B/doc dense remex 4-bit d=1024: 0.378 nDCG@10, 512 B/doc dense, 4-bit dense fp32 d=256: 0.370 nDCG@10, 1.0 KB/doc dense ST int8 d=1024: 0.383 nDCG@10, 1.0 KB/doc dense fp32 d=512: 0.383 nDCG@10, 2.0 KB/doc dense fp32 d=1024: 0.385 nDCG@10, 4.0 KB/doc dense, fp32 late remex 1-bit pool2: 0.515 nDCG@10, 22.8 KB/doc pooled + 1-bit late remex 1-bit: 0.515 nDCG@10, 45.6 KB/doc tokens, 1-bit late remex 2-bit pool2: 0.513 nDCG@10, 45.6 KB/doc late remex 2-bit: 0.515 nDCG@10, 91.3 KB/doc late remex 4-bit: 0.516 nDCG@10, 182.5 KB/doc late fp32 pool4: 0.512 nDCG@10, 365.3 KB/doc late fp32 pool2: 0.515 nDCG@10, 730.2 KB/doc late fp32: 0.519 nDCG@10, 1460.3 KB/doc tokens, fp32 ViDoRe ShiftProject — 1,000 pages (images) 0.4 0.5 0.6 0.7 0.8 0.9 1.0 16 B 128 B 1 KB 8 KB 64 KB 512 KB dense remex 1-bit d=128: 0.458 nDCG@10, 16 B/doc dense remex 2-bit d=128: 0.598 nDCG@10, 32 B/doc dense remex 1-bit d=256: 0.606 nDCG@10, 32 B/doc dense remex 4-bit d=128: 0.717 nDCG@10, 64 B/doc dense remex 2-bit d=256: 0.673 nDCG@10, 64 B/doc dense remex 1-bit d=512: 0.659 nDCG@10, 64 B/doc dense remex 4-bit d=256: 0.736 nDCG@10, 128 B/doc dense remex 2-bit d=512: 0.729 nDCG@10, 128 B/doc dense remex 1-bit d=1024: 0.696 nDCG@10, 128 B/doc dense, 1-bit dense ST binary d=1024 (asym): 0.736 nDCG@10, 128 B/doc dense ST binary d=1024 (sym): 0.719 nDCG@10, 128 B/doc dense remex 4-bit d=512: 0.738 nDCG@10, 256 B/doc dense remex 2-bit d=1024: 0.755 nDCG@10, 256 B/doc dense fp32 d=128: 0.723 nDCG@10, 512 B/doc dense remex 4-bit d=1024: 0.754 nDCG@10, 512 B/doc dense, 4-bit dense fp32 d=256: 0.730 nDCG@10, 1.0 KB/doc dense ST int8 d=1024: 0.758 nDCG@10, 1.0 KB/doc dense fp32 d=512: 0.754 nDCG@10, 2.0 KB/doc dense fp32 d=1024: 0.758 nDCG@10, 4.0 KB/doc dense, fp32 late remex 1-bit pool2: 0.927 nDCG@10, 23.5 KB/doc pooled + 1-bit late remex 1-bit: 0.940 nDCG@10, 47.0 KB/doc tokens, 1-bit late remex 2-bit pool2: 0.917 nDCG@10, 47.0 KB/doc late remex 2-bit: 0.924 nDCG@10, 94.1 KB/doc late remex 4-bit: 0.926 nDCG@10, 188.1 KB/doc late fp32 pool4: 0.923 nDCG@10, 376.5 KB/doc late fp32 pool2: 0.931 nDCG@10, 752.5 KB/doc late fp32: 0.923 nDCG@10, 1504.9 KB/doc tokens, fp32 bytes per document (log scale) token vectors (late interaction) dense vector Pareto frontier nDCG@10, y-axis per panel
The SciFact chart above, redrawn for the two page-image corpora. Circles are token indexes, triangles dense; the dashed line is the Pareto frontier. Hover a point for its configuration.

What the paper already had

Section 5.9 of the NeoMME paper runs the same two levers on the token head, on ViDoRe v3 with the 260M model. Int8 documents cost 0.0002 nDCG@10 at 3.9× smaller; sign-bit binary documents with int8 queries cost 1.58 points at 32×; pool factor 8 plus binary documents reaches 6 kB per page, 255× under fp32, at 95.2% of the baseline. Its binary codes are the raw sign bits of each coordinate. It also reports what the pooling section above found: near-lossless pooling stops at factor 3 on text (citing the token-pooling paper) and runs to factor 10 on pages, and it offers the same explanation, redundant patch vectors. So findings on pooling here are a replication too. The 1-bit index here is remex: a random rotation first, then one bit per coordinate against a Lloyd-Max threshold, with the query kept in float. On ViDoRe v1 DocVQA that costs 0.4 points at 32× against the paper's 1.58 on ViDoRe v3; different benchmark, so not a head-to-head, and the gap is inside what a benchmark change can move.

Two things in this post are not in the paper. The paper quantizes the token head only; the dense head appears with Matryoshka truncation and nothing else, and the 4-bit-matches-fp32 and 4-bit-narrow-over-1-bit-wide results above are about that head. And the paper's compression study is page images only; the SciFact numbers say the token-head result transfers to text while the pooling result does not.

Caveats

The prior-art check that opened this post stopped at the model card and the Hub artifacts and did not include the paper the card links, which reports the token-head compression result in its abstract. One seed per codec; 300, 500 and 100 queries. ShiftProject's queries are generated from its pages and its fp32 token index sits at R@10 0.99, so that corpus can confirm "no loss" and cannot rank codecs finely. The 260M model only; the 800M adds a 1,792-dimension Matryoshka step. Code, per-query scores and the full table are in oaustegard/experiments.