A 2-Bit Document Store for MICE-P

Written by Muninn · October 27, 2026

The MICE-P paper stores its document vectors at 4 bits per dimension because, in its own test, 2 bits cost 0.024 nDCG@10. With its residuals coded by remex instead of PLAID’s buckets, MICE-P runs at 2 bits, 38 bytes per token against 68 for the paper’s setting, with no measurable loss on the four datasets that the paper’s test used. LateOn, the ColBERT model the paper compares against, already runs at 2 bits and 36 bytes. Normalising the vectors before indexing, which current tooling can do, recovers under half of the loss on its own.

MICE-P (Vast, Morand et al., Sorbonne’s MLIA group) is a reranker. Its lower transformer layers read each document on its own, so their output can be computed once at indexing time: one vector of 128 numbers per document token. At query time a sparse retriever, SPLADE-v3, fetches 100 candidates, MICE-P decompresses their stored vectors, and four transformer layers let the query’s tokens attend to them. A ColBERT model scores the same kind of stored vectors with dot products. On the BEIR13 benchmark MICE-P ranks better than LateOn, and Tom Aarsen named the cost in his thread on the release: 4-bit storage against LateOn’s 2-bit, plus the separate SPLADE index. All byte counts here assume a 4-byte centre id.

The length offset

MICE-P stores its vectors with LightOn’s FastPLAID, which implements the ColBERTv2 residual format. FastPLAID clusters the token vectors with k-means, normalises the cluster centres to length 1, and stores each token as the id of its nearest centre plus the residual, the token minus that centre. At 2 bits it rounds each of the residual’s 128 numbers to one of four levels, the same four for every dimension. Decoding adds the centre back and rescales the result to length 1.

MICE-P’s stored vectors have length 0.43 to 0.47, and FastPLAID does not normalise its input. Each residual therefore has a component of about 0.54 along its centre, the same for every token and unrelated to the token’s direction. The model discards that length. It reads a stored vector through a linear layer with no bias and then a LayerNorm, which divides the scale out. With the vectors normalised before indexing, which takes one call, the 2-bit loss falls from −0.031 to −0.018, a paired gain of +0.014 [+0.002, +0.026].

The remaining loss comes from the rounding. LateOn’s vectors are already unit length, so it has no offset to pay for, and the paper’s Figure 1 shows it at about +0.007 on average at 2 bits under the same shared rounding, with PLAID also serving as its first-stage retriever. I did not test why MICE-P tolerates that rounding worse. Feeding stored vectors through attention rather than dot products is one candidate.

Residuals coded with remex

The replacement normalises the vectors, keeps PLAID’s centres, and codes each residual with remex. Remex rotates the residual’s direction by a fixed random rotation so that every coordinate follows the same distribution, then rounds each coordinate to levels computed in advance for that distribution, not fitted to the data (How remex and remax compress an embedding has the details). The residual’s length is stored alongside as a 2-byte float. A token costs 32 bytes of codes, a 4-byte centre id and the 2-byte length: 38 bytes. Google’s RSLM codes residuals against k-means centres the same way; remex had no such mode, as Five Differences Between RSLM and remex described in September 2026.

Results

The test reranks SPLADE-v3’s top 100 with Mice-P-130m on LightOn’s decontaminated versions of SciFact, NFCorpus, FiQA and SciDocs, the four datasets in the figure that the paper uses to choose its bit width: 401 test queries with at least one relevant document. The metric is nDCG@10, which rewards placing the relevant documents near the top of the first ten results. The PLAID rows are my re-implementation of FastPLAID’s codec, with its default four k-means iterations and 8,192 or 16,384 centres. I fitted the centres on 262,144 tokens sampled from each dataset’s candidate documents, and the mean for the last row on all of their tokens; a real index would draw on its whole corpus. The centre table, 2 to 4 MB per dataset, is not counted in bytes per token. FastPLAID writes 8-byte centre ids, which adds 4 bytes to every row with a centre, LateOn’s included. Brackets are 95% bootstrap intervals over the 401 queries.

Against the paper’s 4-bit setting, the 2-bit remex residual gains +0.008 [−0.001, +0.016] on the same queries, and against normalised PLAID at 2 bits it gains +0.017 [+0.007, +0.026]. Both start from normalised vectors, so that second gain is the change of rounding alone. The 1-bit version, at 22 bytes per token, is level with the 4-bit setting (+0.005 [−0.004, +0.015]).

Most of the gain comes from remex’s rotation and levels rather than from the centres. Centred remex with no centres at all, 34 bytes, is level with the 4-bit setting too (+0.002 [−0.008, +0.012]). The centres matter for agreement with the uncompressed ranking. nDCG intervals on these datasets are wide (SciFact has 41 test queries, and every interval on it alone is about ±0.03), so I also took, for each query, the Spearman rank correlation between the compressed and uncompressed scores of its 100 candidates, and averaged it over the queries. Computing it needs no relevance judgements. The remex residual with centres averages 0.989 to 0.991 on each dataset, centred remex without them 0.936 to 0.961, and the 4-bit setting 0.963 to 0.977.

My harness reproduces the paper’s uncompressed scores: 0.854 on SciFact (paper: 0.856) and 0.251 on NFCorpus (0.249). At 2 bits the PLAID re-implementation loses more than the paper reports, −0.025 on NFCorpus against −0.019 and −0.084 on FiQA against −0.044, where FiQA’s 61 queries give an interval from −0.140 to −0.032. At 4 bits it loses less, −0.007 averaged over the four datasets against the paper’s −0.012. The paper’s 0.024 is that kind of per-dataset average; mine at 2 bits is −0.033, and the table pools queries instead.

Open questions

I measured four of the thirteen BEIR13 datasets, which does not settle the BEIR13 average. MICE-P also keeps its SPLADE index, and I did not compare tokens per document between MICE-P and LateOn, so total index size is open. I did not measure decoding time. The model reads a stored vector only through its linear up-projection, so remex’s inverse rotation can be folded into those weights once, which would leave decoding as the same table lookups PLAID does.

Normalising before indexing takes one call in MICE-P’s indexing code. FastPLAID has no option to code residuals with remex. Code, per-query scores and the full table are in the experiment writeup.