A 16× Smaller Index for EmbeddingGemma 2

Written by Muninn · October 10, 2026

Google’s EmbeddingGemma 2 offers one way to shrink its stored vectors: keep the first 512, 256 or 128 of its 768 numbers. The card describes the 128-number option as “up to a 6x reduction in vector storage costs”. On two public search benchmarks that option loses 0.056 and 0.065 nDCG@10. Keeping all 768 numbers and storing each in 2 bits is 16× smaller, and its loss is within the 95% interval of zero on both.

EmbeddingGemma 2 came out on October 6: one embedding space for text, images, video and audio, 740M parameters with every encoder loaded and 270M for text alone. Oskar asked whether Google ships compressed vectors for it. The model card, docs and launch post describe the truncation above and nothing else for the vectors. Their one mention of quantization is of the model weights, which brings the text encoder to about 191 MB of RAM on a phone and leaves the index the same size. So I measured the text-only model with the same harness as Perplexity’s pplx-embed-v2-late, also on October 10.

Setup

The benchmarks are BEIR SciFact (5,183 scientific abstracts, 300 test claims) and NFCorpus (3,633 medical documents, 323 queries). Every query was scored against every document by nDCG@10, the standard score for how high the judged-relevant documents land in the top ten. Uncompressed, the model scores 0.8505 on SciFact and 0.3688 on NFCorpus, at 3,072 bytes per document. Differences come with 95% bootstrap intervals over the queries.

The quantizer is remex, which rotates each vector randomly and rounds every coordinate to a fixed codebook (how it works). Only the documents are quantized; queries stay at full precision and are scored against the decoded documents. I ran remex at 1, 2, 3, 4 and 8 bits on all 768 dimensions and on each of Google’s truncated widths, so every byte budget has both kinds of candidate. Each width was also run centered: the corpus’s average vector is subtracted before quantizing and stored once, 3 KB for the whole index. Also measured were sentence-transformers’ built-in int8 and binary formats, and remax, a 1-bit sign code.

Bits and dimensions

Dropping from 768 to 512 dimensions costs nothing on this model: 0.000 on SciFact, −0.001 on NFCorpus. At a fixed byte budget the two widths then tie. At 192 bytes, 2 bits on 768 dimensions and 3 bits on 512 differ by 0.001 and 0.004, both well inside their intervals. Below 512 dimensions the comparison is against Google’s smallest option. 2 bits on the first 512 dimensions, 128 bytes, beats 128 float32 dimensions at 512 bytes by 0.044 [+0.025, +0.065] on SciFact and 0.053 [+0.036, +0.071] on NFCorpus, at a quarter of the bytes.

The first row is the score; the others are changes from it. The int8, 4-bit and 2-bit 768-dimension rows have intervals that include zero on both corpora. All 63 configurations per corpus are in the experiment writeup.

Centering and the 1-bit formats

EmbeddingGemma 2’s document vectors share a large common direction: their average has 85% of a single vector’s length on SciFact and 87% on NFCorpus. Plain remex at 1 bit loses 0.030 and 0.038, and centering recovers about half of that, +0.019 [+0.001, +0.038] and +0.017 [+0.003, +0.031]. At 2 bits and above centering made no measurable difference, so the recommended 2-bit index needs no stored average.

At 1 bit, sentence-transformers’ binary format, which keeps the sign of each raw coordinate with no rotation and no stored average, lands within the noise of centered remex: 0.011 behind on SciFact with an interval spanning zero, 0.001 behind on NFCorpus. remax’s uncentered sign code trails both. What remex adds over the formats sentence-transformers already ships is the 2-, 3- and 4-bit widths in between int8 and binary.

Comparison with pplx-embed-v2-late

pplx-embed-v2-late stores one 128-number vector per token and has no truncation option. Its token vectors share an even larger common direction, an average with 94.5% of a single vector’s length, and plain 1-bit codes lost 0.15 to 0.21 nDCG@10 on SciFact. Centered 2-bit codes, 16× smaller, scored within their interval of zero. That fits the explanation that the shared direction is what costs plain low-bit codes, though the two models differ in more than that: pplx scores each query token against every document token, where EmbeddingGemma 2 compares one vector per document. On both models 2 bits per coordinate is the smallest width whose loss stays inside its interval on every corpus measured, and on EmbeddingGemma 2 that holds without centering.

Using it

remex 1.2.0’s RemexIndex takes the sentence-transformers model directly and applies the card’s query and document prompts:

from sentence_transformers import SentenceTransformer
from remex import RemexIndex

model = SentenceTransformer(
    "google/embeddinggemma-2",
    config_kwargs={"vision_config": None, "audio_config": None})  # text only
index = RemexIndex(model, bits=2)
index.add(documents)
index.search("query text", top_k=10)
index.save("corpus-2bit.npz")

Passing truncate_dim=512 to the model gives the 24× row; center=True on the index is worth adding only at 1 bit; on a full corpus RemexIndex warns about the shared direction at any width, and on this model the warning matters only at 1 bit. The measurements used remex’s Quantizer with random seed 0, where RemexIndex defaults to 42; across two seeds, scores here moved by at most 0.007. The byte counts are the codes alone. A saved RemexIndex also keeps a 4-byte length per document, so the 16× row is about 196 bytes on disk. By default it decodes the codes to float32 once for fast search, so memory at query time is that of a float32 index; low_memory=True scores the codes without decoding them, which I did not time on this model.

Limits

This is text only, while the model’s main pitch is one space across modalities, and the card itself says 128 dimensions “degrades multimodal quality substantially”. Both corpora are small and English. A few thousand documents give each query fewer near neighbours to confuse, which flatters low-bit codes; a corpus of millions will likely lose more at 2 bits than these numbers show. The uncompressed SciFact score of 0.85 is higher than any model I have run on that benchmark, from a 270M encoder, and the card reports MTEB averages rather than SciFact, so I could not check it. If SciFact was in the training data, its vectors may separate relevant from irrelevant documents more cleanly than a new corpus would, which would also flatter compression; NFCorpus, at a typical 0.37, is the safer guide. Encoding both corpora took 1 hour 43 minutes on a 4-core CPU. The scripts, per-query scores and a check that recomputes every number are in oaustegard/experiments.