Embedding Ops

Chapter 1: Why Spaces Do Not Transfer

The rotation in Chapter 0 looked like a convenient fiction chosen to make a point. It is not. It is the default. If you train the same architecture on the same data with the same recipe twice, changing only the random seed, you get two models whose coordinates do not match — and the reason is a two-line piece of algebra.

The objective cannot see the coordinates

Almost every modern text embedder is trained contrastively. Take a query q, its matching passage p⁺, and a pile of non-matching passages p⁻. Push the matching pair together and the others apart. The loss for one example is

Every term in that expression is an inner product — the sum of the elementwise products of two vectors, written a · b. Notice what is not in the expression: any individual coordinate. The loss never asks “what is the value of dimension 412?” It only ever asks “how aligned are these two vectors?”

That is the whole story, and here is why. An orthogonal matrix R is a square matrix whose columns are unit vectors at right angles to each other. Equivalently, R^TR = I — multiply it by its own transpose and you get the identity. Rotations and reflections are exactly the orthogonal matrices. Now apply R to every embedding in your model and recompute an inner product:

Unchanged. Every inner product in the model is unchanged, so every term of the loss is unchanged, so the loss is unchanged, so the gradient is unchanged. The optimizer has no signal whatsoever distinguishing your model from the rotated copy of your model. They are not similar solutions; they are the same point in loss space wearing different clothes.

Check the algebra with actual numbers

Do not take R^TR = I on trust. Build the 70° rotation from Chapter 0. With cos 70° = 0.342 and sin 70° = 0.940:

First column dotted with itself: 0.342² + 0.940² = 0.1170 + 0.8836 = 1.0006, which is 1 up to the three digits we kept. First column dotted with the second: 0.342(−0.940) + 0.940(0.342) = −0.3215 + 0.3215 = 0. Orthonormal columns, exactly as required.

Now take two embeddings and verify the inner product survives. Let e₁ = [1.000, 0.000] and e₂ = [0.966, 0.259] — that is 0° and 15°, our cat and kitten. Their inner product is

Rotate both. For e₁:

For e₂, one coordinate at a time:

And the inner product of the rotated pair:

0.966 before, 0.9665 after — identical to rounding. Both vectors moved a long way (the first went from [1, 0] to [0.342, 0.940]) and their relationship did not move at all. Every coordinate is different; every angle is the same.

It is not only rotation

Rotation is the clean, provable case. In practice the two spaces differ in a longer list of ways, all of them invisible to a shape check:

Only the last row is the reason you wanted the upgrade. Everything above it is noise you inherit for free, and it is enough on its own to destroy retrieval.

The measurement that is invariant: gram matrices and CKA

If coordinates are arbitrary, how do you ever ask “are these two models representing the same thing?” You throw the coordinates away and keep only the relationships. Take a batch of n documents and embed them with model A into a matrix X of shape n × d. The gram matrix is

Every entry is an inner product, and we just proved inner products are rotation-invariant. Formally:

So K is a fingerprint of the geometry that survives any rotation. Centered kernel alignment (CKA) compares two models by comparing their gram matrices after centering both:

where L = YY^T is model B’s gram matrix, the bar means column-and-row centered, and ⟨·,·⟩_F is just the elementwise dot product of two matrices. It is a cosine similarity between relationship patterns rather than between vectors. CKA of 1 means “these two models agree perfectly about which documents are similar to which,” regardless of how either of them labels its axes.

This is also the technical core of the Platonic Representation Hypothesis line of work: as models get bigger and better, their representations converge in the CKA sense — they carve the world up the same way — even while remaining mutually unreadable coordinate-wise. Convergence of geometry does not buy you interoperability of storage. Your index stores coordinates.

If the difference is a rotation, can we just undo it?

Excellent instinct, and it has a name and a closed-form answer. Take N texts, embed each with both models, giving X (old, N × d) and Y (new, N × d). Find the orthogonal matrix that best carries one to the other:

This is the orthogonal Procrustes problem, solved in 1966. Compute the singular value decomposition of the cross-covariance, X^TY = U Σ V^T, and the answer is

Three lines of NumPy, no training loop:

pythonimport numpy as np

def fit_bridge(X_old, Y_new):        # both [N, 768], L2-normalized rows
    U, S, Vt = np.linalg.svd(X_old.T @ Y_new)   # [768,768] each
    return U @ Vt                            # R: [768, 768], orthogonal

R = fit_bridge(X_old, Y_new)
bridged = X_old_full @ R              # old gallery, pushed toward the new space
bridged /= np.linalg.norm(bridged, axis=1, keepdims=True)

And it partly works. Partly, because the two models are not related by an exact rotation — if they were, v2 would be exactly as good as v1 and there would be no reason to upgrade. The residual ‖XR̂ − Y‖ is the part of the upgrade that is genuinely new knowledge, and no linear map can manufacture it.

So the bridge is a stopgap, and a cheap one. Count the arithmetic. Applying a 768 × 768 matrix to one vector costs two floating-point operations per weight:

Across the whole 4.2-million-chunk corpus:

Compare that to re-embedding. A 110-million-parameter encoder over a 380-token chunk costs roughly 2 × params × tokens:

The linear bridge is about seventy thousand times cheaper than the real migration. It runs in under a minute on one GPU. It also stores in 768 × 768 × 4 bytes = 2.36 MB. That is an extraordinary cost-benefit ratio for a partial fix, which is exactly why it belongs in your incident toolkit: it is what you run at 14:05 on Tuesday to claw recall back from 0.11 to something survivable while the real backfill spins up.

The thirty-second diagnostic

Everything above suggests a test you can run during an incident, before you understand anything. Push the same twenty strings through both code paths and take the cosine of each corresponding pair.

pythonPROBE = ["reset my password", "annual invoice", ...]   # 20 short strings

a = embed_via_service_A(PROBE)     # [20, 768], normalized
b = embed_via_service_B(PROBE)     # [20, 768], normalized
same = (a * b).sum(-1)              # [20] — cosine of each pair with ITSELF
print(same.mean(), same.min())

How do you read the output? Derive the reference points instead of eyeballing them.

If the two paths are the same space, each string maps to the same vector and the cosine is 1.000, or 0.9999 with a precision difference. If the two paths are unrelated spaces, the pairs are effectively two independent random unit vectors in 768 dimensions. The cosine of two independent random unit vectors has mean 0 and standard deviation 1/√d:

So under the “unrelated” hypothesis, three standard deviations is 0.108. Anything inside ±0.11 is indistinguishable from random. Two related-but-different encoders — the realistic case — land in a middle band, typically 0.10 to 0.45, because they share coarse structure without sharing coordinates.

Twenty strings, two API calls, one dot product. This test takes half a minute and settles the single most expensive question in an embedding incident, which is why it belongs in the runbook rather than in a research notebook.

Concept to realization: what this means for storage

The practical conclusion of this chapter is one sentence, and it should change how you design the store:

Teams delete the text more often than you would think, usually because “the vector store is the search index, and the documents live in the CMS” — until the CMS purges revisions, or the chunker changes and the old chunk boundaries are gone, or the source was a PDF that got extracted once by a pipeline that no longer exists. Rebuildability is a property you have to maintain, not one you have by default.

Chapter 2: The Re-embedding Migration

The honest fix is to recompute every vector with the new encoder. That is the migration. Before we discuss patterns, do the arithmetic, because almost everyone’s intuition about the cost is wrong in both directions: they overestimate the compute and badly underestimate everything else.

Step 1: how much is there?

Our corpus is 4.2 million chunks averaging 380 tokens. Total work:

Two ways to buy that computation.

Option A, a hosted embedding API at $0.02 per million tokens:

Thirty-two dollars. The cost is not the problem. The rate limit is: at a typical 1 million tokens per minute ceiling,

Option B, self-hosted on your own GPU. A 110M-parameter encoder at 380 tokens per chunk, fp16, batched, with realistic padding waste, does about 250 chunks per second on one mid-range accelerator. Sanity-check that against physics: 250 chunks/s × 380 tokens = 95,000 tokens/s, and at 2 × params FLOPs per token that is

against a card with roughly 125 TFLOP/s of fp16 peak, i.e. about 17% utilization. That is a believable number for a standard inference stack with padded batches — not a fantasy, not sandbagging. So:

Step 2: what does it weigh?

768 float32 values is 3,072 bytes per vector. Across the corpus:

In float16, half that: 6.45 GB. Then the index structure itself. A graph index with M = 32 stores up to 2M = 64 neighbour ids at the base layer, 4 bytes each:

Fine on its own. Now the migration constraint: a shadow index means both versions are resident at once. Peak footprint during cutover:

Your steady state was 14 GB and your box has 32 GB. You are now within 4 GB of the ceiling, during the exact window when you are also running double the write traffic. This is the number that actually kills migrations, and it is the one nobody computes before the change-freeze meeting.

Step 3: the consistency window

The backfill takes 16,800 seconds. Your corpus is not frozen for 16,800 seconds. At a modest 30 document writes per second:

If you do not handle that, one document in eight is either missing from the v2 index or present at a stale revision. And it is not a random eighth — it is the freshest eighth, which is also the most queried. A naive backfill produces a v2 index that is worst exactly where it matters most, which is why the naive backfill usually passes the golden-set evaluation and fails in production.

There are two clean answers, and you will typically use both:

Dual write is the primary; the watermark sweep is the reconciliation that catches whatever the dual write dropped (a failed enqueue, a retry storm, a partition that lagged). Run the sweep and expect it to find something. If it finds nothing, verify that it is actually running.

Step 4: what happens at scale

Change one number and watch every conclusion invert. Suppose the corpus is 900 million chunks instead of 4.2 million — a large product catalogue, a log corpus, a code index.

Now the cost is real but still not the blocker — the blocker is that a shadow index needs another 1.38 TB of hot memory, and that 42 days of single-GPU backfill against a live corpus means a consistency window longer than most of your documents’ lifetimes. This is the regime where you stop asking “how do we do the backfill” and start asking “how do we avoid needing one,” which is Chapter 3.

The seven-phase cutover

Here is the pattern in full. The organizing principle is one rule, and every phase is chosen to obey it: rollback must stay free until the last possible moment. Free means a config flag, not a restore.

Phase 3 deserves a note. “Newest first” is not aesthetic. Query traffic against a document corpus is brutally skewed toward recent content — in a docs product it is common for the newest 10% of chunks to absorb more than half the retrievals. Backfilling newest-first means that after the first 10% of the work, you have already covered the majority of live traffic, and the shadow-read numbers in phase 5 become meaningful days earlier. Backfilling by primary key, which is what everyone does by accident, gives you a v2 index that is useless for evaluation until it is 100% complete.

Concept to realization: the write path, both versions

Here is what dual write actually looks like, including the part people skip — the failure semantics.

pythondef upsert_chunk(chunk_id: str, text: str):
    # v1 is the serving path: its failure is a user-visible failure.
    v1 = encoders["v1"].embed(text)                 # float32[768]
    store.upsert(chunk_id, v1, fingerprint=FP_V1)   # must succeed

    # v2 is the shadow path: its failure must NOT break serving,
    # but it must be recorded, or the reconcile sweep has nothing to find.
    try:
        v2 = encoders["v2"].embed(text)             # float32[768]
        store.upsert(chunk_id, v2, fingerprint=FP_V2)
    except Exception as e:
        backlog.push(chunk_id)                       # the sweep drains this
        metrics.incr("dualwrite.v2.failed")
        log.warning("v2 shadow write failed", chunk_id=chunk_id, err=str(e))

Three decisions in that snippet, each of which is the difference between a boring migration and a bad week. v1 first and unguarded, so a v2 outage cannot take down search. The v2 failure is caught but enqueued, so silence is impossible — an un-enqueued failure is a document that will be permanently missing from v2 and will never be noticed. And the fingerprint travels with the vector, not with the table, so a single store can hold both versions and a read can filter.

One index with a version column, or two separate indexes? Both work; the tradeoff is concrete:

For a migration specifically, prefer two physical indexes. The filter approach makes the catastrophic mistake (omitting one predicate) both easy and silent, and graph indexes are known to degrade badly under selective filters because the traversal keeps landing on nodes it must discard.

Chapter 3: Backward-Compatible Training

Chapter 2 assumed the gallery must move. Chapter 3 asks the opposite question: what if we constrain the new encoder so that its queries land where the old gallery already is? Then the upgrade ships the day the model finishes training, with zero vectors rewritten, and the backfill becomes a background chore you do whenever it is convenient — or never.

This is backward-compatible training (BCT), introduced for face-recognition galleries where re-embedding was legally and operationally impossible, and it generalizes cleanly to text retrieval.

State the requirement as an equation

Write φ_old and φ_new for the two encoders. The four things you can measure form a 2 × 2 grid, and only one cell is normally optimized:

Backward compatibility is the demand that row three be close to row one. Formally: for every query q and document d, the score under the mixed pair should behave like the score under the old pair,

which, since this must hold for all d, is essentially the demand that φ_new(q) ≈ φ_old(q) as vectors — not just as rankings. That looks like it forbids the new model from being new at all. Resolving that tension is the whole design.

The influence loss, derived

The original BCT construction is beautifully economical. The old system was trained with a classifier head κ_old sitting on top of the old embedding — a matrix of class prototypes, one row w_c per class, living in old-embedding coordinates. Train the new encoder with its own loss plus a second loss in which the new features are pushed through the frozen old head:

Why does that anchor the space? Because κ_old’s prototypes are fixed points in the old coordinate system. Requiring that φ_new(x) be classified correctly by those fixed prototypes requires φ_new(x) to point at the old prototype for its class. And the old gallery vectors also cluster around those same prototypes. Anchor both to the same landmarks and they become comparable.

Do the arithmetic on a toy

Three classes, two dimensions. The old head has prototypes at 0°, 120°, 240° — evenly spread, which is what a well-trained head does. A document of class 1 should embed near 0°.

Case A: the new model drifts 70° (the free, unconstrained outcome). A class-1 input now embeds at 70°. Its cosine against each frozen old prototype:

Class 2 wins. Now turn that into a loss. Cosine classifiers multiply by a scale s before the softmax; take s = 10. Logits are [3.42, 6.43, −9.85], so

Case B: the new model stays put. A class-1 input embeds at 0°. Logits are 10 × [1, cos 120°, cos 240°] = [10, −5, −5]:

A loss of 3.06 versus essentially zero. That gap is a strong, well-shaped gradient pulling the new representation back into the old frame, and it costs nothing at inference — the old head is used only during training and then thrown away.

The text-retrieval version: an anchor loss

Text embedders usually have no classifier head. The analogue is to anchor directly on paired embeddings of the same text:

Understand the geometry of that second term, because its behaviour has a sharp edge. For unit vectors u = φ_new(x) and v = φ_old(x), the gradient of (1 − u · v) with respect to u is simply −v. But u is constrained to the unit sphere, so only the part of that gradient tangent to the sphere can move it. Project out the radial component:

The step therefore moves u along the great circle toward v, with magnitude

where θ is the angle between them. Read that carefully: the pull is strongest at 90°, and it goes to zero at 0° — which is what we want — but it also goes to zero at 180°. The antipode is a stationary point. A dimension of the new model that initialized exactly opposite its old counterpart will not be pulled home; it will sit there contributing a constant loss of 2. In practice the noise of SGD dislodges it, but if you ever see an anchor loss plateau at a suspiciously round number, this is why.

What compatibility costs

Nothing is free. Every unit of λ is a unit of freedom taken away from the new model, and the new model wanted that freedom in order to be better. Here is the sweep from our scenario:

Read the λ = 1 row as an engineer. You keep

while day-one recall against the untouched index is 0.845 — 1.5 points below today’s 0.860, which is a real but survivable dip, and it climbs toward 0.884 as the lazy background backfill progresses. Compare that to the alternative: 42 days of blocked calendar in the large-corpus case, or a shadow index you do not have the memory for.

Then read the λ = 10 row as a warning. Chained hard enough to the old geometry, the new model inherits the old model’s mistakes. Compatibility is a leash, and past a certain tension the leash is what you are optimizing.

The real payoff is not day one — it is the whole middle

Everyone frames BCT as “ship before the backfill.” That undersells it. The more important consequence shows up during the backfill, and you can compute it.

Let m be the fraction of the gallery that has been re-embedded so far. A relevant document is a v2 vector with probability m and a v1 vector otherwise. To a first approximation, recall blends:

Without BCT, plugging in 0.905 and 0.110:

Check the halfway point by hand: 0.5(0.905) + 0.5(0.110) = 0.4525 + 0.055 = 0.5075. That is the number to sit with. At the exact midpoint of your migration, recall is 0.508 against a production baseline of 0.860. The backfill is not a period of gradual improvement; it is a canyon you have to cross, and the canyon is deepest for as long as it takes to backfill the second half. With the large corpus from Chapter 2, that is weeks of production running at roughly half the recall it had.

Now with BCT at λ = 1, plugging in 0.884 and 0.845:

0.5(0.884) + 0.5(0.845) = 0.442 + 0.4225 = 0.8645. Monotone, shallow, and above today’s 0.860 from about the 40% mark onward. There is no canyon. That is the property you are buying with 2.3 points of ceiling, and it is worth far more than the day-one number that gets quoted.

Which gives the concrete operational rule from Chapter 2 a second, independent justification: serve from one version at a time. Build the v2 index alongside, evaluate it whole, and switch atomically. A half-migrated index that is being served is not a compromise between two systems; it is a third system with score-scale pathologies that neither of its parents had, and it is the one you never evaluated.

When BCT is not available, and what to do instead

BCT has one hard prerequisite: you must control the training of the new model, and you must decide to be compatible before you train it. If v2 is a hosted API model, or an off-the-shelf checkpoint, or a model someone else already finished training last quarter, BCT is not on the menu. Your options collapse to:

That last row is a common misconception worth killing. Matryoshka representation learning lets you truncate a 1024-dimensional vector to 256 and keep most of the quality — a genuinely useful property for cost control. It does not make two different models compatible. Truncating solves “the same model, a smaller slice.” Nothing in this lesson is about that problem.

Concept to realization: the training loop

pythonfor batch in loader:                       # batch.text: list[str], len B
    e_new = phi_new(batch.text)              # float32[B, 768]  requires grad
    with torch.no_grad():                    # old model is frozen, eval mode
        e_old = phi_old(batch.text)          # float32[B, 768]  no grad

    e_new = F.normalize(e_new, dim=-1)      # unit sphere, both
    e_old = F.normalize(e_old, dim=-1)

    loss_task   = info_nce(e_new, batch.positives, temperature=0.05)
    loss_anchor = (1.0 - (e_new * e_old).sum(-1)).mean()   # scalar

    (loss_task + LAM * loss_anchor).backward()

Three implementation notes that decide whether this works. Normalize before the anchor, or the loss rewards shrinking the norm instead of turning the vector. Compute e_old under no_grad — it is a fixed target, and letting gradient flow into it turns the anchor into a mutual-collapse objective where both models meet in the middle at a degenerate point. And anchor on the same distribution you will query: if you anchor only on your training corpus but your live queries are short keyword fragments, compatibility holds on the thing you measured and fails on the thing you serve.

Chapter 4: Versioning Discipline

Everything so far has been about surviving a version change. This chapter is about making the Chapter 0 outage structurally impossible — not “unlikely if everyone remembers,” but impossible in the sense that the code refuses to run.

A model name is not a version

The instinct is to store model = "bge-base-en-v1.5" next to each vector and call it done. That field is necessary and nowhere near sufficient, because the same model name produces different spaces under any of the following:

The instruction-prefix row is the one that has burned the most teams. Many retrieval encoders expect the query to be prefixed with something like “Represent this sentence for searching relevant passages:” while passages get no prefix. Two services embed “billing plan”; one prepends the prefix and one does not; both call the same model, both produce 768 floats, and their outputs are in measurably different regions of the space. Same weights. Different space. No error.

Fingerprint the whole pipeline

The unit of versioning is not the model. It is the text-to-vector function, end to end. Hash everything that can change its output:

pythonimport hashlib, json

EMBED_SPEC = {
    "chunker":      {"kind": "recursive", "size": 512, "overlap": 64},
    "normalizer":   {"lowercase": False, "strip_html": True, "unicode": "NFKC"},
    "model_id":     "BAAI/bge-base-en-v1.5",
    "revision":     "a5beb1e",        # pin the commit, never a moving tag
    "pooling":      "cls",
    "normalize":    True,
    "max_seq_len":  512,
    "query_prefix": "Represent this sentence for searching relevant passages: ",
    "doc_prefix":   "",
    "dtype":        "float16",
    "dim":          768,
}

def fingerprint(spec: dict) -> str:
    # sort_keys makes the hash independent of dict ordering
    blob = json.dumps(spec, sort_keys=True, separators=(",", ":"))
    return hashlib.sha256(blob.encode()).hexdigest()[:12]

FP = fingerprint(EMBED_SPEC)     # e.g. "a3f19c4e8b02" — 12 hex chars

Twelve hex characters is 48 bits, so an accidental collision needs on the order of 2²⁴ ≈ 16 million distinct specs before it becomes plausible. You will have dozens. It is enough, and it fits in a log line.

Where the fingerprint has to live

Refuse, do not degrade

Here is the search path, written so that Chapter 0 cannot happen:

pythonclass IncompatibleEmbedding(RuntimeError): pass

def search(index, qvec, qfp, k=8):
    if qfp != index.manifest.fingerprint:
        # 503, page someone, DO NOT fall through to a nearest-neighbour call
        raise IncompatibleEmbedding(
            f"query fp={qfp} vs index fp={index.manifest.fingerprint}; "
            f"refusing to compare vectors from different spaces"
        )
    if qvec.shape[-1] != index.manifest.dim:
        raise IncompatibleEmbedding("dimension mismatch")   # the easy one
    return index.knn(qvec, k)

Two things about the order of those checks. The dimension check is second because it is the weak one: 768 == 768 proves nothing at all, and a system that only checks dimensions has a check that passes in exactly the case that hurts. And the fingerprint mismatch raises rather than logging a warning, because a warning in a retrieval path is a line in a file that nobody reads until the postmortem.

The contract test that catches the upstream you do not control

Pinning a revision protects you from the repo. It does not protect you from your own dependency upgrades: a library bump that changes the default pooling, a tokenizer update that handles a Unicode class differently, a new kernel that changes reduction order. The defence is a golden vector test in CI.

python# tests/test_embedding_contract.py
GOLDEN = np.load("tests/fixtures/golden_a3f19c4e8b02.npy")   # [20, 768]
PHRASES = open("tests/fixtures/golden_phrases.txt").read().splitlines()

def test_space_is_unchanged():
    got = embed_documents(PHRASES)                # [20, 768], normalized
    cos = (got * GOLDEN).sum(-1)                  # [20]
    assert cos.min() > 0.9999, (
        f"embedding space moved (min cos {cos.min():.6f}). "
        f"If this is intentional, bump EMBED_SPEC and regenerate the fixture."
    )

Why a cosine gate at 0.9999 rather than elementwise equality? Because legitimate, harmless variation exists. Running the same fp32 weights in fp16 perturbs each output slightly; measured on a normalized 768-dimensional vector, the cosine between the two typically lands around 0.9999 to 0.99999. An exact-equality test fails on every hardware change and gets deleted within a month. A cosine gate at 0.9999 passes precision noise and catches a pooling change, which moves cosine to something like 0.6.

And the assertion message is doing real work: it tells the engineer that the correct response to a red test is to bump the version, not to loosen the threshold. Half of version discipline is making the right action the obvious one at the moment of failure.

Normalize once, at write

A small operational rule with an outsized payoff. If your metric is cosine, store unit vectors and use inner product as the metric. Three consequences:

1. Cosine and inner product become the same operation, so it is impossible to configure the index with the “wrong” metric.
2. You cannot end up with a store containing a mix of normalized and unnormalized vectors, which produces a ranking dominated by whichever documents happened to have large norms.
3. The per-query square root and division disappear from the hot path.

Put "normalize": true in the spec so it is part of the fingerprint, and assert it at insert time: assert abs(np.linalg.norm(v) - 1.0) < 1e-3. That single assertion has caught more real bugs than most monitoring dashboards.

Make the migration a state machine, not a set of feature flags

Most migrations are governed by four or five independent booleans: dual_write_enabled, backfill_running, shadow_read_pct, v2_traffic_pct, v1_index_loaded. Five booleans is thirty-two configurations, of which perhaps eight are coherent and several are actively dangerous — for instance, v2 traffic above zero while the backfill is at 40%, which Chapter 3 just showed produces a system nobody evaluated.

Collapse them into one field with a legal transition table:

The value of writing it this way is that the dangerous configurations stop being reachable. “Serve v2 while backfilling” is not a flag someone can set at 3 a.m.; it is a transition that does not exist. And the state is one string in one place, so “where are we?” has an answer that fits in a status page rather than requiring an archaeology session across five config stores.

Log enough to have a postmortem at all

One structured line per search, containing the things that let you reconstruct an incident after the fact:

pythonlog.info("search",
    query_fp   = qfp,                  # which encoder made the query vector
    index_id   = index.manifest.index_id,
    index_fp   = index.manifest.fingerprint,
    state      = MIGRATION_STATE,      # the state machine above
    top1_score = float(scores[0]),    # feeds the drift and cache dashboards
    top8_ids   = ids[:8],             # enables overlap@8 after the fact
    latency_ms = dt,
)

Two of those fields are the ones people leave out and then wish they had. top1_score is what builds the score histogram that makes a version mismatch visible at a glance and what recalibrates every threshold in Chapter 7. top8_ids is what lets you compute overlap between two systems retroactively — replay yesterday’s logged results against today’s index and you get a change-magnitude number with no labels, no shadow traffic, and no waiting.

Concept to realization: the index manifest

pythonmanifest = {
    "index_id":      "docs-prod-2026-08-17",
    "fingerprint":   "a3f19c4e8b02",      # the text-to-vector function
    "spec":          EMBED_SPEC,           # full, human-readable, for the postmortem
    "corpus_snapshot": "cms@2026-08-17T04:00Z",
    "dim":           768,
    "metric":        "ip",                # inner product, because vectors are unit
    "count":         4_200_117,
    "build":         {"algo": "hnsw", "M": 32, "ef_construction": 200},
    "built_at":      "2026-08-17T06:14:22Z",
    "golden_recall8": 0.861,           # measured at build, before serving
}

The last field is the quiet hero. An index that records its own golden-set recall at build time means the question “did this index ever work?” has an answer that does not require rerunning anything, and a loader can refuse an index whose build-time recall is below a floor. It converts a class of production mysteries into a build-time failure.

Chapter 5: Drift With No Model Change

Now change nothing. Same encoder, same fingerprint, same index, no deploys for six weeks. Retrieval quality still degrades. This is drift, and it is the failure mode that version discipline cannot touch, because nothing about your system changed — the world did.

Two drifts, different signatures

Content drift is the corpus moving. You launch a product line and 40,000 chunks of new vocabulary land in the index. A support team starts writing in German. An acquisition dumps a differently-structured knowledge base into the same store. The encoder still embeds everything competently, but the distribution of what lives in the index is no longer the one your thresholds, your clusters, and your evaluation set were built on.

Query drift is the users moving. An incident sends everyone asking about one outage. A new mobile surface funnels three-word queries into a system tuned on full sentences. Seasonality arrives. Marketing renames a feature and users start typing the new name, which appears nowhere in the corpus.

Signal 1: centroid shift, with the noise floor derived

The cheapest useful signal. Average all of yesterday’s query embeddings into one vector — the centroid — and compare it to the trailing four-week centroid by cosine. If they diverge, the shape of what people are asking has moved.

The obvious objection: centroids of finite samples wobble, so what counts as a real move? Derive the noise floor rather than guessing it.

Text embeddings are famously anisotropic — they do not fill the sphere evenly but cluster in a cone, so the population mean μ of unit vectors has a substantial norm. Take ‖μ‖ = 0.42, which is typical. Each query is a unit vector, so its total variance around the mean is 1 − ‖μ‖², spread over d = 768 dimensions:

Average n = 50,000 queries and each coordinate’s variance drops by a factor of n:

The total squared length of the noise, summed over all 768 coordinates, and then its square root:

That noise is essentially perpendicular to μ in high dimensions, so the angle it induces is noise divided by signal:

and for a small angle, cos θ ≈ 1 − θ²/2:

If deriving the floor makes you nervous, measure it instead: split last month’s queries into random halves two hundred times, compute the cosine between half-centroids each time, and take the first percentile. That bootstrap gives you the null distribution with no assumptions at all, and it is twenty lines of code.

Signal 2: PSI on a scalar projection

Centroid cosine collapses everything into one number and can miss a shift that moves mass around without moving the mean. The complementary tool is a histogram comparison — but you cannot histogram 768 dimensions, so first project down to a scalar. Three projections that work:

Then bin the scalar into 10 buckets whose edges are the baseline’s deciles — so by construction the baseline puts exactly 10% in each bucket — and compute the population stability index:

with e_i the expected (baseline) share and a_i the actual (current) share.

Where that formula comes from

It is not arbitrary. Expand the two Kullback–Leibler divergences between the distributions:

Add them:

PSI is exactly the symmetrized KL divergence. That explains its two most important properties: it is symmetric (swapping which distribution you call baseline gives the same number), and it diverges to infinity if any bin empties, because ln(0) does. Guard against that with a floor — replace a zero share with 0.5/n — or the day your German traffic vanishes your dashboard reports inf and the alert is useless.

A worked PSI, by hand

Baseline is 0.10 in every bin. This week’s shares, low bin to high:

(They sum to 1.00 — check it: 0.04+0.05+0.07+0.09+0.10 = 0.35, and 0.11+0.12+0.13+0.14+0.15 = 0.65.) Mass has slid from the low bins to the high bins. Term by term, with (a − e) · ln(a/e):

Sum them:

Against the conventional bands from credit-risk monitoring, where PSI has been used for decades:

Our 0.148 is squarely in “investigate.” Notice how lopsided the contributions are: bins 1 and 2 alone contribute 0.0896 of the 0.148, over 60%. PSI is dominated by bins that emptied out, not bins that filled up, because the logarithm punishes proportional loss far harder than proportional gain. That asymmetry is a feature: a bin going from 10% to 4% usually means something stopped working, and a bin going from 10% to 15% usually means something got popular.

What an emptied bin does, and why the floor matters

Change one thing: suppose bin 1 does not shrink to 0.04 but empties completely, because the traffic source that produced those queries was switched off. Then a₁ = 0 and the term is

Your dashboard now reads inf and the alert is useless — it fires, but it tells you nothing about magnitude and it poisons any average you compute over bins or days. Apply the floor: replace a zero share with 0.5/n. With n = 50,000 that is 10⁻⁵:

One empty bin contributes 0.921 all by itself, which swamps the 0.25 “major shift” band by a factor of nearly four. That is the right behaviour, not a bug in the floor: a segment of your traffic vanishing entirely is the largest distribution change that can happen, and PSI should say so loudly. The floor converts an uninterpretable infinity into a large, finite, comparable number.

The corpus side: content drift needs its own measurements

Both signals so far watch queries. Content drift lives on the other side and needs its own instruments, computed on a sample of the index rather than on traffic.

That last row deserves emphasis because it converts a monitoring system into a product input. “Four hundred queries a day about the new billing flow, average top-1 similarity 0.44, nothing in the corpus above 0.55” is not an infrastructure alert. It is a documentation ticket with evidence attached, and it comes free from machinery you built to watch for regressions.

A worked reading. Suppose the corpus centroid cosine week over week is 0.9962 while the query centroid cosine is 0.9971. Both are far below the 0.99995 noise floor, so both moved for real. Two possibilities, and one extra measurement separates them: if coverage held steady, the corpus and the queries moved together — a product launch, working as intended, and the action is to refresh your frozen artifacts. If coverage fell, they moved apart — the questions went somewhere the content did not follow, and the action is content, not engineering.

Signal 3: the out-of-distribution rate

The most directly actionable of the three, because it measures the thing users experience. For each query, record the top-1 similarity. Establish the baseline 5th percentile — say it is 0.61. Then track the fraction of queries that fall below 0.61. By construction it starts at 5%. When it climbs, users are asking things your corpus cannot answer.

Set the alert with statistics, not vibes. The standard error of a proportion p over n samples is √(p(1−p)/n). With p = 0.05 and n = 50,000:

Which reveals how tight this instrument is at scale: a rise from 5.0% to 5.3% is already a three-sigma event. If your alert threshold is “10%,” you have a detector that fires only after the problem has been obvious to users for weeks.

Drift is not always a bug

This is the part teams get wrong emotionally. Every signal above fires when the new product line launches. That launch is the company working. The corpus should contain the new vocabulary; users should be asking about it.

So the correct reading of a drift alarm is not “something is broken” but “the assumptions baked into your frozen artifacts are now stale.” The frozen artifacts are: the golden query set, every calibrated threshold (out-of-distribution cutoff, dedup cutoff, semantic-cache cutoff), the cluster assignments, and any hand-tuned reranker weights. Drift is the signal to refresh those, not to roll anything back.

Concept to realization: the daily job

pythondef daily_drift_report(day_vecs, day_top1, base):
    # day_vecs: float32[n, 768] normalized query embeddings from the last 24h
    # day_top1: float32[n]      best similarity each query achieved
    # base:     baseline artifacts, frozen with the golden set

    c_now = day_vecs.mean(0)
    c_now /= np.linalg.norm(c_now)                       # [768]
    centroid_cos = float(c_now @ base.centroid)          # scalar, floor ~0.99995

    proj  = day_vecs @ base.pc1                             # [n] scalar projection
    share = np.histogram(proj, bins=base.decile_edges)[0] / len(proj)
    share = np.maximum(share, 0.5 / len(proj))            # the ln(0) guard
    psi   = float(((share - 0.1) * np.log(share / 0.1)).sum())

    ood   = float((day_top1 < base.p05_top1).mean())        # starts at 0.05
    se    = math.sqrt(0.05 * 0.95 / len(day_top1))
    return {
        "centroid_cos": centroid_cos, "centroid_alarm": centroid_cos < 0.9995,
        "psi": psi,                   "psi_alarm": psi > 0.10,
        "ood_rate": ood,             "ood_alarm": ood > 0.05 + 3 * se,
    }

Everything in that function is a dot product, a histogram, and a mean over one day of queries. On 50,000 queries it runs in under a second. There is no excuse for not having it, and its absence is why most teams learn about drift from a customer.

Chapter 6: Evaluation in Production

Chapters 2 and 3 gave you ways to change the encoder. Chapter 5 gave you ways to notice the world changing. This chapter is the gate that stands between a candidate and production, and its central claim is uncomfortable: most embedding evaluations cannot detect the regression they are supposed to prevent, because they compare point estimates on samples that are far too small.

Build the golden set correctly

A golden set is a frozen list of queries, each with a judged set of documents that ought to be retrieved. Three construction rules that matter more than the size:

1. Sample from real traffic, stratified by frequency. If you take the top 200 queries you have built a head-only set, and the head is the part that works. Draw from every frequency decile including the tail, where retrieval actually struggles.
2. Judge documents, not chunk ids. Chunk ids change when the chunker changes; a golden set keyed on chunk ids silently dies at the next chunking tweak. Key on a stable document identifier plus a quoted snippet.
3. Version it and record the judging date. Chapter 5 just told you it goes stale. A golden set with no date is a golden set nobody knows to refresh.

Which metric gates the deploy

For a retrieval-augmented generation stack, the primary gate is recall at the exact k you pass to the model. If the language model receives 8 chunks and reranks them itself, the ordering inside those 8 barely matters, but a document that misses the top 8 is invisible forever. Gate on recall@8, watch nDCG as a secondary.

nDCG by hand, once

Worth doing once so the number stops being magic. One query, graded relevance on a 0–3 scale. Your system returns, in order, documents graded 3, 0, 2, 1, then nothing relevant. Discounted cumulative gain uses gain 2^rel − 1 and discount log₂(rank + 1):

The ideal ordering would have put the same three graded documents at ranks 1, 2, 3 as 3, 2, 1:

Notice how forgiving this is: a completely irrelevant document sitting at rank 2, above two relevant ones, cost under five points. That is why nDCG is a poor gate for a candidate-generation stage — it is designed to reward getting the top roughly right, and candidate generation cares about the whole set.

Recall@k by hand, and the definition trap inside it

Recall@k looks too simple to get wrong, and it is routinely gotten wrong. Take one query with three judged relevant documents, of which two appear in the top 8:

Now a second query with one judged relevant document, which does appear in the top 8:

Average over the two queries: (0.667 + 1.000)/2 = 0.833. That is macro recall, averaging the per-query rates. The alternative is micro recall, pooling first:

Same data, two answers eight points apart, and both are called “recall@8.” Macro weights every query equally, which is what you want when queries represent users. Micro weights every judged document equally, which lets a handful of heavily-judged queries dominate. Pick macro, write it down in the evaluation spec, and never compare a number you computed one way against a number someone else computed the other way.

One more trap in the same neighbourhood: if a query has a relevant document that is not in the corpus at all, recall is capped below 1 for reasons that have nothing to do with the retriever. Judge relevance against what you actually indexed, or your gate will punish encoder changes for a content problem.

Overlap@k by hand

The label-free workhorse. Suppose for one query the two systems return

The intersection is {a, c, e}, so

Average that over a few thousand live queries and you have a change-magnitude number that cost nothing, needed no judgments, and can be computed retroactively from the top8_ids you started logging in Chapter 4. An overlap of 0.375 sits in the “different system” band below — a big change, plausible for a genuine encoder upgrade, and an absolute requirement that you have labelled evidence before ramping.

The part everyone skips: is the difference real?

Say your candidate scores nDCG 0.941 against production’s 0.951 on 200 golden queries. A one-point drop. Ship or block?

Neither, yet, because you have not asked whether 0.010 is bigger than the noise. Per-query nDCG varies wildly — a standard deviation around 0.25 is normal, since some queries score 1.0 and some score 0. The standard error of the mean over 200 queries is

The difference you are trying to resolve is 0.010 and your measurement error on each system alone is 0.0177. The comparison is meaningless.

Fix it with pairing. Run both systems on the same queries and analyze the per-query differences, which cancels the enormous query-to-query variation. A typical paired standard deviation is far smaller — take 0.08:

Now 0.010 is 1.77 standard errors, which still does not clear the conventional 1.96. Pairing bought you a factor of three and it is still not enough. So compute the sample size you actually need instead of guessing.

Sizing the golden set from a non-inferiority requirement

State the requirement the way a deploy gate should: “I will ship if I can show the new system is not worse by more than δ = 0.010 nDCG.” That is a non-inferiority test. With an observed drop d and paired standard error SE, the condition is that the upper end of the confidence interval on the drop stays below the margin:

Suppose the observed drop is a small d = 0.004 and the paired sd is 0.08. Solve for n:

Label-free signals you get for free

Judging is expensive, so lean on the two comparisons that need no labels at all.

Overlap@k. Run the same live query through both indexes and measure how many of the top-k are shared. It is a pure change-magnitude measurement, and it is astonishingly informative:

That last row is the reason overlap@k belongs on the migration dashboard. Chapter 0’s outage would have shown overlap@8 near 0.02 within one minute of the deploy, with no labels, no golden set, and no waiting.

Score distribution. Log the top-1 similarity histogram for both systems. A healthy upgrade shifts it slightly. A version mismatch flattens and shifts it in a way that is visible at a glance — and a cache or threshold calibrated on the old histogram is now mis-calibrated, which is the bridge to Chapter 7.

Canary the index, not just the code

A canary deploy of a service routes a small share of traffic to new code. An embedding change needs a canary of the data: a second index, built over the same corpus snapshot, served to a small share of queries, with its own manifest.

And gate on more than relevance. An encoder change moves latency (different sequence length handling), memory (different width), cost per query, and the distribution of similarity scores that every downstream threshold depends on. A candidate that improves recall by two points and doubles p99 has not passed.

Chapter 7: Semantic Caching as an Ops Surface

A semantic cache is the most seductive object in this lesson. Embed the incoming question, look for a stored question whose embedding is within τ cosine, and if you find one, return the stored answer without calling the language model at all. It saves real money and real latency, it is fifty lines of code, and it is built out of exactly the machinery we have spent six chapters learning to distrust.

What makes it different from retrieval is one word: automatic. A retriever returns candidates and a downstream model gets to reject them. A cache returns a finished answer straight to the user. There is no second opinion. So a cache is a retrieval system whose false positives are shipped, and that changes the arithmetic completely.

Model it, then decide

Let p be the share of incoming queries that genuinely have a semantically equivalent entry already cached — the repeats. For those, the best cached neighbour scores high; call that distribution normal with mean 0.93 and standard deviation 0.04. For the other 1 − p, the closest cached entry is related but different, and its score is normal with mean 0.70 and standard deviation 0.08. Those two overlapping bell curves are the entire problem.

With Φ the standard normal cumulative function, at a threshold τ:

The second term of that sum is the false-hit rate: the fraction of all traffic that receives a confident answer to a question it did not ask.

Run the numbers at three thresholds

Take p = 0.22 — 22% of traffic is a genuine repeat, which is a healthy support-desk figure. At τ = 0.90:

Per million queries: 175,000 cache hits, of which 4,844 are answers to a different question.

At τ = 0.95 the same arithmetic gives z_same = 0.50 and z_diff = 3.125, so P(hit|same) = 0.3085 and P(hit|different) = 0.000889. True hits 0.0679, false hits 0.000693, hit rate 6.86%, and 693 wrong answers per million. At τ = 0.85, z_same = −2.0 and z_diff = 1.875, giving a 23.9% hit rate and 23,709 wrong answers per million.

Dropping the threshold from 0.95 to 0.85 multiplies your savings by 3.5 and your wrong answers by 34. Savings grow roughly linearly with the threshold move; false hits grow with the tail of a Gaussian, which is to say explosively. These two quantities are not on the same scale and never will be.

The break-even, derived

Convert both sides to money. Let ΔC be what you save on a hit — skipping a language-model call. A substantive retrieval-augmented answer with 4,000 input tokens at $3 per million and 500 output tokens at $15 per million costs

Let L be what a wrong answer costs you: a support contact, a lost conversion, a small piece of trust. Put $0.50 on it. Net value per query is savings minus losses:

Substitute h = true + false and group:

Since L > ΔC, the second term is a penalty. Set net to zero and divide by h to get the condition on the false-hit share f:

Check it against the table. At τ = 0.90, f = 2.77% < 4%: allowed. At τ = 0.85, f = 9.93% > 4%: forbidden, and indeed the net there is

which is minus seven thousand dollars per million queries, from a feature installed to save money.

The uncomfortable case: when no threshold works

Raise L to $5 — a wrong answer in a medical, legal, or financial context. The condition becomes f < 0.02/5 = 0.4%. Push τ all the way to 0.99: z_same = 1.5 and z_diff = 3.625, giving true hits 0.0147 and false hits 0.000112, so

Still nearly twice the limit — at a threshold so tight that the cache now serves 1.5% of traffic. The different-intent distribution has a tail that does not thin out fast enough, so there is no threshold at which this cache is safe. Raising τ trades away all the value long before it buys enough safety.

The fix is not a number, it is a mechanism: put a cheap verifier between the hit and the user. A small model, or one fast language-model call on a short prompt, asked only “does this stored answer actually answer this new question, yes or no?” It reintroduces cost, but it caps the tail instead of hoping the tail is thin.

Price the verifier

Say the verifier costs C_v = $0.001 per hit, rejects 90% of false hits, and mistakenly rejects 2% of true hits. Every hit pays the verifier; only survivors are served:

Evaluate at τ = 0.90, where true = 0.17014, false = 0.004844, h = 0.17499, and L = $0.50:

Against the $1,078 per million the same threshold earned without a verifier. The verifier nearly triples the value of the cache, which is the opposite of what “adding a cost” sounds like it should do.

Now redo it at L = $5, the case where no threshold worked at all. Without the verifier, τ = 0.90 gives

With the verifier:

Why this belongs in an embedding-ops lesson

Because every number above is a property of one specific embedding space, and the cache is the surface where that dependency bites hardest.

The first row is Chapter 0 replayed inside the cache, and it is even harder to see, because a cache does not log misses as errors and nobody has a dashboard for “answers that were confidently retrieved from the wrong bucket.” Bump the encoder without invalidating the cache and you get a system that serves old-space neighbours to new-space queries, indefinitely, silently.

python# the cache key must name the space, not just the query
def cache_lookup(question: str, fp: str, tau: float):
    q = embed_query(question)                       # float32[768], space = fp
    ns = cache.namespace(fp)                        # one index per fingerprint
    hit, score = ns.nearest(q)
    if hit is None or score < tau:
        return None
    if hit.source_rev != current_rev(hit.source_ids):   # world changed
        ns.evict(hit.key)
        return None
    metrics.observe("cache.hit_score", score)       # feeds recalibration
    return hit.answer

Namespacing by fingerprint is the important line. It makes a version bump degrade into a cold cache — a temporary cost increase, entirely visible on a dashboard — instead of a silent correctness failure. That is the pattern to internalize from this whole lesson: when a space changes, arrange for the system to lose performance rather than truth.

Chapter 8: The Runbook

Everything above, compressed into the document you actually want open when you are shipping an encoder change. Each phase has an exit criterion — a measurable thing that must be true before you move — and a rollback, which must stay cheap until the very end.

Preconditions: do not start without these

If P1 fails, stop and fix that first; nothing else matters. If P6 fails, your options narrow to backward-compatible training or a learned bridge, because a shadow index is not physically available to you.

The decision: which migration are you doing?

The phases, with exit criteria

Post-conditions: the cleanup nobody schedules

A migration is not finished when traffic is at 100%. Every artifact calibrated on the old space is now wrong, and each one fails quietly in its own way:

That table is the argument for keeping the number of things calibrated on raw similarity values as small as you possibly can. Every hard-coded 0.97 in your codebase is a hidden dependency on one model version.

The five that actually go wrong

The one-page checklist

runbook# BEFORE
[ ] source text retained and re-readable for 100% of vectors
[ ] fingerprint on every row; manifest on every index; registry append-only
[ ] read path RAISES on mismatch (verified by a deliberate staging failure)
[ ] golden set sized from the margin: n >= (1.96 * sd_paired / (delta - d))^2
[ ] baselines captured: recall@8, nDCG@10, p99, cost/query, top-1 histogram
[ ] capacity: 2x vectors + 2x graph fits with headroom, during 2x write load

# DURING
[ ] dual write on, v2 branch guarded, failures enqueued not swallowed
[ ] backfill newest-first, cursor checkpointed, idempotent
[ ] reconcile sweep runs twice; second run finds nothing
[ ] shadow read: overlap@8 in band, score histogram shifted not scrambled
[ ] canary 1% -> gates -> 5% -> gates -> 25% -> gates -> 100%
[ ] soak seven days across a full weekly cycle

# AFTER
[ ] cache namespaced by fingerprint and recalibrated
[ ] dedup / OOD / rerank thresholds re-measured in the new space
[ ] drift baselines rebuilt from new-space traffic
[ ] golden set re-judged against the new corpus
[ ] v1 index dropped on a SEPARATE day, fingerprint kept forever

Chapter 9: Connections

One idea has been underneath all nine chapters, and it is worth stating once more in the plainest possible form. An embedding is a coordinate, and coordinates are meaningless without the frame that produced them. Everything else in this lesson — the fingerprints, the dual writes, the anchor losses, the non-inferiority tests, the cache namespaces — is bookkeeping in service of that one sentence.

The strategies, side by side

The fourth row is the one to take seriously. A large share of “our retrieval got worse” incidents have no version change in them at all. Before you plan a migration, check whether the encoder is the thing that moved.

Where to go from here

Understand the object itself. Start at Vector Embeddings if any of the geometry here felt shaky, then Similarity Metrics for why cosine and not something else, and High-Dimensional Geometry for why anisotropy makes a centroid norm of 0.42 the normal case rather than a bug.

Understand where they are trained and stored. Contrastive Learning is the objective whose rotation invariance is the root cause of Chapter 1. Vector Databases covers the index structures whose memory footprint set the migration ceiling in Chapter 2, and Storage and Retrieval is the general theory of derived indexes, of which an embedding store is one instance.

Understand the system they sit inside. RAG is the consumer of everything here; Text Chunking is the pipeline stage that must be inside your fingerprint; Caching and CDNs gives the classical cache-invalidation background that Chapter 7 adapts to a probabilistic key.

Understand how to prove it works. Evaluation Statistics is the machinery behind Chapter 6’s sample-size derivation; Regression Testing for ML is the gate discipline generalized beyond retrieval; Embedding Benchmarks explains what a public leaderboard score does and does not tell you about your corpus; and GenAI Evaluation covers judging the answers rather than the retrieval.

Related operational surfaces. On-Device Embeddings multiplies every problem here by the number of client versions in the wild — a phone that has not updated in six months is a permanent old-space query source, which is the strongest argument for backward-compatible training that exists. AI Evaluation and Agent Evaluation extend the gating story to systems where retrieval is one step of many.

Five things to carry out of here

References

1. Shen, Xiong, Xia, Soatto. “Towards Backward-Compatible Representation Learning.” CVPR, 2020. arXiv:2003.11942 — the influence loss of Chapter 3.
2. Ramanujan, Vasu, Farhadi, Tuzel, Pouransari. “Forward Compatible Training for Large-Scale Embedding Retrieval Systems.” CVPR, 2022. arXiv:2112.02805 — transform the old gallery instead of constraining the new model.
3. Hu et al. “Learning Backward Compatible Embeddings.” KDD, 2022. arXiv:2206.03040 — the recommendation-system framing of the same problem.
4. Kornblith, Norouzi, Lee, Hinton. “Similarity of Neural Network Representations Revisited.” ICML, 2019. arXiv:1905.00414 — CKA and why gram matrices are the invariant object.
5. Huh, Cheung, Wang, Isola. “The Platonic Representation Hypothesis.” ICML, 2024. arXiv:2405.07987 — representations converge in geometry while staying mutually unreadable in coordinates.
6. Schönemann. “A generalized solution of the orthogonal Procrustes problem.” Psychometrika 31(1), 1966 — the closed-form bridge of Chapter 1.
7. Malkov, Yashunin. “Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs.” IEEE TPAMI, 2018. arXiv:1603.09320 — the index whose memory arithmetic drives Chapter 2.
8. Muennighoff, Tazi, Magne, Reimers. “MTEB: Massive Text Embedding Benchmark.” EACL, 2023. arXiv:2210.07316 — where the “four points better” in Chapter 0 comes from, and why it does not transfer to your corpus.
9. Kusupati et al. “Matryoshka Representation Learning.” NeurIPS, 2022. arXiv:2205.13147 — nested widths, which solve truncation and not compatibility.
10. Gao, Yao, Chen. “SimCSE: Simple Contrastive Learning of Sentence Embeddings.” EMNLP, 2021. arXiv:2104.08821 — anisotropy and the alignment/uniformity view behind the centroid-norm assumption.
11. Bang et al. “GPTCache: An Open-Source Semantic Cache for LLM Applications.” NLP-OSS workshop, 2023. arXiv:2308.02669 — the cache design Chapter 7 puts a price on.