Skip to main content
Agentic Enterprise.
ArchitectureLayersDecisionsResearchLibrary
Research

The multi-card retrieval experiments

Five rounds of experiments on public corpora, with human relevance judgements, testing whether several purpose-specific embeddings per document beat one, whether a cheap gate and topic-level processing pay for themselves, and whether telling the index the goal helps. What held, what did not, and the approach the evidence supports.

The question

In short: Does giving each document several purpose-specific embeddings, designed with a model and executed with classical machinery, beat the usual single vector, and does it pay?

The pattern under test came from the authors' production experience in the social media domain, where it replaced reading every post with a frontier model. It is treated here as a hypothesis validated in one domain and tested on public corpora only. Four claims were separated and tested one at a time:

  1. Aspect dilution. A pooled vector of a multi-aspect document matches each aspect weakly; several purpose cards keep each aspect addressable. Tested on a synthetic corpus where the number of aspects is a dial, on a published stress test, and on human-judged scientific abstracts.
  2. The cheap gate and the bill. A relevance gate built from embeddings alone can discard off-purpose material before any expensive step, and processing discovered topics instead of documents makes the cost advantage widen with corpus size. Tested on 20,000 corporate emails.
  3. Diversity for the consumer. The right amount of diversity in a result set depends on whether a model or a person will read it. Tested with a constructed coverage task and a consumer study judged by three model families.
  4. Objective conditioning. Telling the view designer the goal (the research objective, the business context, the schema) produces an index that serves that goal better than a corpus-derived one. Tested in five forms over five rounds, ending on real human-judged instruction-following data.

The five rounds at a glance

One vector, aspects 1 to 10

0.815 to 0.294

nDCG@10 of the pooled index as a document grows from one aspect to ten. Several vectors per document hold it near 0.74.

Cards over matched chunks

+0.188

At ten aspects per document, when queries target one aspect, with 25 percent fewer embeddings than the chunk control.

Cards under matched chunks

−0.032

On human-judged scientific abstracts, where a query concerns a whole finding rather than one aspect.

Objective conditioning

0 of 5

Forms of telling the index the goal that produced a stable benefit. On real data it cost about 0.06 nDCG@10.

What held and what did not

How to read the numbers

In short: Every comparison is a paired test over the same queries, the control arm always gets the same budget as the method, and every headline had to hold across seeds and on a second machine.

  • The retrieval metric is nDCG@10, reported with 95 percent bootstrap intervals and paired permutation p-values, Holm-corrected wherever several comparisons run together. Wins, ties, and losses over the query set are recorded beside every delta.
  • Every experiment carries three arms: one pooled vector, the same text cut into blind fixed-size windows, and the same text split by purpose. The gap between pooled and chunk is what more embeddings buy; the gap between chunk and card is what purposes buy. Only the second gap is the hypothesis, and the chunk control is matched on units per document. Getting that wrong was the single largest error in the programme, and it is described on the method page.
  • The corpora form a ladder of realism, so a technique that works only on the top rung is an artefact of the test rather than a technique.

Reproducibility

In short: One command per experiment, seeded, cached, and byte-identical between runs.

Every number on these pages is produced by one command against committed code and seeds. Two seeded runs must produce byte-identical per-query output, and a test enforces it; reaching that required accumulating similarities in double precision, because a single-precision matrix product varies by around one part in a hundred million with the reduction order, enough to move a value across a rounding boundary. Generative calls are metered and cached; the design and judging calls for a full round cost between a tenth of a cent and ten US cents. Per-query scores are released alongside the aggregates so the significance tests can be recomputed independently. The experiment code and every artifact live in the benchmark repository, multicard-bench, which is released with the write-up.

Two machines were used, an Intel Mac and an Apple Silicon Mac, and the replication found two harness defects that are also documented on the method page: a GPU encoder path that silently produced different vectors, and an unfixed hash seed that changed which training queries a designer saw.

Where to go next

The research behind this page
  • The benchmark repository (multicard-bench), results directory, commit 92c18cb, 2026-08-22: one metrics.json and one per_query.csv per experiment.
  • The programme's hypothesis register and findings summary, 2026-08-20 to 2026-08-22, held privately until the write-up is released.
  • The data-to-memory pipeline and agent data engineering, where the pattern sits in the architecture.

The decision catalog

Twenty-five technology choices usually made by habit, each with a verdict, the condition that decides your case, and what would change it.

Aspect dilution: several vectors beat one, and why

A document about several things gets one embedding that is an average of all of them. The measurement of that loss, the control that separates more vectors from better-aligned vectors, and the stress test that shows how much of the gain is capacity.

On this page

The questionThe five rounds at a glanceWhat held and what did notHow to read the numbersReproducibilityWhere to go next