The cheap gate and the bill
A first pass built from embeddings alone separates on-purpose from off-purpose email at ROC-AUC 0.933, yet discards only a third of a corpus at high recall. The two-pass design is about two hundred times cheaper than reading every document with a model, but the advantage does not widen with corpus size.
Does the gate discriminate?
In short: Yes. Held-out ROC-AUC 0.933 on corporate email against unrelated Usenet posts, using nothing but embeddings and a handful of anchor phrases.
On-purpose items are 4,000 messages from a public corporate email corpus (Enron); off-purpose items are 4,000 Usenet posts from unrelated newsgroups. Provenance gives exact labels at no annotation cost. Three anchor sets describe facets of the purpose, each item scores the mean of its best few similarities to each set, and the combination is fitted on one half and reported on the other. At the target recall of 0.95 the gate keeps 66 percent of the corpus and quarantines the rest; precision at that point is 0.71.
Held-out ROC-AUC
0.933
PR-AUC 0.935. The first pass is a filter, not a sample.
Equal anchor weights
0.931
Fitting the weights is worth 0.002. The discrimination comes from having several anchor sets at all.
Deliberately irrelevant anchors
0.860
Three sets of off-purpose phrases still reach 0.860, because the space is already separable.
Logistic regression, same vectors
0.995
A trivial supervised classifier on the raw embeddings beats the anchor gate by 0.062.
Two qualifications matter more than the headline. Fitting the anchor weights is worth almost nothing: equal weights give 0.931 against 0.933. And the adversarial review ran a null control that the original design lacked. Three sets of deliberately irrelevant anchor phrases reach ROC-AUC 0.860, one purpose-specific set alone reaches 0.852, and plain logistic regression on the raw embeddings reaches 0.995. Most of the gate's discrimination comes from projecting onto a few directions of a space that is already separable, and a supervised classifier on the same vectors is better by a wide margin when labels exist.
How much does the gate remove?
In short: At the recall a production filter must hold, about a third; at 0.99 recall, an eighth.
Measured result
Corpus discarded by the gate, by recall target
Held-out half of the email-against-Usenet corpus. Precision at each point in the table view.
The economic case for a first pass is proportional to the fraction it removes. At recall 0.95 the gate removes a third; at 0.99, an eighth. It contributes a linear factor between one and two, bought directly with recall.
Table view
| Category | Percent of corpus discarded |
|---|---|
| Recall 0.80 (achieved 0.781) | 57.7 |
| Recall 0.90 (achieved 0.901) | 46.5 |
| Recall 0.95 (achieved 0.954) | 33.7 |
| Recall 0.98 (achieved 0.981) | 20.0 |
| Recall 0.99 (achieved 0.993) | 12.1 |
Source: Benchmark repository, results/e2_gate/metrics.json, commit 92c18cb (2026-08-22). On a second machine: ROC-AUC 0.936, 32.3 percent discarded at recall 0.95.
A first pass is usually described as discarding most of a corpus. On this evidence it does not. The order-of-magnitude saving in a two-pass design comes from elsewhere: from attaching generative cost to the number of discovered topics rather than to the number of documents. Attributing the saving to the filter overstates the filter and understates the substitution. A caveat bounds the result from above: separating corporate email from Usenet is easier than separating on-purpose from off-purpose material inside one corpus, which is what a deployed gate faces. Failing here would have been disqualifying; passing here is necessary rather than sufficient.
The bill, measured rather than estimated
In short: At comparable topic granularity the two-pass design costs roughly two hundred times less than sending every document to a model. The claim that the advantage widens with corpus size does not survive.
Nothing here is a rule of thumb. Token counts come from a real tokenizer over the real corpus, the number of generative calls comes from actually running the gate and the clustering at each corpus size, so the topic count K is measured rather than assumed. Only the per-token prices are external: a published price table dated 2026-08-19, reported in the artifact so the reader can edit it. Encoder compute is priced at zero, which flatters the ratio, and is stated as such.
The original run appeared to show a scaling law: topics grew as N to the power 0.52 and the cost advantage widened as N to the power 0.44. Re-running the economics with topic coverage reported alongside cost, at two clustering settings, reversed the direction of the finding.
Measured result
Cost advantage of the two-pass design against corpus size, at two topic granularities
Ratio of full-model cost to two-pass cost, Enron, 500 to 20,000 messages. Coarse clustering (minimum cluster size 25) against useful granularity (minimum cluster size 10).
- Coarse clustering
- Useful granularity
At coarse clustering the advantage grows (N to the power 0.44), because the clusters coarsen as the corpus grows: 14 topics over 10,000 messages, at 485 documents per topic, is not a summary of anything. At useful granularity topics grow faster than the corpus (N to the power 1.25) and the advantage shrinks (N to the power −0.27). What remains is a constant factor of roughly two hundred.
Table view
| Category | Coarse clustering | Useful granularity |
|---|---|---|
| 500 | 325 | 216 |
| 1,000 | 586 | 861 |
| 2,000 | 1481 | 1505 |
| 5,000 | 947 | 418 |
| 10,000 | 1824 | 152 |
| 20,000 | 1841 | 204 |
Source: Benchmark repository, results/e2_economics_mcs25/metrics.json and results/e2_economics_mcs10/metrics.json, commit 92c18cb (2026-08-22). Prices as of 2026-08-19, editable in the artifact.
- Messages
- 500
- Topics, coarse
- 2
- Coverage of survivors, coarse
- 49.6%
- Topics, fine
- 3
- Coverage, fine
- 54.0%
- Documents per topic, fine
- 45
- Messages
- 2,000
- Topics, coarse
- 2
- Coverage of survivors, coarse
- 59.8%
- Topics, fine
- 2
- Coverage, fine
- 78.6%
- Documents per topic, fine
- 393
- Messages
- 5,000
- Topics, coarse
- 7
- Coverage of survivors, coarse
- 97.6%
- Topics, fine
- 16
- Coverage, fine
- 97.1%
- Documents per topic, fine
- 152
- Messages
- 10,000
- Topics, coarse
- 7
- Coverage of survivors, coarse
- 94.8%
- Topics, fine
- 88
- Coverage, fine
- 32.1%
- Documents per topic, fine
- 18
- Messages
- 20,000
- Topics, coarse
- 14
- Coverage of survivors, coarse
- 67.9%
- Topics, fine
- 132
- Coverage, fine
- 26.3%
- Documents per topic, fine
- 20
Coverage also exposes how coarse the coarse setting is, and how the exponents depend on the clustering hyperparameter rather than on the design: on a second machine, where the dimensionality reduction and density clustering produce different topic models, the fitted exponents moved again (0.27 and 0.71 at the original setting). The exponents are a clustering artefact. The constant factor is not.
Two further caveats stand. The two designs produce different deliverables: a topic-level summary of a corpus is not a per-document understanding of it, so the ratio applies to topic discovery rather than to reading. And pricing local encoder compute at zero overstates the saving for anyone who pays for that compute by the hour.
What this establishes
Independently measured Exact provenance labels, measured token counts, dated public prices.
The research behind this page
- Benchmark repository:
results/e2_gate,results/e2_economics,results/e2_economics_mcs10,results/e2_economics_mcs25, commit 92c18cb, 2026-08-22. - The adversarial review of Sprint R1, 2026-08-20, for the null anchor control and the economics addendum.
- TnT-LLM (KDD 2024), Databricks (Jul 2026, vendor) and Distilling Step-by-Step (ACL Findings 2023) for the published economics of letting a model design and classical machinery execute, on the reading list.
Real prose: where purpose views lose
On 5,183 human-judged scientific abstracts, purpose cards score below matched blind chunks, whether the card taxonomy is written by hand or designed by a model from the corpus and the training queries. The domain boundary of the method, measured.
Diversity, and who consumes the results
Within a gated pool, harvesting the least similar tail buys sub-topic coverage about twice as cheaply as maximal marginal relevance and admits fewer answerless documents than a determinantal point process. The claim that the right amount of diversity depends on whether a model or a person reads the results did not survive a cross-family judge.