Hybrid retrieval has a fusion tax: combining retrievers can make results worse
New diagnostics show hybrid fusion degrades 13-48% of queries, retrieval scores track annotation budgets, chunking is a compute decision, and RAG security is tested against the wrong attacker.
- In a preprint, Chauhan and Agarwal report that hybrid retrieval ranks the relevant document below the better of its two components on 13-48% of queries across the datasets they tested. Self-reported m:…8181
- In a preprint, the same authors report the fusion tax persists across encoder sizes from 22M to 596M parameters and with learned-sparse methods like SPLADE. Self-reported m:…8181
- In a preprint on clinical RAG, the authors report that expanding ground-truth annotations increased recognized evidence without changing the underlying retrieval rankings. Self-reported m:…1515
- In a preprint measuring real document revisions, fixed-size chunking forced recomputation of 57-68% of the embedding workload, versus 38-56% for recursive chunking, while retrieval quality differed by less than 2%. Self-reported m:…1323
- In a survey of RAG protection research, the authors report that adaptive multi-turn attackers recovered 48.3% of hidden facts even when verbatim copying was prevented. Self-reported m:…6347
Fusion is not free
Hybrid retrieval — dense plus sparse, usually fused with reciprocal rank fusion — is the default stack for production RAG, built on the assumption that two retrievers beat one. In a preprint posted this week, Aaditya Chauhan and Arun Agarwal measured what fusion actually does per query instead of on average, and the answer is that it frequently makes things worse: hybrid retrieval ranks the relevant document below the better of its two components on 13 to 48 percent of queries across the datasets they tested. They call the degradation the fusion tax. m:…8181
This is not a small-model artifact. The tax persists across encoder sizes from 22M to 596M parameters and with learned-sparse alternatives like SPLADE. Worse for anyone hoping to predict it: fusion failure is only weakly predictable within a corpus and transfers unreliably across corpora, because what matters is the shifting sparse-dense balance of the specific documents you serve. The practical read is that 'don't fuse' should be a first-class deployment option, and fusion decisions should be validated per corpus rather than inherited from a blog post. m:…8181
Your metrics are what you annotate
If fusion is the pipeline's first shaky default, evaluation is the second. Miguel Mochizuki Silva and colleagues built a clinical RAG system over breast-lesion literature and studied how ground-truth annotation coverage moves the scoreboard. Their finding: expanding the annotations increased the recognized evidence without changing the underlying retrieval rankings — the system did not get better, the scoreboard just got friendlier. Retrieval metrics, in other words, are measuring the annotation budget as much as the retriever. In their tests, a hybrid stack of dense embeddings, BM25, and a cross-encoder rerank still came out on top. m:…1515
The same paper reports 384-token chunks with 20 percent overlap as the optimal configuration for its corpus — a concrete anchor for your own chunking experiments, not a universal law to copy. m:…1515
Chunking is a cost decision
The third shaky default is chunking. Yuning Bian and colleagues measured what happens to RAG systems when documents change — the update cost nobody benchmarks. When a document is revised, chunk boundaries shift, and chunks that barely changed get re-embedded anyway. Fixed-size chunking, the default, forces recomputation of 57 to 68 percent of the workload, versus 38 to 56 percent for recursive chunking and 43 to 54 percent for content-defined chunking. The catch: retrieval quality, measured as evidence recall under a 2048-token budget, differs by less than 2 percent across the strategies. Chunking is a cost decision, not a quality decision. m:…1323
Boundary drift — the movement of chunk edges between document versions — is the main driver of the wasted work, the authors report. If your knowledge base updates frequently, the chunking strategy is a cloud-compute line item, not a retrieval-accuracy one. m:…1323
Your defenses are tested against the wrong attacker
The same measurement-minded turn is hitting RAG security. In a preprint survey of the RAG protection literature, Peiyang Liu and colleagues found the field is measuring the wrong endpoint: 73.3 percent of the research targets corpus-use detection, while only 17.8 percent studies content disclosure or reconstruction — the endpoint that matters when someone is trying to pull your documents out. And the defenses being studied do not hold up: verbatim-copying guards like 8-gram filters are insufficient to prevent private-fact recovery, and adaptive multi-turn attackers recovered 48.3 percent of hidden facts in the authors' testbed even with verbatim copying prevented. m:…6347
Four papers, one pattern: the RAG stack's defaults — fuse everything, trust the metrics, chunk however is easiest, guard against verbatim copying — are all being measured and found wanting. The teams that come out ahead will treat retrieval as a system to be measured per corpus, per revision, and per attacker. m:…8181 m:…1515 m:…1323 m:…6347
What is still unresolved
- The fusion-tax diagnostic and the chunking measurements are preprints without independent replication.
- The chunking study is limited to Wikipedia revisions; update costs on enterprise corpora with different edit patterns are unmeasured.
- RAG protection still lacks standardized evaluation metrics, which the survey itself flags as an open problem.
Provenance by Synorb. Every claim links to its source. This story was written by an agent from the Manifests listed below. Read the methodology and limitations, or send a correction.
Free to reuse, including commercially. Credit “Hanging Context, with provenance by Synorb” and link to this story. Keep the cited Manifest ids. Reuse terms v1.