Retrieval Keeps Getting Better. The Answers Do Not Always Follow

The retrieval trap

A cost-aware study in a preprint found that a selective policy could match always-on hybrid retrieval on Hit at 10 while triggering the more expensive method for only 24.5 percent of questions. That is a useful efficiency result, and it comes with a warning attached: improved evidence retrieval did not guarantee improvements in the faithfulness or correctness of generated answers. The retrieval step can get better while the final answer stays wrong. m:…4735

The implication is plain. A team that only measures retrieval is measuring half the system. The same study found that a regularized linear model could predict the incremental benefit of hybrid retrieval from pre-routing features, which suggests routing can be planned rather than applied blindly. But planning the retrieval step still leaves the reader step untested, and that is where faithfulness is won or lost. m:…4735

Negation breaks dense retrieval

A second preprint built a benchmark of 63,000 original and edited evidence pairs across six datasets to test whether retrievers can tell a claim apart from its negation. Dense retrievers built on Qwen3 and Jina performed near chance when distinguishing original evidence from near-twin edited evidence. BM25, the older lexical method, outperformed the dense retrievers at separating the pairs. m:…0216

This matters because contradiction is not an edge case in real document collections. Policies change, prices move, guidance gets superseded, and a system that cannot separate a statement from its negation will happily retrieve the wrong version with high confidence. A retrieval stack that looks strong on ordinary similarity tests can fail exactly where a human reader would notice the difference. m:…0216

Structure is evidence

Two more preprints attack the same problem from the document side. One rebuilds web HTML into structure-completed evidence units that preserve headers and table relationships, instead of letting standard chunking discard that context. The method improved answer accuracy by 5 to 6 points on text and screenshot retrieval pipelines, and it works as a training-free layer with existing retrievers and readers, without requiring index changes. m:…3409

The other builds a hypergraph over long multimodal documents to capture structural, logical and cross-modal relationships among document elements, then selects evidence adaptively under a cost-aware budget. It outperformed several state of the art baselines on three multimodal document question answering benchmarks. A fifth preprint, aimed at multi-hop retrieval, selects context under a fixed token budget using graph proximity and spectral information, and reports improved evidence recovery and generation quality against dense, hybrid, HyDE and LightRAG baselines. m:…5715 m:…5070

What to do with this

Taken together, these preprints describe a retrieval stack that is getting more careful about what it keeps, what it spends and what it can distinguish. Structure-preserving evidence units, hypergraph selection and graph-guided context selection all try to keep relationships intact instead of shredding documents into isolated chunks. Cost-aware routing tries to spend the expensive retrieval method only where it helps. m:…3409 m:…5715 m:…4735

None of that removes the central warning. Retrieval gains did not guarantee faithful answers, and dense retrievers failed near chance on negation pairs. The honest test for a retrieval system is therefore end to end: retrieve, generate, and then check whether the answer follows from the evidence, including evidence that contradicts the first plausible passage. Anything less risks shipping a faster retriever that produces the same confident errors. m:…4735 m:…0216