The cheapest fix in RAG is a second look: reranking lifts precision from 81% to 97%
OpenRouter's benchmark shows a cross-encoder reranker rescuing near-miss retrieval, while preprints tackle cost, privacy and when to stay silent.
- Standard vector search with google/gemini-embedding-2 achieved 81 percent precision@1 on a 274-question benchmark, and adding voyageai/rerank-3-lite raised it to 97 percent. Self-reported m:…2572
- In a preprint, SAG2 achieves Recall@10 of 86.62 percent, 97.43 percent and 99.00 percent on MuSiQue, 2WikiMultiHopQA and HotpotQA, with indexing requiring no model calls. Self-reported m:…5119
- In a preprint, a private dense retrieval protocol adds only 0.38 seconds, or 6.5 percent, of latency to a 128-token Qwen3-32B RAG pipeline. Self-reported m:…5120
- In a preprint, multi-source evidence chaining is the dominant bottleneck for LLM agents in retail workflows across providers and model scales. Self-reported m:…5105
The near-miss problem
Standard vector search often fails to retrieve the correct document because of near misses, where irrelevant chunks are ranked highly. Embedding models compress entire chunks, so similar concepts cluster together in vector space and the wrong document looks like the right one. OpenRouter's walkthrough of reranking in RAG puts numbers on the fix: on a 274-question benchmark, standard vector search with google/gemini-embedding-2 achieved 81 percent precision at one. Passing the top 30 vector search candidates through a cross-encoder model to re-score them against the question, using voyageai/rerank-3-lite, increased precision at one to 97 percent. Going from 10 to 30 candidates improved precision from 94.5 to 96.7 percent, while going from 50 to 100 candidates did not improve it further. m:…2572
The most accurate option is not the one most teams should pick. LLM-based reranking with openai/gpt-6-luna achieved 99.6 percent precision at one but took 4.0 seconds per call, making it better suited to batch processing than live answers. Dedicated rerankers are the production choice for live answers. There is also a small detail with a large effect: omitting the input_type parameter in embedding calls can reduce precision at one by 7 to 12 points. And one honest caveat: reranking can improve result ordering without necessarily improving the final answers provided to users, so the retrieval win does not automatically become a product win. m:…2572
Cheaper, private, and more careful
A preprint this week, SAG2, takes a different route to the same end: embedding-free multi-hop retrieval using lexical scoring and a decision model, with a SQLite lexical index replacing expensive LLM-based graph extraction. It achieves Recall at 10 of 86.62 percent on MuSiQue, 97.43 percent on 2WikiMultiHopQA and 99.00 percent on HotpotQA, and reduces indexing and retrieval costs on MuSiQue to $4.33 compared with $14.85 for its predecessor SAG. Because indexing requires no model calls, the system supports real-time indexing for frequently changing corpora. For teams whose documents change daily, that is arguably more valuable than the recall number. m:…5119
Privacy is the other axis. Another preprint proposes practical private dense retrieval at scale, using learned deep hashing as a private filter to generate a short candidate list before cryptographic reranking, avoiding a full-corpus cryptographic search. The protocol adds only 0.38 seconds, or 6.5 percent, of latency to a 128-token Qwen3-32B RAG pipeline, and the authors say the released code satisfies directional metric differential privacy and reduces embedding-inversion and property-inference leakage. That matters for enterprises whose retrieval corpus is the sensitive asset: the embeddings themselves can leak what they index. m:…5120
A third preprint asks when the agent should answer at all. Its framework applies conformal factuality to agentic RAG, where agents must decide when to stop retrieving. Stricter conformal targets are consistently associated with higher observed response-level factual support, but increasing them reduces claim retention and non-empty response coverage, and the trade-off varies significantly across models and datasets. The honest version of a retrieval agent is one that sometimes says nothing, and this work gives teams a way to price that silence. The retail preprint makes the complementary point: in enterprise retail workflows, multi-source evidence chaining is the dominant bottleneck across providers and model scales, and reliable retail agents require explicit cross-source orchestration rather than scaling model parameters alone. m:…5106 m:…5105
What to change on Monday
The combined advice is refreshingly cheap. Add a reranker and tune the candidate count to around 30, set input_type on embedding calls, and measure whether better retrieval actually improves final answers. If your corpus changes fast, look at embedding-free indexing; if it is sensitive, look at private retrieval; if your agents answer confidently from bad evidence, give them a conformal rule for staying silent. None of the preprint results are peer-reviewed, and their benchmarks are the authors' own, so treat the percentages as direction, not gospel. But the 81-to-97 jump comes from a practitioner's benchmark, not a paper, and it is the kind of gain that shows up in production the same week you ship it. m:…2572 m:…5119 m:…5120 m:…5106
What is still unresolved
- Whether the 81-to-97 precision gain holds on real enterprise corpora rather than a 274-question benchmark.
- How SAG2's recall figures compare with embedding-based retrieval on the same hardware budget.
- Whether conformal factuality thresholds tuned on one model transfer to another.
Provenance by Synorb. Every claim links to its source. This story was written by an agent from the Manifests listed below. Read the methodology and limitations, or send a correction.
Free to reuse, including commercially. Credit “Hanging Context, with provenance by Synorb” and link to this story. Keep the cited Manifest ids. Reuse terms v1.