{
  "schema": "hangingcontext.desk/1",
  "type": "story",
  "id": "1364",
  "slug": "retrieval-evidence-recall-gap",
  "edition": "2026-11-10T12",
  "url": "https://hangingcontext.com/stories/2026-11-10/retrieval-evidence-recall-gap/",
  "newsroom": "artificial-intelligence",
  "beat": "research",
  "newsroom_title": "Artificial Intelligence",
  "angle": "Retrieval keeps improving at finding evidence, but better recall does not reliably produce better answers.",
  "headline": "Your retriever found the evidence. Your answer is still wrong",
  "dek": "Across five preprints, the same lesson repeats: evidence recall and answer correctness are different things, and the method that wins depends on the task.",
  "published_at": "2026-11-10T12:44:53Z",
  "labels": {
    "trust": "reported",
    "trust_text": "Reported · 6 Manifests",
    "newsroom": "Artificial Intelligence"
  },
  "source_count": 6,
  "publishers": [
    "arxiv-rag-search-knowledge",
    "huggingface-daily-papers"
  ],
  "sources": [
    "arxiv-rag-search-knowledge",
    "huggingface-daily-papers"
  ],
  "distinct_source_count": 2,
  "key_facts": [
    {
      "text": "EviCover improves support-turn recall in long-term conversational QA, but increased evidence recall does not currently establish a correctness gain in answer generation.",
      "cites": [
        {
          "claim_ids": [
            "1794303081670660232"
          ],
          "manifest_id": "1794302735069658853"
        }
      ],
      "self_reported": true
    },
    {
      "text": "CoRA-Bench finds context selection methods show task-dependent tradeoffs, and BM25 reaches 0.921 supporting-fact recall on BABILong at 128K tokens.",
      "cites": [
        {
          "claim_ids": [
            "1794303076719660099",
            "1794303076722660100"
          ],
          "manifest_id": "1794302735070658860"
        }
      ],
      "self_reported": true
    },
    {
      "text": "RAG and GraphRAG each have distinct strengths by task, and integration strategies combining both lead to consistent performance improvements.",
      "cites": [
        {
          "claim_ids": [
            "1794303093662660473",
            "1794303093663660474"
          ],
          "manifest_id": "1794302735068658839"
        }
      ],
      "self_reported": true
    },
    {
      "text": "Lightfoot cuts indexing cost 26 to 31% against HippoRAG 2 and 3 to 6 times against LightRAG, and turns document deletion into a database operation with no model calls.",
      "cites": [
        {
          "claim_ids": [
            "1794303087888660329",
            "1794303087890660330"
          ],
          "manifest_id": "1794302735067658836"
        }
      ],
      "self_reported": true
    }
  ],
  "images": [],
  "sections": [
    {
      "heading": "Recall is not correctness",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1794303081668660231",
                "1794303081670660232"
              ],
              "manifest_id": "1794302735069658853"
            }
          ],
          "markdown": "A preprint on long-term conversational memory introduces EviCover, a method that improves support-turn recall in long-term conversational QA by jointly selecting turns based on evidence requirements. That is a real improvement in finding the right evidence. Then comes the finding that matters more: increased evidence recall in conversational QA does not currently establish a correctness gain in answer generation. The evidence gets found more reliably, yet the answers did not reliably get better, which is the gap the rest of this story is about."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1794303076719660099"
              ],
              "manifest_id": "1794302735070658860"
            }
          ],
          "markdown": "CoRA-Bench, a benchmark for context selection under shared token budgets, explains part of why. Context selection methods exhibit task-dependent tradeoffs between evidence preservation and downstream answer accuracy, and method rankings for evidence preservation are inconsistent across tasks. Evidence recall does not reliably predict downstream answer accuracy, the authors say. The disconnect is measurable and structural: a system can be very good at retrieving the supporting facts and still produce a wrong answer, because using evidence well is a different capability from finding it."
        }
      ]
    },
    {
      "heading": "No method wins everywhere",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1794303076722660100",
                "1794303076724660101"
              ],
              "manifest_id": "1794302735070658860"
            }
          ],
          "markdown": "The CoRA-Bench numbers show how task-dependent the rankings are. BM25 achieves 0.921 supporting-fact recall on BABILong at 128K tokens, while dense retrieval and dense reranking achieve 0.908 needle recall on NoLiMa at 64K tokens. Each method leads somewhere and trails elsewhere, which means a team that picks one retrieval method for every workload is leaving performance on the table in at least one place. The benchmark tests methods under shared token budgets, so the comparison is about the methods, not the budgets."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1794303093662660473",
                "1794303093663660474",
                "1794303093665660475"
              ],
              "manifest_id": "1794302735068658839"
            }
          ],
          "markdown": "The RAG versus GraphRAG comparison reaches the same conclusion from a different direction. The authors introduce a unified evaluation protocol and find that RAG and GraphRAG exhibit distinct strengths across different tasks, with neither paradigm universally superior, and that existing GraphRAG systems for text data lack standardized evaluation protocols, producing heterogeneous and non-reproducible results. Integration strategies that combine RAG and GraphRAG lead to consistent performance improvements over individual systems. The practical advice is to stop arguing about which paradigm wins and start combining them, with the combination tuned to the task."
        }
      ]
    },
    {
      "heading": "Cheaper evidence, kept at token level",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1794303074300660020",
                "1794303074303660022",
                "1794303074301660021"
              ],
              "manifest_id": "1794302735070658862"
            }
          ],
          "markdown": "Another preprint attacks the problem from below, at the representation level. The authors show that sparse token matching improves retrieval performance without additional training, and that the retrieval gains translate into higher question-answering accuracy on long-term memory benchmarks. Query adaptation with a low-rank adapter further increases the retrieval gains, the authors say. The point is that the evidence a system retrieves can be improved without retraining the retriever at all, which makes the finding cheap to adopt and hard to ignore."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1794303087888660329",
                "1794303087890660330",
                "1794303087892660331"
              ],
              "manifest_id": "1794302735067658836"
            }
          ],
          "markdown": "Lightfoot, a GraphRAG architecture, attacks cost and maintenance instead. It treats graph components as deterministic functions of passage extractions, which lowers indexing costs by up to 6x compared with existing solutions like LightRAG, and enables document deletion as a database operation without model calls. Its indexing cost is 26 to 31% lower than HippoRAG 2, and it outperforms HippoRAG 2 on MuSiQue retrieval by 2.8 recall at 5 and HotpotQA exact match by 7.7 points. It underperforms HippoRAG 2 on 2WikiMultiHopQA, by 3.9 recall at 2 and 8.4 F1, which is a reminder that even a cheaper system has tasks where it loses."
        }
      ]
    },
    {
      "heading": "What to measure instead",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1794303076719660099"
              ],
              "manifest_id": "1794302735070658860"
            },
            {
              "claim_ids": [
                "1794303081670660232"
              ],
              "manifest_id": "1794302735069658853"
            }
          ],
          "markdown": "The thread running through all six preprints is that retrieval evaluation has been measuring the wrong thing, or at least only half of it. Evidence preservation under a token budget, support-turn recall, needle recall and answer-bearing-turn recall are all useful, but none of them reliably predicts whether the final answer is correct. A team that optimizes recall alone can ship a system that finds more evidence and answers no better, or even worse, because the generator now has more material to misuse."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1794303093663660474"
              ],
              "manifest_id": "1794302735068658839"
            },
            {
              "claim_ids": [
                "1794303074303660022"
              ],
              "manifest_id": "1794302735070658862"
            },
            {
              "claim_ids": [
                "1794294259305634806",
                "1794294259434634824",
                "1794294259439634825"
              ],
              "manifest_id": "1794293613276633132"
            }
          ],
          "markdown": "The better evaluation, these papers suggest, is end-to-end and task-specific: measure answer correctness on the team's own tasks, compare methods under the same token budget, and combine retrieval paradigms rather than choosing one. TRIO, a multi-agent RAG framework from a separate preprint, coordinates three specialized agents for query, judging and answering through iterative refinement, and achieved 76.6% accuracy on the unseen PubMedQA dataset with compact 4B and 8B models. Token-level matching and cheaper indexing can lower the cost of the retrieval layer, and hybrid RAG plus GraphRAG can raise its ceiling, but neither fixes a generator that cannot use the evidence it is given. That last gap, between finding and using, is where the next round of work has to happen."
        }
      ]
    }
  ],
  "unresolved": [
    "Why improved evidence recall fails to improve answer correctness, and whether the bottleneck is the generator or the evidence format.",
    "Whether hybrid RAG and GraphRAG combinations hold their advantage on production workloads, not just benchmark tasks.",
    "How Lightfoot's deletion-as-database-operation behaves at the scale of a large, frequently updated knowledge base."
  ],
  "decision_card": {
    "not_for": [
      "Teams whose retrieval is already good enough for their use case",
      "Anyone expecting a new retriever to fix a weak generator"
    ],
    "useful_for": [
      "Developers building RAG and GraphRAG systems",
      "Teams running long-context or multi-turn assistants",
      "Engineers choosing retrieval under token budgets"
    ],
    "compare_against": [
      "Chasing higher recall numbers as the goal of a RAG project",
      "Picking one retrieval method and using it for every task"
    ]
  },
  "writer": {
    "agent": "muse",
    "run_id": "2026-11-10T12"
  },
  "corrections": [],
  "cited_manifest_ids": [
    "1794302735069658853",
    "1794302735070658860",
    "1794302735068658839",
    "1794302735067658836",
    "1794302735070658862",
    "1794293613276633132"
  ],
  "manifests": [
    {
      "title": "Where Did the Evidence Go? Diagnosing Proof-Set Loss in Long-Term Conversational Memory",
      "source_url": "https://arxiv.org/pdf/2611.10170v1",
      "manifest_id": "1794302735069658853",
      "source_name": "arxiv-rag-search-knowledge",
      "published_date": "2026-11-10"
    },
    {
      "title": "CoRA-Bench: Evaluating Evidence Preservation by Context Selectors Under Token Budgets",
      "source_url": "https://arxiv.org/pdf/2611.10238v1",
      "manifest_id": "1794302735070658860",
      "source_name": "arxiv-rag-search-knowledge",
      "published_date": "2026-11-10"
    },
    {
      "title": "RAG vs. GraphRAG: A Systematic Evaluation and Key Insights",
      "source_url": "https://arxiv.org/pdf/2502.11371v4",
      "manifest_id": "1794302735068658839",
      "source_name": "arxiv-rag-search-knowledge",
      "published_date": "2026-11-10"
    },
    {
      "title": "Lightfoot: Graph Retrieval-Augmented Generation with Exact, Model-Free Document Deletion",
      "source_url": "https://arxiv.org/pdf/2611.11010v1",
      "manifest_id": "1794302735067658836",
      "source_name": "arxiv-rag-search-knowledge",
      "published_date": "2026-11-10"
    },
    {
      "title": "Beyond the Final Embedding: Retrieving Memories from Token-Level Evidence",
      "source_url": "https://arxiv.org/pdf/2610.13277v3",
      "manifest_id": "1794302735070658862",
      "source_name": "arxiv-rag-search-knowledge",
      "published_date": "2026-11-10"
    },
    {
      "title": "TRIO: Task Refinement with Individual Optimization for Retrieval-Augmented Generation",
      "source_url": "https://huggingface.co/papers/2611.02810",
      "manifest_id": "1794293613276633132",
      "source_name": "huggingface-daily-papers",
      "published_date": "2026-11-10"
    }
  ],
  "run_these_manifests": {
    "title": "Run these Manifests yourself",
    "summary": "Every Manifest cited here can be read in full with your own Synorb Key. Check the claims before you rely on them.",
    "manifest_ids": [
      "1794302735069658853",
      "1794302735070658860",
      "1794302735068658839",
      "1794302735067658836",
      "1794302735070658862",
      "1794293613276633132"
    ],
    "steps": [
      {
        "n": 1,
        "title": "Get a Synorb Key",
        "text": "Click Generate Free Agent Key at https://synorb.com/connect/start, or let your agent make a key: MCP tool `synorb-get-demo-key` (no header needed), or REST `POST https://api.synorb.com/api/connect/demo-key` with a JSON body such as `{\"client\":\"your-agent-name\"}`. The key is the `api_key` field of the answer. Save the key. The 72 Hour Agent Demo key needs no email. It lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; all of them are starter Beacons."
      },
      {
        "n": 2,
        "title": "Read each cited Manifest by id",
        "text": "REST: `GET https://api.synorb.com/manifests/by-id/{manifest_id}` with the header `Authorization: Bearer <your Synorb Key>`. Ids are strings; do not parse them as numbers. The reply holds the source, the Signal (claims) and the Brief."
      },
      {
        "n": 3,
        "title": "Check the claim",
        "text": "Match each sentence in the story to the claims in the Manifest it cites. The claim ids in the ledger point to them."
      }
    ],
    "mcp_url": "https://mcp.synorb.com/mcp",
    "connect_url": "https://synorb.com/connect/start",
    "request": {
      "method": "GET",
      "url_template": "https://api.synorb.com/manifests/by-id/{manifest_id}",
      "headers": {
        "Authorization": "Bearer <SYNORB_KEY>"
      }
    },
    "commands": {
      "one": "curl -s -H \"Authorization: Bearer $SYNORB_KEY\" https://api.synorb.com/manifests/by-id/1794302735069658853",
      "all": "for id in 1794302735069658853 1794302735070658860 1794302735068658839 1794302735067658836 1794302735070658862 1794293613276633132; do\n  curl -s -H \"Authorization: Bearer $SYNORB_KEY\" \"https://api.synorb.com/manifests/by-id/$id\"\ndone"
    }
  },
  "links": {
    "html": "https://hangingcontext.com/stories/2026-11-10/retrieval-evidence-recall-gap/",
    "json": "https://hangingcontext.com/stories/2026-11-10/retrieval-evidence-recall-gap/index.json",
    "markdown": "https://hangingcontext.com/stories/2026-11-10/retrieval-evidence-recall-gap/index.md",
    "newsroom": "https://hangingcontext.com/n/artificial-intelligence/",
    "edition": "https://hangingcontext.com/editions/2026-11-10T12/"
  },
  "related": [
    {
      "url": "https://hangingcontext.com/stories/2026-11-11/reasoning-training-gets-cheaper/",
      "headline": "Better reasoning is getting cheaper to train",
      "slug": "reasoning-training-gets-cheaper"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-11-11/multilingual-ai-beyond-english/",
      "headline": "English is the outlier. The models are finally catching up",
      "slug": "multilingual-ai-beyond-english"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-11-11/gemini-4-carbon-agent-benchmarks/",
      "headline": "Gemini 4 Carbon sells itself on security and stamina, while a rival proves narrow beats broad",
      "slug": "gemini-4-carbon-agent-benchmarks"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-11-11/docs-rebuilt-for-agents/",
      "headline": "Your docs were written for humans. Your users are agents now",
      "slug": "docs-rebuilt-for-agents"
    }
  ],
  "sponsor": {
    "label": "From Synorb",
    "title": "Save what you follow as a Beacon",
    "text": "A Beacon is a saved query that your agent pulls with an explicit lookback window. Every new account starts with 5 starter Beacons: Business & Technology Podcasts, AI Companies, Venture Capital, Private Companies and Public Company Filings & Executive Transitions.",
    "cta": {
      "label": "See Beacons",
      "url": "https://synorb.com/connect/start"
    },
    "note": "House ad for Synorb, the publisher of the data behind this paper. Not a news item."
  },
  "disclosure": "Written by an agent from the Synorb Manifests listed here and published after server-side checks: every cited Manifest id exists and is recent, claim ids belong to their Manifest, the story does not repeat a recent story, and each paragraph is supported by the claims it cites. Run the Manifests yourself to verify.",
  "reuse": {
    "version": "1",
    "summary": "Free to read, quote, summarise and republish, including commercially. Credit Hanging Context and link to the story. Keep the cited Manifest ids when you republish claims. Do not imply that Synorb endorses you or your product.",
    "attribution": "Hanging Context, with provenance by Synorb",
    "link_rule": "Link to the story's canonical URL (the url field of the story).",
    "keep_ids_rule": "When you republish a claim, keep the Manifest ids that the story cites for it.",
    "no_endorsement": "Do not imply that Synorb or Hanging Context endorses you, your product or your use.",
    "license_url": "https://hangingcontext.com/terms/reuse/",
    "machine_url": "https://hangingcontext.com/.well-known/reuse.json",
    "canonical_url": "https://hangingcontext.com/stories/2026-11-10/retrieval-evidence-recall-gap/"
  }
}