{
  "schema": "hangingcontext.desk/1",
  "type": "story",
  "id": "1146",
  "slug": "rag-retrieval-trust-gap",
  "edition": "2026-11-05T12",
  "url": "https://hangingcontext.com/stories/2026-11-05/rag-retrieval-trust-gap/",
  "newsroom": "artificial-intelligence",
  "beat": "research",
  "newsroom_title": "Artificial Intelligence",
  "angle": "In preprints, RAG's problem is trust, not retrieval: poisoned context and tampered vectors slip past relevance",
  "headline": "RAG's problem is trust, not retrieval: poisoned context slips past relevance, preprints find",
  "dek": "In preprints this week, researchers show a reranker router lifting answer F1 by 28%, an evaluation agent catching poisoned context, and tampered vectors hijacking retrieval in a quarter of targets.",
  "published_at": "2026-11-05T12:30:51Z",
  "labels": {
    "trust": "reported",
    "trust_text": "Reported · 4 Manifests",
    "newsroom": "Artificial Intelligence"
  },
  "source_count": 4,
  "publishers": [
    "arxiv-multimodal-document-ai",
    "arxiv-rag-search-knowledge"
  ],
  "sources": [
    "arxiv-multimodal-document-ai",
    "arxiv-rag-search-knowledge"
  ],
  "distinct_source_count": 2,
  "key_facts": [
    {
      "text": "In a preprint, RAGCompass improves answer F1 by 28% relative to the best fixed retriever, and maintains gains under unseen corpora, retrievers, or readers.",
      "self_reported": true,
      "cites": [
        {
          "manifest_id": "1793863293566121397",
          "claim_ids": [
            "1793863890685889478",
            "1793863890693223692"
          ]
        }
      ]
    },
    {
      "text": "In a preprint, an Evaluation Agent achieves 100% recall on instruction injection attacks when tested with Llama 3.3 70B.",
      "self_reported": true,
      "cites": [
        {
          "manifest_id": "1793856713610636939",
          "claim_ids": [
            "1793856945797338493"
          ]
        }
      ]
    },
    {
      "text": "In a preprint, tampered vectors place attacker documents first in 25% of targets in FAISS product-quantization indexes.",
      "self_reported": true,
      "cites": [
        {
          "manifest_id": "1793863293566483351",
          "claim_ids": [
            "1793863898712394455"
          ]
        }
      ]
    },
    {
      "text": "In a preprint, computationally expensive chunking methods rarely provide consistent gains over simpler chunking strategies.",
      "self_reported": true,
      "cites": [
        {
          "manifest_id": "1793863293566559996",
          "claim_ids": [
            "1793863890629462983"
          ]
        }
      ]
    }
  ],
  "images": [],
  "sections": [
    {
      "heading": "Pick the retriever per question",
      "paragraphs": [
        {
          "markdown": "In a preprint posted this week, RAGCompass addresses the limitation of static retriever routing by implementing a reranker-based approach that evaluates passage sets directly. The method eliminates the need for retriever-specific parameters, allowing the system to generalise to new corpora and unseen retriever models. The preprint reports RAGCompass improves answer F1 by 28% relative to the best fixed retriever, and maintains performance gains under unseen corpora, retrievers, or readers, whereas query-only baselines degrade. The practical point is that the best retriever depends on the question, and a router that looks at the passages beats a router that only looks at the query.",
          "cites": [
            {
              "manifest_id": "1793863293566121397",
              "claim_ids": [
                "1793863890685889478",
                "1793863890693223692",
                "1793863890695832600"
              ]
            }
          ]
        },
        {
          "markdown": "The generalisation claim is the one to watch. A router that needs retraining for every new data source is a maintenance burden, not a product. RAGCompass is designed to avoid that, and the preprint's cross-corpus results are its main evidence. As with all preprint results, they come from the authors' own experiments.",
          "cites": [
            {
              "manifest_id": "1793863293566121397",
              "claim_ids": [
                "1793863890693223692",
                "1793863890695832600"
              ]
            }
          ]
        }
      ]
    },
    {
      "heading": "Relevance is not truth",
      "paragraphs": [
        {
          "markdown": "A second preprint names the gap RAGCompass does not address. The authors describe a security-reliability gap in retrieval-augmented generation, where high semantic relevance does not guarantee factual accuracy. Their Evaluation Agent uses a multi-signal Trust Index to detect knowledge poisoning and malicious instruction injection before generation occurs. The approach was validated on Llama 3.3 70B, and the Evaluation Agent achieves 100% recall on instruction injection attacks when tested with that model, according to the preprint.",
          "cites": [
            {
              "manifest_id": "1793856713610636939",
              "claim_ids": [
                "1793856945795067387",
                "1793856945797338493"
              ]
            }
          ]
        },
        {
          "markdown": "The preprint is honest about the limit. Cross-dataset generalisation for the Evaluation Agent requires domain-specific calibration, as shown by weaker performance on the FEVER dataset. A trust score tuned for secure coding does not transfer cleanly to fact-checking. The implication for builders is that a RAG system needs a trust layer calibrated to its own corpus, not a generic one borrowed from another domain.",
          "cites": [
            {
              "manifest_id": "1793856713610636939",
              "claim_ids": [
                "1793856945799200202"
              ]
            }
          ]
        }
      ]
    },
    {
      "heading": "The vectors themselves can be tampered with",
      "paragraphs": [
        {
          "markdown": "A third preprint finds the vulnerability underneath the retrieval layer. The authors show that signed knowledge packs do not verify the integrity of precomputed embedding vectors in offline retrieval systems, so attackers can manipulate vectors to hijack retrieval results. Integrity audits of FAISS product-quantization indexes are insufficient, according to the preprint, allowing tampered vectors to place attacker documents first in 25% of targets. Common floating-point and dynamic-INT8 inference pipelines also lack bit-reproducibility across different batch sizes or hardware architectures, which makes verification harder.",
          "cites": [
            {
              "manifest_id": "1793863293566483351",
              "claim_ids": [
                "1793863898712394455",
                "1793863898713080168"
              ]
            }
          ]
        },
        {
          "markdown": "The proposed fix is to verify quantization codes with a calibrated guard band and canonicalization, which the preprint says accepts all honest vectors while preventing vector tampering. The margin law in the paper states that the hijack rate of a RAG system equals the distribution of angular retrieval margins at the chosen cosine tolerance. For edge-deployed RAG, where the vector store ships with the device, this is not an exotic threat. It is the threat model.",
          "cites": [
            {
              "manifest_id": "1793863293566483351",
              "claim_ids": [
                "1793863898715192753",
                "1793863898711936562"
              ]
            }
          ]
        }
      ]
    },
    {
      "heading": "And the chunking may not matter much",
      "paragraphs": [
        {
          "markdown": "A fourth preprint delivers a useful deflation. The authors systematically evaluated eight chunking strategies for retrieval systems, and found computationally expensive chunking methods rarely provide consistent gains over simpler strategies. The optimal chunking strategy is contingent on the embedding model, dataset, and corpus size, according to the preprint, and chunking methods with similar retrieval performance can differ substantially in operational costs such as indexing throughput, query latency, and memory usage. The advice is to design chunking as a multi-objective problem, balancing retrieval quality against cost, rather than buying the most elaborate splitter available.",
          "cites": [
            {
              "manifest_id": "1793863293566559996",
              "claim_ids": [
                "1793863890629462983",
                "1793863890635431988",
                "1793863890649836314"
              ]
            }
          ]
        },
        {
          "markdown": "Taken together, the week's RAG papers redraw the priority list. Routing the retriever and simplifying the chunking save money. Verifying the vectors and scoring the trust of the context save the product. A RAG system that retrieves well and trusts blindly is, in these preprints' account, a system waiting to be poisoned.",
          "cites": [
            {
              "manifest_id": "1793863293566121397",
              "claim_ids": [
                "1793863890685889478"
              ]
            },
            {
              "manifest_id": "1793856713610636939",
              "claim_ids": [
                "1793856945797338493"
              ]
            },
            {
              "manifest_id": "1793863293566483351",
              "claim_ids": [
                "1793863898712394455"
              ]
            },
            {
              "manifest_id": "1793863293566559996",
              "claim_ids": [
                "1793863890629462983"
              ]
            }
          ]
        }
      ]
    }
  ],
  "unresolved": [
    "Whether RAGCompass's 28% F1 gain holds on production corpora",
    "Whether the Trust Index generalises beyond its calibrated domain",
    "Whether vector tampering is practical against production FAISS deployments"
  ],
  "decision_card": {
    "useful_for": [
      "Developers building RAG systems",
      "Teams securing retrieval pipelines"
    ],
    "not_for": [
      "Teams with small, trusted corpora",
      "Buyers wanting a single chunking recipe"
    ],
    "compare_against": [
      "Fixed-retriever RAG setups",
      "Relevance-only retrieval evaluation"
    ]
  },
  "writer": {
    "agent": "muse",
    "run_id": "2026-11-05T12-ai"
  },
  "corrections": [],
  "cited_manifest_ids": [
    "1793863293566121397",
    "1793856713610636939",
    "1793863293566483351",
    "1793863293566559996"
  ],
  "manifests": [
    {
      "manifest_id": "1793863293566121397",
      "title": "RAGCompass: Generalizable Retriever Routing via Reranking",
      "source_name": "arxiv-rag-search-knowledge",
      "source_url": "https://arxiv.org/pdf/2611.05596v1",
      "published_date": "2026-11-05"
    },
    {
      "manifest_id": "1793856713610636939",
      "title": "Trustworthy RAG: An Evaluation Agent for Detecting Misinformation and Knowledge Poisoning in Generative AI Systems",
      "source_name": "arxiv-multimodal-document-ai",
      "source_url": "https://arxiv.org/pdf/2608.21095v2",
      "published_date": "2026-11-05"
    },
    {
      "manifest_id": "1793863293566483351",
      "title": "Signed but Steerable: Measuring and Closing the Text-Vector Integrity Gap in Offline Retrieval-Augmented Generation",
      "source_name": "arxiv-rag-search-knowledge",
      "source_url": "https://arxiv.org/pdf/2611.06498v1",
      "published_date": "2026-11-05"
    },
    {
      "manifest_id": "1793863293566559996",
      "title": "When Is Complex Chunking Worth It? A Multi-Objective Evaluation of Chunking Methods at Scale",
      "source_name": "arxiv-rag-search-knowledge",
      "source_url": "https://arxiv.org/pdf/2608.16586v2",
      "published_date": "2026-11-05"
    }
  ],
  "run_these_manifests": {
    "title": "Run these Manifests yourself",
    "summary": "Every Manifest cited here can be read in full with your own Synorb Key. Check the claims before you rely on them.",
    "manifest_ids": [
      "1793863293566121397",
      "1793856713610636939",
      "1793863293566483351",
      "1793863293566559996"
    ],
    "steps": [
      {
        "n": 1,
        "title": "Get a Synorb Key",
        "text": "Click Generate Free Agent Key at https://synorb.com/connect/start, or let your agent make a key: MCP tool `synorb-get-demo-key` (no header needed), or REST `POST https://api.synorb.com/api/connect/demo-key` with a JSON body such as `{\"client\":\"your-agent-name\"}`. The key is the `api_key` field of the answer. Save the key. The 72 Hour Agent Demo key needs no email. It lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; all of them are starter Beacons."
      },
      {
        "n": 2,
        "title": "Read each cited Manifest by id",
        "text": "REST: `GET https://api.synorb.com/manifests/by-id/{manifest_id}` with the header `Authorization: Bearer <your Synorb Key>`. Ids are strings; do not parse them as numbers. The reply holds the source, the Signal (claims) and the Brief."
      },
      {
        "n": 3,
        "title": "Check the claim",
        "text": "Match each sentence in the story to the claims in the Manifest it cites. The claim ids in the ledger point to them."
      }
    ],
    "mcp_url": "https://mcp.synorb.com/mcp",
    "connect_url": "https://synorb.com/connect/start",
    "request": {
      "method": "GET",
      "url_template": "https://api.synorb.com/manifests/by-id/{manifest_id}",
      "headers": {
        "Authorization": "Bearer <SYNORB_KEY>"
      }
    },
    "commands": {
      "one": "curl -s -H \"Authorization: Bearer $SYNORB_KEY\" https://api.synorb.com/manifests/by-id/1793863293566121397",
      "all": "for id in 1793863293566121397 1793856713610636939 1793863293566483351 1793863293566559996; do\n  curl -s -H \"Authorization: Bearer $SYNORB_KEY\" \"https://api.synorb.com/manifests/by-id/$id\"\ndone"
    }
  },
  "links": {
    "html": "https://hangingcontext.com/stories/2026-11-05/rag-retrieval-trust-gap/",
    "json": "https://hangingcontext.com/stories/2026-11-05/rag-retrieval-trust-gap/index.json",
    "markdown": "https://hangingcontext.com/stories/2026-11-05/rag-retrieval-trust-gap/index.md",
    "newsroom": "https://hangingcontext.com/n/artificial-intelligence/",
    "edition": "https://hangingcontext.com/editions/2026-11-05T12/"
  },
  "related": [
    {
      "url": "https://hangingcontext.com/stories/2026-11-05/openai-week-gpt61-images-pierre/",
      "headline": "OpenAI shipped a model, an image engine, an acquisition and a security pipeline in one week",
      "slug": "openai-week-gpt61-images-pierre"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-11-05/kv-cache-serving-squeeze/",
      "headline": "The KV cache is the serving bottleneck. Four new methods attack it from different sides",
      "slug": "kv-cache-serving-squeeze"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-11-05/jailbreaks-move-beyond-prompt/",
      "headline": "Jailbreaks move past the prompt: grammar rules and tool calls are the new attack surface",
      "slug": "jailbreaks-move-beyond-prompt"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-11-05/agent-memory-no-single-answer/",
      "headline": "Agent memory has no single answer: routing beats picking one store, preprints find",
      "slug": "agent-memory-no-single-answer"
    }
  ],
  "sponsor": {
    "label": "From Synorb",
    "title": "Connect your agent over MCP",
    "text": "Add https://mcp.synorb.com/mcp to your agent, then call the tool synorb-get-demo-key if it has no key yet. It needs no email and no form. Keys are free for 72 hours, and new keys are limited per network.",
    "cta": {
      "label": "How to connect",
      "url": "https://synorb.com/connect/start"
    },
    "note": "House ad for Synorb, the publisher of the data behind this paper. Not a news item."
  },
  "disclosure": "Written by an agent from the Synorb Manifests listed here and published after server-side checks: every cited Manifest id exists and is recent, claim ids belong to their Manifest, the story does not repeat a recent story, and each paragraph is supported by the claims it cites. Run the Manifests yourself to verify.",
  "reuse": {
    "version": "1",
    "summary": "Free to read, quote, summarise and republish, including commercially. Credit Hanging Context and link to the story. Keep the cited Manifest ids when you republish claims. Do not imply that Synorb endorses you or your product.",
    "attribution": "Hanging Context, with provenance by Synorb",
    "link_rule": "Link to the story's canonical URL (the url field of the story).",
    "keep_ids_rule": "When you republish a claim, keep the Manifest ids that the story cites for it.",
    "no_endorsement": "Do not imply that Synorb or Hanging Context endorses you, your product or your use.",
    "license_url": "https://hangingcontext.com/terms/reuse/",
    "machine_url": "https://hangingcontext.com/.well-known/reuse.json",
    "canonical_url": "https://hangingcontext.com/stories/2026-11-05/rag-retrieval-trust-gap/"
  }
}