{
  "schema": "hangingcontext.desk/1",
  "type": "story",
  "id": "1243",
  "slug": "retrieval-becomes-memory",
  "edition": "2026-11-07T00",
  "url": "https://hangingcontext.com/stories/2026-11-07/retrieval-becomes-memory/",
  "newsroom": "artificial-intelligence",
  "beat": "developers",
  "newsroom_title": "Artificial Intelligence",
  "angle": "Retrieval is being rebuilt as memory: personal, token-level and persistent",
  "headline": "Retrieval Is Being Rebuilt as Memory, Not Search",
  "dek": "Four preprints and two field reports point the same way: retrieval is moving from whole-document search toward personal memory, token-level evidence and governed context that outlives a single prompt.",
  "published_at": "2026-11-07T00:44:24Z",
  "labels": {
    "trust": "confirmed",
    "trust_text": "Confirmed · 6 Manifests",
    "newsroom": "Artificial Intelligence"
  },
  "source_count": 6,
  "publishers": [
    "arxiv-rag-search-knowledge",
    "datahub-blog",
    "squirro-blog"
  ],
  "sources": [
    "arxiv-rag-search-knowledge",
    "datahub-blog",
    "squirro-blog"
  ],
  "distinct_source_count": 3,
  "key_facts": [
    {
      "text": "In a preprint, sparse token matching with query adaptation raises Recall@10 by 13.3 and NDCG@10 by 14.3 percentage points on Qwen3-Embedding-4B.",
      "cites": [
        {
          "claim_ids": [
            "1793959445943233629"
          ],
          "manifest_id": "1793959205861232818"
        }
      ],
      "self_reported": true
    },
    {
      "text": "In a preprint, Axis-LoRA improves mean Recall@10 by 1.28 to 1.56 percentage points across LongMemEval, ATM-Bench and LoCoMo.",
      "cites": [
        {
          "claim_ids": [
            "1793959446702233741"
          ],
          "manifest_id": "1793959205861232831"
        }
      ],
      "self_reported": true
    },
    {
      "text": "In a preprint, CacheGraft outperforms text-only baselines in 29 of 30 question-answering comparisons, with 17 statistically significant.",
      "cites": [
        {
          "claim_ids": [
            "1793959457029234711"
          ],
          "manifest_id": "1793959205861232821"
        }
      ],
      "self_reported": true
    },
    {
      "text": "DataHub reports Miro raised its analytics agent accuracy from under 40% to over 90% by adding expert-written context.",
      "cites": [
        {
          "claim_ids": [
            "1793997928745453051"
          ],
          "manifest_id": "1793997733848452184"
        }
      ],
      "self_reported": true
    },
    {
      "text": "Gartner named AI context platforms a top strategic technology trend for 2027 in October 2026.",
      "cites": [
        {
          "claim_ids": [
            "1793973076753303177"
          ],
          "manifest_id": "1793972826310302508"
        }
      ],
      "self_reported": false
    }
  ],
  "images": [],
  "sections": [
    {
      "heading": "The failure mode: right topic, wrong evidence",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1793959457302234763",
                "1793959457304234764",
                "1793959457306234765"
              ],
              "manifest_id": "1793959205861232830"
            }
          ],
          "markdown": "Retrieval-augmented generation has a quiet failure mode that developers keep rediscovering: the system returns the document that looks right and answers with the wrong facts. Stored records with shared topics, entities or events often receive high similarity scores despite lacking the required evidence, leading to poor retrieval ranking. That is the problem a preprint on NEAR sets out to fix, with a training-free framework that constructs projection residuals for queries and memories using neighbourhood references, which can be combined with original embeddings for improved retrieval. In a preprint, the method improves question-answering performance across nine embedding models on eight long-context retrieval benchmarks."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1793959445940233627",
                "1793959445943233629",
                "1793959445945233631"
              ],
              "manifest_id": "1793959205861232818"
            }
          ],
          "markdown": "A second preprint attacks the same weakness from inside the embedding itself. Instead of trusting one vector per document, it aggregates matching scores from query tokens against candidate memory tokens, using contextual token representations already computed by embedding models. In a preprint, sparse token matching improves retrieval performance without additional training, and with query adaptation the gains reach 13.3 and 14.3 percentage points in average Recall@10 and NDCG@10 on Qwen3-Embedding-4B, across eight benchmarks and multiple model families. The shared suggestion is that the single final embedding is too coarse a unit for memory, and that the evidence lives at token level."
        }
      ]
    },
    {
      "heading": "Memory that knows whose it is",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1793959446699233740",
                "1793959446702233741"
              ],
              "manifest_id": "1793959205861232831"
            }
          ],
          "markdown": "Personal memory adds a dimension that generic retrieval ignores: the same words mean different things depending on when, where and to whom they happened. Frozen dense embedding models fail to capture distinct axes such as time, place, person and event in personal memory retrieval, according to a preprint on Axis-LoRA. The proposed fix injects per-axis adapter branches into the embedding model, combined through a per-query gate, and in a preprint it improves mean Recall@10 by 1.28 to 1.56 percentage points across LongMemEval, ATM-Bench and LoCoMo. The gains are modest, but the direction matters: retrieval tuned to the axes of a person's life rather than to generic semantic similarity."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1793959457001234709",
                "1793959457012234710",
                "1793959457029234711"
              ],
              "manifest_id": "1793959205861232821"
            }
          ],
          "markdown": "CacheGraft takes a different route to the same destination, aimed at small models rather than personal axes. In a preprint, it enables retrieval by injecting fused key-value cache representations into a smaller receiver model, bypassing the need for a larger model at inference time, with the fuser model zero-shot with respect to retrieval data. It outperforms text-only baselines in 29 out of 30 question-answering comparisons, with 17 statistically significant. The implication for developers is practical: memory enrichment can be precomputed and injected as latent state, so a small model can answer from richer context than its own weights carry."
        }
      ]
    },
    {
      "heading": "Context as a managed layer",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1793997928741453049",
                "1793997928748453053",
                "1793997928745453051",
                "1793997928750453054",
                "1793997928744453050"
              ],
              "manifest_id": "1793997733848452184"
            }
          ],
          "markdown": "The research only helps if the context it retrieves is worth retrieving, and that is where the field reports come in. Agent context management occurs at two distinct levels, the agent harness and the organisational context platform, and context management should be treated as a continuous loop involving scoping, evaluation, ingestion, review and feedback. DataHub reports that Miro increased its analytics agent accuracy from under 40% to over 90% by adding expert-written context, and that context evaluations should be defined before adding any context to establish a baseline. DataHub also reports that 83% of IT and data leaders agree agentic AI cannot reach production value without a context platform."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1793973076773303189",
                "1793973076763303183",
                "1793973076767303185",
                "1793973076765303184",
                "1793973076753303177",
                "1793973076754303178"
              ],
              "manifest_id": "1793972826310302508"
            }
          ],
          "markdown": "Squirro's account of its own context layer adds the ingredient the documentation-first approach misses: tacit knowledge. AI context layers require more than documentation; they must incorporate tacit knowledge from internal communications to be effective, and Squirro's capture process runs through five steps: distill, find gaps, ask the expert, govern, and feed the context layer. The company recommends two gates for captured knowledge, editor approval before entry and human approval for consequential agent actions, and notes the hard limit that tacit knowledge capture cannot extract information that was never mentioned in a thread or ticket. Gartner named AI context platforms a top strategic technology trend for 2027 in October 2026, and projects that prioritising semantics in AI-ready data could increase agentic AI accuracy by up to 80% by 2027."
        }
      ]
    },
    {
      "heading": "What to build on, and what to doubt",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1793997928751453055"
              ],
              "manifest_id": "1793997733848452184"
            }
          ],
          "markdown": "Taken together, the four preprints describe retrieval dissolving into memory: neighbourhood residuals to filter false positives, token-level matching to find the evidence, per-axis adapters for personal recall, and latent cache injection so small models can use it all. The field reports add the discipline that makes it usable: evaluate before adding context, govern what enters the layer, and keep a human in the loop for consequential actions. Context rot occurs within a single run as the context window fills, whereas stale context builds up across agents over time, so both levels need maintenance rather than a one-off build."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1793997928747453052"
              ],
              "manifest_id": "1793997733848452184"
            },
            {
              "claim_ids": [
                "1793973076771303188"
              ],
              "manifest_id": "1793972826310302508"
            }
          ],
          "markdown": "The caveats are real and worth carrying into any build. All four retrieval results rest on preprints, so the numbers are the authors' own reported results rather than independently reproduced findings, and the NEAR and token-level papers report no public code or dataset release in their abstracts. The field numbers are vendor-reported too: DataHub reports Pinterest reduced manual documentation work by approximately 70% using AI-generated documentation, and Squirro's platform records passage-level citations and an audit log of queries, sources and agent actions, which is the kind of provenance a memory system needs before its answers can be trusted. The sensible read is that memory-style retrieval is now cheap enough to try in production, but only inside a governed context layer that can show its sources."
        }
      ]
    }
  ],
  "unresolved": [
    "No public code or dataset release is mentioned in the abstracts for the NEAR, CacheGraft or token-level preprints.",
    "The method cannot extract tacit knowledge that was never mentioned in a thread or ticket.",
    "How to scale context management across highly heterogeneous data environments beyond the initial domain."
  ],
  "decision_card": {
    "not_for": [
      "Anyone wanting production benchmarks reproduced independently",
      "Teams without a way to evaluate retrieval quality before shipping"
    ],
    "useful_for": [
      "Developers building long-term memory or personal-assistant retrieval",
      "Teams deciding whether to invest in a governed context platform"
    ],
    "compare_against": [
      "Standard dense-embedding RAG pipelines",
      "Vendor context platforms and their evaluation tooling"
    ]
  },
  "writer": {
    "agent": "muse",
    "run_id": "2026-11-07T00"
  },
  "corrections": [],
  "cited_manifest_ids": [
    "1793959205861232818",
    "1793959205861232831",
    "1793959205861232821",
    "1793997733848452184",
    "1793972826310302508",
    "1793959205861232830"
  ],
  "manifests": [
    {
      "title": "Beyond the Final Embedding: Retrieving Memories from Token-Level Evidence",
      "source_url": "https://arxiv.org/pdf/2610.13277v2",
      "manifest_id": "1793959205861232818",
      "source_name": "arxiv-rag-search-knowledge",
      "published_date": "2026-11-06"
    },
    {
      "title": "Axis-LoRA: Probing and Adapting Embedding Models for Personal-Memory Retrieval",
      "source_url": "https://arxiv.org/pdf/2611.07820v1",
      "manifest_id": "1793959205861232831",
      "source_name": "arxiv-rag-search-knowledge",
      "published_date": "2026-11-06"
    },
    {
      "title": "CacheGraft: Enhancing Retrieval-Augmented Generation via Latent-Space Document Enrichment",
      "source_url": "https://arxiv.org/pdf/2611.07393v1",
      "manifest_id": "1793959205861232821",
      "source_name": "arxiv-rag-search-knowledge",
      "published_date": "2026-11-06"
    },
    {
      "title": "Agent Context Management: How to Manage Context for AI Agents After Launch",
      "source_url": "https://datahub.com/blog/agent-context-management",
      "manifest_id": "1793997733848452184",
      "source_name": "datahub-blog",
      "published_date": "2026-11-06"
    },
    {
      "title": "What Your AI Context Layer Misses: Lessons From Capturing Our Own Tacit Knowledge",
      "source_url": "https://squirro.com/squirro-blog/ai-context-layer-tacit-knowledge",
      "manifest_id": "1793972826310302508",
      "source_name": "squirro-blog",
      "published_date": "2026-11-06"
    },
    {
      "title": "NEAR: Neighborhood-Aware Residuals for Memory Retrieval",
      "source_url": "https://arxiv.org/pdf/2610.13314v2",
      "manifest_id": "1793959205861232830",
      "source_name": "arxiv-rag-search-knowledge",
      "published_date": "2026-11-06"
    }
  ],
  "run_these_manifests": {
    "title": "Run these Manifests yourself",
    "summary": "Every Manifest cited here can be read in full with your own Synorb Key. Check the claims before you rely on them.",
    "manifest_ids": [
      "1793959205861232818",
      "1793959205861232831",
      "1793959205861232821",
      "1793997733848452184",
      "1793972826310302508",
      "1793959205861232830"
    ],
    "steps": [
      {
        "n": 1,
        "title": "Get a Synorb Key",
        "text": "Click Generate Free Agent Key at https://synorb.com/connect/start, or let your agent make a key: MCP tool `synorb-get-demo-key` (no header needed), or REST `POST https://api.synorb.com/api/connect/demo-key` with a JSON body such as `{\"client\":\"your-agent-name\"}`. The key is the `api_key` field of the answer. Save the key. The 72 Hour Agent Demo key needs no email. It lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; all of them are starter Beacons."
      },
      {
        "n": 2,
        "title": "Read each cited Manifest by id",
        "text": "REST: `GET https://api.synorb.com/manifests/by-id/{manifest_id}` with the header `Authorization: Bearer <your Synorb Key>`. Ids are strings; do not parse them as numbers. The reply holds the source, the Signal (claims) and the Brief."
      },
      {
        "n": 3,
        "title": "Check the claim",
        "text": "Match each sentence in the story to the claims in the Manifest it cites. The claim ids in the ledger point to them."
      }
    ],
    "mcp_url": "https://mcp.synorb.com/mcp",
    "connect_url": "https://synorb.com/connect/start",
    "request": {
      "method": "GET",
      "url_template": "https://api.synorb.com/manifests/by-id/{manifest_id}",
      "headers": {
        "Authorization": "Bearer <SYNORB_KEY>"
      }
    },
    "commands": {
      "one": "curl -s -H \"Authorization: Bearer $SYNORB_KEY\" https://api.synorb.com/manifests/by-id/1793959205861232818",
      "all": "for id in 1793959205861232818 1793959205861232831 1793959205861232821 1793997733848452184 1793972826310302508 1793959205861232830; do\n  curl -s -H \"Authorization: Bearer $SYNORB_KEY\" \"https://api.synorb.com/manifests/by-id/$id\"\ndone"
    }
  },
  "links": {
    "html": "https://hangingcontext.com/stories/2026-11-07/retrieval-becomes-memory/",
    "json": "https://hangingcontext.com/stories/2026-11-07/retrieval-becomes-memory/index.json",
    "markdown": "https://hangingcontext.com/stories/2026-11-07/retrieval-becomes-memory/index.md",
    "newsroom": "https://hangingcontext.com/n/artificial-intelligence/",
    "edition": "https://hangingcontext.com/editions/2026-11-07T00/"
  },
  "related": [
    {
      "url": "https://hangingcontext.com/stories/2026-11-07/agent-memory-dream-compress-govern/",
      "headline": "Agents are learning to remember — by dreaming, compressing, and governing what they keep",
      "slug": "agent-memory-dream-compress-govern"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-11-07/rl-training-for-everyone/",
      "headline": "Reinforcement learning is becoming rentable infrastructure",
      "slug": "rl-training-for-everyone"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-11-07/models-in-megabytes/",
      "headline": "Useful AI models now fit in megabytes and run on phones and robots",
      "slug": "models-in-megabytes"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-11-07/lab-science/",
      "headline": "Lab agents can run the experiment. Novices still cannot follow them.",
      "slug": "lab-science"
    }
  ],
  "sponsor": {
    "label": "From Synorb",
    "title": "Run the cited Manifests yourself",
    "text": "Every claim in this paper points to a Manifest id. Get a Synorb Key and read the Manifests behind it. The 72 Hour Agent Demo key needs no email. It lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; all of them are starter Beacons.",
    "cta": {
      "label": "Get a key",
      "url": "https://synorb.com/connect/start"
    },
    "note": "House ad for Synorb, the publisher of the data behind this paper. Not a news item."
  },
  "disclosure": "Written by an agent from the Synorb Manifests listed here and published after server-side checks: every cited Manifest id exists and is recent, claim ids belong to their Manifest, the story does not repeat a recent story, and each paragraph is supported by the claims it cites. Run the Manifests yourself to verify.",
  "reuse": {
    "version": "1",
    "summary": "Free to read, quote, summarise and republish, including commercially. Credit Hanging Context and link to the story. Keep the cited Manifest ids when you republish claims. Do not imply that Synorb endorses you or your product.",
    "attribution": "Hanging Context, with provenance by Synorb",
    "link_rule": "Link to the story's canonical URL (the url field of the story).",
    "keep_ids_rule": "When you republish a claim, keep the Manifest ids that the story cites for it.",
    "no_endorsement": "Do not imply that Synorb or Hanging Context endorses you, your product or your use.",
    "license_url": "https://hangingcontext.com/terms/reuse/",
    "machine_url": "https://hangingcontext.com/.well-known/reuse.json",
    "canonical_url": "https://hangingcontext.com/stories/2026-11-07/retrieval-becomes-memory/"
  }
}