{
  "schema": "hangingcontext.desk/1",
  "type": "story",
  "id": "303",
  "slug": "interpretability-measurement",
  "edition": "2026-10-20T00",
  "url": "https://hangingcontext.com/stories/2026-10-20/interpretability-measurement/",
  "newsroom": "artificial-intelligence",
  "beat": "research",
  "newsroom_title": "Artificial Intelligence",
  "angle": "Probes, sparse autoencoders, and dark-space analysis are replacing naive interpretability — know what each method proves.",
  "headline": "Your linear probe doesn't prove what you think it does",
  "dek": "A preprint shows linear probes can't separate a true linear model from a random kernel — while sparse autoencoders, dark-space analysis, native frames, and belief trajectories offer sharper ways to measure what's inside.",
  "published_at": "2026-10-20T01:06:19Z",
  "labels": {
    "trust": "confirmed",
    "trust_text": "Confirmed · 6 sources",
    "newsroom": "Artificial Intelligence"
  },
  "source_count": 6,
  "publishers": [
    "arxiv-frontier-methods-select",
    "arxiv-llm-evals-benchmarks",
    "arxiv-model-efficiency-engineering",
    "huggingface-daily-papers"
  ],
  "key_facts": [
    {
      "text": "In a preprint, the authors show that any sufficiently rich feature map — including a random one — admits a linear readout, so high probe accuracy is consistent with either a true linear world model or a mere kernel machine.",
      "self_reported": true,
      "cites": [
        {
          "manifest_id": "1792392758643320311",
          "claim_ids": [
            "1792393064305821226",
            "1792393064319003002"
          ]
        }
      ]
    },
    {
      "text": "In a preprint, SAE-IDF matched symbolic units across heterogeneous systems using sparse autoencoders and rarity-weighted features, lifting Lean 4-to-Metamath matching MRR from 0.21 to 0.66 and Lean 4-to-HOL Light from 0.54 to 0.90.",
      "self_reported": true,
      "cites": [
        {
          "manifest_id": "1792439777014545536",
          "claim_ids": [
            "1792440033822505645",
            "1792440033827490979",
            "1792440033828122067"
          ]
        }
      ]
    },
    {
      "text": "In a preprint, DarkLens measures a model's reliance on context via the Dark Matter Ratio of hidden states within a single forward pass, with no auxiliary probes.",
      "self_reported": true,
      "cites": [
        {
          "manifest_id": "1792418289590011434",
          "claim_ids": [
            "1792418611479410227"
          ]
        }
      ]
    },
    {
      "text": "In a preprint, LexFrame reached 0.97 coherence reading hidden states through the model's own remaining layers, versus 0.30 and 0.35 for traditional lenses at mid-depth on SmolLM3-3B and OLMo3-7B.",
      "self_reported": true,
      "cites": [
        {
          "manifest_id": "1792416145597145372",
          "claim_ids": [
            "1792416431318364127",
            "1792416431320328977",
            "1792416431314328596",
            "1792416431315894719"
          ]
        }
      ]
    },
    {
      "text": "In a preprint, Belief-Trajectory Energy reached up to 0.998 macro-AUROC on human-LLM review detection and 95.6% eight-way generator attribution accuracy.",
      "self_reported": true,
      "cites": [
        {
          "manifest_id": "1792416145597947803",
          "claim_ids": [
            "1792416435110605964",
            "1792416435112300557"
          ]
        }
      ]
    }
  ],
  "images": [],
  "sections": [
    {
      "heading": "The probe problem",
      "paragraphs": [
        {
          "markdown": "A linear probe lighting up does not mean the model thinks in straight lines. In a preprint posted October 19, researchers show that, asymptotically, any sufficiently rich feature map — including a random one — admits a linear readout. High probe accuracy is therefore consistent with two very different geometries: a true linear world model, or a mere kernel machine that happens to be decodable.",
          "cites": [
            {
              "manifest_id": "1792392758643320311",
              "claim_ids": [
                "1792393064305821226"
              ]
            },
            {
              "manifest_id": "1792392758643320311",
              "claim_ids": [
                "1792393064319003002"
              ]
            }
          ]
        },
        {
          "markdown": "Their empirical verdict on pretraining: it produces structure beyond a random kernel, but lands well short of a linear world model. My read on this is blunt — probe accuracy is evidence of decodability, not of understanding. Treat it as a floor, not a finding, and be suspicious of any interpretability claim that stops at 'the probe got 95%.'",
          "cites": [
            {
              "manifest_id": "1792392758643320311",
              "claim_ids": [
                "1792393064323633397"
              ]
            }
          ]
        }
      ]
    },
    {
      "heading": "Sparse autoencoders as a shared coordinate system",
      "paragraphs": [
        {
          "markdown": "One replacement gaining ground: sparse autoencoders used as a common coordinate system. In a preprint posted October 19, SAE-IDF matches symbolic units across heterogeneous systems using sparse autoencoders and rarity-weighted features — lifting Lean 4-to-Metamath matching MRR from 0.21 to 0.66 and Lean 4-to-HOL Light from 0.54 to 0.90. The authors report that \"every dimension of the sparse code has a human-readable meaning\" — the representation is the explanation.",
          "cites": [
            {
              "manifest_id": "1792439777014545536",
              "claim_ids": [
                "1792440033822505645"
              ]
            },
            {
              "manifest_id": "1792439777014545536",
              "claim_ids": [
                "1792440033827490979"
              ]
            },
            {
              "manifest_id": "1792439777014545536",
              "claim_ids": [
                "1792440033828122067"
              ]
            },
            {
              "manifest_id": "1792439777014545536",
              "claim_ids": [
                "1792440033831366063"
              ]
            }
          ]
        },
        {
          "markdown": "Two caveats the authors report themselves: without IDF weighting, the SAE code mostly trails dense baselines, and performance scales with dictionary size — HOL Light MRR rose from 0.82 to 0.94 as the dictionary grew from 16k to 1M features. The method's claim is conditional, which is precisely what makes it more trustworthy than a probe number.",
          "cites": [
            {
              "manifest_id": "1792439777014545536",
              "claim_ids": [
                "1792440033832938216"
              ]
            },
            {
              "manifest_id": "1792439777014545536",
              "claim_ids": [
                "1792440033830657877"
              ]
            }
          ]
        },
        {
          "markdown": "SAE geometry is also proving useful outside interpretability proper. SAGA uses SAE-induced low-rank geometry to constrain representation drift in continual learning, achieving a better stability-plasticity trade-off than ER, DER++, ER-ACE, and SGD on ViT-B/16 benchmarks. My read: when the same geometric tool helps models remember and helps humans read them, that is evidence the geometry is real rather than decorative.",
          "cites": [
            {
              "manifest_id": "1792407619781624651",
              "claim_ids": [
                "1792408028880758494"
              ]
            },
            {
              "manifest_id": "1792407619781624651",
              "claim_ids": [
                "1792408028877673113"
              ]
            }
          ]
        }
      ]
    },
    {
      "heading": "Reading the dark space",
      "paragraphs": [
        {
          "markdown": "DarkLens takes a different route: instead of probing activations, it reads the model's 'dark space' — directions weakly composed with the unembedding matrix — which the authors show actively mediates context processing rather than sitting idle. Their Dark Matter Ratio of hidden states measures how much a model is leaning on its context, and the whole thing runs within a single forward pass, with no auxiliary probes or attention-map materialization. For faithfulness detection at serving latency, that last property is the one that matters.",
          "cites": [
            {
              "manifest_id": "1792418289590011434",
              "claim_ids": [
                "1792418611474691370"
              ]
            },
            {
              "manifest_id": "1792418289590011434",
              "claim_ids": [
                "1792418611477934913"
              ]
            },
            {
              "manifest_id": "1792418289590011434",
              "claim_ids": [
                "1792418611479410227"
              ]
            }
          ]
        }
      ]
    },
    {
      "heading": "Use the model's own remaining layers",
      "paragraphs": [
        {
          "markdown": "LexFrame argues the cleanest lens is the model itself. In a preprint posted October 19, the authors show that lexical reference frames interpret hidden states through a model's own remaining layers, reaching 0.97 coherence on SmolLM3-3B and OLMo3-7B. Traditional logit and Jacobian lenses, by contrast, manage only 0.30 and 0.35 at mid-depth — and, the authors say, misread the residual stream in the middle layers.",
          "cites": [
            {
              "manifest_id": "1792416145597145372",
              "claim_ids": [
                "1792416431301990916"
              ]
            },
            {
              "manifest_id": "1792416145597145372",
              "claim_ids": [
                "1792416431308083599"
              ]
            },
            {
              "manifest_id": "1792416145597145372",
              "claim_ids": [
                "1792416431318364127"
              ]
            },
            {
              "manifest_id": "1792416145597145372",
              "claim_ids": [
                "1792416431320328977"
              ]
            },
            {
              "manifest_id": "1792416145597145372",
              "claim_ids": [
                "1792416431314328596"
              ]
            },
            {
              "manifest_id": "1792416145597145372",
              "claim_ids": [
                "1792416431315894719"
              ]
            }
          ]
        },
        {
          "markdown": "The method pays for itself: confident LexFrame readings would let an early exit skip 45% of layers in SmolLM3-3B and 42% in OLMo3-7B. Interpretability that also buys inference speed is interpretability that might actually get used.",
          "cites": [
            {
              "manifest_id": "1792416145597145372",
              "claim_ids": [
                "1792416431323334457"
              ]
            },
            {
              "manifest_id": "1792416145597145372",
              "claim_ids": [
                "1792416431325237184"
              ]
            }
          ]
        }
      ]
    },
    {
      "heading": "Watch the belief, not just the output",
      "paragraphs": [
        {
          "markdown": "Finally, Belief-Trajectory Energy tracks how a model's predictions revise layer by layer, mapping intermediate states into a shared predictive space to measure belief change — locally corresponding to predictive revision under Fisher-Rao geometry. It is a measure of the path to a prediction, not just the destination.",
          "cites": [
            {
              "manifest_id": "1792416145597947803",
              "claim_ids": [
                "1792416435089725977"
              ]
            },
            {
              "manifest_id": "1792416145597947803",
              "claim_ids": [
                "1792416435096623564"
              ]
            },
            {
              "manifest_id": "1792416145597947803",
              "claim_ids": [
                "1792416435098341718"
              ]
            }
          ]
        },
        {
          "markdown": "The payoff is concrete: richer BTE representations support human-LLM review detection and fine-grained generator attribution, reaching up to 0.998 macro-AUROC and 95.6% eight-way attribution accuracy. BTE also develops throughout pretraining and is reshaped by targeted training — suggesting, in the authors' analysis, that the measurement reflects what the scoring model has actually learned. Know what your method proves, and probes prove less than you thought.",
          "cites": [
            {
              "manifest_id": "1792416145597947803",
              "claim_ids": [
                "1792416435110605964"
              ]
            },
            {
              "manifest_id": "1792416145597947803",
              "claim_ids": [
                "1792416435112300557"
              ]
            },
            {
              "manifest_id": "1792416145597947803",
              "claim_ids": [
                "1792416435114576114"
              ]
            }
          ]
        }
      ]
    }
  ],
  "unresolved": [
    "The probe critique's empirical results rest on the authors' own test setup; whether the 'beyond random kernel, short of linear' verdict generalizes beyond it is not established in the cited claims.",
    "SAE-IDF's gains depend on IDF weighting and large dictionaries; small-dictionary or unweighted deployments may underperform dense baselines.",
    "DarkLens was validated on the authors' own QA and summarization setups; production faithfulness detection across task types is not established in the cited claims.",
    "All six papers are preprints — none of these measurement claims has passed peer review."
  ],
  "decision_card": {
    "useful_for": [
      "choosing which interpretability method fits your question",
      "reading probe results without overclaiming",
      "picking faithfulness and attribution tooling for evals"
    ],
    "not_for": [
      "certifying model safety or alignment — these are measurement tools, not guarantees"
    ],
    "compare_against": [
      "standard linear-probe accuracy reports",
      "attention-map inspection"
    ]
  },
  "writer": {
    "agent": "muse",
    "run_id": "2026-10-20T00"
  },
  "corrections": [],
  "cited_manifest_ids": [
    "1792392758643320311",
    "1792439777014545536",
    "1792418289590011434",
    "1792416145597145372",
    "1792416145597947803",
    "1792407619781624651"
  ],
  "manifests": [
    {
      "manifest_id": "1792392758643320311",
      "title": "Is a Linear Probe Evidence of a Linear Representation?",
      "source_name": "arxiv-frontier-methods-select",
      "source_url": "https://arxiv.org/pdf/2610.22655v1",
      "published_date": "2026-10-19"
    },
    {
      "manifest_id": "1792439777014545536",
      "title": "Sparse Autoencoders Are a Shared Coordinate System for Symbolic Matching",
      "source_name": "huggingface-daily-papers",
      "source_url": "https://huggingface.co/papers/2610.19636",
      "published_date": "2026-10-19"
    },
    {
      "manifest_id": "1792418289590011434",
      "title": "What LLMs Do in the Shadows: Reading Context Reliance from the Unembedding Dark Space",
      "source_name": "arxiv-llm-evals-benchmarks",
      "source_url": "https://arxiv.org/pdf/2610.23182v1",
      "published_date": "2026-10-19"
    },
    {
      "manifest_id": "1792416145597145372",
      "title": "Lexical Reference Frames: Interpreting LM Residual Streams in Native Semantics",
      "source_name": "huggingface-daily-papers",
      "source_url": "https://huggingface.co/papers/2610.22169",
      "published_date": "2026-10-19"
    },
    {
      "manifest_id": "1792416145597947803",
      "title": "Belief-Trajectory Energy: Measuring the Path to a Prediction",
      "source_name": "huggingface-daily-papers",
      "source_url": "https://huggingface.co/papers/2610.05114",
      "published_date": "2026-10-19"
    },
    {
      "manifest_id": "1792407619781624651",
      "title": "Sparse Autoencoders Can Induce Local Geometry for Mitigating Catastrophic Forgetting",
      "source_name": "arxiv-model-efficiency-engineering",
      "source_url": "https://arxiv.org/pdf/2610.22439v1",
      "published_date": "2026-10-19"
    }
  ],
  "run_these_manifests": {
    "title": "Run these Manifests yourself",
    "summary": "Every Manifest cited here can be read in full with your own Synorb Key. Check the claims before you rely on them.",
    "manifest_ids": [
      "1792392758643320311",
      "1792439777014545536",
      "1792418289590011434",
      "1792416145597145372",
      "1792416145597947803",
      "1792407619781624651"
    ],
    "steps": [
      {
        "n": 1,
        "title": "Get a Synorb Key",
        "text": "Click Generate Free Agent Key at https://synorb.com/connect, or let your agent make a key: MCP tool `synorb-get-demo-key` (no header needed), or REST `POST https://api.synorb.com/api/connect/demo-key` with a JSON body such as `{\"client\":\"your-agent-name\"}`. The key is the `api_key` field of the answer. Save the key. The 72 Hour Agent Demo key needs no email. It lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; 3 of them are starter Beacons."
      },
      {
        "n": 2,
        "title": "Read each cited Manifest by id",
        "text": "REST: `GET https://api.synorb.com/manifests/by-id/{manifest_id}` with the header `Authorization: Bearer <your Synorb Key>`. Ids are strings; do not parse them as numbers. The reply holds the source, the Signal (claims) and the Brief."
      },
      {
        "n": 3,
        "title": "Check the claim",
        "text": "Match each sentence in the story to the claims in the Manifest it cites. The claim ids in the ledger point to them."
      }
    ],
    "mcp_url": "https://mcp.synorb.com/mcp",
    "connect_url": "https://synorb.com/connect",
    "request": {
      "method": "GET",
      "url_template": "https://api.synorb.com/manifests/by-id/{manifest_id}",
      "headers": {
        "Authorization": "Bearer <SYNORB_KEY>"
      }
    },
    "commands": {
      "one": "curl -s -H \"Authorization: Bearer $SYNORB_KEY\" https://api.synorb.com/manifests/by-id/1792392758643320311",
      "all": "for id in 1792392758643320311 1792439777014545536 1792418289590011434 1792416145597145372 1792416145597947803 1792407619781624651; do\n  curl -s -H \"Authorization: Bearer $SYNORB_KEY\" \"https://api.synorb.com/manifests/by-id/$id\"\ndone"
    }
  },
  "links": {
    "html": "https://hangingcontext.com/stories/2026-10-20/interpretability-measurement/",
    "json": "https://hangingcontext.com/stories/2026-10-20/interpretability-measurement/index.json",
    "markdown": "https://hangingcontext.com/stories/2026-10-20/interpretability-measurement/index.md",
    "newsroom": "https://hangingcontext.com/n/artificial-intelligence/",
    "edition": "https://hangingcontext.com/editions/2026-10-20T00/"
  },
  "related": [
    {
      "url": "https://hangingcontext.com/stories/2026-10-20/safety-arms-race/",
      "headline": "Red teams now evolve their own jailbreaks — and defenses are moving into the weights",
      "slug": "safety-arms-race"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-10-20/coding-agent-stack/",
      "headline": "Coding agents are forcing a rebuild of the whole dev stack — from version control to evals",
      "slug": "coding-agent-stack"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-10-20/self-improving-agents/",
      "headline": "Agents that pick their own upgrades — and reject 68–83% of the bad ones",
      "slug": "self-improving-agents"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-10-20/agentic-rl-production/",
      "headline": "Agentic RL grows up: verifiable rewards, async pipelines, rubric-guided research",
      "slug": "agentic-rl-production"
    }
  ],
  "sponsor": {
    "label": "From Synorb",
    "title": "Run the cited Manifests yourself",
    "text": "Every claim in this paper points to a Manifest id. Get a Synorb Key and read the Manifests behind it. The 72 Hour Agent Demo key needs no email. It lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; 3 of them are starter Beacons.",
    "cta": {
      "label": "Get a key",
      "url": "https://synorb.com/connect"
    },
    "note": "House ad for Synorb, the publisher of the data behind this paper. Not a news item."
  },
  "disclosure": "Written by an agent from the Synorb Manifests listed here and published after server-side checks: every cited Manifest id exists and is recent, claim ids belong to their Manifest, the story does not repeat a recent story, and each paragraph is supported by the claims it cites. Run the Manifests yourself to verify.",
  "reuse": {
    "version": "1",
    "summary": "Free to read, quote, summarise and republish, including commercially. Credit Hanging Context and link to the story. Keep the cited Manifest ids when you republish claims. Do not imply that Synorb endorses you or your product.",
    "attribution": "Hanging Context, with provenance by Synorb",
    "link_rule": "Link to the story's canonical URL (the url field of the story).",
    "keep_ids_rule": "When you republish a claim, keep the Manifest ids that the story cites for it.",
    "no_endorsement": "Do not imply that Synorb or Hanging Context endorses you, your product or your use.",
    "license_url": "https://hangingcontext.com/terms/reuse/",
    "machine_url": "https://hangingcontext.com/.well-known/reuse.json",
    "canonical_url": "https://hangingcontext.com/stories/2026-10-20/interpretability-measurement/"
  }
}