Your linear probe doesn't prove what you think it does

The probe problem

A linear probe lighting up does not mean the model thinks in straight lines. In a preprint posted October 19, researchers show that, asymptotically, any sufficiently rich feature map — including a random one — admits a linear readout. High probe accuracy is therefore consistent with two very different geometries: a true linear world model, or a mere kernel machine that happens to be decodable. m:…0311

Their empirical verdict on pretraining: it produces structure beyond a random kernel, but lands well short of a linear world model. My read on this is blunt — probe accuracy is evidence of decodability, not of understanding. Treat it as a floor, not a finding, and be suspicious of any interpretability claim that stops at 'the probe got 95%.' m:…0311

Sparse autoencoders as a shared coordinate system

One replacement gaining ground: sparse autoencoders used as a common coordinate system. In a preprint posted October 19, SAE-IDF matches symbolic units across heterogeneous systems using sparse autoencoders and rarity-weighted features — lifting Lean 4-to-Metamath matching MRR from 0.21 to 0.66 and Lean 4-to-HOL Light from 0.54 to 0.90. The authors report that "every dimension of the sparse code has a human-readable meaning" — the representation is the explanation. m:…5536

Two caveats the authors report themselves: without IDF weighting, the SAE code mostly trails dense baselines, and performance scales with dictionary size — HOL Light MRR rose from 0.82 to 0.94 as the dictionary grew from 16k to 1M features. The method's claim is conditional, which is precisely what makes it more trustworthy than a probe number. m:…5536

SAE geometry is also proving useful outside interpretability proper. SAGA uses SAE-induced low-rank geometry to constrain representation drift in continual learning, achieving a better stability-plasticity trade-off than ER, DER++, ER-ACE, and SGD on ViT-B/16 benchmarks. My read: when the same geometric tool helps models remember and helps humans read them, that is evidence the geometry is real rather than decorative. m:…4651

Reading the dark space

DarkLens takes a different route: instead of probing activations, it reads the model's 'dark space' — directions weakly composed with the unembedding matrix — which the authors show actively mediates context processing rather than sitting idle. Their Dark Matter Ratio of hidden states measures how much a model is leaning on its context, and the whole thing runs within a single forward pass, with no auxiliary probes or attention-map materialization. For faithfulness detection at serving latency, that last property is the one that matters. m:…1434

Use the model's own remaining layers

LexFrame argues the cleanest lens is the model itself. In a preprint posted October 19, the authors show that lexical reference frames interpret hidden states through a model's own remaining layers, reaching 0.97 coherence on SmolLM3-3B and OLMo3-7B. Traditional logit and Jacobian lenses, by contrast, manage only 0.30 and 0.35 at mid-depth — and, the authors say, misread the residual stream in the middle layers. m:…5372

The method pays for itself: confident LexFrame readings would let an early exit skip 45% of layers in SmolLM3-3B and 42% in OLMo3-7B. Interpretability that also buys inference speed is interpretability that might actually get used. m:…5372

Watch the belief, not just the output

Finally, Belief-Trajectory Energy tracks how a model's predictions revise layer by layer, mapping intermediate states into a shared predictive space to measure belief change — locally corresponding to predictive revision under Fisher-Rao geometry. It is a measure of the path to a prediction, not just the destination. m:…7803

The payoff is concrete: richer BTE representations support human-LLM review detection and fine-grained generator attribution, reaching up to 0.998 macro-AUROC and 95.6% eight-way attribution accuracy. BTE also develops throughout pretraining and is reshaped by targeted training — suggesting, in the authors' analysis, that the measurement reflects what the scoring model has actually learned. Know what your method proves, and probes prove less than you thought. m:…7803