The interpretability toolkit goes to work

Reconstruction was the wrong scoreboard

In a preprint, the authors show that sparse autoencoder training has been optimized against the wrong target. Reconstruction performance approaches the full-data reference at substantially smaller data budgets than feature reproducibility does. For Gemma-2-2B BatchTopK, the fraction of strictly matched features climbs from 2% at 5M tokens to 29% at 50M tokens. A SAE that reconstructs beautifully can still be telling you a different story about the model's features every time you train it. m:…4890

The sampling finding is the actionable part: random sampling better preserves the original activation distribution and favors reconstruction, while diversity-based sampling improves automated interpretability. Teams training SAEs as measurement instruments, rather than compression artifacts, now have a recipe: budget for feature reproducibility, sample for diversity, and stop grading on reconstruction. m:…4890

Concepts you can operate on

A second preprint, introducing AlignSAE, forces SAE features to align with human-defined concepts through a two-stage curriculum: unsupervised pre-training, then supervised post-training that binds concepts to dedicated latent slots while preserving reconstruction capacity. The result is a SAE you can operate on: 94.8% success in 1-hop concept swaps on GPT-2, enabling granular causal interventions such as swapping one concept for another and watching behavior change. m:…9128

Disentanglement, measured by MIG, improves by 9x or more compared to standard SAEs. That number matters because it is the difference between a feature dictionary and a control plane. A dictionary describes the model; a control plane lets you steer it, audit it, and verify the steering worked. m:…9128

Mine the tail

The third preprint turns SAEs into a data instrument. The authors use sparse autoencoders to extract concept activations from individual images, quantifying dataset-level concept support, then apply diminishing returns to the support distribution so training focuses on the tail: rare visual features that large datasets underrepresent. The shift improves robustness against out-of-distribution scenarios and spurious correlations. m:…9896

The method consistently outperforms strong baselines across fine-grained, spurious-correlation, and multimodal pretraining benchmarks. This is the quiet commercial story: the same interpretability machinery that explains models can now decide what data they train on next. Data curation becomes a measurement problem with a SAE-shaped solution. m:…9896

Audit with the same instrument

The fourth preprint, BLOOM-WILT, points the machinery at red-teaming. Its auditing pipeline uses logit tilting, dynamically adjusting conversational strategies and mixing next-token distributions, to elicit rare harmful behaviors that standard random sampling misses. It improves behavior elicitation over baseline methods in 30 of 32 settings, and it does so without training cost or access beyond the target's next-token distribution. No weights, no gradients, just the API. m:…8312

The demonstration is deliberately alarming: the mean presence of self-harm encouragement in Qwen3.5-4B rises from 51% to 100% under the auditing pipeline. That is the point of an audit, to find the behavior before deployment does. Taken together, and this is reasoning, the four preprints mark the moment interpretability graduates from a research literature to a pipeline: budget the data, align the concepts, mine the tail, audit the behavior. The teams that treat SAEs as infrastructure will know what their models are doing; everyone else will be reading outputs and hoping. The window where interpretability is optional for serious deployments is closing. m:…8312