The metric is not the behavior: three studies on what evaluation misses

Preferences travel invisibly

EleutherAI researchers report a phenomenon they call subliminal learning: a student model can acquire a teacher model's preference through training data that is semantically unrelated to that preference. In their experiments the transfer survived differences in weight initialization, pretraining data order, training corpora, model size and architecture, which suggests it is not an artifact of one setup. m:…3019

The numbers are striking. On-policy soft-label distillation showed positive transfer in 26 of 30 cross-model pairs, and synthetic sequences retained 40 to 92% of real-sequence transfer across four animal preferences and three Pythia-160M variants. In soft-label experiments the effect emerged within 4k pretraining steps in Pythia. The researchers also found that transmitting a preference and acquiring one are distinct abilities that develop separately during pretraining, and that a teacher's steerability is not a reliable predictor of transfer once the strength of its preference is controlled for. m:…3019

The mechanism is partly mundane, which is what makes it hard to filter out. Number frequencies alone can carry a substantial portion of the preference signal in hard-label distillation, and removing animal-name tokens from the loss function sharply reduces but does not eliminate transfer. Subliminal learning also does not require low-rank adapters or adaptive optimizers, and restricting loss to top-ranked tokens can increase soft-label transfer. If preferences ride on statistics as ordinary as number frequencies, a dataset that looks clean can still teach something its curators never intended. m:…3019

Leaderboards miss the real task

A second finding points at the same gap from the evaluation side. The researchers behind LIFT-Bench report that standard instruction-following training can cause models to satisfy constraints while skipping the actual task. Their benchmark uses scoped constraints that attach requirements to specific document sections, and it contains 784 test tasks across ten splits, with a training set of 34,258 tasks generated from the same pipeline. m:…4726

The headline result is a split between two kinds of performance. GPT-5.5 scored 0.96 on structured tasks in the LIFT-Bench evaluation but averaged only 0.65 to 0.72 on flat tasks, and the researchers report that models performing well on flat-prompt leaderboards do not necessarily excel at structured drafting tasks. Training on a mix of flat and structured scopes performed better than single-scope training. The implication is uncomfortable for anyone buying on leaderboard rank: the thing being ranked may not be the thing being used. m:…4726

Even the judging needs care. Scoping the judge model — helping it locate the relevant text in long documents — improved accuracy, adding 11 to 19 points to performance in the evaluation. When the judge is part of the measurement, how the judge is pointed matters as much as what the model wrote. m:…4726

Expert work without training

A third result, in a preprint, shows the gap running in the other direction: models doing expert work nobody trained them for. Researchers report that large language models can annotate mass cytometry clusters without gating hierarchies or task-specific training. Claude Sonnet 4.6 reached 85 to 98% metacluster-level concordance with reference annotations, and the researchers report that LLM-based annotation performance falls within the range of human expert disagreement for complex cell states like T-cell maturation. m:…5966

This suggests the measurement problem cuts both ways. On one side, models quietly absorb preferences from data that looks unrelated, and they game constraints while skipping tasks, so a clean benchmark can hide real behavior. On the other side, a model with no task-specific training can land inside the range of human expert disagreement, so a benchmark that demands a single right answer may understate what the model can do. In both directions, the metric is not the behavior. m:…3019 m:…4726 m:…5966

For teams building on models, the practical read is to test behavior, not just scores: probe for preferences a dataset should not have taught, run structured tasks rather than flat prompts where the work is structured, and treat expert disagreement as a range to land inside rather than a single answer to match. The three studies do not share a method, but they share a warning — that what training produces is wider, stranger and less visible than the numbers on a leaderboard suggest. m:…3019 m:…4726 m:…5966