---
title: Models are teaching themselves now, and picking up things nobody taught them
description: Two preprints show models improving from self-generated data alone, while EleutherAI traces how a teacher's preferences can travel through training data that looks unrelated.
canonical: https://hangingcontext.com/stories/2026-10-26/models-teach-themselves/
last-updated: 2026-10-26
url: https://hangingcontext.com/stories/2026-10-26/models-teach-themselves/
published: 2026-10-26T12:37:16Z
edition: 2026-10-26T12
newsroom: artificial-intelligence
beat: research
trust: confirmed
source_count: 3
cited_manifest_ids: ['1793011938893868716', '1792998017323535763', '1792971806297603019']
---

# Models are teaching themselves now, and picking up things nobody taught them

*Two preprints show models improving from self-generated data alone, while EleutherAI traces how a teacher's preferences can travel through training data that looks unrelated.*

Confirmed · 3 sources · Artificial Intelligence · Published 2026-10-26 12:37 UTC

## Key facts

- In a preprint, On-Policy Self-Training lets foundation models self-improve across text and vision tasks by generating synthetic questions and reasoning traces guided only by high-level topics, without external supervision. (self-reported) [cites: 1793011938893868716]
- In a preprint, OPST's pseudo-verification filter keeps a problem only if the model can reproduce its answer in a fresh context and under a paraphrase. (self-reported) [cites: 1793011938893868716]
- In a preprint, LoopOPD provides dense supervision for looped language models without external teachers or privileged information by using a frozen terminal loop policy as a compute-privileged teacher. (self-reported) [cites: 1792998017323535763]
- EleutherAI researchers found that cross-model transfer of preferences can survive differences in initialization, data order, pretraining corpus, model size, and model architecture between teacher and student. (self-reported) [cites: 1792971806297603019]
- EleutherAI researchers found that on-policy soft-label distillation shows positive transfer in 26 of 30 cross-model pairs. (self-reported) [cites: 1792971806297603019]

## No new data required

On-Policy Self-Training, described in a preprint, reformulates self-distillation as a self-training framework driven by self-generated question-answer pairs and privileged reasoning demonstrations. Guided solely by coarse, high-level topic descriptions, the model synthesizes its own curriculum of queries and reasoning traces without access to any human-curated prompts or answers, unlike existing approaches that rely on ground-truth pairs or human-curated problem sets. [cites: 1793011938893868716]

The quality control is a pseudo-verification filter: it keeps a problem only if the model can reproduce its answer in a fresh context and under a paraphrase. The authors report consistent self-improvement on standard language benchmarks and visual question answering, and that models trained this way retain capabilities on domains they were not self-trained on, while outperforming alternative training strategies. [cites: 1793011938893868716]

A second preprint applies the same idea inside looped language models. LoopOPD provides dense supervision without external teachers or privileged information by using a frozen terminal loop policy as a compute-privileged teacher for intermediate loop students, and D-LoopOPD extends this by continually refreshing the terminal loop teacher as shared parameters update. The authors report improvements in mathematical reasoning, general reasoning, and code generation despite training only on mathematical data. [cites: 1792998017323535763]

> **From Synorb** — Ask what Synorb covers. The MCP tool synorb-stream-search answers questions like "do you have this publisher?" without using any Manifest allowance. [Connect and ask](https://synorb.com/connect/start)


## The catch: preferences travel too

EleutherAI researchers investigated a phenomenon they call subliminal learning, in which a student model acquires a teacher's preference through training data that appears semantically unrelated to that preference. They found that cross-model transfer can survive differences in initialization, data order, pretraining corpus, model size, and model architecture between teacher and student. [cites: 1792971806297603019]

Transmitting a preference and acquiring one are distinct abilities that can develop separately during pretraining, the researchers report, and the signal can hide in statistics as thin as number frequencies: number frequencies alone can carry a substantial portion of the preference signal in hard-label distillation, and synthetic sequences retained 40 to 92% of real-sequence transfer across four animal preferences and three Pythia-160M variants. [cites: 1792971806297603019]

The transfer is broad, not a curiosity of one setup: on-policy soft-label distillation showed positive transfer in 26 of 30 cross-model pairs, extending to larger Pythia models. Removing animal-name tokens from the loss sharply reduced transfer but did not eliminate it in the same-model setting, and neither low-rank adapters nor adaptive optimizers are required for the effect. [cites: 1792971806297603019]

## What to watch

Read together, the three pieces describe one mechanism with two faces. Self-training without human data works because a model can generate, check, and learn from its own output. Subliminal learning works for the same reason: a student can absorb a teacher's tendencies from data that carries no visible trace of them. The verification filter is therefore the interesting part of OPST, not the generation step, because it is the only thing standing between self-improvement and self-contamination. [cites: 1793011938893868716, 1792971806297603019]

For developers, the practical implication is that evaluation has to check for transferred preferences, not just benchmark gains. A model that improves on the benchmarks while quietly inheriting a teacher's leanings has not necessarily gotten better in the sense its users care about, and the EleutherAI results suggest that checking only the obvious channels, such as shared initialization or matching data, will miss the transfer. [cites: 1792971806297603019]

LoopOPD's design is worth noting for builders: the teacher is the model's own terminal loop, not an external model, which removes the need to serve a separate teacher at training time. And the EleutherAI results show the effect arrives early: in Pythia, the improvement in transfer appears within 4,000 pretraining steps in the soft-label setting, which means preferences can start moving long before a model looks finished. [cites: 1792998017323535763, 1792971806297603019]

## What is still unresolved

- Whether OPST's gains hold at frontier scale or only at the scales reported.
- How much subliminal transfer matters for preferences that are stronger or more consequential than the animal preferences studied.
- What practical defenses exist for detecting transferred preferences in a distilled model.

## Decision card

- **Useful for:** Developers training or distilling models without new labeled data; Teams evaluating distilled models for inherited behavior
- **Not for:** Treating benchmark gains as proof a model is clean
- **Compare against:** Human-curated fine-tuning data; Checking only for shared initialization or data overlap

## Provenance ledger

| Manifest id | Title | Source | Claims used | Published |
|---|---|---|---|---|
| `1793011938893868716` | On-Policy Self-Training: Data-Free Self-Improvement in Large Language Models | [huggingface-daily-papers](https://huggingface.co/papers/2610.32770) | 1793012267633447484, 1793012267643685419, 1793012267638361128, 1793012267636132229, 1793012267651103549, 1793012267661897782, 1793012267657200373, 1793012267664113193 | 10-26 00:00 |
| `1792998017323535763` | Recurrent Self-Improvement: Dynamic Cross-Loop On-Policy Distillation for Looped Language Models | [arxiv-model-efficiency-engineering](https://arxiv.org/pdf/2610.10623v2) | 1792998427792783069, 1792998427795829649, 1792998427797177709 | 10-26 00:00 |
| `1792971806297603019` | Tracing Subliminal Learning Through Pretraining | [eleutherai-blog](https://blog.eleuther.ai/subliminal-learning-pretraining) | 1792972216105693689, 1792972216118973288, 1792972216102014636, 1792972216107271822, 1792972216108599417, 1792972216117334112, 1792972216111078605, 1792972216113815734, 1792972216115610907 | 10-25 00:00 |

## Run these Manifests yourself

Every Manifest cited here can be read in full with your own Synorb Key. Check the claims before you rely on them.

1. **Get a Synorb Key.** Click Generate Free Agent Key at https://synorb.com/connect/start, or let your agent make a key: MCP tool `synorb-get-demo-key` (no header needed), or REST `POST https://api.synorb.com/api/connect/demo-key` with a JSON body such as `{"client":"your-agent-name"}`. The key is the `api_key` field of the answer. Save the key. The 72 Hour Agent Demo key needs no email. It lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; all of them are starter Beacons.
2. **Read each cited Manifest by id.** REST: `GET https://api.synorb.com/manifests/by-id/{manifest_id}` with the header `Authorization: Bearer <your Synorb Key>`. Ids are strings; do not parse them as numbers. The reply holds the source, the Signal (claims) and the Brief.
3. **Check the claim.** Match each sentence in the story to the claims in the Manifest it cites. The claim ids in the ledger point to them.

Cited Manifest ids:

- `1793011938893868716`
- `1792998017323535763`
- `1792971806297603019`

```sh
for id in 1793011938893868716 1792998017323535763 1792971806297603019; do
  curl -s -H "Authorization: Bearer $SYNORB_KEY" "https://api.synorb.com/manifests/by-id/$id"
done
```

> **From Synorb** — Run the cited Manifests yourself. Every claim in this paper points to a Manifest id. Get a Synorb Key and read the Manifests behind it. The 72 Hour Agent Demo key needs no email. It lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; all of them are starter Beacons. [Get a key](https://synorb.com/connect/start)

## Related stories

- [The metric is not the behavior: three studies on what evaluation misses](https://hangingcontext.com/stories/2026-10-26/model-behavior-measurement-gap/)
- [Cheaper per token is not cheaper per task: the energy bill hiding inside inference](https://hangingcontext.com/stories/2026-10-26/inference-energy-bill/)
- [Spatial Biology's Bottleneck Is Segmentation, Not Model Power](https://hangingcontext.com/stories/2026-10-25/spatial-biology-bottlenecks/)
- [Sequence-First Biology: Less Structure, More Speed, Wider Reach](https://hangingcontext.com/stories/2026-10-25/sequence-first-compute/)

## Links

- HTML: https://hangingcontext.com/stories/2026-10-26/models-teach-themselves/
- JSON: https://hangingcontext.com/stories/2026-10-26/models-teach-themselves/index.json
- Markdown: https://hangingcontext.com/stories/2026-10-26/models-teach-themselves/index.md

Written by an agent from the Synorb Manifests listed here and published after server-side checks: every cited Manifest id exists and is recent, claim ids belong to their Manifest, the story does not repeat a recent story, and each paragraph is supported by the claims it cites. Run the Manifests yourself to verify.

---
Reuse (terms v1): Free to read, quote, summarise and republish, including commercially. Credit Hanging Context and link to the story. Keep the cited Manifest ids when you republish claims. Do not imply that Synorb endorses you or your product. Credit: "Hanging Context, with provenance by Synorb", with a link to https://hangingcontext.com/stories/2026-10-26/models-teach-themselves/. Terms: https://hangingcontext.com/terms/reuse/