The new training trick is letting the model teach itself — and learn from its failures
A self-improving system now holds the top rank on GPU kernel benchmarks, a framework turns papers into agents that beat coding assistants, and a training method that learns from failed attempts lifts professional-task scores.
- In a preprint, CompoundingKernel ranks first on all three public SOL-ExecBench leaderboards for NVIDIA B200 hardware as of November 11, with kernels 4.40 times faster than the benchmark's reference baseline on 209 public-board problems. Self-reported m:…8093
- In a preprint, AutoMaster improves mean performance over vanilla supervised fine-tuning by 1.42 to 8.99 percentage points across Terminal, Search and software engineering tasks. Self-reported m:…8089
- Snorkel says Paper2Agent scored 91.2% accuracy on 300 benchmark questions, against 80.3% for Claude Code on the same questions, and turned 74 of 100 computational biology papers into working agents. Self-reported m:…9669
- Snorkel says its binary-gated GRPO training lifted the Qwen3.8-27B model's pass rate on professional GDPval tasks from 31.9% to 40.7%, and its deliverables beat human expert references in 44.0% of head-to-head comparisons, up from 36.4% for the base model. Self-reported m:…9670
- In a preprint, researchers find that withholding training data delays but does not prevent language models from acquiring skills, and that models regain withheld skills with a smaller fraction of the data needed from initialization as they improve. Self-reported m:…4448
The teacher improves itself
AutoMaster, in a preprint posted this week, starts from an awkward fact about training: the teacher model is usually fixed, and its bad habits get baked into the student. The framework uses Skill-FT, or skill-guided fine-tuning, to refine the teacher model's own trajectories before they are used for training — using natural-language skills to guide the teacher's rollouts, suppressing unproductive behaviours and reinforcing effective ones. The idea is that a fixed model can be made into a more effective teacher by shaping what it demonstrates. m:…8089
In a preprint, the authors report that AutoMaster improves mean performance over vanilla supervised fine-tuning by 1.42 to 8.99 percentage points across Terminal, Search and software engineering tasks, using 4B and 9B target models. Retaining the skill instructions during fine-tuning improved performance by a further 2.15 to 4.01 points over removing them. The gains are modest in absolute terms, but the direction is the point: the improvement comes from better teaching, not from a bigger model or more data. m:…8089
CompoundingKernel applies the same instinct to a harder job. In a preprint, it is a self-improving agent system that writes GPU kernels by searching from a problem's PyTorch definition, with no reference kernels or official scores to start from. As of November 11, its generated kernels rank first on all three public SOL-ExecBench leaderboards for NVIDIA B200 hardware, and it has held first place on the L1 board since October 27. On 209 public-board problems, its kernels are 4.40 times faster than the benchmark's reference baseline, and it beat a coded tree-search baseline on 35 of 38 problems. The authors note that the rate of reliable measurements limits how far the search scales, rather than the number of agents — the bottleneck is verifying results, not generating candidates. m:…8093
Learning from failure
Snorkel AI's contribution this week is about what training usually throws away. The company trained the Qwen3.8-27B model on 1,105 professional tasks from the GDPval+ series — real deliverables like financial models and regulatory filings, drawn from nine sectors of the US economy — using a method called binary-gated GRPO. Standard reinforcement learning with a binary pass-or-fail signal barely improves the base model on this data, because under standard GRPO, 79% of generated groups contained no passing rollout at all. There was nothing to learn from. m:…9670
Binary-gated GRPO uses a fallback training score when all rollouts in a group fail, so the model still gets a signal from its failures. With that change, 95% of generated groups carried a training signal, compared with 21% under standard GRPO. Snorkel says the model's pass rate on the GDPval gold set rose from 31.9% to 40.7% after 102,000 training rollouts, and the share of attempts that ran out of response budget fell from 17.2% to 7.0%. A pass on GDPval means satisfying at least 90% of the rubric items, so this is a demanding bar — and the company's claim is that the failures were the training data all along. m:…9670
The same company also showed what happens when the model is the reader rather than the writer. Paper2Agent turns a research paper and its codebase into an interactive AI agent, packaged as a Model Context Protocol server, so users can query scientific methods in natural language and the agent can execute methods, access data and reproduce results. Snorkel says these paper agents scored 91.2% accuracy on 300 benchmark questions, against 80.3% for Claude Code on the same questions, and that 74 out of 100 computational biology papers were successfully turned into working agents. The company also says the agents can collaborate across papers to perform novel scientific analysis — and that some papers cannot be turned into agents at all, because their code is incomplete or poorly documented. m:…9669
The data question
A third preprint this week asks a question that sits underneath all of this: how much does specific training data actually matter? The researchers withheld particular skills from training and found that withholding data delays but does not prevent models from acquiring skills — as models improve, they need a smaller fraction of the data to regain a withheld capability. A 7-billion-parameter model regained a withheld programming language when exposed to the data, and a chess model never explicitly trained to respond to check ranked legal responses almost as well as a trained model. m:…4448
A related preprint, JevRSI, makes the same point about cost. It decouples real-time action selection from slower reasoning, building a decision-model harness that evolves through environmental feedback, so expensive reasoning is amortised over time. The authors report that the system consistently improves decision quality, matching or surpassing LLM agents in several settings, while the need for slow, deliberate interventions decreases as the harness evolves. Put together, the week's papers describe a shift in where the work happens: less in collecting the perfect dataset, more in building systems that improve themselves once they are running. m:…1519
What is still unresolved
- Whether AutoMaster's gains hold on larger models and on tasks beyond Terminal, Search and software engineering.
- Whether CompoundingKernel's leaderboard results transfer to kernels written for real production workloads rather than benchmark problems.
- Whether Paper2Agent's accuracy holds for papers outside computational biology, and for papers with no usable codebase.
- Whether binary-gated GRPO's gains survive independent evaluation on professional tasks, and whether the human-expert comparisons were judged blind.
Provenance by Synorb. Every claim links to its source. This story was written by an agent from the Manifests listed below. Read the methodology and limitations, or send a correction.
Free to reuse, including commercially. Credit “Hanging Context, with provenance by Synorb” and link to this story. Keep the cited Manifest ids. Reuse terms v1.