AI is starting to do the research, and the tests can barely keep up
METR's latest update finds agent time horizons doubling every 3.6 months, while new benchmarks for AI scientists show frontier models scoring 45 out of 100 — and struggling to run their own training.
- METR's Q4 2026 update finds frontier model time horizons doubling roughly every 3.6 months, with Opus 5.5 solving 77% of medium to large programs in the MirrorCode suite and GPT-6 Astra scoring 98% on FrontierMath Tier 4. m:…4775
- In a preprint, TwinLab-Bench tests AI scientists on 52 tasks drawn from real laboratory records, and the strongest model evaluated, Claude Fable 5.2, averaged a score of 45.0. Self-reported m:…3726
- In a preprint, Agent^2 RL-Bench shows agents improving ALFWorld performance from 4.85 to 93.28 using supervised warm-up and online rollouts, yet stable agent-driven RL post-training remains rare under fixed budgets. Self-reported m:…8050
- In a preprint, Paper2Lean formalised 20 mathematical optimisation papers with 100% compilation success, producing 110,910 lines of verified Lean code. Self-reported m:…1709
The capability curve
METR's Q4 2026 update puts numbers on a trend that has been hard to argue with. Frontier models now match or outperform skilled humans on coding and optimisation tasks, and the time horizons agents can handle are doubling roughly every 3.6 months. Opus 5.5 solves 77 percent of medium to large programs in the MirrorCode suite, and GPT-6 Astra scored 98 percent on FrontierMath Tier 4. m:…4775
The same update draws a line the models have not crossed. They do worse than human experts on judgement tasks, the kind where choices have no clear empirical feedback, and they do worse on tasks with higher lifecycle cost, even when task length is controlled for. METR says current public evidence is insufficient to rule out drastic AI research acceleration in the next couple of years, which is a careful way of saying the acceleration is plausible but unproven. m:…4775
The tests are struggling too
A preprint introducing TwinLab-Bench tries to measure the thing METR is warning about: AI agents acting as scientists. The benchmark covers 52 tasks from physical sciences, life sciences and engineering, and tests three abilities, workflow execution, measurement selection and experimental design. Each task uses a TwinLab agent built from real laboratory records to execute the experiments the tested agent designs. The strongest model evaluated, Claude Fable 5.2, averaged a score of 45.0. m:…3726
Two more preprints fill in the picture. Agent^2 RL-Bench asks whether agents can run their own reinforcement-learning post-training end to end, and finds they can lift ALFWorld performance from 4.85 to 93.28 with supervised warm-up and online rollouts, but stable agent-driven training under fixed budgets remains rare. Paper2Lean, an agentic system that turns maths papers into formal Lean proofs, reached 100 percent compilation success on 20 optimisation papers and produced 110,910 lines of verified Lean code, yet semantic fidelity remains a central challenge even when the proofs verify. m:…8050 m:…1709
The judge problem
There is a second-order problem hiding in the measurement. METR finds that AI judges systematically overestimate the quality of tasks made by models stronger than themselves, and that agents now build borderline-useful evaluation tasks, but not yet ones METR considers especially informative. If the judge cannot tell a good test from a bad one, the loop that is supposed to improve the model starts grading its own homework. m:…4775
TwinLab-Bench tries to defend against this by grounding its environments in real laboratory records and using closed-book probes to control for answers recalled from memory. The design choice matters: as training environments increasingly resemble testing benchmarks, METR warns that current evaluation methods may be losing their predictive power. A benchmark that a model can pass by memorising the lab is not measuring research ability. m:…4775 m:…3726
What it means
For anyone running a lab, the practical split is now clear. Hand agents the verifiable work, coding, optimisation, formalisation, where they already match or beat skilled humans. Keep humans on the judgement calls, the experimental design and the interpretation, where the best model in the hardest test scores 45 out of 100. That is not a temporary gap. It is the shape of the capability curve as it stands. m:…4775 m:…3726
What to watch is whether the next update closes the gap on high lifecycle cost tasks, the expensive, slow-feedback work that METR uses as a proxy for evaluation difficulty. METR says it remains uncertain about the relative growth rates there. Until those tasks start moving, talk of runaway research acceleration is a forecast, not a fact. m:…4775
What is still unresolved
- Whether time-horizon growth continues at the current rate on high lifecycle cost tasks
- Whether TwinLab-Bench scores improve once models are trained specifically for lab work
- Whether agent-driven RL post-training becomes stable without supervised warm-up pipelines
Provenance by Synorb. Every claim links to its source. This story was written by an agent from the Manifests listed below. Read the methodology and limitations, or send a correction.
Free to reuse, including commercially. Credit “Hanging Context, with provenance by Synorb” and link to this story. Keep the cited Manifest ids. Reuse terms v1.