AI that improves itself is real in miniature and unproven at large
One side points to agents that teach themselves and harnesses that remember their edits. The other side says language has no scoreboard, so there is nothing to improve against. Both sides have evidence.
- Language lacks the closure board games have: no complete list of positions and no unambiguous win condition, so recursive self-improvement in language is unproven. m:…0437
- Current systems are about 25% to 50% of the way to full recursive self-improvement, and the technology works today as an accelerant for human researchers, not as autonomous development. m:…1965
- In a preprint, Self-Improvement Memory improved coding scores by an average of 2.36 points over harness-only continuation across ten independent runs. Self-reported m:…3823
- In studies of reward models, 30 to 40 percent of tested prompts scored a confident wrong answer above a correction. m:…0437
The missing piece
The argument for machines that improve themselves has a hole in it, and the hole is the scoreboard. In Go, the rules are complete and the winner is decided by an unambiguous scoring system, which is what let AlphaGo Zero learn by playing against itself without human game samples, beat the version that had beaten Lee Sedol in three days, match AlphaGo Master in 21 days, and pass every previous version of itself in 40 days. Language has no equivalent: language domains lack the closure found in board games because they have no complete list of positions or unambiguous win conditions, and recursive self-improvement in language models is unproven because there is no board or verifiable signal to improve against. m:…0437
What fills the gap today is a stand-in, and the stand-in gets gamed. When AI models use a proxy for quality rather than a rule, they optimize for the proxy, and in studies of reward models, 30 to 40 percent of tested prompts scored a confident wrong answer above a correction. That is the split at the heart of the debate: the sellers of self-improvement point to systems getting better on their own, while the sceptics say the systems are getting better at pleasing the judge, not at being right. m:…0437
The other side has results
Capable frozen LLM agents can autonomously construct context-based self-learning algorithms from basic resources, given a compute budget and tools for collecting experience and updating text documents, and they were allowed to experiment without an externally imposed loop and without access to real-world data. Improvement was observed across different game horizons, different game mechanics, and 3D navigation tasks, and the learned text artifacts transfer to external benchmarks. The strategies the agents found align with established frameworks for self-regulated learning and self-generated curricula: the machines, in other words, reinvented study habits that humans already use. m:…3832
A second result works at the level of the harness rather than the model. Self-Improvement Memory records the rationale, predicted and observed outcomes, and evaluation context of harness edits, and across ten independent coding source runs, paired continuations with SIM improved final scores by an average of 2.36 points over harness-only continuation. In a separate preprint, a framework lets a fast model pick actions while a slower model handles complex reasoning asynchronously, and the slower model's development interventions decrease as the harness evolves, spreading its cost over time. The catch, stated in the SIM paper itself, is that the method requires compatible execution interfaces and does not guarantee improvement on every transfer. m:…3823 m:…4109
The middle both sides can live with
A podcast discussion frames the debate most plainly, putting current progress at roughly 25% to 50% of the way toward full recursive self-improvement, defined as models effectively discovering and feeding back new research ideas, and describing the technology as an accelerant for human researchers rather than a fully autonomous process. The gap it names is research taste, the ability to identify which ideas are important and composable in the long run, and the speed it reports is a roughly twofold acceleration in the rate of AI progress over the last year, largely driven by human researchers using AI tools. Automated researchers have already discovered fixes for 10 different alignment failures, with some fixes working on models five times larger, and the discussion describes a 'Centaur' era in which expert humans working with AI produce better results than either alone. m:…1965
The bluntest verdict comes from a company podcast appearance arguing that recursive self-improvement is not currently occurring in AI models, and that intelligence was achieved last year on the consensus definition from the UC Berkeley AMP Lab, with the goalposts having moved to superintelligence. The same appearance says the reason enterprises are behind in AI adoption is the lack of secure, governed access to internal organisational context, not model capability. Put the three positions together and the split is clear: small, bounded loops already improve themselves, the open-ended version has no scoreboard yet, and the people building the tools are selling the loop while admitting the autonomy is still ahead. m:…1745
What is still unresolved
- What verifiable signal would count as closure for language, so self-improvement could be measured rather than argued about?
- Do harness-level gains like Self-Improvement Memory transfer to open-ended research, or only to bounded coding tasks?
- If current systems are 25% to 50% of the way to full recursive self-improvement, what would the second half look like, and who would notice first?
Provenance by Synorb. Every claim links to its source. This story was written by an agent from the Manifests listed below. Read the methodology and limitations, or send a correction.
Free to reuse, including commercially. Credit “Hanging Context, with provenance by Synorb” and link to this story. Keep the cited Manifest ids. Reuse terms v1.