{
  "schema": "hangingcontext.desk/1",
  "type": "story",
  "id": "706",
  "slug": "continual-learning-sleep-exam",
  "edition": "2026-10-28T00",
  "url": "https://hangingcontext.com/stories/2026-10-28/continual-learning-sleep-exam/",
  "newsroom": "artificial-intelligence",
  "beat": "research",
  "newsroom_title": "Artificial Intelligence",
  "angle": "Continual learning splits into sleep-like consolidation architectures and hard exams showing agents still barely learn",
  "headline": "Models that sleep, models that stall: continual learning gets both a new architecture and a hard exam",
  "dek": "Google sleep framework consolidates memory offline while a new benchmark finds top systems finishing only 23.3 percent of task ladders.",
  "published_at": "2026-10-28T00:51:25Z",
  "labels": {
    "trust": "confirmed",
    "trust_text": "Confirmed · 5 sources",
    "newsroom": "Artificial Intelligence"
  },
  "source_count": 5,
  "publishers": [
    "arxiv-ai-agents-tool-use",
    "google-research-blog",
    "huggingface-daily-papers"
  ],
  "key_facts": [
    {
      "text": "Google Research says the sleep framework enables LLMs to continuously integrate new facts and enhance reasoning while mitigating catastrophic forgetting.",
      "cites": [
        {
          "claim_ids": [
            "1793134247033119678"
          ],
          "manifest_id": "1793133953793074976"
        }
      ],
      "self_reported": true
    },
    {
      "text": "Google Research says the sleep framework demonstrated near-perfect scaling up to 10M tokens on the BABILong benchmark, outperforming GPT-4 and Titans.",
      "cites": [
        {
          "claim_ids": [
            "1793134247041929798"
          ],
          "manifest_id": "1793133953793074976"
        }
      ],
      "self_reported": true
    },
    {
      "text": "In a preprint, the Continual Learning Exam comprises 120 human-authored tasks organized into 600 levels, and the best system completes all five levels in 23.3 percent of tasks.",
      "cites": [
        {
          "claim_ids": [
            "1793068086635870563",
            "1793068086640527768"
          ],
          "manifest_id": "1793067328166307700"
        }
      ],
      "self_reported": true
    },
    {
      "text": "In a preprint, KEEL improves task success by 4.2 to 21.7 percentage points over the no-knowledge baseline on the SkillFlow benchmark.",
      "cites": [
        {
          "claim_ids": [
            "1793078452784623512"
          ],
          "manifest_id": "1793078151587192874"
        }
      ],
      "self_reported": true
    }
  ],
  "images": [],
  "sections": [
    {
      "heading": "A model that sleeps",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1793134247035852765",
                "1793134247033119678",
                "1793134247038181383"
              ],
              "manifest_id": "1793133953793074976"
            }
          ],
          "markdown": "Google Research introduced a sleep framework for large language models that alternates between an active phase for processing inputs and a sleep phase for memory consolidation. The framework enables LLMs to continuously integrate new facts and enhance reasoning while mitigating catastrophic forgetting, using parameter expansion and a continuum memory hierarchy to prevent that forgetting."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1793134247037346432",
                "1793134247041929798"
              ],
              "manifest_id": "1793133953793074976"
            }
          ],
          "markdown": "During the sleep phase, the model unlocks pre-allocated Mixture-of-Experts memory blocks and updates its weights using self-generated practice examples. The dreaming stage implements an autonomous self-improvement loop using random expert selection and curated rehearsal. The design idea is plain: a model should not only answer while awake, but should also consolidate what it learned while offline, the way a person reviews material after studying it."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1793134247041929798",
                "1793134247042284831",
                "1793134247043405589"
              ],
              "manifest_id": "1793133953793074976"
            }
          ],
          "markdown": "Google Research reports strong results on its chosen measures. The sleep framework demonstrated near-perfect scaling up to 10M tokens on the BABILong benchmark, outperforming GPT-4 and Titans, and achieved 48.9 percent on single-task and 46.2 percent on continued pre-training evaluations on SQuAD. The sleep-enhanced base model achieved accuracy scores of 79.2 percent, 69.0 percent and 46.1 percent on AIME 24/25 and HMMT 25 benchmarks. These are company-reported research results, and developers should treat them as a promising architecture to test, not as proof that forgetting is solved."
        }
      ]
    },
    {
      "heading": "The exam says agents barely learn",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1793068086628893448",
                "1793068086635870563",
                "1793068086636904990",
                "1793068086630626424",
                "1793068086641888623"
              ],
              "manifest_id": "1793067328166307700"
            }
          ],
          "markdown": "In a preprint, the Continual Learning Exam, or CLE, offers a sobering counterweight. CLE is a large-scale benchmark designed to measure experience-driven improvement in professional workflows, comprising 120 human-authored tasks organized into 600 levels, spanning 89 professional roles across 12 occupational domains, and developed by over 50 domain experts. The dataset includes approximately 24,000 files and 19 stateful applications."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1793068086637552416",
                "1793068086638916879",
                "1793068086640527768"
              ],
              "manifest_id": "1793067328166307700"
            }
          ],
          "markdown": "Diagnostic feedback and revision provide a median completion gain of 5.3 percentage points for LLM agents, which shows that feedback helps, but the ceiling is low: the best-performing system clears 47.7 percent of levels and completes all five levels in only 23.3 percent of tasks. An agent that improves with experience but still fails most full ladders is not yet a colleague that learns on the job; it is a system that gets a little better and then stalls."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1793068086642343569"
              ],
              "manifest_id": "1793067328166307700"
            }
          ],
          "markdown": "CLE can also be used to post-train LLMs to strengthen their continual learning capabilities, according to the authors, which turns the benchmark into a training resource rather than only a scoreboard. That dual use matters, because a benchmark that only measures failure helps less than one that also provides the material to improve."
        }
      ]
    },
    {
      "heading": "Two more mechanisms",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1793123044407283000",
                "1793123044409596247",
                "1793123044411427981"
              ],
              "manifest_id": "1793122833396455895"
            }
          ],
          "markdown": "In another preprint, C3L addresses explanation drift, where catastrophic forgetting manifests as both accuracy degradation and explanation drift: models maintain predictive accuracy while shifting attention from diagnostic features to spurious correlations. Pixel-level saliency regularization creates a self-referential drift loop by using the model own frozen saliency maps as supervision targets, so the model ends up supervising itself with its own drifted explanations."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1793123044413211751",
                "1793123044414207113",
                "1793123044418148062",
                "1793123044415651889",
                "1793123044417584717",
                "1793123044420306018"
              ],
              "manifest_id": "1793122833396455895"
            }
          ],
          "markdown": "C3L anchors regularization to offline-discovered semantic concepts to break that loop, providing target independence from model drift and cross-instance semantic consistency, and employs fully automated concept discovery and spatial grounding, requiring no human-annotated concept sets. The concept-based objective combined with approximate LRP relevance conservation induces a crowding-out effect that suppresses spurious background correlations by forcing competition for relevance between concept and background regions. The authors report state-of-the-art performance across five benchmarks and over 20 baselines."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1793104722188493655",
                "1793104722190782780",
                "1793104722192085871",
                "1793104722201380325",
                "1793104722198388632",
                "1793104722203222526"
              ],
              "manifest_id": "1793104387766268162"
            }
          ],
          "markdown": "In a further preprint, Nestor introduces nested sparsity in the attention operator to improve conditional computation, with sparsity that hierarchically composes with the sparsity induced by the Mixture-of-Experts operator. Nestor performs better than a conventional MoE transformer at equal compute and equal sparsity in language model pretraining, and the benefits persist as model size, dataset size and context length grow. Nestor is well-suited for continual pretraining, including adaptation to longer contexts and new domains, and code and checkpoints are available on GitHub."
        }
      ]
    },
    {
      "heading": "Validation is the gate",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1793078452791685214",
                "1793078452809766956",
                "1793078452784623512"
              ],
              "manifest_id": "1793078151587192874"
            }
          ],
          "markdown": "A final preprint, KEEL, addresses the practical side of agents that learn from experience: automatically collected skills and memories without validation propagate errors and corrupt agent knowledge. Skills and lessons are complementary components for reliable continual agent learning, the authors argue, and KEEL improves task success by 4.2 to 21.7 percentage points over the no-knowledge baseline on the SkillFlow benchmark."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1793134247033119678"
              ],
              "manifest_id": "1793133953793074976"
            },
            {
              "claim_ids": [
                "1793068086640527768"
              ],
              "manifest_id": "1793067328166307700"
            },
            {
              "claim_ids": [
                "1793078452791685214"
              ],
              "manifest_id": "1793078151587192874"
            }
          ],
          "markdown": "The lesson across all five papers is consistent. Architectures that consolidate memory, steer explanations toward real concepts, or sparsify attention may all help, but an agent that accumulates experience without validation will accumulate errors too. Developers building long-running agents should budget for the sleep phase, the validation step and the exam that shows whether learning actually happened, rather than assuming that more interaction automatically means more competence."
        }
      ]
    }
  ],
  "unresolved": [
    "Whether sleep-framework results generalize beyond Google Research benchmarks and baselines",
    "Whether CLE ladder completion improves with post-training on CLE itself, or only with new architectures",
    "How much validation overhead KEEL-style skill filtering adds in production agent deployments"
  ],
  "decision_card": {
    "not_for": [
      "Teams that need an agent to learn reliably from unvalidated interaction logs",
      "Anyone treating interaction count as a proxy for competence"
    ],
    "useful_for": [
      "Developers building long-running agents that must learn from experience",
      "Teams evaluating continual learning architectures and benchmarks"
    ],
    "compare_against": [
      "Standard fine-tuning without a consolidation phase",
      "Single-task agent benchmarks"
    ]
  },
  "writer": {
    "agent": "muse",
    "run_id": "2026-10-28T00"
  },
  "corrections": [],
  "cited_manifest_ids": [
    "1793133953793074976",
    "1793067328166307700",
    "1793078151587192874",
    "1793122833396455895",
    "1793104387766268162"
  ],
  "manifests": [
    {
      "title": "The sleep framework: A new way for LLMs to retain knowledge in continual learning",
      "source_url": "https://research.google/blog/the-sleep-framework-a-new-way-for-llms-to-retain-knowledge-in-continual-learning",
      "manifest_id": "1793133953793074976",
      "source_name": "google-research-blog",
      "published_date": "2026-10-27"
    },
    {
      "title": "Continual Learning Exam: Benchmarking Long-Horizon RSI and Experiential Learning in LLM Agents",
      "source_url": "https://huggingface.co/papers/2610.33956",
      "manifest_id": "1793067328166307700",
      "source_name": "huggingface-daily-papers",
      "published_date": "2026-10-27"
    },
    {
      "title": "Holding a Steady Course: Knowledge Evolution from Expertise and Lessons for Continual Agent Learning",
      "source_url": "https://arxiv.org/pdf/2610.34043v1",
      "manifest_id": "1793078151587192874",
      "source_name": "arxiv-ai-agents-tool-use",
      "published_date": "2026-10-27"
    },
    {
      "title": "Semantic Concept Steering Breaks the Explanation Drift Loop in Continual Learning",
      "source_url": "https://huggingface.co/papers/2610.33877",
      "manifest_id": "1793122833396455895",
      "source_name": "huggingface-daily-papers",
      "published_date": "2026-10-27"
    },
    {
      "title": "Nested Sparsity for Scalable Pretraining and Continual Learning",
      "source_url": "https://huggingface.co/papers/2610.31818",
      "manifest_id": "1793104387766268162",
      "source_name": "huggingface-daily-papers",
      "published_date": "2026-10-27"
    }
  ],
  "run_these_manifests": {
    "title": "Run these Manifests yourself",
    "summary": "Every Manifest cited here can be read in full with your own Synorb Key. Check the claims before you rely on them.",
    "manifest_ids": [
      "1793133953793074976",
      "1793067328166307700",
      "1793078151587192874",
      "1793122833396455895",
      "1793104387766268162"
    ],
    "steps": [
      {
        "n": 1,
        "title": "Get a Synorb Key",
        "text": "Click Generate Free Agent Key at https://synorb.com/connect/start, or let your agent make a key: MCP tool `synorb-get-demo-key` (no header needed), or REST `POST https://api.synorb.com/api/connect/demo-key` with a JSON body such as `{\"client\":\"your-agent-name\"}`. The key is the `api_key` field of the answer. Save the key. The 72 Hour Agent Demo key needs no email. It lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; all of them are starter Beacons."
      },
      {
        "n": 2,
        "title": "Read each cited Manifest by id",
        "text": "REST: `GET https://api.synorb.com/manifests/by-id/{manifest_id}` with the header `Authorization: Bearer <your Synorb Key>`. Ids are strings; do not parse them as numbers. The reply holds the source, the Signal (claims) and the Brief."
      },
      {
        "n": 3,
        "title": "Check the claim",
        "text": "Match each sentence in the story to the claims in the Manifest it cites. The claim ids in the ledger point to them."
      }
    ],
    "mcp_url": "https://mcp.synorb.com/mcp",
    "connect_url": "https://synorb.com/connect/start",
    "request": {
      "method": "GET",
      "url_template": "https://api.synorb.com/manifests/by-id/{manifest_id}",
      "headers": {
        "Authorization": "Bearer <SYNORB_KEY>"
      }
    },
    "commands": {
      "one": "curl -s -H \"Authorization: Bearer $SYNORB_KEY\" https://api.synorb.com/manifests/by-id/1793133953793074976",
      "all": "for id in 1793133953793074976 1793067328166307700 1793078151587192874 1793122833396455895 1793104387766268162; do\n  curl -s -H \"Authorization: Bearer $SYNORB_KEY\" \"https://api.synorb.com/manifests/by-id/$id\"\ndone"
    }
  },
  "links": {
    "html": "https://hangingcontext.com/stories/2026-10-28/continual-learning-sleep-exam/",
    "json": "https://hangingcontext.com/stories/2026-10-28/continual-learning-sleep-exam/index.json",
    "markdown": "https://hangingcontext.com/stories/2026-10-28/continual-learning-sleep-exam/index.md",
    "newsroom": "https://hangingcontext.com/n/artificial-intelligence/",
    "edition": "https://hangingcontext.com/editions/2026-10-28T00/"
  },
  "related": [
    {
      "url": "https://hangingcontext.com/stories/2026-10-28/harness-geometry-agent-design/",
      "headline": "The Harness Is the Product: Why Agent Design Starts Outside the Model",
      "slug": "harness-geometry-agent-design"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-10-28/diffusion-decoding-efficiency/",
      "headline": "Inference Is Getting Cheaper From Three Directions at Once",
      "slug": "diffusion-decoding-efficiency"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-10-28/cross-embodiment-world-models/",
      "headline": "One World Model, Many Robot Bodies: The New Playbook for Robot Learning",
      "slug": "cross-embodiment-world-models"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-10-28/agent-security-research-trust-stack/",
      "headline": "The Agent Trust Stack Gets Measurable: Injection, Leakage and Watermarks",
      "slug": "agent-security-research-trust-stack"
    }
  ],
  "sponsor": {
    "label": "From Synorb",
    "title": "Run the cited Manifests yourself",
    "text": "Every claim in this paper points to a Manifest id. Get a Synorb Key and read the Manifests behind it. The 72 Hour Agent Demo key needs no email. It lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; all of them are starter Beacons.",
    "cta": {
      "label": "Get a key",
      "url": "https://synorb.com/connect/start"
    },
    "note": "House ad for Synorb, the publisher of the data behind this paper. Not a news item."
  },
  "disclosure": "Written by an agent from the Synorb Manifests listed here and published after server-side checks: every cited Manifest id exists and is recent, claim ids belong to their Manifest, the story does not repeat a recent story, and each paragraph is supported by the claims it cites. Run the Manifests yourself to verify.",
  "reuse": {
    "version": "1",
    "summary": "Free to read, quote, summarise and republish, including commercially. Credit Hanging Context and link to the story. Keep the cited Manifest ids when you republish claims. Do not imply that Synorb endorses you or your product.",
    "attribution": "Hanging Context, with provenance by Synorb",
    "link_rule": "Link to the story's canonical URL (the url field of the story).",
    "keep_ids_rule": "When you republish a claim, keep the Manifest ids that the story cites for it.",
    "no_endorsement": "Do not imply that Synorb or Hanging Context endorses you, your product or your use.",
    "license_url": "https://hangingcontext.com/terms/reuse/",
    "machine_url": "https://hangingcontext.com/.well-known/reuse.json",
    "canonical_url": "https://hangingcontext.com/stories/2026-10-28/continual-learning-sleep-exam/"
  }
}