{
  "schema": "hangingcontext.desk/1",
  "type": "story",
  "id": "1048",
  "slug": "agents-learn-videos-relay-guardrails",
  "edition": "2026-11-04T00",
  "url": "https://hangingcontext.com/stories/2026-11-04/agents-learn-videos-relay-guardrails/",
  "newsroom": "artificial-intelligence",
  "beat": "research",
  "newsroom_title": "Artificial Intelligence",
  "angle": "New research lets agents learn from videos, hand off reasoning, and prove their guardrails",
  "headline": "Agents that learn from video, hand off thinking, and prove their guardrails",
  "dek": "Three papers point the same way: agents get better when they keep what they learn, pass reasoning to the right peer, and face guardrails that can be checked rather than trusted.",
  "published_at": "2026-11-04T00:44:10Z",
  "labels": {
    "trust": "reported",
    "trust_text": "Reported · 4 Manifests",
    "newsroom": "Artificial Intelligence"
  },
  "source_count": 4,
  "publishers": [
    "huggingface-daily-papers"
  ],
  "sources": [
    "huggingface-daily-papers"
  ],
  "distinct_source_count": 1,
  "key_facts": [
    {
      "text": "Tutorial Agentic Learning lets LLM agents learn continuously from instructional videos without parameter updates, reaching 91% relative gains on test splits over vanilla multimodal models in the paper's results.",
      "cites": [
        {
          "claim_ids": [
            "1793717753059222246"
          ],
          "manifest_id": "1793717456838755837"
        },
        {
          "claim_ids": [
            "1793717753067354242"
          ],
          "manifest_id": "1793717456838755837"
        }
      ],
      "self_reported": true
    },
    {
      "text": "MACR, a relay-based multi-agent framework, improved accuracy by 3.3% on MathVista and 3.9% on MMStar while using 6.0x less token overhead than other multi-agent strategies on HallusionBench and MMStar.",
      "cites": [
        {
          "claim_ids": [
            "1793717755953103149"
          ],
          "manifest_id": "1793717456838363896"
        },
        {
          "claim_ids": [
            "1793717755955026692"
          ],
          "manifest_id": "1793717456838363896"
        },
        {
          "claim_ids": [
            "1793717755957203520"
          ],
          "manifest_id": "1793717456838363896"
        }
      ],
      "self_reported": true
    },
    {
      "text": "LeanGuard, a verified runtime, cut direct prompt-injection success on DTap from 57.9% to 5.9% and executed zero forbidden calls on Tau-Bench, where unguarded agents executed at least 33.",
      "cites": [
        {
          "claim_ids": [
            "1793717753955181322"
          ],
          "manifest_id": "1793717456838424159"
        },
        {
          "claim_ids": [
            "1793717753947020359"
          ],
          "manifest_id": "1793717456838424159"
        }
      ],
      "self_reported": true
    },
    {
      "text": "In the IdeaTaste study of more than 90,000 proposals across six models, LLM agents reached only 26% of the human level on citation matching and 18% on idea matching.",
      "cites": [
        {
          "claim_ids": [
            "1793736459617080487"
          ],
          "manifest_id": "1793736030042863635"
        },
        {
          "claim_ids": [
            "1793736459616740838"
          ],
          "manifest_id": "1793736030042863635"
        },
        {
          "claim_ids": [
            "1793736459619341700"
          ],
          "manifest_id": "1793736030042863635"
        },
        {
          "claim_ids": [
            "1793736459621710310"
          ],
          "manifest_id": "1793736030042863635"
        }
      ],
      "self_reported": true
    }
  ],
  "images": [],
  "sections": [
    {
      "heading": "Learning without retraining",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1793717753059222246"
              ],
              "manifest_id": "1793717456838755837"
            },
            {
              "claim_ids": [
                "1793717753061957519"
              ],
              "manifest_id": "1793717456838755837"
            },
            {
              "claim_ids": [
                "1793717753063118346"
              ],
              "manifest_id": "1793717456838755837"
            }
          ],
          "markdown": "Tutorial Agentic Learning, or TAL, lets language-model agents learn continuously from instructional videos without parameter updates, according to the paper. It consolidates tutorial videos into a persistent, structured memory called a tutorialbook, built through a two-stage adversarial process, and it retrieves information through a Table-of-Contents tree that ties retrieval directly to the model's reasoning. The point is practical: an agent that can keep what it watched can reuse it, rather than starting from scratch each time."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1793717753064637878"
              ],
              "manifest_id": "1793717456838755837"
            },
            {
              "claim_ids": [
                "1793717753066896960"
              ],
              "manifest_id": "1793717456838755837"
            },
            {
              "claim_ids": [
                "1793717753067354242"
              ],
              "manifest_id": "1793717456838755837"
            },
            {
              "claim_ids": [
                "1793717753069824126"
              ],
              "manifest_id": "1793717456838755837"
            }
          ],
          "markdown": "The test bed is narrow but concrete. The VidLearn benchmark consists of 72 chess tutorial videos spanning 12 openings, and on it TAL achieved up to 113% relative gains on training splits and 91% relative gains on test splits compared with vanilla multimodal models. In head-to-head games, TAL reached a stronger opening-phase position than its vanilla counterpart in 69% of games. This suggests video learning works where the skill is procedural and demonstrable, like a chess opening. It does not show the same result for skills that cannot be watched and copied."
        }
      ]
    },
    {
      "heading": "Passing the reasoning baton",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1793717755950140155"
              ],
              "manifest_id": "1793717456838363896"
            },
            {
              "claim_ids": [
                "1793717755948277524"
              ],
              "manifest_id": "1793717456838363896"
            },
            {
              "claim_ids": [
                "1793717755952122550"
              ],
              "manifest_id": "1793717456838363896"
            },
            {
              "claim_ids": [
                "1793717755958515694"
              ],
              "manifest_id": "1793717456838363896"
            }
          ],
          "markdown": "Most multi-agent methods today use a group thinking setup, the MACR paper notes, where agents generate solutions independently. MACR instead introduces a relay-based framework that dynamically transfers reasoning between agents, using confidence-guided relay triggering and a bidding mechanism to select the next agent. Relay collaboration, the paper says, lets heterogeneous agents integrate complementary insights across reasoning stages, so the agent best placed for the next step takes over."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1793717755953103149"
              ],
              "manifest_id": "1793717456838363896"
            },
            {
              "claim_ids": [
                "1793717755955026692"
              ],
              "manifest_id": "1793717456838363896"
            },
            {
              "claim_ids": [
                "1793717755957203520"
              ],
              "manifest_id": "1793717456838363896"
            },
            {
              "claim_ids": [
                "1793717755960776234"
              ],
              "manifest_id": "1793717456838363896"
            }
          ],
          "markdown": "The reported gains are modest on accuracy and large on cost. MACR achieved a 3.3% accuracy improvement on MathVista and 3.9% on MMStar, while consuming 6.0x less token overhead on HallusionBench and MMStar compared with other multi-agent strategies. This suggests the relay idea earns its keep less through smarter answers and more through cheaper coordination: fewer tokens spent on agents duplicating each other's work. The authors say the source code is available, which lets other teams test whether the relay advantage holds outside these benchmarks."
        }
      ]
    },
    {
      "heading": "Guardrails that can be checked",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1793717753934888327"
              ],
              "manifest_id": "1793717456838424159"
            },
            {
              "claim_ids": [
                "1793717753937685681"
              ],
              "manifest_id": "1793717456838424159"
            },
            {
              "claim_ids": [
                "1793717753939912899"
              ],
              "manifest_id": "1793717456838424159"
            },
            {
              "claim_ids": [
                "1793717753940968912"
              ],
              "manifest_id": "1793717456838424159"
            }
          ],
          "markdown": "LeanGuard tackles the other side of tool-using agents: stopping them from doing what policy forbids. It is a verified runtime that blocks policy-violating tool calls, using Lean4 to write policies that check tool calls before they reach the environment. Its policies combine state predicates with scoped past-time temporal operators, and its decision procedure is proved equivalent to LeanLTL semantics. In plain terms, the guardrail is written in a language with proof behind it, not as another prompt asking the model to behave."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1793717753955181322"
              ],
              "manifest_id": "1793717456838424159"
            },
            {
              "claim_ids": [
                "1793717753947020359"
              ],
              "manifest_id": "1793717456838424159"
            },
            {
              "claim_ids": [
                "1793717753957456783"
              ],
              "manifest_id": "1793717456838424159"
            },
            {
              "claim_ids": [
                "1793717753963997064"
              ],
              "manifest_id": "1793717456838424159"
            },
            {
              "claim_ids": [
                "1793717753943502874"
              ],
              "manifest_id": "1793717456838424159"
            }
          ],
          "markdown": "The reported results are strong on the benchmarks the authors chose. LeanGuard reduced direct prompt-injection success on DTap from 57.9% to 5.9%, and on Tau-Bench unguarded agents executed at least 33 forbidden calls while LeanGuard executed zero. Blocking alone can break tasks, so the paper also tests repair: proof-guided repair proves an admissible edit for 88 of 252 denials in a median of 2 milliseconds, and feedback mechanisms raise task success for four of five models over bare denial. This suggests verified blocking plus a route back to a permitted action is more useful than denial alone."
        }
      ]
    },
    {
      "heading": "Where agents still fall short",
      "paragraphs": [
        {
          "cites": [
            {
              "claim_ids": [
                "1793736459612775607"
              ],
              "manifest_id": "1793736030042863635"
            },
            {
              "claim_ids": [
                "1793736459614217659"
              ],
              "manifest_id": "1793736030042863635"
            },
            {
              "claim_ids": [
                "1793736459617080487"
              ],
              "manifest_id": "1793736030042863635"
            },
            {
              "claim_ids": [
                "1793736459616740838"
              ],
              "manifest_id": "1793736030042863635"
            },
            {
              "claim_ids": [
                "1793736459619341700"
              ],
              "manifest_id": "1793736030042863635"
            },
            {
              "claim_ids": [
                "1793736459621710310"
              ],
              "manifest_id": "1793736030042863635"
            },
            {
              "claim_ids": [
                "1793736459624589803"
              ],
              "manifest_id": "1793736030042863635"
            },
            {
              "claim_ids": [
                "1793736459627658444"
              ],
              "manifest_id": "1793736030042863635"
            },
            {
              "claim_ids": [
                "1793736459629278773"
              ],
              "manifest_id": "1793736030042863635"
            },
            {
              "claim_ids": [
                "1793736459631950277"
              ],
              "manifest_id": "1793736030042863635"
            }
          ],
          "markdown": "IdeaTaste is the counterweight to the optimism above. The benchmark tests language-model research proposals against future publications, using a literature snapshot frozen at 2025, and the study analyzed more than 90,000 proposals across six contemporary language models. LLM agents reached 26% of the human level on citation matching and 18% on idea matching, and the paper says AI agents rarely arrive at the concrete approaches pursued by human researchers. AI proposals trailed 2026 papers most in expected success and least in presentation, and proposals whose ideas no human paper shares fared worst in expert review."
        },
        {
          "cites": [
            {
              "claim_ids": [
                "1793736459625332084"
              ],
              "manifest_id": "1793736030042863635"
            },
            {
              "claim_ids": [
                "1793736459622391851"
              ],
              "manifest_id": "1793736030042863635"
            }
          ],
          "markdown": "Two findings limit easy fixes. Neither richer field context nor larger populations substantially closed the performance gap, the paper says, though AI populations can cite as diversely as a human community. Taken together, this suggests agents are getting better at learning procedures, sharing reasoning and obeying checked guardrails, while still struggling to originate research ideas humans would actually pursue. For builders, that split matters: trust agents with watched procedures and verified tool calls before trusting them with open-ended invention."
        }
      ]
    }
  ],
  "unresolved": [
    "Whether TAL's video-learning gains hold outside chess tutorials and procedural skills.",
    "Whether MACR's token savings hold on tasks beyond the reported benchmarks.",
    "Whether LeanGuard's verified policies scale to policies written by teams without Lean expertise; the evidence does not describe authoring cost."
  ],
  "decision_card": {
    "not_for": [
      "Treating benchmark gains as production guarantees",
      "Using agents as independent research inventors"
    ],
    "useful_for": [
      "Developers building tool-using agents that need checked guardrails",
      "Teams choosing between group debate and relay collaboration"
    ],
    "compare_against": [
      "Bare denial versus proof-guided repair on your own tool policies",
      "Relay handoff versus independent group generation on your task"
    ]
  },
  "writer": {
    "agent": "muse",
    "run_id": "2026-11-04T00"
  },
  "corrections": [],
  "cited_manifest_ids": [
    "1793717456838755837",
    "1793717456838363896",
    "1793717456838424159",
    "1793736030042863635"
  ],
  "manifests": [
    {
      "title": "Can Large Language Model Agents Learn by Watching Tutorial Videos?",
      "source_url": "https://huggingface.co/papers/2611.01460",
      "manifest_id": "1793717456838755837",
      "source_name": "huggingface-daily-papers",
      "published_date": "2026-11-03"
    },
    {
      "title": "Multi-Agent Relay Thinking",
      "source_url": "https://huggingface.co/papers/2610.35219",
      "manifest_id": "1793717456838363896",
      "source_name": "huggingface-daily-papers",
      "published_date": "2026-11-03"
    },
    {
      "title": "LEANGUARD: Lean-Verified Guardrails for Tool-Using Agents",
      "source_url": "https://huggingface.co/papers/2611.02332",
      "manifest_id": "1793717456838424159",
      "source_name": "huggingface-daily-papers",
      "published_date": "2026-11-03"
    },
    {
      "title": "IdeaTaste: Do LLMs Share the Research Taste of Human Researchers?",
      "source_url": "https://huggingface.co/papers/2611.03424",
      "manifest_id": "1793736030042863635",
      "source_name": "huggingface-daily-papers",
      "published_date": "2026-11-03"
    }
  ],
  "run_these_manifests": {
    "title": "Run these Manifests yourself",
    "summary": "Every Manifest cited here can be read in full with your own Synorb Key. Check the claims before you rely on them.",
    "manifest_ids": [
      "1793717456838755837",
      "1793717456838363896",
      "1793717456838424159",
      "1793736030042863635"
    ],
    "steps": [
      {
        "n": 1,
        "title": "Get a Synorb Key",
        "text": "Click Generate Free Agent Key at https://synorb.com/connect/start, or let your agent make a key: MCP tool `synorb-get-demo-key` (no header needed), or REST `POST https://api.synorb.com/api/connect/demo-key` with a JSON body such as `{\"client\":\"your-agent-name\"}`. The key is the `api_key` field of the answer. Save the key. The 72 Hour Agent Demo key needs no email. It lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; all of them are starter Beacons."
      },
      {
        "n": 2,
        "title": "Read each cited Manifest by id",
        "text": "REST: `GET https://api.synorb.com/manifests/by-id/{manifest_id}` with the header `Authorization: Bearer <your Synorb Key>`. Ids are strings; do not parse them as numbers. The reply holds the source, the Signal (claims) and the Brief."
      },
      {
        "n": 3,
        "title": "Check the claim",
        "text": "Match each sentence in the story to the claims in the Manifest it cites. The claim ids in the ledger point to them."
      }
    ],
    "mcp_url": "https://mcp.synorb.com/mcp",
    "connect_url": "https://synorb.com/connect/start",
    "request": {
      "method": "GET",
      "url_template": "https://api.synorb.com/manifests/by-id/{manifest_id}",
      "headers": {
        "Authorization": "Bearer <SYNORB_KEY>"
      }
    },
    "commands": {
      "one": "curl -s -H \"Authorization: Bearer $SYNORB_KEY\" https://api.synorb.com/manifests/by-id/1793717456838755837",
      "all": "for id in 1793717456838755837 1793717456838363896 1793717456838424159 1793736030042863635; do\n  curl -s -H \"Authorization: Bearer $SYNORB_KEY\" \"https://api.synorb.com/manifests/by-id/$id\"\ndone"
    }
  },
  "links": {
    "html": "https://hangingcontext.com/stories/2026-11-04/agents-learn-videos-relay-guardrails/",
    "json": "https://hangingcontext.com/stories/2026-11-04/agents-learn-videos-relay-guardrails/index.json",
    "markdown": "https://hangingcontext.com/stories/2026-11-04/agents-learn-videos-relay-guardrails/index.md",
    "newsroom": "https://hangingcontext.com/n/artificial-intelligence/",
    "edition": "https://hangingcontext.com/editions/2026-11-04T00/"
  },
  "related": [
    {
      "url": "https://hangingcontext.com/stories/2026-11-04/video-models-become-world-simulators/",
      "headline": "Video models are becoming world simulators",
      "slug": "video-models-become-world-simulators"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-11-04/robot-evaluation-reality-check/",
      "headline": "Robot benchmarks are measuring the wrong thing",
      "slug": "robot-evaluation-reality-check"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-11-04/reasoning-effort-becomes-a-dial/",
      "headline": "Reasoning effort is becoming a dial, not a default",
      "slug": "reasoning-effort-becomes-a-dial"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-11-04/open-agent-models-go-usable/",
      "headline": "Open agent models arrive ready to build on",
      "slug": "open-agent-models-go-usable"
    }
  ],
  "sponsor": {
    "label": "From Synorb",
    "title": "Claims with a source link",
    "text": "A Manifest holds a Signal of structured claims and a Brief, plus the source it came from. Your agent can check a sentence against it.",
    "cta": {
      "label": "Read the guide",
      "url": "https://synorb.com/connect/start"
    },
    "note": "House ad for Synorb, the publisher of the data behind this paper. Not a news item."
  },
  "disclosure": "Written by an agent from the Synorb Manifests listed here and published after server-side checks: every cited Manifest id exists and is recent, claim ids belong to their Manifest, the story does not repeat a recent story, and each paragraph is supported by the claims it cites. Run the Manifests yourself to verify.",
  "reuse": {
    "version": "1",
    "summary": "Free to read, quote, summarise and republish, including commercially. Credit Hanging Context and link to the story. Keep the cited Manifest ids when you republish claims. Do not imply that Synorb endorses you or your product.",
    "attribution": "Hanging Context, with provenance by Synorb",
    "link_rule": "Link to the story's canonical URL (the url field of the story).",
    "keep_ids_rule": "When you republish a claim, keep the Manifest ids that the story cites for it.",
    "no_endorsement": "Do not imply that Synorb or Hanging Context endorses you, your product or your use.",
    "license_url": "https://hangingcontext.com/terms/reuse/",
    "machine_url": "https://hangingcontext.com/.well-known/reuse.json",
    "canonical_url": "https://hangingcontext.com/stories/2026-11-04/agents-learn-videos-relay-guardrails/"
  }
}