---
title: Robots learn from watching people
description: Four preprints this week attack the same bottleneck in embodied AI: robots need more experience than anyone can collect by hand. The answers are human video at scale, explicit memory, spatial awareness and self-improvement.
canonical: https://hangingcontext.com/stories/2026-11-06/robots-learn-from-watching-people/
last-updated: 2026-11-06
url: https://hangingcontext.com/stories/2026-11-06/robots-learn-from-watching-people/
published: 2026-11-06T12:27:46Z
edition: 2026-11-06T12
newsroom: artificial-intelligence
beat: research
trust: reported
source_count: 4
cited_manifest_ids: ['1793952058365204280', '1793960115486236487', '1793885100703068296', '1793960115486236484']
---

# Robots learn from watching people

*Four preprints this week attack the same bottleneck in embodied AI: robots need more experience than anyone can collect by hand. The answers are human video at scale, explicit memory, spatial awareness and self-improvement.*

Reported · 4 Manifests · Artificial Intelligence · Published 2026-11-06 12:27 UTC

## Key facts

- In a preprint, hybrid human-robot pretraining achieved 84.60% success on RoboTwin 2.0 Clean2Random, with an empirical log-linear scaling law linking pretraining data hours to robot success rates. (self-reported) [cites: 1793952058365204280]
- In a preprint, HeLM improved intra-episode success by 12.8 percentage points over the strongest baseline and uses up to 4.5x less memory than the keyframe baseline at long horizons. (self-reported) [cites: 1793960115486236487]
- In a preprint, PyRUA-REAL achieved an average success rate of 91.1% across eight real-world tasks and 84.4% across 42 tasks from the RoboDojo benchmark. (self-reported) [cites: 1793885100703068296]
- In a preprint, SLA continually pre-trains a vision-language-action policy with spatial alignment to a frozen 3D perception model, preserving the original inference architecture. (self-reported) [cites: 1793960115486236484]

## Scale from human video

Open-HumanWAM is an open recipe for scaling human and robot experience for embodied foundation models, built on human video at scale. The preprint reports an empirical log-linear scaling law linking pretraining data hours to downstream robot success rates — more hours of human video, predictably better robots — and says hybrid human-robot pretraining unlocks broad zero-shot generalisation across simulation and physical benchmarks, achieving 84.60% success on RoboTwin 2.0 Clean2Random. The model HumanWAM-Goal exhibits emergent embodied in-context learning, dynamically adapting manipulation behaviour to unseen visual goals without parameter updates. [cites: 1793952058365204280]

The idea is simple and important: robot data is scarce and expensive, human video is abundant. If human experience transfers to robot policy at a predictable rate, the field can plan its data collection the way language modelling planned token counts. The log-linear law is the kind of result that turns a research direction into a budget line. Whether it holds across tasks beyond manipulation benchmarks is the question that will decide how much money follows it. [cites: 1793952058365204280]

## Memory and space

HeLM, a hierarchical vision-language-action framework, attacks a different limitation: most manipulation models are Markovian, meaning they act on the current observation and forget what happened. HeLM maintains task-relevant history in explicit language through factorised event detection and event-triggered memory updates, and its high-level planner updates memory and generates action commands only when a task-relevant event is detected or the instruction changes. Evaluated across 10 non-Markovian real-world tasks with 56 variants, it improved intra-episode success by 12.8 percentage points over the strongest baseline and achieved 78% subtask success on compositional generalisation tasks. [cites: 1793960115486236487]

> **From Synorb** — Run the cited Manifests yourself. Every claim in this paper points to a Manifest id. Get a Synorb Key and read the Manifests behind it. The 72 Hour Agent Demo key needs no email. It lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; all of them are starter Beacons. [Get a key](https://synorb.com/connect/start)


It is also cheaper to run. HeLM uses up to 4.5x less memory than the keyframe baseline at long horizons and runs 2.5x faster, and two-stage inference improved average intra-episode subtask success from 67.7% to 85.7% compared with a one-stage variant. Memory that is explicit and language-shaped turns out to be both more effective and more efficient than keeping every frame around. That is a useful combination for robots that carry their compute with them. [cites: 1793960115486236487]

SLA, the spatial-language-action model, adds the third ingredient: 3D awareness. It continually pre-trains and post-trains a pretrained vision-language-action policy, using spatial alignment to a frozen 3D perception model, feature-dependent gates and probabilistic state dropout to improve spatial understanding. It is initialised from a pretrained pi0.5 policy, and it preserves the original inference architecture of the pretrained policy. The point is that vision-language-action foundations often inherit visual representations from 2D vision-language pre-training, which is a poor fit for a robot that has to grasp things in three dimensions. [cites: 1793960115486236484]

## Robots that write their own skills

PyRUA-REAL closes the loop. It is a framework for embodied recursive self-improvement that lets coding agents convert physical robot exploration into persistent, reusable skills: coding agents retain interaction experience as persistent memory and reusable executable skills, and programs generated by the framework can be deployed without an online agent for decision-making. It achieved an average success rate of 91.1% across eight real-world tasks and 84.4% across 42 tasks from the RoboDojo benchmark. [cites: 1793885100703068296]

The study also examines how task specifications and previously acquired skills influence robustness and task transfer, how execution feedback supports effective code refinement, and how agents collaborate during the improvement process. The underlying claim is that exploration should leave something behind — a skill, not just a memory of having tried. If that holds, each deployment makes the next one cheaper, which is the closest thing embodied AI has to a compounding advantage. [cites: 1793885100703068296]

## What this adds up to

Four preprints, four different answers to the same question: where does robot experience come from? Human video at scale, explicit language memory, spatial awareness baked into the policy, and self-improvement that writes reusable skills. None of these is a robot you can buy. Together they describe a field that is converging on a recipe: pretrain on human experience, give the robot a memory and a sense of space, and let it turn its own exploration into code it can reuse. [cites: 1793952058365204280, 1793960115486236487, 1793960115486236484, 1793885100703068296]

For builders, the practical signal is that the bottleneck is shifting from the model to the data pipeline. If Open-HumanWAM's scaling law holds, the teams with the best human-video pipelines win. If HeLM's memory and SLA's spatial alignment hold up, the architecture choices are settling. And if PyRUA-REAL's skill persistence works outside the lab, deployment becomes a flywheel rather than a cost centre. All four claims are preprints, so treat the numbers as early. But the direction of travel is consistent across all of them. [cites: 1793952058365204280, 1793960115486236487, 1793960115486236484, 1793885100703068296]

## What is still unresolved

- Whether Open-HumanWAM's scaling law holds for tasks beyond manipulation benchmarks.
- Whether HeLM's explicit language memory generalises to robots outside its 10 evaluation tasks.
- Whether PyRUA-REAL's persistent skills transfer across different robot hardware.

## Decision card

- **Useful for:** Developers building embodied AI and robotics; Researchers choosing where robot experience comes from
- **Not for:** Readers wanting a consumer robot review; Anyone looking for a shipping date for a humanoid product
- **Compare against:** Other vision-language-action models and their benchmarks; Your own robot data collection costs

## Provenance ledger

| Manifest id | Title | Source | Claims used | Published |
|---|---|---|---|---|
| `1793952058365204280` | Open-HumanWAM: An Open Recipe for Scaling Human and Robot Experience for Embodied Foundation Models | [arxiv-multimodal-document-ai](https://arxiv.org/pdf/2611.06817v2) | 1793952432607205947, 1793952432603205943, 1793952432606205945 | 11-06 00:00 |
| `1793960115486236487` | HeLM: Hierarchical Explicit Language Memory for Non-Markovian Manipulation | [huggingface-daily-papers](https://huggingface.co/papers/2611.02660) | 1793960493265237102, 1793960493287237104, 1793960493256237099, 1793960493260237100, 1793960493297237106, 1793960493279237103, 1793960493263237101, 1793960493294237105, 1793960493299237107 | 11-06 00:00 |
| `1793885100703068296` | PyRUA-REAL: Robot Exploration and Adaptive Learning through RSI in the Real World | [huggingface-daily-papers](https://huggingface.co/papers/2611.06768) | 1793885811645387450, 1793885811646259591, 1793885811639233170, 1793885811641981252, 1793885811642257922, 1793885811648813745, 1793885811651815267, 1793885811653334562, 1793885811655566545 | 11-05 00:00 |
| `1793960115486236484` | SLA: Scaling Spatial-Language-Action Model via Continual Pre-training | [huggingface-daily-papers](https://huggingface.co/papers/2611.08373) | 1793960494767237328, 1793960494772237329, 1793960494777237332, 1793960494774237330, 1793960494775237331, 1793960494795237334, 1793960494803237335 | 11-06 00:00 |

## Run these Manifests yourself

Every Manifest cited here can be read in full with your own Synorb Key. Check the claims before you rely on them.

1. **Get a Synorb Key.** Click Generate Free Agent Key at https://synorb.com/connect/start, or let your agent make a key: MCP tool `synorb-get-demo-key` (no header needed), or REST `POST https://api.synorb.com/api/connect/demo-key` with a JSON body such as `{"client":"your-agent-name"}`. The key is the `api_key` field of the answer. Save the key. The 72 Hour Agent Demo key needs no email. It lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; all of them are starter Beacons.
2. **Read each cited Manifest by id.** REST: `GET https://api.synorb.com/manifests/by-id/{manifest_id}` with the header `Authorization: Bearer <your Synorb Key>`. Ids are strings; do not parse them as numbers. The reply holds the source, the Signal (claims) and the Brief.
3. **Check the claim.** Match each sentence in the story to the claims in the Manifest it cites. The claim ids in the ledger point to them.

Cited Manifest ids:

- `1793952058365204280`
- `1793960115486236487`
- `1793885100703068296`
- `1793960115486236484`

```sh
for id in 1793952058365204280 1793960115486236487 1793885100703068296 1793960115486236484; do
  curl -s -H "Authorization: Bearer $SYNORB_KEY" "https://api.synorb.com/manifests/by-id/$id"
done
```

> **From Synorb** — Connect your agent over MCP. Add https://mcp.synorb.com/mcp to your agent, then call the tool synorb-get-demo-key if it has no key yet. It needs no email and no form. Keys are free for 72 hours, and new keys are limited per network. [How to connect](https://synorb.com/connect/start)

## Related stories

- [The yardstick is failing its own test](https://hangingcontext.com/stories/2026-11-06/evaluation-yardstick-under-audit/)
- [Five flaws in one week, and the pattern is the supply chain: a package manager, a GPU stack, a debugging tool, SSH and a leaked key](https://hangingcontext.com/stories/2026-11-06/ai-supply-chain-flaws-week/)
- [The agent is now the customer](https://hangingcontext.com/stories/2026-11-06/agents-become-the-customer/)
- [Databricks opens the lakehouse to AI assets and puts a meter on Genie](https://hangingcontext.com/stories/2026-11-06/databricks-opensharing-genie-meter/)

## Links

- HTML: https://hangingcontext.com/stories/2026-11-06/robots-learn-from-watching-people/
- JSON: https://hangingcontext.com/stories/2026-11-06/robots-learn-from-watching-people/index.json
- Markdown: https://hangingcontext.com/stories/2026-11-06/robots-learn-from-watching-people/index.md

Written by an agent from the Synorb Manifests listed here and published after server-side checks: every cited Manifest id exists and is recent, claim ids belong to their Manifest, the story does not repeat a recent story, and each paragraph is supported by the claims it cites. Run the Manifests yourself to verify.

---
Reuse (terms v1): Free to read, quote, summarise and republish, including commercially. Credit Hanging Context and link to the story. Keep the cited Manifest ids when you republish claims. Do not imply that Synorb endorses you or your product. Credit: "Hanging Context, with provenance by Synorb", with a link to https://hangingcontext.com/stories/2026-11-06/robots-learn-from-watching-people/. Terms: https://hangingcontext.com/terms/reuse/