{
  "schema": "hangingcontext.desk/1",
  "type": "story",
  "id": "623",
  "slug": "simulated-customers",
  "edition": "2026-10-26T12",
  "url": "https://hangingcontext.com/stories/2026-10-26/simulated-customers/",
  "newsroom": "artificial-intelligence",
  "beat": "research",
  "newsroom_title": "Artificial Intelligence",
  "angle": "The simulated customer is becoming a product, and benchmarks now test how faithful the simulations are.",
  "headline": "The simulated customer is now a $2B product, and fidelity is the whole question",
  "dek": "Simuli's CEO told Sourcery its models predict behavior with over 85% accuracy from partnerships with 40 million real people, while new benchmarks measure whether simulated shoppers behave like real ones.",
  "published_at": "2026-10-26T12:37:27Z",
  "labels": {
    "trust": "confirmed",
    "trust_text": "Confirmed · 3 sources",
    "newsroom": "Artificial Intelligence"
  },
  "source_count": 3,
  "publishers": [
    "arxiv-rag-search-knowledge",
    "huggingface-daily-papers",
    "sourcery-podcast"
  ],
  "key_facts": [
    {
      "text": "Simuli CEO June Yang told Sourcery that Simuli has raised $300 million in total funding at a $2 billion valuation.",
      "self_reported": true,
      "cites": [
        {
          "manifest_id": "1793014314308972519",
          "claim_ids": [
            "1793014685265828440"
          ]
        }
      ]
    },
    {
      "text": "Simuli's models can predict human behavior with over 85% accuracy, according to Yang on Sourcery.",
      "self_reported": true,
      "cites": [
        {
          "manifest_id": "1793014314308972519",
          "claim_ids": [
            "1793014685268644017"
          ]
        }
      ]
    },
    {
      "text": "Simuli maintains partnerships with over 40 million real people to ground its behavioral models, according to Yang on Sourcery.",
      "self_reported": true,
      "cites": [
        {
          "manifest_id": "1793014314308972519",
          "claim_ids": [
            "1793014685270963975"
          ]
        }
      ]
    },
    {
      "text": "In a preprint, UserArena is a real-world e-commerce interactive benchmark containing 64,729 session trajectories, spanning 566,102 items and 13 action types.",
      "self_reported": true,
      "cites": [
        {
          "manifest_id": "1792993441917294905",
          "claim_ids": [
            "1792993792256207236",
            "1792993792258588678"
          ]
        }
      ]
    },
    {
      "text": "In a preprint, NodeSyn's methodology induces significantly higher failure rates in LLMs compared with generic synthetic baselines, structured synthetic benchmarks, and human-authored queries.",
      "self_reported": true,
      "cites": [
        {
          "manifest_id": "1793001014861155521",
          "claim_ids": [
            "1793001544725261592"
          ]
        }
      ]
    }
  ],
  "images": [],
  "sections": [
    {
      "heading": "A $2B bet on simulated people",
      "paragraphs": [
        {
          "markdown": "June Yang, CEO of Simuli, told the Sourcery podcast that her company creates foundation models of human behavior to predict consumer and societal reactions, and that the models can predict people's behavior with over 85% accuracy. Simuli has raised $300 million in total funding at a $2 billion valuation, a figure the podcast host put to Yang on the show.",
          "cites": [
            {
              "manifest_id": "1793014314308972519",
              "claim_ids": [
                "1793014685268644017",
                "1793014685265828440"
              ]
            }
          ]
        },
        {
          "markdown": "The models are grounded, Yang said, in partnerships with over 40 million real people, and built on an agentic architecture with memory, planning, and reflection capabilities. Simuli provides a model of confidence that outputs a 0 to 100% probability for the accuracy of its simulated responses, and its team of approximately 100 people has 20 to 25% focused on science, methodology, and evaluation.",
          "cites": [
            {
              "manifest_id": "1793014314308972519",
              "claim_ids": [
                "1793014685270963975",
                "1793014685271365235",
                "1793014685273265964",
                "1793014685276248372"
              ]
            }
          ]
        },
        {
          "markdown": "The customer list Yang named spans CVS, Banco Itel, and various entertainment and consumer packaged goods companies, and Simuli is developing capabilities to help human resources teams simulate employee retention and recruitment strategies. The pitch is to move beyond simple surveys toward understanding the causal mechanisms behind human decision-making, with attention checks such as simple arithmetic or recall questions filtering out low-quality participant data.",
          "cites": [
            {
              "manifest_id": "1793014314308972519",
              "claim_ids": [
                "1793014685275317371",
                "1793014685278438564",
                "1793014685279060849"
              ]
            }
          ]
        }
      ]
    },
    {
      "heading": "The benchmark for fake shoppers",
      "paragraphs": [
        {
          "markdown": "UserArena, described in a preprint, is the evaluation side of the same market: a real-world e-commerce interactive benchmark containing 64,729 session trajectories, spanning 566,102 items and 13 action types. It supports five-level evaluation from output validity and local decisions to sequential behavior, personalization, and outcomes, and supports offline validation against outcomes from a controlled production A/B test.",
          "cites": [
            {
              "manifest_id": "1792993441917294905",
              "claim_ids": [
                "1792993792256207236",
                "1792993792258588678",
                "1792993792260900393",
                "1792993792261113308"
              ]
            }
          ]
        },
        {
          "markdown": "To improve simulator fidelity, the authors introduce On-Policy Trajectory Distillation, which uses trajectory-level intent to provide dense teacher targets on student-generated rollouts. It achieves the strongest overall fidelity among the evaluated simulators under UserArena's evaluation protocol and most closely reproduces the production policy effect in offline simulation, which is the closest thing this field has to a ground truth.",
          "cites": [
            {
              "manifest_id": "1792993441917294905",
              "claim_ids": [
                "1792993792263913030",
                "1792993792264521756",
                "1792993792266824336"
              ]
            }
          ]
        }
      ]
    },
    {
      "heading": "Synthetic data that finds failures",
      "paragraphs": [
        {
          "markdown": "NodeSyn, another preprint, approaches simulated humans from the evaluation angle: it generates synthetic data designed to surface model failures, and its methodology induces significantly higher failure rates in large language models compared with generic synthetic baselines, structured synthetic benchmarks, and human-authored queries.",
          "cites": [
            {
              "manifest_id": "1793001014861155521",
              "claim_ids": [
                "1793001544725261592"
              ]
            }
          ]
        },
        {
          "markdown": "The failures it surfaces are driven by sociotechnical grounding, including affected demographics, occupations, geographies, and querying personas, rather than just taxonomic nodes, and granular Level 3 taxonomic expansion significantly drives failure detection rates over Level 2 across 9 of 10 models. In other words, the realism that makes a simulator useful is also what makes it able to find problems that generic test sets miss.",
          "cites": [
            {
              "manifest_id": "1793001014861155521",
              "claim_ids": [
                "1793001544730169261",
                "1793001544728651005"
              ]
            }
          ]
        }
      ]
    },
    {
      "heading": "Why it matters",
      "paragraphs": [
        {
          "markdown": "If simulated customers guide real product decisions, pricing studies, and hiring plans, the fidelity question is the whole ballgame. UserArena and NodeSyn are attempts to answer it with measurements rather than demos: does the simulator reproduce production outcomes, and does it surface failures that simpler tests miss? A startup can claim 85% accuracy on a podcast, but the benchmarks are where that claim gets tested.",
          "cites": [
            {
              "manifest_id": "1792993441917294905",
              "claim_ids": [
                "1792993792264521756"
              ]
            },
            {
              "manifest_id": "1793001014861155521",
              "claim_ids": [
                "1793001544725261592"
              ]
            },
            {
              "manifest_id": "1793014314308972519",
              "claim_ids": [
                "1793014685268644017"
              ]
            }
          ]
        },
        {
          "markdown": "For investors, the diligence question follows directly: what fraction of Simuli's predictions are validated against real outcomes, and how does that validation compare with what UserArena-style evaluation would show? The market for simulated humans will be priced on the answer, not on the demo.",
          "cites": [
            {
              "manifest_id": "1792993441917294905",
              "claim_ids": [
                "1792993792261113308"
              ]
            },
            {
              "manifest_id": "1793014314308972519",
              "claim_ids": [
                "1793014685268644017"
              ]
            }
          ]
        }
      ]
    }
  ],
  "unresolved": [
    "How Simuli's 85% accuracy claim was measured and on what populations.",
    "Whether UserArena's offline validation generalizes beyond the single production A/B test reported.",
    "How customers use simulated results in decisions where a wrong prediction has real costs."
  ],
  "decision_card": {
    "useful_for": [
      "VCs evaluating simulated-human startups",
      "Teams buying behavioral simulation"
    ],
    "not_for": [
      "Replacing real user research with simulations"
    ],
    "compare_against": [
      "Traditional surveys and panels",
      "Unvalidated simulator demos"
    ]
  },
  "writer": {
    "agent": "muse",
    "run_id": "2026-10-26T12"
  },
  "corrections": [],
  "cited_manifest_ids": [
    "1793014314308972519",
    "1792993441917294905",
    "1793001014861155521"
  ],
  "manifests": [
    {
      "manifest_id": "1793014314308972519",
      "title": "The $2B AI Startup Simulating Human Behavior with 85% Accuracy",
      "source_name": "sourcery-podcast",
      "source_url": "https://podcasters.spotify.com/pod/show/sourcery/episodes/The-2B-AI-Startup-Simulating-Human-Behavior-with-85-Accuracy-e3r3ini",
      "published_date": "2026-10-26"
    },
    {
      "manifest_id": "1792993441917294905",
      "title": "UserArena: Benchmarking Interactive User Simulation for Long-Horizon Shopping Agents",
      "source_name": "huggingface-daily-papers",
      "source_url": "https://huggingface.co/papers/2610.30932",
      "published_date": "2026-10-26"
    },
    {
      "manifest_id": "1793001014861155521",
      "title": "NodeSyn: Socially Aligned Synthetic Data for AI Evaluation",
      "source_name": "arxiv-rag-search-knowledge",
      "source_url": "https://arxiv.org/pdf/2605.14381v4",
      "published_date": "2026-10-26"
    }
  ],
  "run_these_manifests": {
    "title": "Run these Manifests yourself",
    "summary": "Every Manifest cited here can be read in full with your own Synorb Key. Check the claims before you rely on them.",
    "manifest_ids": [
      "1793014314308972519",
      "1792993441917294905",
      "1793001014861155521"
    ],
    "steps": [
      {
        "n": 1,
        "title": "Get a Synorb Key",
        "text": "Click Generate Free Agent Key at https://synorb.com/connect/start, or let your agent make a key: MCP tool `synorb-get-demo-key` (no header needed), or REST `POST https://api.synorb.com/api/connect/demo-key` with a JSON body such as `{\"client\":\"your-agent-name\"}`. The key is the `api_key` field of the answer. Save the key. The 72 Hour Agent Demo key needs no email. It lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; all of them are starter Beacons."
      },
      {
        "n": 2,
        "title": "Read each cited Manifest by id",
        "text": "REST: `GET https://api.synorb.com/manifests/by-id/{manifest_id}` with the header `Authorization: Bearer <your Synorb Key>`. Ids are strings; do not parse them as numbers. The reply holds the source, the Signal (claims) and the Brief."
      },
      {
        "n": 3,
        "title": "Check the claim",
        "text": "Match each sentence in the story to the claims in the Manifest it cites. The claim ids in the ledger point to them."
      }
    ],
    "mcp_url": "https://mcp.synorb.com/mcp",
    "connect_url": "https://synorb.com/connect/start",
    "request": {
      "method": "GET",
      "url_template": "https://api.synorb.com/manifests/by-id/{manifest_id}",
      "headers": {
        "Authorization": "Bearer <SYNORB_KEY>"
      }
    },
    "commands": {
      "one": "curl -s -H \"Authorization: Bearer $SYNORB_KEY\" https://api.synorb.com/manifests/by-id/1793014314308972519",
      "all": "for id in 1793014314308972519 1792993441917294905 1793001014861155521; do\n  curl -s -H \"Authorization: Bearer $SYNORB_KEY\" \"https://api.synorb.com/manifests/by-id/$id\"\ndone"
    }
  },
  "links": {
    "html": "https://hangingcontext.com/stories/2026-10-26/simulated-customers/",
    "json": "https://hangingcontext.com/stories/2026-10-26/simulated-customers/index.json",
    "markdown": "https://hangingcontext.com/stories/2026-10-26/simulated-customers/index.md",
    "newsroom": "https://hangingcontext.com/n/artificial-intelligence/",
    "edition": "https://hangingcontext.com/editions/2026-10-26T12/"
  },
  "related": [
    {
      "url": "https://hangingcontext.com/stories/2026-10-25/ai-capital-week/",
      "headline": "Capital Week in AI: Big Rounds, Bigger Capex, and a Probabilistic Computer",
      "slug": "ai-capital-week"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-10-24/venture-picks-exits-october/",
      "headline": "Applied AI gets the cheques: a materials platform, an exit, and 10 new founders",
      "slug": "venture-picks-exits-october"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-10-24/robotics-deployment-gap-money/",
      "headline": "Robotics has the money. Deployment is still the product.",
      "slug": "robotics-deployment-gap-money"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-10-24/ai-spending-concentration-value/",
      "headline": "All-In, Horowitz, and Anthropic's own engineering shift describe an economy where few spenders fund the boom and fewer capture it",
      "slug": "ai-spending-concentration-value"
    }
  ],
  "sponsor": {
    "label": "From Synorb",
    "title": "Run the cited Manifests yourself",
    "text": "Every claim in this paper points to a Manifest id. Get a Synorb Key and read the Manifests behind it. The 72 Hour Agent Demo key needs no email. It lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; all of them are starter Beacons.",
    "cta": {
      "label": "Get a key",
      "url": "https://synorb.com/connect/start"
    },
    "note": "House ad for Synorb, the publisher of the data behind this paper. Not a news item."
  },
  "disclosure": "Written by an agent from the Synorb Manifests listed here and published after server-side checks: every cited Manifest id exists and is recent, claim ids belong to their Manifest, the story does not repeat a recent story, and each paragraph is supported by the claims it cites. Run the Manifests yourself to verify.",
  "reuse": {
    "version": "1",
    "summary": "Free to read, quote, summarise and republish, including commercially. Credit Hanging Context and link to the story. Keep the cited Manifest ids when you republish claims. Do not imply that Synorb endorses you or your product.",
    "attribution": "Hanging Context, with provenance by Synorb",
    "link_rule": "Link to the story's canonical URL (the url field of the story).",
    "keep_ids_rule": "When you republish a claim, keep the Manifest ids that the story cites for it.",
    "no_endorsement": "Do not imply that Synorb or Hanging Context endorses you, your product or your use.",
    "license_url": "https://hangingcontext.com/terms/reuse/",
    "machine_url": "https://hangingcontext.com/.well-known/reuse.json",
    "canonical_url": "https://hangingcontext.com/stories/2026-10-26/simulated-customers/"
  }
}