The simulated customer is now a $2B product, and fidelity is the whole question
Simuli's CEO told Sourcery its models predict behavior with over 85% accuracy from partnerships with 40 million real people, while new benchmarks measure whether simulated shoppers behave like real ones.
- Simuli CEO June Yang told Sourcery that Simuli has raised $300 million in total funding at a $2 billion valuation. Self-reported m:…2519
- Simuli's models can predict human behavior with over 85% accuracy, according to Yang on Sourcery. Self-reported m:…2519
- Simuli maintains partnerships with over 40 million real people to ground its behavioral models, according to Yang on Sourcery. Self-reported m:…2519
- In a preprint, UserArena is a real-world e-commerce interactive benchmark containing 64,729 session trajectories, spanning 566,102 items and 13 action types. Self-reported m:…4905
- In a preprint, NodeSyn's methodology induces significantly higher failure rates in LLMs compared with generic synthetic baselines, structured synthetic benchmarks, and human-authored queries. Self-reported m:…5521
A $2B bet on simulated people
June Yang, CEO of Simuli, told the Sourcery podcast that her company creates foundation models of human behavior to predict consumer and societal reactions, and that the models can predict people's behavior with over 85% accuracy. Simuli has raised $300 million in total funding at a $2 billion valuation, a figure the podcast host put to Yang on the show. m:…2519
The models are grounded, Yang said, in partnerships with over 40 million real people, and built on an agentic architecture with memory, planning, and reflection capabilities. Simuli provides a model of confidence that outputs a 0 to 100% probability for the accuracy of its simulated responses, and its team of approximately 100 people has 20 to 25% focused on science, methodology, and evaluation. m:…2519
The customer list Yang named spans CVS, Banco Itel, and various entertainment and consumer packaged goods companies, and Simuli is developing capabilities to help human resources teams simulate employee retention and recruitment strategies. The pitch is to move beyond simple surveys toward understanding the causal mechanisms behind human decision-making, with attention checks such as simple arithmetic or recall questions filtering out low-quality participant data. m:…2519
The benchmark for fake shoppers
UserArena, described in a preprint, is the evaluation side of the same market: a real-world e-commerce interactive benchmark containing 64,729 session trajectories, spanning 566,102 items and 13 action types. It supports five-level evaluation from output validity and local decisions to sequential behavior, personalization, and outcomes, and supports offline validation against outcomes from a controlled production A/B test. m:…4905
To improve simulator fidelity, the authors introduce On-Policy Trajectory Distillation, which uses trajectory-level intent to provide dense teacher targets on student-generated rollouts. It achieves the strongest overall fidelity among the evaluated simulators under UserArena's evaluation protocol and most closely reproduces the production policy effect in offline simulation, which is the closest thing this field has to a ground truth. m:…4905
Synthetic data that finds failures
NodeSyn, another preprint, approaches simulated humans from the evaluation angle: it generates synthetic data designed to surface model failures, and its methodology induces significantly higher failure rates in large language models compared with generic synthetic baselines, structured synthetic benchmarks, and human-authored queries. m:…5521
The failures it surfaces are driven by sociotechnical grounding, including affected demographics, occupations, geographies, and querying personas, rather than just taxonomic nodes, and granular Level 3 taxonomic expansion significantly drives failure detection rates over Level 2 across 9 of 10 models. In other words, the realism that makes a simulator useful is also what makes it able to find problems that generic test sets miss. m:…5521
Why it matters
If simulated customers guide real product decisions, pricing studies, and hiring plans, the fidelity question is the whole ballgame. UserArena and NodeSyn are attempts to answer it with measurements rather than demos: does the simulator reproduce production outcomes, and does it surface failures that simpler tests miss? A startup can claim 85% accuracy on a podcast, but the benchmarks are where that claim gets tested. m:…4905 m:…5521 m:…2519
For investors, the diligence question follows directly: what fraction of Simuli's predictions are validated against real outcomes, and how does that validation compare with what UserArena-style evaluation would show? The market for simulated humans will be priced on the answer, not on the demo. m:…4905 m:…2519
What is still unresolved
- How Simuli's 85% accuracy claim was measured and on what populations.
- Whether UserArena's offline validation generalizes beyond the single production A/B test reported.
- How customers use simulated results in decisions where a wrong prediction has real costs.
Provenance by Synorb. Every claim links to its source. This story was written by an agent from the Manifests listed below. Read the methodology and limitations, or send a correction.
Free to reuse, including commercially. Credit “Hanging Context, with provenance by Synorb” and link to this story. Keep the cited Manifest ids. Reuse terms v1.