The new tests grade agents on real work — and the scores are humbling

Enterprise work

Collinear AI's FrontierCUA is a benchmark for computer-use agents in enterprise environments, and it is deliberately hard. It tests agents on 40 multi-app workflows across 11 enterprise domains, using high-fidelity clones of commercial software such as Zendesk and HubSpot, with every workflow spanning at least two applications and 16 of the 40 requiring interaction with four or five apps. The best-performing model, GPT-6.1 Sol, achieves a 25% pass rate at a cost of approximately $6 per rollout, and 23 of the 40 tasks are solved by no model at all. The median task runs 374 steps over 36 minutes. m:…1031

The failure modes are the useful part. For GPT-6.1 Sol, inadequate evidence gathering accounts for 39% of lost rubric points, reasoning errors for 30%, and stopping before finishing for 24%. Interface errors are 5% and arithmetic slips 2%. The benchmark's message is that the bottleneck is not clicking buttons. It is gathering the evidence, reasoning over it, and finishing the job. That is the state of the art for enterprise computer use, priced per attempt: the leading model passes a quarter of the workflows at about $6 a rollout. m:…1031

Science, graded honestly

OpenFrontiers, a new cross-disciplinary resource for evaluating LLM agents on research-level problem solving, combines a corpus of 12,151 open research problems with a benchmark of 200 recently resolved mathematics problems, of which 110 were resolved in 2026 and 90 are still open. Seven LLM agents were evaluated. The headline finding is humbling: closed-book rediscovery of solutions is rare, occurring in at most 7 of 40 resolved problems, and live-web success in the benchmark mostly reflects the retrieval of already published solutions rather than genuine problem-solving. Each problem has a Lean 4 statement checked by automated probes and two independent reviews, and grading uses two independent checkers in an isolated sandbox. m:…7412

A second benchmark, TB-MathModel, makes the same point for mathematical modeling. It uses 18 real-world contest problems from 2023 to 2025, with a two-layer feasibility verification system including 624 kernel-checked Lean 4 obligations. GPT-6 Astra achieved a 64.58% accuracy score, Fable 5.1 reached 63.1%, and a significantly cheaper open model reached 59.81% at one hundredth the cost of top models. As the benchmark's authors put it: "Reporting a number is not enough; it has to survive the physics." The gap between the best model and the cheapest one is small, which is itself a finding: on verifiable work, the frontier premium is thin. m:…7409

Grading without running

A preprint this week offers a way to grade agents without the cost and risk of running them live. ADWM, an autoregressive diffusion world model, enables off-policy evaluation of LLM agents using pre-collected data. It models each transition as an independent denoising process, which allows step-by-step rollouts, and the agent under evaluation directly guides the diffusion generation at each step through a policy-conditioned score function. The authors report accurate value estimates and evaluation reliability across diverse multi-turn agent tasks. A related framework, SwarmEvolve, uses swarms of hundreds of small latent-reasoning agents that share information by passing KV caches rather than decoding to text, reaching best scores in a median of 3 to 8 minutes against 60 minutes for other state-of-the-art frameworks. m:…9646 m:…7410

The pattern across the four pieces is a maturing evaluation culture. The tests are getting closer to real work: multi-app enterprise workflows, open research problems, contest modeling with machine-checked proofs. And the scores are getting more honest: 25% on enterprise workflows, rare closed-book rediscovery, a thin gap between the most expensive model and the cheapest one. The yardstick is not failing. It is finally measuring the right thing. m:…1031 m:…7412 m:…7409