Your coding agent passes the tests. It still did not finish the job

Mostly right is not done

Co-Design Bench, a benchmark published on 26 Oct 2026 and created by researchers at Alibaba and five universities, measures something most coding benchmarks skip: whether the agent actually finished the task, not just whether it passed some checks. It consists of 273 tasks and 2,331 hidden acceptance criteria, curated from 37,463 deduplicated queries across 26,314 real user coding sessions. The best-performing model, GPT-6 Astra, passed 69.9% of criteria per task on average, but fully finished only 26.4% of tasks. In 73.6% of tasks, at least one criterion stayed unmet. Opus 5 fully finished 18.3% of tasks, and even the lowest-scoring model passed 51.1% of criteria. m:…7246

The gap matters because of how code review works. Diff-based reviews show what changed, so a requirement the agent quietly skipped is easy to miss, and incomplete code gets merged looking complete. Astra finished 41.0% of 100 multimodal editing tasks, which is better than its overall rate but still leaves most tasks partly undone. The benchmark also exposes how fragile rankings are: swapping harnesses moved Qwen3.8-27B between 56.0% and 74.9% on repository edits, so a leaderboard number says as much about the harness as about the model. m:…7246

Debugging help helps, until it misleads

A preprint on agentic debugging of distributed systems, DDBench, tests a different part of the same problem: finding the bug in the first place. Distributed debugging exercises a reasoning dimension that single-process benchmarks do not surface, and bounded debugging context lifts the aggregate pass rate by 19.0 percentage points. That is a real gain, and it comes with a warning attached: even faithful debugging context can anchor a strong model on the wrong layer of the system when its observation sits far from the fault. More context is not the same as the right context, and context that looks helpful can lock the agent into a wrong theory. m:…6712

The practical read is that debugging agents need curation, not volume. A team that dumps logs and traces into the prompt and hopes the model sorts it out is as likely to get a confident wrong answer as a correct one. The benchmark's point is that distributed bugs are a distinct skill, and a model that is strong on single-file fixes can still be weak when the fault lives across services. m:…6712

Writing more tests does not fix it

Another preprint this week asks whether agent-generated tests help at all, and the answer is uncomfortable for teams that treat test-writing as a safety net. The authors found that test-writing frequency is model-dependent and does not correlate with task resolution success. Agent-generated tests prioritize printing runtime values over logical assertions, which means they often confirm the code ran rather than confirming it is right. Forcing test generation significantly increases API and token usage without improving resolution outcomes, so the habit costs money and buys little. m:…6717

A related preprint on patch correctness makes the same point from the evaluation side. Reported detection rates in automated program repair and coding agents are significantly altered by changes in candidate populations, oracle rules, or execution budgets, and repository-level coding agents require distinct evaluation protocols that separate issue reproduction from candidate-patch discrimination. In plain terms, a passing test suite is a weak claim about a patch, and the number moves when the rules of the test move. m:…6710

The coverage gap underneath

Longtail-SWE, a preprint benchmark of 202 issue-resolution tasks, shows where the measurement itself is thinnest. It covers 13 programming languages with limited coverage in existing repository-level benchmarks: Clojure, Dart, Elixir, Erlang, F#, Haskell, Julia, Kotlin, Lua, OCaml, R, Scala, and Swift. The authors built it with a solver-judge-repairer pipeline, evaluated nine model configurations including frontier models, and released executable environments, construction and evaluation tooling, and agent trajectories, plus an easier set of tasks for smaller models. If your stack lives outside the mainstream languages, the headline benchmark numbers were never really about you. m:…1889

Taken together, the week points to one decision for teams shipping with coding agents. Completion, not pass rate, is the number to track, and it should be measured against hidden acceptance criteria the agent never saw, on your own repositories, in your own languages. Tests the agent wrote itself, and reviews that only show the diff, will both flatter a result that is not finished. m:…7246