Before Agents Ship, Teams Are Learning to Measure the Work Itself

Does the code match the spec

Qodo can now follow links from pull requests to tickets and then to specifications, reviewing code against those requirements, with spec sources including Asana, Confluence, Notion, ServiceNow and JSM Assets. The problem it targets is specific: pull requests can pass compilation and tests while still missing critical requirements defined in linked specifications, and the company says its review catches code that functions correctly but fails to deliver the intended behavior or constraints. m:…4358

The security version of that gap is concrete: Qodo flags missing account-level authorization checks as security issues when the specification requires them but the code omits them. The capability also integrates into the Kiro spec-driven workflow, where specifications become plans, tasks and code, so the check sits inside the development loop rather than arriving as a separate audit after the fact. m:…4358

What is each training task worth

Patronus AI built an internal post-training stack to quantify the value of training tasks, automating five stages from training and checkpointing through endpoint deployment, evaluation and leaderboard updates. In experiments using Qwen3.8-27B as the base model, supervised fine-tuning followed by reinforcement learning lifted GitHub task performance from 35.6 percent to 47.8 percent, while reinforcement learning alone reached 45.4 percent. m:…0759

The Unreal Engine results point the same way: fine-tuning plus reinforcement learning moved performance from 64.3 percent to 71.5 percent, while reinforcement learning alone reached 73.1 percent, and the team says reinforcement learning alone performed about as well as the combined approach. The team audited 50 passing runs to verify no reward hacking occurred, and plans a data flywheel to store and analyze trajectories, which suggests the stack is meant to price training effort rather than to celebrate any single run. m:…0759

Watching work in production

Mastra now supports span queries that filter and group individual spans across traces, which are more granular than trace queries because they search within each trace and return matching spans. Queries can combine up to 100 conditions, filter by time range, type, name, status, error and duration, return up to 1,000 spans per page, and aggregate matching spans into measures such as counts, error rates and latency. m:…7325

Google Agents CLI evaluation loop covers the same need from the testing side, with continuous testing and grading designed to catch prompt regressions, side-by-side trace comparisons for model backend swaps, nightly scheduled runs and threshold-based alerting to track quality over time. Taken together, the four releases describe a pipeline: check the work against the spec, price the training task, then watch the traces and alert when quality drifts, instead of discovering drift from users. m:…9566

The decision this creates

The practical decision these tools force is a budget question disguised as an engineering question: which agent work deserves more training, more review, or retirement. Patronus plans a data flywheel to store and analyze trajectories and is exploring partial credit rewards and adaptive task choice to optimize compute, while Google threshold alerting tracks quality over time so drift triggers action rather than anecdote. A team that can price a training task, check code against its spec, and watch production spans has three independent signals before it scales any agent further. m:…0759 m:…9566

Reproducibility is the quieter half of the measurement story: Patronus snapshots code and records git commits for reproducibility and links runs to Weights and Biases pages, while Mastra span queries work on self-hosted projects using DuckDB, PostgreSQL or ClickHouse. Measurement that cannot be rerun or replayed from the same traces will not survive an audit, which is why both teams put the record-keeping next to the metric. m:…0759 m:…7325