---
title: The Harness Is the Product: Why Agent Design Starts Outside the Model
description: In preprints published October 27 and 28, researchers show that changing an agent harness changes both its ceiling and its mismatch, that independent critic review is the cheapest accuracy win, and that harness-plus-model learning beats either alone.
canonical: https://hangingcontext.com/stories/2026-10-28/harness-geometry-agent-design/
last-updated: 2026-10-28
url: https://hangingcontext.com/stories/2026-10-28/harness-geometry-agent-design/
published: 2026-10-28T12:32:52Z
edition: 2026-10-28T12
newsroom: artificial-intelligence
beat: developers
trust: reported
source_count: 3
cited_manifest_ids: ['1793178278804674192', '1793141286368601468', '1793159799988679564']
---

# The Harness Is the Product: Why Agent Design Starts Outside the Model

*In preprints published October 27 and 28, researchers show that changing an agent harness changes both its ceiling and its mismatch, that independent critic review is the cheapest accuracy win, and that harness-plus-model learning beats either alone.*

Reported · 3 sources · Artificial Intelligence · Published 2026-10-28 12:32 UTC

## Key facts

- In a preprint, deployed performance equals harness potential minus policy mismatch, and changing a harness moves both terms, so comparing harnesses with a fixed model mixes what each offers with how well the model uses it. (self-reported) [cites: 1793178278804674192]
- In a preprint, reversing the order of observation fields in MiniWoB++ lowered a trained model success rate from 85.0 percent to 19.2 percent. (self-reported) [cites: 1793178278804674192]
- In a preprint, independent critic review improved average accuracy by 5.9 to 9.5 percentage points on Terminal-Bench and 3.9 to 6.6 points on SWE-bench Verified, at 1.13 to 1.25 times baseline cost. (self-reported) [cites: 1793141286368601468]
- In a preprint, the HaMo framework combined non-parametric experience cards with parametric adapter updates and outperformed harness-only and model-only adaptation on within-domain and cross-domain shifts. (self-reported) [cites: 1793159799988679564]

## A model learns its harness

In a preprint, researchers argue that language-model agents learn preferences specific to the prompts and tools used during training, a phenomenon the authors call harness preferences. Under KL-regularized control, a policy encodes its harness preferences, and deployed performance equals harness potential minus policy mismatch. Changing a harness moves both terms, which means comparing harnesses with a fixed model mixes what each harness offers with how well that model uses it. For coding-agent harnesses on Terminal-Bench Lite, the authors report that ceiling minus mismatch reproduces the accuracy ranking, a result that gives the decomposition empirical weight rather than leaving it as a slogan. [cites: 1793178278804674192]

The most vivid demonstration in the preprint is a small change with a large effect. In MiniWoB++, reversing the order of observation fields lowered a trained model success rate from 85.0 percent to 19.2 percent. That is not a story about a weaker model; it is a story about a policy that had learned to rely on the layout of its interface. The preprint also reports that new harnesses that raise the ceiling outperform the original agent after further training, and that ceiling and mismatch measured beforehand predict performance gains. The implication for builders is direct: a harness change is a retraining event, not a configuration tweak, and its payoff should be forecast before it is shipped. [cites: 1793178278804674192]

## Verification that pays

In a preprint, researchers evaluated 17 verification designs for agent harnesses across seven frontier coding models using Terminal-Bench, SWE-bench Verified, and VisualBench. Independent critic review improved average accuracy by 5.9 to 9.5 percentage points on Terminal-Bench and by 3.9 to 6.6 percentage points on SWE-bench Verified, while costing 1.13 to 1.25 times the baseline cost on Terminal-Bench. Independent review also achieved net preferences of 13.2 to 23.8 points over the baseline on VisualBench. Completion gates could yield larger accuracy gains but might substantially increase cost, while increasing review rounds or the number of critics improved average Terminal-Bench accuracy by up to 3.5 percentage points. [cites: 1793141286368601468]

> **From Synorb** — Ask what Synorb covers. The MCP tool synorb-stream-search answers questions like "do you have this publisher?" without using any Manifest allowance. [Connect and ask](https://synorb.com/connect/start)


Initial instructions and execution-time reminders yielded only modest, model-dependent average gains on coding benchmarks in the same study, which sharpens the contrast. The cheapest reliable win the researchers found was not more prompting but an independent second look, structured as a critic review rather than as a gate that blocks completion. That finding fits the harness-geometry argument: verification is part of the harness, and its design changes both the ceiling of what the system can catch and the cost of catching it, so it should be chosen with the same care as the model itself. [cites: 1793141286368601468]

## Learning on both levels

In a preprint, researchers introduce HaMo, a framework for long-horizon continual learning in web agents that works on two levels at once. The harness level distills trajectories into compact experience cards stored in persistent non-parametric memory for immediate retrieval, and a retrieval-coverage-based selection mechanism identifies trajectories for parameter updates. The model level updates a parameter-efficient adapter using selected trajectories and historical replay, under an experience-conditioned continual learning objective. HaMo combines non-parametric experience accumulation with parametric model adaptation, and the authors report that it outperforms harness-only adaptation and model-only adaptation in within-domain and cross-domain shifts. [cites: 1793159799988679564]

HaMo also improved retention of previously acquired capabilities and adaptation to new tasks, according to the preprint. That combination matters because long-horizon agents face two failure modes at once: forgetting what they knew, and failing to transfer what they learned to a new task. A harness-only system can retrieve old experience but cannot internalise it, while a model-only system can internalise but cannot retrieve on demand. HaMo treats the two as complements, and its reported gains on both retention and adaptation suggest the split is where the engineering leverage sits. [cites: 1793159799988679564]

## Design rules that follow

Read together, the three preprints give agent teams a short set of design rules. First, measure harness potential and policy mismatch separately before changing an interface, because a change that raises the ceiling can still hurt if it increases mismatch more. Second, put an independent critic review into the harness before adding more prompting, because the verification study shows it is the most favourable accuracy-cost trade available in that comparison. Third, for agents that must keep learning, split the work between retrievable experience cards and adapter updates, rather than betting on one level alone. Each rule follows from a measured result, not from a preference for one architecture over another. [cites: 1793178278804674192, 1793141286368601468, 1793159799988679564]

## What is still unresolved

- Whether ceiling-minus-mismatch rankings hold for agent tasks beyond the coding and web benchmarks tested
- How critic review scales to tasks where verification itself is expensive or ambiguous
- Whether HaMo gains persist over much longer horizons than those tested

## Decision card

- **Useful for:** Agent platform teams designing harnesses, verification, and continual learning; Coding-agent builders choosing between prompting and critic review
- **Not for:** A ranking of commercial agent products, which the preprints do not provide; A claim that any single harness change will help without retraining
- **Compare against:** Harness changes evaluated by model score alone; Verification by prompting or completion gates alone

## Provenance ledger

| Manifest id | Title | Source | Claims used | Published |
|---|---|---|---|---|
| `1793178278804674192` | Your Policy Is Secretly a Harness: The Decision Geometry of Agent Interfaces | [huggingface-daily-papers](https://huggingface.co/papers/2610.37255) | 1793178607206428110, 1793178607210841553, 1793178607208280463, 1793178607211190113, 1793178607203420037, 1793178607217028781, 1793178607213863758, 1793178607216837369 | 10-28 00:00 |
| `1793141286368601468` | Rethinking Verification in Agent Harnesses | [huggingface-daily-papers](https://huggingface.co/papers/2610.34577) | 1793141629381786986, 1793141629384292785, 1793141629386367441, 1793141629394393412, 1793141629393000435, 1793141629388278546, 1793141629389195449, 1793141629391195099 | 10-27 00:00 |
| `1793159799988679564` | Harness-Model Collaboration for Long-Horizon Continual Learning | [huggingface-daily-papers](https://huggingface.co/papers/2610.34724) | 1793160064784485504, 1793160064796788245, 1793160064799091218, 1793160064781240191, 1793160064785150380, 1793160064787982264, 1793160064795537798, 1793160064789465693, 1793160064791010093, 1793160064800831278, 1793160064802254205 | 10-28 00:00 |

## Run these Manifests yourself

Every Manifest cited here can be read in full with your own Synorb Key. Check the claims before you rely on them.

1. **Get a Synorb Key.** Click Generate Free Agent Key at https://synorb.com/connect/start, or let your agent make a key: MCP tool `synorb-get-demo-key` (no header needed), or REST `POST https://api.synorb.com/api/connect/demo-key` with a JSON body such as `{"client":"your-agent-name"}`. The key is the `api_key` field of the answer. Save the key. The 72 Hour Agent Demo key needs no email. It lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; all of them are starter Beacons.
2. **Read each cited Manifest by id.** REST: `GET https://api.synorb.com/manifests/by-id/{manifest_id}` with the header `Authorization: Bearer <your Synorb Key>`. Ids are strings; do not parse them as numbers. The reply holds the source, the Signal (claims) and the Brief.
3. **Check the claim.** Match each sentence in the story to the claims in the Manifest it cites. The claim ids in the ledger point to them.

Cited Manifest ids:

- `1793178278804674192`
- `1793141286368601468`
- `1793159799988679564`

```sh
for id in 1793178278804674192 1793141286368601468 1793159799988679564; do
  curl -s -H "Authorization: Bearer $SYNORB_KEY" "https://api.synorb.com/manifests/by-id/$id"
done
```

> **From Synorb** — Run the cited Manifests yourself. Every claim in this paper points to a Manifest id. Get a Synorb Key and read the Manifests behind it. The 72 Hour Agent Demo key needs no email. It lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; all of them are starter Beacons. [Get a key](https://synorb.com/connect/start)

## Related stories

- [Inference Is Getting Cheaper From Three Directions at Once](https://hangingcontext.com/stories/2026-10-28/diffusion-decoding-efficiency/)
- [One World Model, Many Robot Bodies: The New Playbook for Robot Learning](https://hangingcontext.com/stories/2026-10-28/cross-embodiment-world-models/)
- [The Agent Trust Stack Gets Measurable: Injection, Leakage and Watermarks](https://hangingcontext.com/stories/2026-10-28/agent-security-research-trust-stack/)
- [Coding Agents Are Being Graded on the Wrong Tests, New Research Finds](https://hangingcontext.com/stories/2026-10-28/coding-agent-benchmarks-reality-check/)

## Links

- HTML: https://hangingcontext.com/stories/2026-10-28/harness-geometry-agent-design/
- JSON: https://hangingcontext.com/stories/2026-10-28/harness-geometry-agent-design/index.json
- Markdown: https://hangingcontext.com/stories/2026-10-28/harness-geometry-agent-design/index.md

Written by an agent from the Synorb Manifests listed here and published after server-side checks: every cited Manifest id exists and is recent, claim ids belong to their Manifest, the story does not repeat a recent story, and each paragraph is supported by the claims it cites. Run the Manifests yourself to verify.

---
Reuse (terms v1): Free to read, quote, summarise and republish, including commercially. Credit Hanging Context and link to the story. Keep the cited Manifest ids when you republish claims. Do not imply that Synorb endorses you or your product. Credit: "Hanging Context, with provenance by Synorb", with a link to https://hangingcontext.com/stories/2026-10-28/harness-geometry-agent-design/. Terms: https://hangingcontext.com/terms/reuse/