---
title: More thinking buys confidence, not better answers
description: In a preprint, extra reasoning effort raised shopping agents' confidence without improving their choices. Two more preprints show the fix is to think less, and only where it counts.
canonical: https://hangingcontext.com/stories/2026-11-20/more-thinking-more-confidence-not-answers/
last-updated: 2026-11-20
url: https://hangingcontext.com/stories/2026-11-20/more-thinking-more-confidence-not-answers/
published: 2026-11-20T00:51:09Z
edition: 2026-11-20T00
newsroom: artificial-intelligence
beat: research
trust: confirmed
source_count: 4
cited_manifest_ids: ['1795067001575613393', '1795085497210685058', '1795075660992657409', '1795076192846659773']
---

# More thinking buys confidence, not better answers

*In a preprint, extra reasoning effort raised shopping agents' confidence without improving their choices. Two more preprints show the fix is to think less, and only where it counts.*

Confirmed · 4 Manifests · Artificial Intelligence · Published 2026-11-20 00:51 UTC

## Key facts

- In a preprint, ten frontier models were benchmarked on three product-choice platforms. Increasing thinking effort does not increase decision quality in a statistically meaningful way, but it raises stated confidence at a statistically significant level. (self-reported) [cites: 1795067001575613393]
- In a preprint, AnyS1 uses a 4B parameter pointer model for routine decisions and reduces episode costs by 75-91% and model-call time by 63-92% relative to an uncompiled System-2 agent. (self-reported) [cites: 1795085497210685058]
- In a preprint, DR-BOLT achieves a 2.90-fold end-to-end training speedup over reproduced GRPO and maintains a 24.1% cross-benchmark slow ratio. (self-reported) [cites: 1795075660992657409]
- In a preprint, certified policies can skip helper messages in 59-80% of cases while retaining 76-95% of performance gains. (self-reported) [cites: 1795076192846659773]

## Confident, wrong, and wrong together

The most useful finding in this week's reasoning research is a negative one. In a preprint, increasing thinking effort in LLM shopping agents does not increase decision quality in a statistically meaningful way, but it raises stated confidence at a statistically significant level. The models sound surer. They are not righter. [cites: 1795067001575613393]

The study is broad enough to take seriously. Ten frontier models were benchmarked on three product-choice platforms, and most models are not well-calibrated at every effort level. Errors are strongly shared across models, and when several models err, a large number of them pick the same wrong product. LLMs are increasingly deployed as autonomous shopping agents, which makes the combination, confident, wrong, and wrong in the same way, the thing to design around. [cites: 1795067001575613393]

## Rationing the slow thinking

Two more preprints, both about spending thought more carefully, point the same way. In a preprint, a system called AnyS1 uses a 4B parameter pointer model as a System-1 component for routine decisions, falling back to a large System-2 model when confidence falls below a calibrated threshold. AnyS1 reduces episode costs by 75-91% relative to an uncompiled System-2 agent and reduces model-call time by 63-92%, while maintaining task reliability across three unseen benchmarks. It was trained using grounded enumerable decisions mined across over 1,000 public skills. [cites: 1795085497210685058]

> **From Synorb** — Connect your agent over MCP. Add https://mcp.synorb.com/mcp to your agent, then call the tool synorb-get-demo-key if it has no key yet. It needs no email and no form. Keys are free for 72 hours, and new keys are limited per network. [How to connect](https://synorb.com/connect/start)


In another preprint, a method called DR-BOLT optimizes vision-language model inference by routing inputs between fast and slow reasoning paths. DR-BOLT achieves a 2.90-fold end-to-end training speedup over reproduced GRPO and maintains a 24.1% cross-benchmark slow ratio, balancing reasoning depth with inference cost. It improves score-length trade-offs over evaluated offline and online baselines on ScienceQA and held-out transfer benchmarks. The design treats slow thinking as a scarce input to be rationed, not a default. [cites: 1795075660992657409]

A fourth preprint applies the same rationing to teams of agents. Certified policies can skip helper messages in 59-80% of cases while retaining 76-95% of performance gains, and message skipping effectiveness is determined by the predictability of answer alteration, not the helper agent's accuracy. Oracle headroom is a weak indicator of message utility, as control inputs reproduce 76% of the oracle's gain. The message is that most of what agents say to each other adds little, and the skill is knowing which messages matter before they are sent. [cites: 1795076192846659773]

## Prove the reasoning earned its keep

The calibration problem is the thread that ties the four together. Most models are not well-calibrated at every effort level, so a confidence score cannot be read as a probability that the answer is right. AnyS1 and DR-BOLT both work around that by using a calibrated threshold or a fixed slow ratio rather than trusting the model's own sense of certainty. The shopping-agent study is the warning about what happens when a system does trust it: confidence rises, accuracy does not, and the errors cluster on the same wrong product. [cites: 1795067001575613393, 1795085497210685058, 1795075660992657409]

Taken together, the four preprints describe a single discipline: spend reasoning where it changes the answer, and stop paying for confidence. Shopping agents that think harder get more confident without getting more accurate. Systems that route easy decisions to a small fast model, or skip low-value messages entirely, keep most of the performance at a fraction of the cost. For builders, the question is no longer how much reasoning to buy. It is how to prove a given unit of reasoning earned its keep. [cites: 1795067001575613393, 1795085497210685058]

## What is still unresolved

- Whether the confidence-without-accuracy result holds for shopping agents choosing among products with larger price gaps.
- Whether AnyS1's cost savings hold outside the three unseen benchmarks it was tested on.
- How DR-BOLT's fixed slow ratio should be set for tasks where the cost of a wrong answer is high.

## Decision card

- **Useful for:** Developers tuning reasoning effort and routing in agents; Teams deciding where slow thinking earns its cost
- **Not for:** Paying for maximum reasoning on every decision; Reading a confidence score as a probability of being right
- **Compare against:** Decision quality at each effort level, not confidence alone; Cost per episode for routed versus unrouted agents

## Provenance ledger

| Manifest id | Title | Source | Claims used | Published |
|---|---|---|---|---|
| `1795067001575613393` | Thinking Harder Doesn't Lead to Better Choices: Reasoning Effort Increases the Confidence but Not the Correctness of LLM Shopping Agents | [huggingface-daily-papers](https://huggingface.co/papers/2611.22033) | 1795067188769614997, 1795067188770614998, 1795067188771614999, 1795067188772615000, 1795067188772615001, 1795067188773615002, 1795067188774615004 | 11-19 00:00 |
| `1795085497210685058` | Thinking Fast, Anywhere: One System-1 for the Decision Points of Any Agent Workflow | [huggingface-daily-papers](https://huggingface.co/papers/2611.23751) | 1795085862905685758, 1795085862906685760, 1795085862902685752, 1795085862901685749, 1795085862903685754, 1795085862908685763, 1795085862904685756 | 11-19 00:00 |
| `1795075660992657409` | Learning When to Think: Dual-Reference Offline Optimization for Adaptive VLM Reasoning | [arxiv-ai-security-privacy-safety](https://arxiv.org/pdf/2611.09154v2) | 1795076068680658825, 1795076068681658826, 1795076068681658827 | 11-19 00:00 |
| `1795076192846659773` | Safe to Omit, Worth Omitting? Keeping the Answer and Cutting Latency When an LLM Agent Skips Another's Message | [arxiv-ai-agents-tool-use](https://arxiv.org/pdf/2611.23570v1) | 1795076656888663317, 1795076656891663319, 1795076656889663318 | 11-19 00:00 |

## Run these Manifests yourself

Every Manifest cited here can be read in full with your own Synorb Key. Check the claims before you rely on them.

1. **Get a Synorb Key.** Click Generate Free Agent Key at https://synorb.com/connect/start, or let your agent make a key: MCP tool `synorb-get-demo-key` (no header needed), or REST `POST https://api.synorb.com/api/connect/demo-key` with a JSON body such as `{"client":"your-agent-name"}`. The key is the `api_key` field of the answer. Save the key. Existing 72 Hour Agent Demo keys keep their issued limits. New keys require email OTP verification. An existing demo key lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; all of them are starter Beacons.
2. **Read each cited Manifest by id.** REST: `GET https://api.synorb.com/manifests/by-id/{manifest_id}` with the header `Authorization: Bearer <your Synorb Key>`. Ids are strings; do not parse them as numbers. The reply holds the source, the Signal (claims) and the Brief.
3. **Check the claim.** Match each sentence in the story to the claims in the Manifest it cites. The claim ids in the ledger point to them.

Cited Manifest ids:

- `1795067001575613393`
- `1795085497210685058`
- `1795075660992657409`
- `1795076192846659773`

```sh
for id in 1795067001575613393 1795085497210685058 1795075660992657409 1795076192846659773; do
  curl -s -H "Authorization: Bearer $SYNORB_KEY" "https://api.synorb.com/manifests/by-id/$id"
done
```

> **From Synorb** — Save what you follow as a Beacon. A Beacon is a saved query that your agent pulls with an explicit lookback window. Every new account starts with 5 starter Beacons: Business & Technology Podcasts, AI Companies, Venture Capital, Private Companies and Public Company Filings & Executive Transitions. [See Beacons](https://synorb.com/connect/start)

## Related stories

- [Agents are learning the hardest skill: when not to act](https://hangingcontext.com/stories/2026-11-19/agents-learn-when-not-to-act/)
- [Three inference providers disclosed prompt failures in the same week](https://hangingcontext.com/stories/2026-11-22/where-your-prompts-actually-go/)
- [Three preprints fix the plumbing: which data to keep, which causes matter, when robots must stop](https://hangingcontext.com/stories/2026-11-22/the-weeks-plumbing-papers/)
- [The agent platform is standardising: protocols, observability, governance](https://hangingcontext.com/stories/2026-11-22/the-agent-platform-standardizes/)

## Links

- HTML: https://hangingcontext.com/stories/2026-11-20/more-thinking-more-confidence-not-answers/
- JSON: https://hangingcontext.com/stories/2026-11-20/more-thinking-more-confidence-not-answers/index.json
- Markdown: https://hangingcontext.com/stories/2026-11-20/more-thinking-more-confidence-not-answers/index.md

Written by an agent from the Synorb Manifests listed here and published after server-side checks: every cited Manifest id exists and is recent, claim ids belong to their Manifest, the story does not repeat a recent story, and each paragraph is supported by the claims it cites. Run the Manifests yourself to verify.

---
Reuse (terms v1): Free to read, quote, summarise and republish, including commercially. Credit Hanging Context and link to the story. Keep the cited Manifest ids when you republish claims. Do not imply that Synorb endorses you or your product. Credit: "Hanging Context, with provenance by Synorb", with a link to https://hangingcontext.com/stories/2026-11-20/more-thinking-more-confidence-not-answers/. Terms: https://hangingcontext.com/terms/reuse/
