Your agent was paying for tools it never used
Voxel51 cut tool-discovery tokens 97.7% with keyword search. Baseten made coding agents 62.5% cheaper by loading tools late. Google now ships the same idea in the Gemini API.
- Keyword-based search for MCP tool discovery achieved a 97.7% token reduction on the Voxel51 Agent, cutting tool discovery from 21,777 tokens per call to under 700 tokens. Self-reported m:…3615
- Deferred tool loading makes coding agents up to 62.5% cheaper and up to 26.2% faster, while agentic benchmarks saved an average of 51.1% in costs. Self-reported m:…2889
- Google is launching Tool Retrieval in the Gemini API in preview, and it works with Gemini 3.6 and later models. Self-reported m:…7635
The waste nobody counted
Every tool an agent can call has to be described to the model, and those descriptions add up fast. Voxel51 reports that its agent discovery process previously consumed 21,777 tokens per call for tool definitions, covering a catalog of 512 operators across 40 plugin namespaces. That is context spent before the agent does anything useful, and it grows with every tool a team adds. The company says implementing a keyword search step reduced tool discovery token usage to under 700 tokens per call, and keyword-based search for MCP tool discovery achieved a 97.7% token reduction on the Voxel51 Agent. m:…3615
The obvious worry is accuracy: search might return the wrong tool. The test result is reassuring, because a keyword-only search correctly identified the human-preferred operator in five out of six test queries. The ranking step adds 2.67ms of latency on the external MCP gateway, which is small against the tokens saved, and the company says ranking for MCP tool discovery must occur after permission filtering to ensure security, so search never exposes restricted operators. Voxel51 recommends building a keyword-based search before attempting embeddings-based semantic ranking, and it notes that Anthropic's Tool Tool reports an 85% token reduction using a search step instead of a flat list. m:…3615
The same fix, measured in dollars
Baseten measures the same idea in money rather than tokens, and the numbers line up. The company says deferred tool loading makes coding agents up to 62.5% cheaper and up to 26.2% faster, and agentic benchmarks saved an average of 51.1% in costs and an average of 14.9% in time. Deferred loading works by passing tool names plus a tool_search tool initially, so the full schemas arrive only when the agent actually needs them, and the company describes it as a form of progressive disclosure applied to tools to reduce wasted context. m:…2889
The scale of the waste explains the size of the saving. Eager loading with 3 MCP servers added about 141k tokens to the system prompt, because eager loading injects the full list of tool schemas into the system prompt, while deferred loading reduced tool schema tokens to about 1.3k. OpenAI and Anthropic APIs typically support deferred tool loading, though the company notes that Claude Code turns off deferred tool loading when using third-party open model providers, so builders on open models should check the setting rather than assume it is on. m:…2889
The platform catches up
Google is launching Tool Retrieval in the Gemini API in preview, which turns the same pattern into a platform feature. Tool Retrieval allows agents to load tool parameter schemas on demand, and it is enabled by adding a tool_search tool and setting defer_loading to true. Tool Retrieval works with Gemini 3.6 and later models, it reduces input token counts and latency for complex agent tasks, and it has no separate charge beyond standard token rates. It also supports remote MCP servers in the Gemini API, and client-side retrieval can be implemented by setting execution to client in the tool_search declaration. m:…7635
The lesson for builders is simple, and it is now backed by independent sources. A flat list of tools is a tax on every call, and the fix is to search for tools the way agents search for anything else. Teams with large catalogs should treat the tool list as a retrieval problem, check that permission filtering runs before ranking, and confirm deferred loading is actually enabled for their provider, because the savings are large enough to change which models and harnesses are affordable. m:…3615 m:…2889 m:…7635
What is still unresolved
- Whether keyword search misses tools that semantic search would catch
- How permission filtering interacts with lazy loading
- Whether deferred loading helps small tool sets
Provenance by Synorb. Every claim links to its source. This story was written by an agent from the Manifests listed below. Read the methodology and limitations, or send a correction.
Free to reuse, including commercially. Credit “Hanging Context, with provenance by Synorb” and link to this story. Keep the cited Manifest ids. Reuse terms v1.