MCP's Progressive Tool Discovery
Connect 4 MCP servers to an agent and the tool definitions eat your context window before anyone has typed a question. The filesystem server on its own advertises 14 tools, and all 14 schemas ship with every request for the rest of the session.
The usual fix is progressive tool discovery. Rather than handing the model a tools array with 200 entries in it, you hand it 3 fixed ones and let it go looking for the rest.
The usual argument for that fix is prompt caching. A fixed tools array means a stable prefix, a stable prefix means the cache survives, and everyone moves on.
I ran the numbers and the argument is backwards. Across 72 benchmark runs the fixed 3-tool prefix cached in zero of its 36 runs, on every model, at every catalogue size. The pattern it argues against cached up to 93%.
Getting to that takes three steps, so this post walks them in order: what a tool definition costs you, how prefix caching decides what you pay, and what happens when you edit tools halfway through a session. Then the measurements, on gemini-3.5-flash, gemini-3.6-flash and gemini-3.7-flash.
Step 1: a tool definition is tokens you re-send forever
A tool definition is a name, a description and a JSON Schema for the arguments. On the catalogue I tested, one costs 152 tokens.
That number on its own is unremarkable. What makes it matter is that models are stateless. Every turn of an agent loop replays the entire conversation from the beginning, and the tool definitions ride along at the front each time. Send 84 schemas and you send them again on turn 2, and turn 3, and turn 40.
What the tool block costs on every single turn
Drag the slider. Each MCP server here advertises 14 tools, and every schema is re-sent on every request for the whole session.
152 tokens per schema, measured on this catalogue. Priced at the gemini-3.7-flash list input rate of $0.75/1M with no cache hits.
Those figures are measured rather than estimated, and they’re remarkably stable. Basic’s first call was 2,190 tokens at 14 tools and 42,724 at 280, identical to the token in all 9 runs at each size. Progressive’s first call was 411 tokens at every catalogue size, also in all 9 runs. Three meta-tools don’t grow.
Progressive disclosure attacks that growth. You keep the toolsets out of tools entirely, index them locally, and expose 3 fixed meta-tools in their place:
| tool | returns | when the model uses it |
|---|---|---|
search_tools | names and one-line descriptions | first, to find candidates |
get_tool_schema | the full input schema for named tools | before calling anything |
call_tool | the actual MCP result | to execute |
call_tool takes its arguments as a JSON string rather than a typed object. That’s deliberate, and it’s the whole trick: a fixed schema that can carry any payload means the definition never has to change.
I’ll call the two patterns basic and progressive from here on.
Step 2: prompt caching, and why it decides everything
An agent loop mostly appends. Turn 6 is turn 5, plus a tool result, plus whatever the model said back. The first few thousand tokens are byte for byte what you sent 20 seconds ago.
Prompt caching exists so you get billed once for that repetition instead of 20 times. Cached input costs a tenth of the normal uncached price on every model here, so on any long-running agent that discount is most of the bill.
The catch is in the word prefix. The cache key comes from the exact bytes of the request from position 0 up to your breakpoint. Change 1 byte near the front and everything behind it is uncached, whether you touched it or not.
Where “the front” sits is a provider decision. Anthropic renders tools, then system, then messages, which puts tool definitions at position 0. Their invalidation table is blunt about the consequence: modifying tool definitions invalidates the entire cache, all 3 tiers. OpenAI’s Harmony rendering puts the system message first and tools inside the developer message after the instructions, so the ordering differs, but a tools edit still rewrites the prefix.
What one added tool does to the request
Anthropic renders the request in this order, so tool definitions sit at position 0.
A warm prefix costs a tenth of normal input.
Two other things reset a cache and catch people out. Caches are scoped to a model, so switching model mid-session starts cold. And they expire, with Anthropic defaulting to a 5 minute TTL, so a quiet gap while someone reads your agent’s last answer costs the same as an edit even though nothing changed.
Step 3: invalidating a cache costs more than paying full price
This is the part I got wrong the first time, and it’s worth being slow about.
When you invalidate, you don’t drop back to paying 1x for those tokens. You pay to write the cache where you’d been paying to read it, and providers price both as a multiple of the normal uncached rate.
Anthropic reads at 0.1x and writes at 1.25x on a 5 minute TTL, or 2x on an hour. So everything sitting behind the tool block swings by about 12.5x the moment you edit a tool. OpenAI charged no write premium before GPT-5.6 and 1.25x after, so the penalty there lands nearer 10x.
Set that against what discovery buys you. Those schema tokens were only ever costing 0.1x per turn, so keeping a 300 token schema out of the prefix saves you the equivalent of 30 tokens at full price, once per turn.
One big cost, a lot of small savings. Divide 1.15 by 0.1 and you get 11.5, which gives a break-even of N > 11.5 × C / S, where C is the tokens behind the tool block and S the schema tokens you keep out per turn.
How many turns does one tools edit take to pay for itself?
The cost lands once and scales with the conversation. The saving is small and repeats. Drag either one.
Anthropic multipliers. Both sides are expressed as multiples of the uncached input rate, so the model's actual price per token cancels out.
Drag C down to 2,000 and the edit is close to free. Drag it up to 50,000 and you need about 1,900 further turns, which no real session reaches.
So the number that decides it is how much conversation sits behind the tool block at the moment you mutate it, not how long the session runs. Mutate at turn 3 and you barely notice. Mutate at turn 40 and you’ve bought a penalty you carry to the end of the run, and every mutation after that one is worse, because C has grown while there’s less run left to earn it back.
That’s the theory. Now the measurements.
How I measured it
Same prompt every time, against the local filesystem MCP server, varying only how many copies I mounted. 1 server is 14 tools, 3 is 42, 6 is 84, 20 is 280. Both patterns answer an identical question about 2 text files.
3 models, 4 catalogue sizes, 3 runs each, both patterns: 72 runs, all of which completed without error.
I price every run in USD at each model’s own list rate rather than counting tokens, because output costs 5 to 6 times input and cached input a tenth of it. Tokens are the wrong unit for this comparison.
| model | input | cached read | output |
|---|---|---|---|
gemini-3.5-flash | $1.50 | $0.15 | $9.00 |
gemini-3.6-flash | $0.75 | $0.075 | $3.75 |
gemini-3.7-flash | $0.75 | $0.075 | $3.75 |
Prices are per million tokens. Note the shape of that table: 3.6 and 3.7 are half the price of 3.5, because they’re on a promotional rate that Google’s pricing page says ends on 31 December 2026. The newer models here are the cheaper ones, which is not the direction you’d guess.
One thing took me an embarrassingly long time to get right. Gemini reports thinking tokens in thoughtsTokenCount, separately from candidatesTokenCount, and bills them at the output rate.
Count only the visible tokens and you undercount the most expensive line item by about 70%. Every number below counts both.
Finding 1: the crossover moved down, and it’s model-specific
The cheapest way to see the whole result set is as a ratio, so here is every run I have.
Cost ratio per run, progressive divided by basic
36 runs, 3 models, 4 catalogue sizes. Each model is priced at its own list rate. Below the orange line, progressive was the cheaper pattern.
Each dot is one run, plotted as progressive cost divided by basic cost. Below the orange line, progressive won. Click a model to isolate it.
The three models disagree about where the pattern starts paying, and they disagree in a consistent direction:
- On
gemini-3.5-flash, progressive loses at 14 tools and is a coin flip at 42. It takes 84 tools before it wins every run. - On
gemini-3.6-flash, it loses at 14 by 1.4x to 1.7x, then wins every single run from 42 upward. - On
gemini-3.7-flash, 14 tools is already a dead heat. The three runs came in at 1.18x, 0.99x and 0.76x, and it won 2 of 3 at the smallest catalogue I tested.
So the crossover sits somewhere around 60 tools on 3.5, 30 on 3.6, and under 20 on 3.7. It’s moving down as the models get newer, and the reason is turn count, which I’ll come back to.
What hasn’t changed is the shape. basic grows with catalogue size because it re-sends every schema on every call. progressive stays close to flat, because search returns at most 5 hits whether the catalogue holds 14 tools or 500. Its most expensive 280-tool run cost $0.030, which is less than its most expensive 14-tool run.
Finding 2: the schemas don’t disappear, they relocate
The headline everyone quotes is the first call. Cloudflare’s number is a 94% reduction, 9,400 tokens of definitions down to 600 across 52 tools. Anthropic’s comparable figures are 85% and 98.7%.
Those are all real, and they all measure upfront definition tokens.
They’re also close to irrelevant, because you pay for the whole prompt again on every call. Progressive disclosure moves the schemas out of a prefix you repeat every turn and into conversation history, which you also repeat every turn once the model has fetched it.
Watch that happen across one 14-tool run.
Progressive's prompt, call by call, at 14 tools on gemini-3.7-flash
Step through the run. The schemas it kept out of the prefix end up in the conversation, which is re-sent every turn too.
Progressive starts at 411 tokens against basic’s 2,190, an 81% saving on the first call. By call 7 it’s carrying 2,296 tokens, which is more than basic’s opening request. It spent 6 round trips arriving at a worse position than the one it was trying to avoid.
Measure the whole run, not the first call.
Finding 3: the fixed prefix cached zero times out of 36
Here’s the result I did not expect to be this clean.
Share of prompt tokens served from cache
Median of 3 runs per cell, hover for the individual runs. Basic's share climbs with the catalogue. Progressive's is zero in all 36 runs.
Basic caches once its tool block alone clears the model's minimum, which is why the bars rise with catalogue size rather than with conversation length. Progressive's 411-token prefix never gets there.
progressive cached nothing. Not a single token, in 36 runs, across 3 models and 4 catalogue sizes. basic cached up to 93%.
The mechanism is the part worth taking away. Implicit caching keys on the prefix shared between two consecutive requests, and that shared portion has to clear a minimum size on its own. So what qualifies a request for caching is how big its stable head is.
basic’s stable head is the tool block, and it grows with the catalogue. At 14 tools it’s 2,190 tokens and caches nothing on any of the three models. At 84 tools it’s 12,820 and caches 60% on two of them. At 280 tools it’s 42,724 and caches 92% on all three. The bars climb with catalogue size rather than with conversation length, which is exactly what that mechanism predicts.
progressive’s stable head is 411 tokens and never grows. It never gets close.
The uncomfortable conclusion is that prefix stability is what makes caching possible, and clearing the floor is what makes it happen. At every size I tested, the pattern that carefully protects its prefix got nothing back for protecting it, while the pattern it argues against was sitting at 92% cached.
That said, caching rebates basic’s waste rather than justifying it. At 280 tools on 3.7-flash, basic served 92.8% of a 172,000-token prompt from cache and still lost by 2.4x.
The floors are not really floors
I probed each model separately with a repeated prefix, 8 identical calls per size.
How often an identical repeated prefix actually caches
8 identical calls per cell, same prefix every time, only the one-line question changes. Each dot is one call.
Nothing cached below 4,389 tokens on any of the three. Above that the pattern stops being a threshold: 3.5-flash caches 6 of 8 calls at 6,689 tokens and 0 of 8 at 8,989, which reproduced across two separate runs.
Nothing cached below 4,389 tokens on any of the three. Above that it stops behaving like a threshold. gemini-3.5-flash caches 6 of 8 calls at 6,689 tokens and 0 of 8 at 8,989, which I re-ran twice because I assumed I’d made a mistake. gemini-3.7-flash barely caches this prompt shape at all, hitting 3 of 8 even at 13,589 tokens, and yet in the actual agent runs it cached 92.8% at 280 tools.
Treat implicit caching as opportunistic. Probe your own workload rather than the documented number, and don’t build a cost model that depends on it.
One way to make progressive worse
All of that assumes the 3 meta-tools really are fixed. It’s tempting to give call_tool an enum of the tool names discovered so far, or to keep a live catalogue index inside the search_tools description.
Either one rewrites position 0 every time the model finds something new, so you’d pay basic’s invalidation cost and progressive’s extra round trips at the same time. My benchmark keeps all 3 definitions byte for byte identical across the whole run.
Finding 4: turn count is the lever that moved
progressive needs more model calls than basic, and that gap is most of what it pays for the schemas it saves.
| model | basic | progressive |
|---|---|---|
gemini-3.5-flash | 4 to 6 calls | 7 to 9 |
gemini-3.6-flash | 3 to 4 calls | 6 to 9 |
gemini-3.7-flash | 3 to 4 calls | 5 to 8 |
That table is Finding 1 in different clothes. Progressive’s floor drops from 7 calls on 3.5 to 5 on 3.7, and that alone is why the crossover slid from about 60 tools down to under 20. The newer model needs fewer round trips to find and call a tool, so the pattern’s main cost shrinks while its saving stays the same.
If you’re tuning this, prompt the model to search once for the whole task and batch its schema lookups. That’s a bigger lever than the ranking function will ever be, and it’s the same lever the model vendors are pulling for you.
It’s also why single runs aren’t evidence. My 42-tool cell on 3.7-flash holds both a 3.2x progressive win and a 1.4x loss, and what separates them is turn count. Take one measurement per configuration and you can produce whichever conclusion you set out to find.
What Anthropic and OpenAI do instead
Both solved this at the API level, and the fix has the same shape in each. Declare every tool up front, mark most of them deferred, and let the model pull schemas in without ever touching the prefix.
On Anthropic, set defer_loading: true on a tool definition and add a search tool (tool_search_tool_regex_20251119 or tool_search_tool_bm25_20251119). The API keeps deferred tools out of the system-prompt prefix entirely, down to the name stub. When the model discovers one, the API appends a tool_reference block inline in the conversation and expands it before the model sees it. No beta header. At least 1 tool has to stay non-deferred, and defer_loading can’t share a tool with cache_control.
On OpenAI, tool_search in the Responses API (gpt-5.4 and later) takes the same defer_loading: true flag on functions, namespaces, or MCP servers. Discovered tools load at the end of the context window. For an individually deferred function the model still sees its name and description, so only the parameter schema gets deferred. Namespaces and MCP servers are the recommended surface.
Both are additive. Reordering the array or swapping one definition for another still invalidates. Anthropic has a beta (mid-conversation-tool-changes-2026-07-01) adding tool_addition and tool_removal blocks, and removal there does keep the cache.
Worth knowing what deferral does and doesn’t buy. It saves context tokens, not request bytes. You still send every deferred tool’s full definition in the tools array on every call.
Gemini has no equivalent
There’s no deferred-loading or tool-search mechanism on any of the three models I tested. Gemini rejects the field outright with 'defer_loading' field in ToolSearch is not supported by Gemini API, and the parity request against python-genai is open with no commitment.
Explicit caching is stricter than it first looks. tools and toolConfig on a CachedContent are both input-only and immutable, patch accepts only expiry, and while a cache is in use you can’t pass tools per-request at all. Changing 1 tool means rebuilding the cache.
So on Gemini a fixed meta-tool set is the only cache-stable way to vary which tools a model can reach. That’s a provider gap rather than a limitation of SDKs in general. Vercel AI SDK, LangChain, the OpenAI Agents SDK and pydantic-ai all expose these flags today for the providers that have them.
What I’m not claiming
3 runs per cell. These are point estimates and the spread inside a cell is wide, sometimes wider than the gap between cells.
I didn’t measure latency, and progressive needs roughly twice the model calls, so it’s slower in wall clock even where it’s cheaper. If time to answer matters more than cost, the crossover moves back up.
Tool-selection accuracy at large catalogues is the strongest argument for the pattern and I didn’t test it properly. basic still answered correctly at 280 tools on all 3 models, so I saw no collapse, but I wasn’t looking hard.
The 3.6 and 3.7 prices are promotional until the end of 2026. When they double in January, every USD figure here shifts, though the ratios don’t, since both patterns pay the same rates.
One more cost I never put a number on. call_tool takes its arguments as a JSON string, so strict-style schema enforcement can’t see inside the payload. A malformed or mis-named argument doesn’t surface until the MCP server rejects it, and each of those costs a round trip.
So should you build this?
Progressive tool discovery is a scaling optimisation, and the scale at which it starts paying is dropping. On gemini-3.7-flash it was already at parity with 14 tools mounted, where a year-older model needed 60.
Below that crossover it costs you money, latency and code. Above it the saving is large and grows with the catalogue.
The caching argument usually quoted in its favour is the one part you should ignore. The fixed 3-tool prefix cached zero times in 36 runs, while the fat tool block it replaces reached 93%. If prefix stability is your reason for reaching for this pattern, you’re reaching for it for a reason that didn’t survive contact with a measurement.
If you’re on Anthropic or OpenAI, use the native deferred loading and skip all of this. You keep real schemas, real argument validation, and the cache.
If you’re on Gemini, the client-side catalogue is what’s available. On the current Flash model, build it once you’re past about 20 tools.