# MCP's Progressive Tool Discovery

Source: https://tpiros.dev/blog/progressive-tool-discovery-prompt-caching

Connect 4 MCP servers to an agent and the tool definitions eat your context window before anyone has typed a question. The filesystem server on its own advertises 14 tools, and all 14 schemas ship with every request for the rest of the session.

The usual fix is progressive tool discovery. Rather than handing the model a `tools` array with 200 entries in it, you hand it 3 fixed ones and let it go looking for the rest.

The usual argument for that fix is prompt caching. A fixed `tools` array means a stable prefix, a stable prefix means the cache survives, and everyone moves on.

I ran the numbers and the argument is backwards. Across 72 benchmark runs the fixed 3-tool prefix cached in **zero** of its 36 runs, on every model, at every catalogue size. The pattern it argues against cached up to 93%.

Getting to that takes three steps, so this post walks them in order: what a tool definition costs you, how prefix caching decides what you pay, and what happens when you edit `tools` halfway through a session. Then the measurements, on `gemini-3.5-flash`, `gemini-3.6-flash` and `gemini-3.7-flash`.

## Step 1: a tool definition is tokens you re-send forever

A tool definition is a name, a description and a JSON Schema for the arguments. On the catalogue I tested, one costs 152 tokens.

That number on its own is unremarkable. What makes it matter is that models are stateless. Every turn of an agent loop replays the entire conversation from the beginning, and the tool definitions ride along at the front each time. Send 84 schemas and you send them again on turn 2, and turn 3, and turn 40.

Those figures are measured rather than estimated, and they're remarkably stable. Basic's first call was 2,190 tokens at 14 tools and 42,724 at 280, identical to the token in all 9 runs at each size. Progressive's first call was 411 tokens at every catalogue size, also in all 9 runs. Three meta-tools don't grow.

Progressive disclosure attacks that growth. You keep the toolsets out of `tools` entirely, index them locally, and expose 3 fixed meta-tools in their place:

| tool | returns | when the model uses it |
| --- | --- | --- |
| `search_tools` | names and one-line descriptions | first, to find candidates |
| `get_tool_schema` | the full input schema for named tools | before calling anything |
| `call_tool` | the actual MCP result | to execute |

`call_tool` takes its arguments as a JSON string rather than a typed object. That's deliberate, and it's the whole trick: a fixed schema that can carry any payload means the definition never has to change.

I'll call the two patterns `basic` and `progressive` from here on.

## Step 2: prompt caching, and why it decides everything

An agent loop mostly appends. Turn 6 is turn 5, plus a tool result, plus whatever the model said back. The first few thousand tokens are byte for byte what you sent 20 seconds ago.

Prompt caching exists so you get billed once for that repetition instead of 20 times. Cached input costs a tenth of the normal uncached price on every model here, so on any long-running agent that discount is most of the bill.

The catch is in the word prefix. The cache key comes from the exact bytes of the request from position 0 up to your breakpoint. Change 1 byte near the front and everything behind it is uncached, whether you touched it or not.

Where "the front" sits is a provider decision. Anthropic renders `tools`, then `system`, then `messages`, which puts tool definitions at position 0. Their invalidation table is blunt about the consequence: modifying tool definitions invalidates the entire cache, all 3 tiers. OpenAI's Harmony rendering puts the system message first and tools inside the developer message after the instructions, so the ordering differs, but a tools edit still rewrites the prefix.

Two other things reset a cache and catch people out. Caches are scoped to a model, so switching model mid-session starts cold. And they expire, with Anthropic defaulting to a 5 minute TTL, so a quiet gap while someone reads your agent's last answer costs the same as an edit even though nothing changed.

## Step 3: invalidating a cache costs more than paying full price

This is the part I got wrong the first time, and it's worth being slow about.

When you invalidate, you don't drop back to paying 1x for those tokens. You pay to *write* the cache where you'd been paying to *read* it, and providers price both as a multiple of the normal uncached rate.

Anthropic reads at 0.1x and writes at 1.25x on a 5 minute TTL, or 2x on an hour. So everything sitting behind the tool block swings by about 12.5x the moment you edit a tool. OpenAI charged no write premium before GPT-5.6 and 1.25x after, so the penalty there lands nearer 10x.

Set that against what discovery buys you. Those schema tokens were only ever costing 0.1x per turn, so keeping a 300 token schema out of the prefix saves you the equivalent of 30 tokens at full price, once per turn.

One big cost, a lot of small savings. Divide 1.15 by 0.1 and you get 11.5, which gives a break-even of `N > 11.5 × C / S`, where `C` is the tokens behind the tool block and `S` the schema tokens you keep out per turn.

Drag `C` down to 2,000 and the edit is close to free. Drag it up to 50,000 and you need about 1,900 further turns, which no real session reaches.

So the number that decides it is how much conversation sits behind the tool block at the moment you mutate it, not how long the session runs. Mutate at turn 3 and you barely notice. Mutate at turn 40 and you've bought a penalty you carry to the end of the run, and every mutation after that one is worse, because `C` has grown while there's less run left to earn it back.

That's the theory. Now the measurements.

## How I measured it

Same prompt every time, against the local filesystem MCP server, varying only how many copies I mounted. 1 server is 14 tools, 3 is 42, 6 is 84, 20 is 280. Both patterns answer an identical question about 2 text files.

3 models, 4 catalogue sizes, 3 runs each, both patterns: 72 runs, all of which completed without error.

I price every run in USD at each model's own list rate rather than counting tokens, because output costs 5 to 6 times input and cached input a tenth of it. Tokens are the wrong unit for this comparison.

| model | input | cached read | output |
| --- | --- | --- | --- |
| `gemini-3.5-flash` | $1.50 | $0.15 | $9.00 |
| `gemini-3.6-flash` | $0.75 | $0.075 | $3.75 |
| `gemini-3.7-flash` | $0.75 | $0.075 | $3.75 |

Prices are per million tokens. Note the shape of that table: 3.6 and 3.7 are half the price of 3.5, because they're on a promotional rate that Google's pricing page says ends on 31 December 2026. The newer models here are the cheaper ones, which is not the direction you'd guess.

One thing took me an embarrassingly long time to get right. Gemini reports thinking tokens in `thoughtsTokenCount`, separately from `candidatesTokenCount`, and bills them at the output rate.

Count only the visible tokens and you undercount the most expensive line item by about 70%. Every number below counts both.

## Finding 1: the crossover moved down, and it's model-specific

The cheapest way to see the whole result set is as a ratio, so here is every run I have.

Each dot is one run, plotted as progressive cost divided by basic cost. Below the orange line, progressive won. Click a model to isolate it.

The three models disagree about where the pattern starts paying, and they disagree in a consistent direction:

- On `gemini-3.5-flash`, progressive loses at 14 tools and is a coin flip at 42. It takes 84 tools before it wins every run.
- On `gemini-3.6-flash`, it loses at 14 by 1.4x to 1.7x, then wins every single run from 42 upward.
- On `gemini-3.7-flash`, 14 tools is already a dead heat. The three runs came in at 1.18x, 0.99x and 0.76x, and it won 2 of 3 at the smallest catalogue I tested.

So the crossover sits somewhere around 60 tools on 3.5, 30 on 3.6, and under 20 on 3.7. It's moving down as the models get newer, and the reason is turn count, which I'll come back to.

What hasn't changed is the shape. `basic` grows with catalogue size because it re-sends every schema on every call. `progressive` stays close to flat, because search returns at most 5 hits whether the catalogue holds 14 tools or 500. Its most expensive 280-tool run cost $0.030, which is less than its most expensive 14-tool run.

## Finding 2: the schemas don't disappear, they relocate

The headline everyone quotes is the first call. Cloudflare's number is a 94% reduction, 9,400 tokens of definitions down to 600 across 52 tools. Anthropic's comparable figures are 85% and 98.7%.

Those are all real, and they all measure upfront definition tokens.

They're also close to irrelevant, because you pay for the whole prompt again on every call. Progressive disclosure moves the schemas out of a prefix you repeat every turn and into conversation history, which you also repeat every turn once the model has fetched it.

Watch that happen across one 14-tool run.

Progressive starts at 411 tokens against basic's 2,190, an 81% saving on the first call. By call 7 it's carrying 2,296 tokens, which is *more* than basic's opening request. It spent 6 round trips arriving at a worse position than the one it was trying to avoid.

Measure the whole run, not the first call.

## Finding 3: the fixed prefix cached zero times out of 36

Here's the result I did not expect to be this clean.

`progressive` cached nothing. Not a single token, in 36 runs, across 3 models and 4 catalogue sizes. `basic` cached up to 93%.

The mechanism is the part worth taking away. Implicit caching keys on the prefix shared between two consecutive requests, and that shared portion has to clear a minimum size on its own. So what qualifies a request for caching is how big its stable head is.

`basic`'s stable head is the tool block, and it grows with the catalogue. At 14 tools it's 2,190 tokens and caches nothing on any of the three models. At 84 tools it's 12,820 and caches 60% on two of them. At 280 tools it's 42,724 and caches 92% on all three. The bars climb with catalogue size rather than with conversation length, which is exactly what that mechanism predicts.

`progressive`'s stable head is 411 tokens and never grows. It never gets close.

The uncomfortable conclusion is that prefix stability is what makes caching *possible*, and clearing the floor is what makes it *happen*. At every size I tested, the pattern that carefully protects its prefix got nothing back for protecting it, while the pattern it argues against was sitting at 92% cached.

That said, caching rebates `basic`'s waste rather than justifying it. At 280 tools on 3.7-flash, `basic` served 92.8% of a 172,000-token prompt from cache and still lost by 2.4x.

### The floors are not really floors

I probed each model separately with a repeated prefix, 8 identical calls per size.

Nothing cached below 4,389 tokens on any of the three. Above that it stops behaving like a threshold. `gemini-3.5-flash` caches 6 of 8 calls at 6,689 tokens and 0 of 8 at 8,989, which I re-ran twice because I assumed I'd made a mistake. `gemini-3.7-flash` barely caches this prompt shape at all, hitting 3 of 8 even at 13,589 tokens, and yet in the actual agent runs it cached 92.8% at 280 tools.

Treat implicit caching as opportunistic. Probe your own workload rather than the documented number, and don't build a cost model that depends on it.

### One way to make progressive worse

All of that assumes the 3 meta-tools really are fixed. It's tempting to give `call_tool` an enum of the tool names discovered so far, or to keep a live catalogue index inside the `search_tools` description.

Either one rewrites position 0 every time the model finds something new, so you'd pay `basic`'s invalidation cost and `progressive`'s extra round trips at the same time. My benchmark keeps all 3 definitions byte for byte identical across the whole run.

## Finding 4: turn count is the lever that moved

`progressive` needs more model calls than `basic`, and that gap is most of what it pays for the schemas it saves.

| model | basic | progressive |
| --- | --- | --- |
| `gemini-3.5-flash` | 4 to 6 calls | 7 to 9 |
| `gemini-3.6-flash` | 3 to 4 calls | 6 to 9 |
| `gemini-3.7-flash` | 3 to 4 calls | 5 to 8 |

That table is Finding 1 in different clothes. Progressive's floor drops from 7 calls on 3.5 to 5 on 3.7, and that alone is why the crossover slid from about 60 tools down to under 20. The newer model needs fewer round trips to find and call a tool, so the pattern's main cost shrinks while its saving stays the same.

If you're tuning this, prompt the model to search once for the whole task and batch its schema lookups. That's a bigger lever than the ranking function will ever be, and it's the same lever the model vendors are pulling for you.

It's also why single runs aren't evidence. My 42-tool cell on 3.7-flash holds both a 3.2x progressive win and a 1.4x loss, and what separates them is turn count. Take one measurement per configuration and you can produce whichever conclusion you set out to find.

## What Anthropic and OpenAI do instead

Both solved this at the API level, and the fix has the same shape in each. Declare every tool up front, mark most of them deferred, and let the model pull schemas in without ever touching the prefix.

On Anthropic, set `defer_loading: true` on a tool definition and add a search tool (`tool_search_tool_regex_20251119` or `tool_search_tool_bm25_20251119`). The API keeps deferred tools out of the system-prompt prefix entirely, down to the name stub. When the model discovers one, the API appends a `tool_reference` block inline in the conversation and expands it before the model sees it. No beta header. At least 1 tool has to stay non-deferred, and `defer_loading` can't share a tool with `cache_control`.

On OpenAI, `tool_search` in the Responses API (gpt-5.4 and later) takes the same `defer_loading: true` flag on functions, namespaces, or MCP servers. Discovered tools load at the end of the context window. For an individually deferred function the model still sees its name and description, so only the parameter schema gets deferred. Namespaces and MCP servers are the recommended surface.

Both are additive. Reordering the array or swapping one definition for another still invalidates. Anthropic has a beta (`mid-conversation-tool-changes-2026-07-01`) adding `tool_addition` and `tool_removal` blocks, and removal there does keep the cache.

Worth knowing what deferral does and doesn't buy. It saves context tokens, not request bytes. You still send every deferred tool's full definition in the `tools` array on every call.

## Gemini has no equivalent

There's no deferred-loading or tool-search mechanism on any of the three models I tested. Gemini rejects the field outright with `'defer_loading' field in ToolSearch is not supported by Gemini API`, and the parity request against `python-genai` is open with no commitment.

Explicit caching is stricter than it first looks. `tools` and `toolConfig` on a `CachedContent` are both input-only and immutable, `patch` accepts only expiry, and while a cache is in use you can't pass tools per-request at all. Changing 1 tool means rebuilding the cache.

So on Gemini a fixed meta-tool set is the only cache-stable way to vary which tools a model can reach. That's a provider gap rather than a limitation of SDKs in general. Vercel AI SDK, LangChain, the OpenAI Agents SDK and pydantic-ai all expose these flags today for the providers that have them.

## What I'm not claiming

3 runs per cell. These are point estimates and the spread inside a cell is wide, sometimes wider than the gap between cells.

I didn't measure latency, and `progressive` needs roughly twice the model calls, so it's slower in wall clock even where it's cheaper. If time to answer matters more than cost, the crossover moves back up.

Tool-selection accuracy at large catalogues is the strongest argument for the pattern and I didn't test it properly. `basic` still answered correctly at 280 tools on all 3 models, so I saw no collapse, but I wasn't looking hard.

The 3.6 and 3.7 prices are promotional until the end of 2026. When they double in January, every USD figure here shifts, though the ratios don't, since both patterns pay the same rates.

One more cost I never put a number on. `call_tool` takes its arguments as a JSON string, so `strict`-style schema enforcement can't see inside the payload. A malformed or mis-named argument doesn't surface until the MCP server rejects it, and each of those costs a round trip.

## So should you build this?

Progressive tool discovery is a scaling optimisation, and the scale at which it starts paying is dropping. On `gemini-3.7-flash` it was already at parity with 14 tools mounted, where a year-older model needed 60.

Below that crossover it costs you money, latency and code. Above it the saving is large and grows with the catalogue.

The caching argument usually quoted in its favour is the one part you should ignore. The fixed 3-tool prefix cached zero times in 36 runs, while the fat tool block it replaces reached 93%. If prefix stability is your reason for reaching for this pattern, you're reaching for it for a reason that didn't survive contact with a measurement.

If you're on Anthropic or OpenAI, use the native deferred loading and skip all of this. You keep real schemas, real argument validation, and the cache.

If you're on Gemini, the client-side catalogue is what's available. On the current Flash model, build it once you're past about 20 tools.