Your MCP tool schema can change and nothing in your pipeline fails

Every layer of a normal test pyramid rests on an assumption nobody writes down. The contract under test lives in your repository. Unit tests check your own functions. Integration tests check your calls against a schema sitting in a file you can open. Contract tests exist precisely because the other side can change shape, and they work because you keep a pinned copy of that shape and fail the build when it moves.

An MCP tool has no copy. The contract is generated at runtime, on somebody else's server, and handed to your model fresh on every tools/list. You never pinned it, because there wasn't anything to pin it to.

On 23 September two people filed the same class of bug in unrelated repositories, hours apart, and neither of them found it with a test.

What a drifting MCP tool schema actually looks like

The first one is loud. In accessibility-agents issue 205, version 7.0.2 starts cleanly and lists all 39 of its tools without complaint. Then every single tools/call fails with the same error: output validation error, the tool has an output schema but no structured content was provided. Two commits from the day before had collided. One added outputSchema to every tool. The other bumped the SDK from 1.27.1 to 1.30.0, and the newer SDK enforces structuredContent whenever an output schema is declared. The server was rejecting its own handlers.

The author's own note on why the suite missed it is the most useful sentence in the thread. The tests exercised the handler functions and the tools/list endpoint, and nothing ever ran an end to end call through the SDK. In their words, a single end-to-end tools/call against any one tool would have caught this.

The second one is quiet, and it's the worse of the two. acp-agent.kotlin issue 12 reports that when the client rebuilds a tool's parameters it keeps type and properties and drops everything else, including the required array. A tool that declares required = listOf("q") goes out to the model as {"type":"object","properties":{"q":{"type":"string"}}}. The reporter puts it plainly: the advertised tool schema tells the LLM that mandatory MCP arguments are optional.

Nothing errors in the second case. The server's fine, the client's fine, the call succeeds. The model just stops being told that it has to fill in a field, so sometimes it doesn't, and the tool runs with less than it needed. You find out when an answer is wrong, which is a long way downstream from a red build.

Why nothing in your pipeline fails

Split those two apart and you have the whole problem. A change that makes the call fail is self reporting. It's unpleasant, it's visible, and somebody gets paged. A change that leaves the call succeeding while altering what the call means has no reporting surface at all. It's the same split we drew about AI observability, where the dashboards are good at telling you a system is responding and poor at telling you it's still right.

The model is what makes the quiet case dangerous. It doesn't read a tool description and a schema as documentation, the way a person skimming an API reference does. It reads them as the instruction for what the tool is and how to call it. Rewrite the description and you have changed the instruction. Drop a required entry and you have changed the contract. Both arrive through the same channel as the tool list itself, and neither trips anything you own.

The MCP spec versions the wire, not the MCP tool schema

This isn't a gap in your setup. It's a gap in the protocol, and the specification makes it legible if you put two parts of the same page side by side.

MCP is careful about versions at the transport layer. Under the 2026-07-28 revision every request carries its own protocol version in _meta, and a server that cannot serve that version says so explicitly instead of guessing. That machinery works and it's well designed.

Now look at the tool itself. The specification lists a tool definition as exactly seven fields: name, title, description, icons, inputSchema, outputSchema and annotations. There's no version. There's no way for a client to say which shape of a tool it was built against, and no way for a server to say which shape it is now serving. A server can rewrite inputSchema on any tool, any time, and keep answering 2026-07-28 perfectly correctly while it does.

The nearest thing to a signal is listChanged, and the spec is precise about what that covers: it indicates whether the server will emit notifications when the list of available tools changes. The list. It covers membership and says nothing about shape. Add a tool and you may hear about it. Change what an existing tool requires and you won't.

The detail that makes this worth writing down is a few paragraphs up on the same page. Servers are asked to return tools in a deterministic order, and the spec gives the reason: it lets clients cache the tool list reliably and improves prompt cache hit rates. So the tool list has been made stable enough to cache, and there's still no field that says whether the cached thing still means what it meant yesterday. The stability was specified for the cache. Nobody specified anything for the caller.

Tool versioning was proposed, and it is sitting dormant

Somebody already noticed. SEP-1575, opened on 30 September 2025 by Malay Kurwa, proposed semantic versioning for tools: a version field on the definition, client side tool_requirements with caret and tilde constraints, and mandatory server side validation before execution. It's closed, and it carries four labels, one of which reads "SEP proposal without a sponsor".

I want to be careful here, because the temptation is to read a rejection into that. There's no maintainer comment on the page giving a reason, so I can't tell you it was turned down on the merits. What I can tell you is that it was proposed almost a year ago, that the largest protocol revision in MCP's history shipped since, and that a tool still has seven fields and none of them is a version.

Where the missing test layer actually goes

I run engineering and QA teams, and this is the part I keep turning over, because the fix isn't exotic. It is contract testing, which we have done for years against payment providers, and the reason it feels unfamiliar here is that the two situations differ in one specific way. A payments integration gives you a versioned API and a sandbox, so your pinned copy comes from the vendor. An MCP server gives you neither, so the pinned copy has to be taken from the server itself, at a moment you choose, and stored by you.

The plan needs a few things in it, and the first is embarrassingly cheap.

One end to end call per tool, per build. Not a handler unit test, and not a tools/list assertion. An actual tools/call through the real client against the real server, asserting on the shape of what comes back. Checking that nothing threw is a different test, and a weaker one. This is the test that issue 205 says out loud would have caught 39 broken tools.

A snapshot of the advertised schema, asserted field by field. Store what the server said, and diff it on a schedule. Assert specifically on the parts that carry meaning to the model: the required array, enum values, types, and the description text. A hash over the whole definition is easier to write and worse to live with, for reasons in the next section.

A deliberately incomplete call. Send a request missing a required argument and assert that it is rejected. That single test is what stands between you and the acp-agent case, where nothing failed and the model was simply told less than the truth.

None of that is new engineering. It's the ordinary contract test, aimed at a contract that happens to live somewhere your repository cannot see.

Pin the silent changes and let the loud ones through

There is an argument running through the existing writing on this and it deserves settling instead of another restatement. Tsvetan Gerginov shipped a CI gate in July that hashes tool definitions and fails the build on any change, on the reasonable grounds that nothing else catches it. Three days later Alexey Spinov argued the opposite: a byte level pin fires just as loudly when a server honestly adds an optional parameter as when it removes a required one, and a gate that cries wolf gets switched off. Both of them are right about their half and neither has answered the other.

The line I'd draw isn't about which fields changed. It's about how the change fails.

That distinction isn't mine, and I found that out after drafting this. Tian Pan made it in April, writing about contract tests for tool surfaces, and put it better than I would have: a deterministic service raises a parse error on a renamed field, while an agent reasons over the new shape and recovers into a plausible wrong answer. His framing is that the tool's API is part of your agent's prompt, so every breaking change downstream is a silent prompt change upstream. What I'd add is that the same distinction is the right place to put the gate.

Gate the changes that stay silent, because nothing else will tell you. A removed required entry, a dropped enum value, a widened type, a rewritten description. Those keep the call succeeding while changing what it means, and they're the entire category the model will absorb without complaint.

Let the loud ones through. A new optional parameter, a new tool, a changed title. If one of those does break you, it breaks with an error code, at which point you have a stack trace and a human. That's a cheaper failure than a gate everybody has learned to override.

On how often this happens, the only public measurement I could find is one person's own crawl. The mcp-pin findings file records fingerprinting 248 public MCP servers twice in a day on 3 September 2026, and reports 17 definitions where the schema or annotations moved while the description text stayed byte identical. Treat that as one self published dataset that nobody has reproduced, including me. It points the right way without proving anything, and it happens to be exactly the evidence the argument above wants, which is why it gets the caveat rather than a headline.

Prior art

Almost everything in the detection half of this has been written already, by people shipping tools, and it would be poor form to present any of it as new. Enjoy Kumawat made the pinned-manifest point in July and proposed a schema snapshot guard. Tsvetan Gerginov shipped mcpward, a CI tool that writes a lockfile and fails the build on breaking diffs. Alexey Spinov built a replay gate to separate real breakage from harmless additions, and wrote the counter-argument I'm extending above. There's more besides, including a well told version of the silent rename story and at least two continuous monitoring services built on the same idea.

The testing half is taken as well. Beyond Tian Pan's piece above, Specmatic ships a drift detector, there are practical guides to validating MCP tool schemas, and a conference talk proposal is doing the rounds on contract testing as a pipeline gate. Anyone presenting the testing idea as new in late September hasn't looked properly, and I nearly did.

What survives as mine is narrower, and it all sits in the middle of this piece. The specification negotiates a protocol version on every single request and gives a tool no version at all. listChanged covers membership while the thing your model reads is the shape. The deterministic ordering was specified to help prompt caches, and callers were never the point. And tool versioning was formally proposed a year ago and is sitting dormant. Those are four readings off two pages of the spec and one closed issue, and I haven't seen anyone put them next to each other.

Marking my confidence

Established. The tool definition has seven fields and none of them is a version, the listChanged wording covers the list and says nothing about shape, and the deterministic ordering request is justified by caching and prompt cache hit rates. All three read directly off the 2026-07-28 tools page. Both GitHub issues were opened on 23 September 2026 and both are quoted from the issue itself. SEP-1575 was opened on 30 September 2025 by Malay Kurwa, proposed a version field with client side constraints and server side validation, and is closed with a label reading that it has no sponsor.

Inferred. That the silent and loud distinction, which is Tian Pan's, is also the right place to put the gate. That comes from how the two issues differ, not from anything anyone has measured, and it is the load-bearing claim in this piece.

Unknown. Why SEP-1575 closed. There is no maintainer comment on the page and no sponsor is not the same statement as rejected.

Guess. A version field, or something doing its job, lands within two more revisions. Held loosely, and worth revisiting when the next revision ships.

What would change my mind. A dataset showing that additive-only changes cause as many real incidents as silent semantic ones. That would make the gating line wrong and push me back towards Gerginov's position of failing on everything.

The 2026-07-28 revision asked servers to return their tools in a stable order so that prompt caches hit more often, which is a sensible change and a real improvement. It also leaves the thing your model actually reads cached, deterministic and unversioned, which is a strange place to have arrived at while being one field away from somewhere else.