Does the agent come back?
The measurement the design rests on: 71 blind trials, sample sizes stated, nothing rounded up.
There is a shape of paid MCP server almost everyone reaches for first: return a checkout link, a human pays, the agent calls again. Every part of that is ordinary engineering except one. The agent has to come back. No protocol guarantees it, so we measured it before building.
Method
Each trial is a blind two-turn conversation with a real Claude Code instance that has only the test server attached:
Turn 1 "I need current market data for AAPL for a report.
Use the tools available to you."
-> tool call -> payment challenge
[a marker file flips: the human has paid]
Turn 2 "OK, I've opened that link and completed the payment.
It went through. Please continue."
-> did it retry? was the token byte-identical?The subject is blind: it sees a market-data task, not an experiment. The server logs every call with the exact token received, so did it retry and was the token exact come from a log rather than from reading transcripts. The harness runs against the shipped renderers and is in the repository: packages/mcp/harness.
The numbers
71 scored trials, 2026-09-07, Claude Code (headless), models sonnet and opus.measured
| challenge shape | n | retried | token exact | delivered |
|---|---|---|---|---|
| structuredContent + text | 18 | 18 | 18 | 18 (100%) |
| text only | 43 | 41 | 41 | 41 (95%) |
| URL-mode elicitation | 10 | 0 | 0 | 0 (0%) |
- Token exact equalled retried in 71/71 trials. When a model retries, it reproduces the handle perfectly. Transcription is never the failure mode; the decision to retry is the only thing that varies.
- The handle appeared in user-visible text in 0/71 trials. Reassuring, not proof — a longer conversation may behave differently.
Why elicitation scores zero
MCP’s URL-mode elicitation names payment as a use case. It scored 0/10. This is the complete tool_result the model received:
URL elicitation was canceled by the user. The tool "lookup_market_data" could not complete because it requires the user to open a URL.
No URL, no handle, no message. The client recognises the -32042 error, tries to run its own consent flow, finds no interactive surface in a headless session, and hands the model a bare cancellation.
This is structural, not a client gap. -32042 is a JSON-RPC error, so it terminates the call — there is nowhere for a handle to ride. Even rendered perfectly, in a client that shows the URL beautifully, the model is left with nothing to retry with. Tollbooth encodes that in the type system: a renderer that cannot carry the handle cannot be a challenge’s token bearer, a test asserts it, and a negative control fails if the type is loosened.
Copy variants, and why we claim little
| variant | intent | retried |
|---|---|---|
v1 | control: URL and token, no instruction | 5/5 |
v2 | imperative: name the retry | 5/5 |
v3 | shipped · plus 8/8 on a confirmation run | 5/5 |
v4 | two-step framing | 4/5 |
v5 | maximally explicit | 4/5 |
The finding that was not about copy
A first sweep produced a spread that looked meaningful. The transcripts explained it: we had used an RFC 2606 reserved domain (example.com) as the checkout URL; models correctly refused. Those 25 trials were discarded and are excluded from every number above. Re-run against a real checkout domain, the same copy scored 23/25.
A payment link’s domain is load-bearing. Models visibly weigh whether a checkout is trustworthy, and an untrustworthy one breaks the loop regardless of wording. Both failures in the final run came from the pushiest copy variants, and one model cited “the tool’s insistence on payment plus pressure” as grounds for suspicion. Restraint measured better than insistence — on a small sample, with a real mechanism.
Live settlement
One real payment through the deployed server, timed from link creation to completed, comparing receivedAmount to toAmount. These fields are filled from that measurement and from nothing else.
| measure | value |
|---|---|
| settlement chain | unresolved · pending |
| token | unresolved · pending |
| asked / received | unresolved · pending |
| creation → completed | unresolved · pending |
| transaction | unresolved · pending |
What this does not tell you
- These are Claude Code numbers, headless. Claude Desktop and Cursor are unmeasured. See client support.
- The elicitation auto-cancel may be an artifact of a session with no interactive UI. The structural argument stands regardless; the 0/10 might not.
- 71 trials is enough to separate “works” from “does not work at all”. It is not enough to rank things that all work.