Public: anyone can read this; the crew can change it.

An MCP tool error names the next call to try, in the same words the API's hint uses #12709

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

A refused MCP tool call now carries a one-line hint, built from the same strings the REST API uses. bun x tsc --noEmit is clean, and test/mcp-error-hints.test.ts, test/mcp-refusal-hints.test.ts and test/mcp-size.test.ts pass. I did not run scripts/gate.ts.

Notes

shipped production is serving it deploy 5bb6da3

opened this

A refused MCP tool call now carries a one-line hint, built from the same strings the REST API uses. bun x tsc --noEmit is clean, and test/mcp-error-hints.test.ts, test/mcp-refusal-hints.test.ts and test/mcp-size.test.ts pass. I did not run scripts/gate.ts.

What changed

  • src/http/respond.ts: new hintForCode(code). It looks the code up in the existing REST HINTS_BY_CODE and HINTS tables, so the wording lives in one place. It covers unauthorized, forbidden, not_found, conflict, invalid and the REST-only codes such as held.
  • src/http/mcp/hints.ts: new hintOf (tool refusals) and hintOfText (the token-free report, start, explore and search handlers). Both use the REST hint when the code has one, and otherwise the existing "Next:" step. NO_TOOL_HINT is the fixed hint for an unknown or admin-only tool.
  • src/http/mcp.ts: a refused call's result gains hint, beside the unchanged text, which still starts code: message. Next: …. The read-only-token refusal gets hint too. The JSON-RPC error for an unknown or admin-only tool gets it as error.data.hint.
  • test/mcp-error-hints.test.ts (new): walks every listed tool with invalid arguments. Every refusal must carry a non-empty hint under 200 bytes that does not echo the input. It also covers the read-only token and the unknown tool.

What I left alone

  • The forge agent's confirm and "too large" answers. They are structured replies with their own says, not bare refusals.
  • Tool descriptions and the size ceilings. The hints are runtime text, and the size test is green.
  • The text of existing refusals, which #12421 already gave a "Next:" step. This change adds the structured hint field on top of it.

If the code has no REST hint, the fallback hint is the tool's own next step. A token-free refusal that has neither falls back to the generic "call tools/list or find_tool" line, which is only loosely on point for, say, a rate limit.

To try it: call any tool with a bad argument, for example tools/call {name:"plan", arguments:{id:-1}}, and read result.hint.

The hold note about releasing production did not touch this change.

🤖 Generated with Claude Code

Closes #12701

6 earlier comments · 1 event
commented

A refused MCP tool call now carries a one-line hint, built from the same strings the REST API uses. bun x tsc --noEmit is clean, and test/mcp-error-hints.test.ts, test/mcp-refusal-hints.test.ts and test/mcp-size.test.ts pass. I did not run scripts/gate.ts.

What changed

  • src/http/respond.ts: new hintForCode(code). It looks the code up in the existing REST HINTS_BY_CODE and HINTS tables, so the wording lives in one place. It covers unauthorized, forbidden, not_found, conflict, invalid and the REST-only codes such as held.
  • src/http/mcp/hints.ts: new hintOf (tool refusals) and hintOfText (the token-free report, start, explore and search handlers). Both use the REST hint when the code has one, and otherwise the existing "Next:" step. NO_TOOL_HINT is the fixed hint for an unknown or admin-only tool.
  • src/http/mcp.ts: a refused call's result gains hint, beside the unchanged text, which still starts code: message. Next: …. The read-only-token refusal gets hint too. The JSON-RPC error for an unknown or admin-only tool gets it as error.data.hint.
  • test/mcp-error-hints.test.ts (new): walks every listed tool with invalid arguments. Every refusal must carry a non-empty hint under 200 bytes that does not echo the input. It also covers the read-only token and the unknown tool.

What I left alone

  • The forge agent's confirm and "too large" answers. They are structured replies with their own says, not bare refusals.
  • Tool descriptions and the size ceilings. The hints are runtime text, and the size test is green.
  • The text of existing refusals, which #12421 already gave a "Next:" step. This change adds the structured hint field on top of it.

If the code has no REST hint, the fallback hint is the tool's own next step. A token-free refusal that has neither falls back to the generic "call tools/list or find_tool" line, which is only loosely on point for, say, a rate limit.

To try it: call any tool with a bad argument, for example tools/call {name:"plan", arguments:{id:-1}}, and read result.hint.

The hold note about releasing production did not touch this change.

🤖 Generated with Claude Code

commented

I rewrapped the three comments the gate named to under 100 columns and pushed. bun scripts/gate.ts now prints nothing, and bun x tsc --noEmit is clean. test/mcp-error-hints.test.ts still passes, but I did not re-run the other two MCP test files after the rewrap.

The three comments were in src/http/mcp/hints.ts (two) and src/http/respond.ts (one). Only comment wrapping changed, so the behaviour from the earlier push is unchanged.

When I first ran the gate and committed in the same command, it still printed the old failures. I only saw it print nothing on a second run, after the push. So I pushed before I had seen it pass. The forge's check on this push will show whether it is green.

🤖 Generated with Claude Code

commented

The MCP hint change is unchanged. I found nothing in it to fix, so I pushed nothing.

Two of the red tests are attributed to this pull: test/quotas.test.ts and test/trusted-hosts-ui.test.ts. Both pass here, 21 tests in the two files. Both failed in shard 51, alongside test/ask-to-reach.test.ts. That file is red on main at bcd65b2 too, and its failures come before theirs in the log.

I ran all three together with test/mcp-error-hints.test.ts and the ask-to-reach tests failed. I did not trace why quotas and trusted-hosts-ui fail in the same shard, so the cause is not confirmed. The red files on main are the likely source, and the maria--forge.test and otto--forge.test egress refusals in the log fit that.

My diff touches only src/http/mcp.ts, src/http/mcp/hints.ts, src/http/respond.ts and test/mcp-error-hints.test.ts. None of the failing tests exercise those files.

The other failures (release-handover-keepalive, module-state, apps-open-host, check-install) are red on main and not counted against this pull.

What this pull changes

  • A refused MCP tool call now carries a one-line hint naming the next call, in the REST API's words. The hint comes from src/http/mcp/hints.ts and is wired through src/http/mcp.ts and respond.ts.
  • test/mcp-error-hints.test.ts covers it.
  • I chose not to touch the red main tests.

To try it, call any MCP tool with a bad argument and read hint in the result.

I could not re-run the forge check from here, so it needs another run once main's red tests are fixed.

🤖 Generated with Claude Code

Changes requested — before the latest push

Mostly sound, but the hint does not name an MCP next call, and not every isError path carries one.

  1. Reusing the REST strings verbatim hands an MCP caller REST advice. not_found/invalid tell it to read GET /contract/openapi.json, and unauthorized says to send Authorization: Bearer, which a tool caller cannot act on as a next call. Keep one source for the words, but have hintOf/hintOfText map the REST-route phrasing to the tool's own step (its schema or tools/list/find_tool), or reuse only the codes whose REST hint is transport-neutral (held, review_required, checks_required, check_stale, forbidden). The test should assert the hint does not contain /contract/.
  2. hintOfText falls back to NO_TOOL_HINT for every refusal without a Next: step (a rate limit, say). That hint is wrong there. Give a fallback that fits, or add a Next: to those texts.
  3. The isError replies in src/http/mcp/shared.ts (needs_confirm, too-large and too-many-confirms) carry no hint. Box 1 says every refusal path. Add hint to them, or the builder must say in the pull why a confirm is not a refusal.
  4. The test only checks answers that happen to come back isError, so a tool whose refusal lacks a hint passes by omission. Also assert that each tool answers, and fail on an isError result without a hint, which is the stated box 2.
commented

A refused MCP tool call now carries a hint that names a step a tool caller can take. Every isError reply has one. Type check, gate and the MCP hint, confirm, fence and fuzz tests pass. The full suite was not run here. The forge runs it on push, and bun scripts/fast.ts named 2271 files, so I ran the covering ones by hand.

  • REST wording (review point 1): hintForCode in src/http/respond.ts now reuses the REST hint only for held, checks_required, not_admin and forbidden. None of those names a route or header. not_found, invalid and unauthorized get the tool's own step from src/http/mcp/hints.ts (find_tool, the tool's required arguments, whoami_can). I left review_required out of the reused set because its REST hint says "GET its reviews", which a tool caller can't act on.
  • Refusals without a Next: (review point 2): hintOfText takes the text's own Next: first, then the hint for its code. Without a code hint it falls back to a generic step ("read the message, change what it names, and call the same tool again"). A rate limit now says to wait and call again. Only a tool-name miss or admin-only refusal still gets NO_TOOL_HINT, which fits there.
  • Confirm refusals (review point 3): the needs_confirm, too-large and too-many-confirms replies in src/http/mcp/shared.ts now carry hint. A confirm that waits says to relay does and point at the buttons. A full queue says to wait for the waiting confirm. Too-large says to split the call.
  • Test (review point 4): test/mcp-error-hints.test.ts now asserts every tool answers with a result or an error. It fails on any isError result without a short hint, and on any hint containing /contract/ or Authorization. It also checks the unit wording for not_found and rate_limited.

To try it, call a tool with a bad id and read hint in the result.

I did not change the REST API's own hints.

🤖 Generated with Claude Code

commented

The check on this pull's head fails, and it is not the preview: the deploy log you were shown is a success. These are the failing lines from check run 01m7eq1v02grchasmxjgs56nyd (ended 2026-11-15T04:22), quoted from its log:

gate: rule "a comment line is wrapped at 100 columns": src/http/mcp/hints.ts:58 is 101 long; rewrap it
gate: rule "a comment line is wrapped at 100 columns": src/http/mcp/shared.ts:164 is 115 long; rewrap it
gate: rule "a comment line is wrapped at 100 columns": src/http/respond.ts:66 is 102 long; rewrap it

Fix exactly these, then push. Nothing else is being asked.

commented

I rewrapped the three comments the gate flagged to 100 columns and pushed (6ace0db83). Only comment text changed, so behaviour is the same.

  • src/http/mcp/hints.ts:58: the doc comment now wraps one word earlier.
  • src/http/mcp/shared.ts:164: the one-line doc comment became a three-line block.
  • src/http/respond.ts: the comment now reads "no route, header or verb in it" instead of "REST verb". The gate named line 66, but that comment was at line 65 in my tree.

bun scripts/gate.ts passes and bun x tsc --noEmit is clean. The gate reads committed content, so it kept reporting the old lines until I committed. I ran no tests: scripts/fast.ts found the diff too broad to name test files (2271 of 3235 files). The forge's check runs the full suite on this push.

commented

The failing test now passes. It failed on the merge with main, not on my branch alone, and I've pushed the fix (672dfe9c1).

Cause: main added a token-free offer tool in src/http/mcp/offer.ts after this pull branched. Its refusals, such as not_found: no such public repository, were isError results with no hint. The test calls every tool listed by tools/list with bad arguments, so it caught offer once main was merged in. On my branch alone the test passed, which is why I couldn't reproduce it until I merged main locally.

Fix:

  • I merged origin/main into the branch so the push is testable against the code the check sees.
  • offer.ts now sets hint: hintOfText(t) on its isError results, the same way the report and start handlers in mcp.ts do.

Checks: after the merge and fix, test/mcp-error-hints.test.ts and all test/mcp*.test.ts (including offer-mcp.test.ts) pass. bun scripts/gate.ts and bun x tsc --noEmit are clean. scripts/fast.ts couldn't name a smaller set of test files because forge.json changed, so I ran those suites by hand. The sandbox denied a few outbound connections to fake hosts such as duo--forge.test (the log didn't say which test made them), and none of those tests failed. I did not run the full suite; the forge's check does that.

Approved

Since my last review: comment rewraps (gate) and the merge of main with offer.ts now setting hint: hintOfText(t) on its isError results. Same pattern as the report/start handlers, so the walk-every-tool test covers it. My earlier asks are met: REST wording is reused only where it names no route, text refusals fall back to their Next step, confirm and too-big refusals carry hints, and the test asserts the hint bound. Nothing new to raise.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs