Public: anyone can read this; the crew can change it.

One table-driven test: every MCP tool that writes refuses a read-only token #11501

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

Adds `test/mcp-ro-refuses-writes.test.ts`, so a single test now holds the read-only/read-write line on `/mcp` for every tool. No tool was found writing for a read-only (`ro`) token, so nothing under `src/` changes.

shipped production is serving it deploy a5a2512

opened this

Adds test/mcp-ro-refuses-writes.test.ts, so a single test now holds the read-only/read-write line on /mcp for every tool. No tool was found writing for a read-only (ro) token, so nothing under src/ changes.

The test boots one forge with an admin, a repository and an ro token for the admin, so admin-only tools are listed too. It walks tools/list as that token and checks:

  • Tools without readOnlyHint: each is called with arguments that get past its own read paths (dryRun: false, apply: true, an action that isn't list, every readsWithout key set). It must answer forbidden: this token may read, not act and no table may change its row count. Failures name the tool and the tables that moved.
  • Read-path coverage: a separate check fails, naming the tool, if those arguments still look like a read to any tool's readsWhen. This catches a new tool whose read path the test's arguments don't get past. I confirmed it by temporarily making sessions.readsWhen always true; the test failed and named sessions.
  • Tools with readOnlyHint: true: each is called with {} and with { repo }. No table may gain or lose a row.
  • Admin-scope tools: these are never listed to an ro token. Each one called directly must answer no tool named <name> and write nothing.

Things to look at:

  • Two tables are left out of the row count: route_timing and usage_count. Every request adds to them and a timer writes them about a second later, so they would wrongly blame whichever tool was running at the time (they showed up against take_back on the first run).
  • report and start are skipped. They can be called without any token, so an ro token gives no less access; test/fuzz-mcp.test.ts already covers them.
  • Limit: each writing tool gets one generic set of arguments, so a tool whose writes need specific fixtures is only checked up to the refusal. That's enough here, because the refusal happens before the tool runs.

The new test passes (6 tests, about 4.7 s, under the 10 s limit). bun scripts/gate.ts, bun x tsc --noEmit and bun test test/mcp-size.test.ts also pass. I couldn't update the plan on the issue: calling the keep_plan tool on $FORGE_URL/mcp got a 404.

Try it: bun test test/mcp-ro-refuses-writes.test.ts

🤖 Generated with Claude Code

Closes #11493

commented

Adds test/mcp-ro-refuses-writes.test.ts, so a single test now holds the read-only/read-write line on /mcp for every tool. No tool was found writing for a read-only (ro) token, so nothing under src/ changes.

The test boots one forge with an admin, a repository and an ro token for the admin, so admin-only tools are listed too. It walks tools/list as that token and checks:

  • Tools without readOnlyHint: each is called with arguments that get past its own read paths (dryRun: false, apply: true, an action that isn't list, every readsWithout key set). It must answer forbidden: this token may read, not act and no table may change its row count. Failures name the tool and the tables that moved.
  • Read-path coverage: a separate check fails, naming the tool, if those arguments still look like a read to any tool's readsWhen. This catches a new tool whose read path the test's arguments don't get past. I confirmed it by temporarily making sessions.readsWhen always true; the test failed and named sessions.
  • Tools with readOnlyHint: true: each is called with {} and with { repo }. No table may gain or lose a row.
  • Admin-scope tools: these are never listed to an ro token. Each one called directly must answer no tool named <name> and write nothing.

Things to look at:

  • Two tables are left out of the row count: route_timing and usage_count. Every request adds to them and a timer writes them about a second later, so they would wrongly blame whichever tool was running at the time (they showed up against take_back on the first run).
  • report and start are skipped. They can be called without any token, so an ro token gives no less access; test/fuzz-mcp.test.ts already covers them.
  • Limit: each writing tool gets one generic set of arguments, so a tool whose writes need specific fixtures is only checked up to the refusal. That's enough here, because the refusal happens before the tool runs.

The new test passes (6 tests, about 4.7 s, under the 10 s limit). bun scripts/gate.ts, bun x tsc --noEmit and bun test test/mcp-size.test.ts also pass. I couldn't update the plan on the issue: calling the keep_plan tool on $FORGE_URL/mcp got a 404.

Try it: bun test test/mcp-ro-refuses-writes.test.ts

🤖 Generated with Claude Code

Approved

A test-only pull: one table-driven walk of tools/list as an ro token. Writing tools must answer the plain refusal and move no table; readOnlyHint tools must move no table; admin-scope tools are refused by name. It names the offending tool, and has a guard that fails when a tool's readsWhen swallows the write arguments. No src change is needed, since no tool was found writing. It adds no page, so the TUI and docs rules do not apply, and README is untouched. Two next steps: (1) the empty catch in rows() hides a table that cannot be counted, so count it or list the skipped names in an assertion; (2) the report/start skip is hard-coded, so derive it from the tools that need no token, so a third token-free tool does not need a second edit.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs