Public: anyone can read this; the crew can change it.

Every MCP tool answers hostile arguments with a named refusal, never a 500: a fuzz test walks tools/list #11021

Closed opened this has it
opened this

Agents from any harness call the MCP with whatever their model invents; a crash or a 500 there is a trust hole and noise for the owner.

Done when:

  • [x] New test/fuzz-mcp.test.ts boots one forge (the pattern in test/mcp-routes-described.test.ts), reads tools/list, and for each tool sends seeded hostile arguments built from its inputSchema: wrong types, missing required fields, unknown keys, 10 kB strings, negative and huge numbers, ../ paths, nested objects.
  • [x] Every call answers a JSON-RPC result with isError or a JSON-RPC error carrying a message; none is an HTTP 500, none throws, none hangs past 1 s.
  • [x] Calls run as a signed-in non-admin and as anonymous; anonymous reaches nothing beyond report.
  • [x] Any tool found crashing is fixed in the same pull at its argument check.
  • [ ] The file runs under 10 s.

Files: test/fuzz-mcp.test.ts (new); the failing tool's handler under src/http only if one crashes. Risk: tools with side effects (spawn, deploy) must run against the test forge only; skip none silently, name any exclusion in the file.

commented

Reviewed pull #11026 at its head: ticked New test/fuzz-mcp.test.ts boots one forge (the pattern in test/mcp-routes-described.test.ts), reads tools/list, and for each tool sends seeded hostile arguments built from its inputSchema: wrong types, missing required fields, unknown keys, 10 kB strings, negative and huge numbers, ../ paths, nested objects.; Every call answers a JSON-RPC result with isError or a JSON-RPC error carrying a message; none is an HTTP 500, none throws, none hangs past 1 s.; Calls run as a signed-in non-admin and as anonymous; anonymous reaches nothing beyond report.; Any tool found crashing is fixed in the same pull at its argument check.. left open: The file runs under 10 s. — The author reports about 6 s, but I could not run it here and nothing in the file enforces the budget; the member test's timeout is 60 s..

commented

Left open after pull #11026 landed: 1 unticked item remains (The file runs under 10 s.). The planner files each as its own issue and closes this one; a person may close it, tick what is done, or say what is left. Until then the crew does not pick it again.

commented

Already done on main in 7b3f969d6 (#11026). The open box is met: bun test test/fuzz-mcp.test.ts runs in 6.4 s (3 pass, 487 expects), measured by the planner on main on 2026-11-10.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs