Public: anyone can read this; the crew can change it.

docs/MCP.md names every tool the MCP lists: /.well-known/mcp counts 202 tools, a test compares the doc to tools/list #14397

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

I added test/mcp-doc-matches-tools.test.ts. docs/MCP.md already named every tool the MCP lists, so the doc needed no edit. The new test and test/mcp-size.test.ts pass, as does bun x tsc --noEmit, and scripts/gate.ts --fix changed nothing.

Notes

shipped production is serving it deploy 4ec2c16

opened this

I added test/mcp-doc-matches-tools.test.ts. docs/MCP.md already named every tool the MCP lists, so the doc needed no edit. The new test and test/mcp-size.test.ts pass, as does bun x tsc --noEmit, and scripts/gate.ts --fix changed nothing. I did not run the fast-tier set or the full suite here. scripts/fast.ts named 105 test files, which it estimates at over 8 minutes, so the forge's check will run it.

  • Doc against tools: the test walks every page of tools/list as an admin with an admin-scope token and compares the names with the ## Tools table in docs/MCP.md. A tool missing from either side fails with its name.
  • Manifest against server: it also checks that /.well-known/mcp reports the same number of tools as tools/list. The 202 in the issue is the count when the SSH switch is on. With it off, as in the test, both say 201.
  • Exception I chose: ssh_keys is the one tool that exists only while the SSH switch is on, so the test leaves its doc row out of the comparison. The row stays in the doc.
  • Why this should hold: the doc, tools/list and the manifest all follow the tool registry, so a tool added without a row, or removed without deleting its row, now fails by name.
  • Left out: the test does not check the per-tool descriptions in the doc, only that each tool has a row. The ceilings are untouched.

To try it, run bun test test/mcp-doc-matches-tools.test.ts. It lives in test/, so the fast tier picks it up.

🤖 Generated with Claude Code

Closes #14393

commented

I added test/mcp-doc-matches-tools.test.ts. docs/MCP.md already named every tool the MCP lists, so the doc needed no edit. The new test and test/mcp-size.test.ts pass, as does bun x tsc --noEmit, and scripts/gate.ts --fix changed nothing. I did not run the fast-tier set or the full suite here. scripts/fast.ts named 105 test files, which it estimates at over 8 minutes, so the forge's check will run it.

  • Doc against tools: the test walks every page of tools/list as an admin with an admin-scope token and compares the names with the ## Tools table in docs/MCP.md. A tool missing from either side fails with its name.
  • Manifest against server: it also checks that /.well-known/mcp reports the same number of tools as tools/list. The 202 in the issue is the count when the SSH switch is on. With it off, as in the test, both say 201.
  • Exception I chose: ssh_keys is the one tool that exists only while the SSH switch is on, so the test leaves its doc row out of the comparison. The row stays in the doc.
  • Why this should hold: the doc, tools/list and the manifest all follow the tool registry, so a tool added without a row, or removed without deleting its row, now fails by name.
  • Left out: the test does not check the per-tool descriptions in the doc, only that each tool has a row. The ceilings are untouched.

To try it, run bun test test/mcp-doc-matches-tools.test.ts. It lives in test/, so the fast tier picks it up.

🤖 Generated with Claude Code

Approved

A tests-only change: it fails by tool name in both directions and checks the manifest count against tools/list. The doc already had every row, so nothing needed fixing and no ceiling moved. I did not run the new test, because the review checkout is the base and does not contain it. Next: (1) derive the ssh_keys exception from the SSH switch (or add a second run with it on) so the 202 case is covered, not only 201; (2) check the descriptions column too, or at least that no row is empty.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs