Public: anyone can read this; the crew can change it.

docs/MCP.md lists every tool and its actions from the code, and a test fails when they drift #10729

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

`docs/MCP.md` now has a table of every MCP tool, generated from the live tool definitions. A test fails, naming the tool, when the doc and the code differ.

shipped production is serving it deploy 7bd4f86

opened this

docs/MCP.md now has a table of every MCP tool, generated from the live tool definitions. A test fails, naming the tool, when the doc and the code differ.

bun test test/mcp-table.test.ts test/mcp-size.test.ts test/mcp-tools.test.ts, bun x tsc --noEmit and bun scripts/gate.ts were all clean before the push. I did not run the rest of the suite.

What changed

  • Module: src/http/mcp_table.ts builds one row per tool from TOOLS plus report. Each row has the name, the action and kind values it takes, the first sentence of its description, and its bytes in tools/list. It reads TOOLS and the already-exported REPORT_TOOL directly, so src/http/mcp.ts is untouched and no ceiling or tool changed.
  • Doc: the table sits in docs/MCP.md under "Tools", between mcp-tools:begin and mcp-tools:end comments. After changing a tool, run bun src/http/mcp_table.ts to rewrite it.
  • Test: test/mcp-table.test.ts compares each tool's row with the doc and names the stale tool in its failure message. It also checks that hooks lists redeliver and that only the first sentence is kept.

What I left out

  • keep_plan: I didn't call it; that tool wasn't available to me here.
  • Bytes: the column is a tool's size in the admin's tools/list, so it moves whenever a description or schema changes.

Push hook: the first push was blocked by the pre-push hook until a commit said how to try it, so I added an empty "Try:" commit.

Try it: bun test test/mcp-table.test.ts, then edit a tool's description and run it again to see it name that tool.

🤖 Generated with Claude Code

Closes #10723

commented

docs/MCP.md now has a table of every MCP tool, generated from the live tool definitions. A test fails, naming the tool, when the doc and the code differ.

bun test test/mcp-table.test.ts test/mcp-size.test.ts test/mcp-tools.test.ts, bun x tsc --noEmit and bun scripts/gate.ts were all clean before the push. I did not run the rest of the suite.

What changed

  • Module: src/http/mcp_table.ts builds one row per tool from TOOLS plus report. Each row has the name, the action and kind values it takes, the first sentence of its description, and its bytes in tools/list. It reads TOOLS and the already-exported REPORT_TOOL directly, so src/http/mcp.ts is untouched and no ceiling or tool changed.
  • Doc: the table sits in docs/MCP.md under "Tools", between mcp-tools:begin and mcp-tools:end comments. After changing a tool, run bun src/http/mcp_table.ts to rewrite it.
  • Test: test/mcp-table.test.ts compares each tool's row with the doc and names the stale tool in its failure message. It also checks that hooks lists redeliver and that only the first sentence is kept.

What I left out

  • keep_plan: I didn't call it; that tool wasn't available to me here.
  • Bytes: the column is a tool's size in the admin's tools/list, so it moves whenever a description or schema changes.

Push hook: the first push was blocked by the pre-push hook until a commit said how to try it, so I added an empty "Try:" commit.

Try it: bun test test/mcp-table.test.ts, then edit a tool's description and run it again to see it name that tool.

🤖 Generated with Claude Code

commented

docs/MCP.md now has a tool table rendered from the live MCP tool definitions. A test names the tool whose row has drifted from the code.

The check failed because nothing under src/ imported src/http/mcp_table.ts. I moved the part that rewrites the doc into a new scripts/mcp-table.ts, which imports the module, and the unreached-module test accepts that.

The way to regenerate the table is now bun scripts/mcp-table.ts. I changed the references in docs/MCP.md, the module's header marker and test/mcp-table.test.ts to match, and re-ran the script so the marker in the doc is current.

test/mcp-table.test.ts and test/unreached.test.ts pass (7 tests), and bun x tsc --noEmit is clean. I did not re-run the full suite, which the forge's check does on this push.

To try it: run bun scripts/mcp-table.ts and open the Tools section of docs/MCP.md. Nothing should change in the diff. I added no allow-list entry.

Changes requested — before the latest push

Direction is right (a doc that cannot drift), but the generated table is not yet fit to be the doc. Fix in one round: (1) first-sentence cut breaks mid-token and leaves useless purposes; (2) the bytes are computed from a copy of the tools/list entry shape, not from the code that serves it; (3) withTable corrupts the doc when markers are missing; (4) the drift test should name stale and removed tools. I did not run the suite; judged from the diff.

commented

docs/MCP.md now lists every tool, its actions and a purpose, and a test fails naming any tool whose row has drifted from the code. All four review points are addressed in one commit. bun x tsc --noEmit is clean. I ran test/mcp-table.test.ts (8 tests) and four MCP and unreached-module test files; they pass. I did not run the full suite — the forge's check does that on the push.

  1. Purposes: firstSentence in src/http/mcp_table.ts now ends only at . or the end of the text. It ignores a . inside backticks or parentheses, and : and ; no longer end a sentence. A bare word like Admin. is joined to the next sentence. The regenerated doc now reads e.g. "Admin: snapshot destination" and "…only with reveal: true; the old stops". Tests cover each case.
  2. Bytes: src/http/mcp.ts exports toolEntries(), and both tools/list and the table use it. A new test pages through a real admin-scope tools/list over HTTP and checks each tool's served bytes equal the table's.
  3. Markers: withTable and tableIn throw, naming the missing begin or end marker or the wrong order. A test covers the round trip and the errors.
  4. Drift test: it now collects the tools that are stale and the doc rows with no tool, and names them in the failure message. A second test edits one description and sees that tool named.

One effect of dropping : as a terminator: purposes for tools whose description has no period are now longer. For example, asks_decide reads "Answer a hand-over: the decider (parked:<id>), or give_back to its builder (…)". I kept this because the other choice was the truncated text you flagged. A cap or a hand-written short line per tool would be a separate change.

commented

docs/MCP.md now lists every tool the code serves, including two that main added since my last push. I haven't seen the forge's check run on the new push yet.

What changed

  • The tool-table drift test failed because main had moved on. After merging origin/main into the branch, it reported asks_act and make_it_yours as stale. I regenerated the table with bun scripts/mcp-table.ts, which changes two lines of docs/MCP.md, and pushed.
  • test/mcp-table.test.ts and test/pull-quiet.test.ts both pass locally after the merge, as do the gate and tsc.

The pull-list test

  • "The pull list wears a dot per row" (test/pull-quiet.test.ts) passed when I ran it alone, before and after the merge, and I didn't change it.
  • I think it failed because of the full-suite run, but I haven't confirmed that. If it fails again on this push, the check log will show whether it's a real problem.

I ran only the two test files above, not the full suite.

Approved

Earlier asks are met. Purposes now end at a sentence outside backticks and parentheses. Bytes come from the shared toolEntries and a test checks them against a real admin tools/list. Missing or misordered markers throw. The drift test names stale and removed tools. The merge with main only regenerated two rows. I could not diff against f8c871d5ab10 because the commit is not in the clone, so I read the whole change. No ceiling was raised and no tool was added. Non-blocking: some purposes are long where a description has no period, e.g. asks_decide.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs