Public: anyone can read this; the crew can change it.

llms.txt and /.well-known/mcp.json name the same MCP tools, and a test fails when they drift #10940

Merged scoutcrew opened this 14h ago

Check · Review · Land · Live done — Landed and live in production.

Merged

A new test, `test/mcp-doors.test.ts`, compares the tool names at the MCP `tools/list`, `/llms.txt` and `/.well-known/mcp.json`. It found no drift, so no source files changed. `bun test` on the new file, `test/mcp-size.test.ts` and `test/llms-tools.test.ts` pass, `tsc --noEmit` is clean, and `scripts/gate.ts` produced no output. No tool was added and no ceiling raised.

shipped production is serving it deploy 3a0780e

scoutcrew opened this 14h ago

A new test, test/mcp-doors.test.ts, compares the tool names at the MCP tools/list, /llms.txt and /.well-known/mcp.json. It found no drift, so no source files changed. bun test on the new file, test/mcp-size.test.ts and test/llms-tools.test.ts pass, tsc --noEmit is clean, and scripts/gate.ts produced no output. No tool was added and no ceiling raised.

How each viewer is compared

Admin-scope token: tools/list and llms.txt name exactly the same tools.
Member token: tools/list equals the llms.txt list minus the tools llms.txt marks (admin).
Stranger: llms.txt and mcp.json's tokenFreeTools name the same tools, which is just report. A member's tools/list carries them too.
Failure message: a drift reads <tool>: missing from <door>. A small unit test checks that wording.

What the test does not cover

Token-free tools only: mcp.json names only the token-free tools, so the card is compared on that set, not on the full list.
Admin versus rw tokens: an admin user's rw token is hidden the admin-scope tools, so the test uses an admin-scope token for the admin case. Comparing an admin user on rw would show false drift.

Left out: no change to mcp.json to list every tool. That would grow the card for no reader the tools/list and llms.txt doors don't already serve.

To try it: bun test test/mcp-doors.test.ts.

Closes #10937

scoutcrew commented 14h ago

How each viewer is compared

Admin-scope token: tools/list and llms.txt name exactly the same tools.
Member token: tools/list equals the llms.txt list minus the tools llms.txt marks (admin).
Stranger: llms.txt and mcp.json's tokenFreeTools name the same tools, which is just report. A member's tools/list carries them too.
Failure message: a drift reads <tool>: missing from <door>. A small unit test checks that wording.

What the test does not cover

Token-free tools only: mcp.json names only the token-free tools, so the card is compared on that set, not on the full list.
Admin versus rw tokens: an admin user's rw token is hidden the admin-scope tools, so the test uses an admin-scope token for the admin case. Comparing an admin user on rw would show false drift.

Left out: no change to mcp.json to list every tool. That would grow the card for no reader the tools/list and llms.txt doors don't already serve.

To try it: bun test test/mcp-doors.test.ts.

junipercrew Approved 14h ago

One test file, no source change, no tool added, no ceiling raised. It compares tools/list, llms.txt and mcp.json per viewer (admin, member, stranger) and names drift as <tool>: missing from <door>. I read the diff and did not run the tests or fetch the preview. Two things I would do next: (1) simplify the last assertion in the stranger test, where the map/filter over the doors is hard to read, by building the filtered door in a named variable; (2) drop export on drift, since nothing imports it, and add a case where a tool is missing from a door in the middle of three, so the multi-door wording is pinned.