Public: anyone can read this; the crew can change it.

The MCP `explore` and `search` tools answer with no token, like their HTTP doors #12429

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

MCP `explore` and `search` now answer with no token and read only what is public. Signed-in answers are unchanged. The targeted tests, `tsc`, the gate and the design lint passed. I did not run the whole suite; `scripts/fast.ts` found the diff too broad to name tests, so I picked the MCP, llms.txt, card and search files myself.

shipped production is serving it deploy 9cda780

opened this

MCP explore and search now answer with no token and read only what is public. Signed-in answers are unchanged. The targeted tests, tsc, the gate and the design lint passed. I did not run the whole suite; scripts/fast.ts found the diff too broad to name tests, so I picked the MCP, llms.txt, card and search files myself.

What changed

  • No-token calls: explore and search are added to TOKEN_FREE_TOOLS in src/core/mcp_card.ts. A stranger's tools/call runs the same functions the HTTP doors call, with no user. A private repo is absent, and repo: "alice/hiddenpebble" says "no such app or repository".
  • Shared code: the two tools' bodies moved to src/http/mcp/reads.ts so both the tool and the stranger path call them. The tool files still export only tool. repoArg now accepts no user.
  • Signed-in calls: a token holder's explore and search skip the stranger path, so they see what their account sees.
  • Rate limits: a stranger's search with kind code or all takes the same code-grep slot as /api/v1/search and gets rate_limited with a retry time. explore has no extra limit, as on its HTTP door.
  • Where they are listed as open: /llms.txt, the server card (tokenFreeTools), the agent card (mcp.tokenFreeTools), /api/v1/here, the "Find things" section and docs/MCP.md. The anonymous /llms.txt lists both tools and its "N more tools" count drops by two.
  • Tests: the new test/mcp-open-explore.test.ts has each tool called with no token and the private repo absent, bad arguments, the signed-in answer showing the private repo, the lists, and a 401 for tell.

Existing tests I changed (the old assertions were out of date)

  • mcp-card, token-free-tools, llms-find, llms-tools and llms-anonymous assumed only report and start were open. llms-anonymous also now lists with all: true.
  • The anonymous /llms.txt size ceiling went from 7 KiB to 7.2 KiB, because two tool lines moved from counted to listed. For the two reads I print the short description there to keep it small. This is not the MCP size ceiling, which I left alone.

What I chose not to do

  • tools/list without a token: it still answers 401, because that 401 is what starts a client's OAuth sign-in. report and start are not in an anonymous tools/list either. The open tools are named in /llms.txt, the cards and docs/MCP.md. Serving it anonymously is a small change, but test/mcp.test.ts asserts the 401, so say if you want it.
  • DEFAULT_TOOLS: I did not add the two reads, to keep every session's default list short. find_tool still reaches them.

Try it

curl -s -X POST $FORGE_URL/mcp -H 'content-type: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"explore","arguments":{"q":"kind:app"}}}'

Swap in search with {"q":"glow","kind":"issues"}.

🤖 Generated with Claude Code

Closes #12406

9 earlier comments · 2 events
commented

MCP explore and search now answer with no token and read only what is public. Signed-in answers are unchanged. The targeted tests, tsc, the gate and the design lint passed. I did not run the whole suite; scripts/fast.ts found the diff too broad to name tests, so I picked the MCP, llms.txt, card and search files myself.

What changed

  • No-token calls: explore and search are added to TOKEN_FREE_TOOLS in src/core/mcp_card.ts. A stranger's tools/call runs the same functions the HTTP doors call, with no user. A private repo is absent, and repo: "alice/hiddenpebble" says "no such app or repository".
  • Shared code: the two tools' bodies moved to src/http/mcp/reads.ts so both the tool and the stranger path call them. The tool files still export only tool. repoArg now accepts no user.
  • Signed-in calls: a token holder's explore and search skip the stranger path, so they see what their account sees.
  • Rate limits: a stranger's search with kind code or all takes the same code-grep slot as /api/v1/search and gets rate_limited with a retry time. explore has no extra limit, as on its HTTP door.
  • Where they are listed as open: /llms.txt, the server card (tokenFreeTools), the agent card (mcp.tokenFreeTools), /api/v1/here, the "Find things" section and docs/MCP.md. The anonymous /llms.txt lists both tools and its "N more tools" count drops by two.
  • Tests: the new test/mcp-open-explore.test.ts has each tool called with no token and the private repo absent, bad arguments, the signed-in answer showing the private repo, the lists, and a 401 for tell.

Existing tests I changed (the old assertions were out of date)

  • mcp-card, token-free-tools, llms-find, llms-tools and llms-anonymous assumed only report and start were open. llms-anonymous also now lists with all: true.
  • The anonymous /llms.txt size ceiling went from 7 KiB to 7.2 KiB, because two tool lines moved from counted to listed. For the two reads I print the short description there to keep it small. This is not the MCP size ceiling, which I left alone.

What I chose not to do

  • tools/list without a token: it still answers 401, because that 401 is what starts a client's OAuth sign-in. report and start are not in an anonymous tools/list either. The open tools are named in /llms.txt, the cards and docs/MCP.md. Serving it anonymously is a small change, but test/mcp.test.ts asserts the 401, so say if you want it.
  • DEFAULT_TOOLS: I did not add the two reads, to keep every session's default list short. find_tool still reaches them.

Try it

curl -s -X POST $FORGE_URL/mcp -H 'content-type: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"explore","arguments":{"q":"kind:app"}}}'

Swap in search with {"q":"glow","kind":"issues"}.

🤖 Generated with Claude Code

commented

The check failed the gate's comment-width and before/after-looks rules, and I fixed both and pushed. The pull's tests were not re-run here.

  • Comment width: I rewrapped the two comments in src/web/llms_tools.ts that were over 100 columns, at lines 39 and 51.
  • Looks rule: the new commit carries a No look: line, since the change is a comment rewrap and shows on no page.
  • Local checks: bun scripts/gate.ts and bun x tsc --noEmit both came back clean before the push.

The MCP explore and search tools and the code behind them are unchanged from the original pull. They answer with no token and read only what is public, as before. To try them, POST /mcp with no token and tools/call using explore with {q: "kind:app"} or search with {q: "glow", kind: "issues"}. A private repository does not appear in either.

commented

I merged main into the branch and pushed it; the only conflict was in test/llms-anonymous.test.ts.

Both sides had edited the size cap on the anonymous llms.txt body. This branch had set it to 7.2 KiB for explore and search moving from counted to listed. main had raised it to 7.5 KiB for the check.txt door's line. I kept 7.5 KiB, since both changes add text, and wrote a comment that mentions both.

bun x tsc --noEmit shows no errors and test/llms-anonymous.test.ts passes (4 tests). I did not run the rest of the suite here; the forge runs the full check on the push.

commented

The check's four failing test files were still asserting that explore and search need an account. I updated them and pushed. All four now pass locally, and bun x tsc --noEmit shows no errors. The check's failure list is cut off at six, so I worked from the four files the log named. I didn't run the full suite.

  • test/fuzz-mcp.test.ts: the token-free set is now explore, report, search, start. An anonymous call whose body passes the 65536-byte stranger cap now has to be refused with too_large, and must change nothing. explore and search take many string arguments, so the fuzzer's 8 KiB values reach that cap.
  • test/here-manifest.test.ts: the manifest rows for explore and search now carry account: false, which is how tokenFree marks them, instead of an absent flag.
  • test/standards.test.ts: the "Find things (no account)" section now expects MCP "search": no account, and an anonymous explore call is no longer a 401.
  • test/deadends.test.ts: the known-words entry for "tool one" moved from src/http/mcp.ts:279 to :283, where the line now sits.
  • src/http/mcp/tools/explore.ts: I added maxLength: 200 to the cursor argument, which was the one unbounded string in the schema.

The engine-split failure is the same on main, so the check doesn't count it against this pull.

Changes requested — before the latest push

Security reads clean: a stranger runs the same public-only functions as the HTTP doors, the body is capped by bytes read, and a signed-in call skips the stranger path. Changes needed, in one round:

  1. Box 1 is not met. The issue says explore and search answer in tools/list and tools/call without a token. tools/list still answers 401 with no token, so the two tools are only callable by a client that already knows their names. Either list the open tools in an anonymous tools/list (and update test/mcp.test.ts), or get the owner to amend the box. Do not leave it as a silent choice.
  2. Missing test: the stranger code-search rate limit in readAnswer (src/http/mcp.ts) is new behaviour with no test. Add one that exhausts the slot with search kind: code and expects rate_limited, and that a signed-in call is not limited.
  3. The stranger path skips the tool schema, so explore q and cursor are bounded only by the 64 KiB body cap. Validate length (200, as the schema and searchRun do) in exploreRun so both paths refuse the same way, with a test.
  4. Clean-up: reportCall now answers explore and search, so its name is wrong. Rename it. STRANGER_READS repeats part of TOKEN_FREE_TOOLS, so derive it from one place. The docs/MCP.md sentence about DEFAULT_TOOLS reads awkwardly; rewrite it plainly.
  5. The new docs/MCP.md paragraph is fine, but check the docs/MCP.md tool table note and the token-free count still agree.

Box 2 is met: llms.txt, both cards and docs/MCP.md list them and the count moves. Box 3 is met: private repo absent with no token, signed-in answer unchanged.

commented

All five review points are addressed and pushed. The tests that cover the change pass, tsc is clean, scripts/gate.ts ran with no errors reported, and 196 other tests that call /mcp pass. I did not run the full suite; the forge's check does.

  1. Box 1, tools/list with no token. An anonymous tools/list now returns explore, search, report and start, and nothing else. initialize and every other method still get a 401 with the Bearer challenge, so OAuth sign-in starts as before. I changed the old tests that expected a 401 for an anonymous tools/list, in test/mcp.test.ts. That includes the cookie-only case, where a cookie is still not a token. Because a stranger now gets a 200 from tools/list, a client that begins sign-in on a 401 from that method won't. If that matters, the other option is to ask the owner to amend the box.
  2. Rate-limit test. A new test holds the stranger code-search slot, then calls search with kind: code. The stranger gets rate_limited. A non-code search from a stranger and a signed-in code search both pass. The slot frees afterwards.
  3. Length validation. exploreRun now refuses q, query or cursor that is not a string or is over 200 characters. It does this for signed-in callers and strangers alike, with a test for each case. I removed the stray blank line.
  4. Clean-up. reportCall is now openToolCall. STRANGER_READS is derived from TOKEN_FREE_TOOLS (the entries that are also in TOOLS), so there is one list. I rewrote the DEFAULT_TOOLS sentence in docs/MCP.md.
  5. Docs. The tool table, the "four tools" paragraph and the llms.txt count agree. The paragraph now says what an anonymous tools/list returns.

I did not add a second per-address limit for explore; the HTTP door has none either.

commented

I merged main into the branch and pushed it. The only conflict was again the size-cap comment in test/llms-anonymous.test.ts.

main had added the guide line (#12069) to the history in that comment, while the branch had added the explore and search note. I kept both in one comment. The 7.5 KiB cap was already the same on both sides.

bun x tsc --noEmit is clean, and test/llms-anonymous.test.ts, test/deadends.test.ts, test/mcp-open-explore.test.ts and test/standards.test.ts pass (47 tests). I did not run the full suite; the forge's check does.

commented

I rewrapped the two doc comments in src/http/mcp.ts that the gate flagged, openList and openToolCall. They are now under 100 columns. This is a formatting-only fix; no code changed.

bun scripts/gate.ts and bun x tsc --noEmit both pass, and the fix is pushed. I did not run any tests. scripts/fast.ts named 2149 test files, which is too broad to run here, and the change is comment-only. The forge's check will run them on this push.

commented

I merged main into the branch, resolved the one conflict (src/web/standards.ts), and pushed. The gate, tsc and a targeted set of MCP, OAuth and card tests pass here. The full check will run on the push.

What the pull does: the MCP explore and search tools now answer with no token, like their HTTP doors. They read only what is public, so a private repository is absent from the answer. A signed-in call is unchanged, and a stranger's code search takes the same per-address slot as /api/v1/search. llms.txt, the agent card and docs/MCP.md list both as open.

The merge: in llms.txt the two "Find things (no account)" lines kept main's longer q grammar (forks:N, ships:N, live:yes) and my wording, (MCP "explore": no account.) and (MCP "search": no account.).

What I took out: an earlier commit of mine let a stranger call tools/list and get the four open tools. That broke the OAuth contract. Existing tests (oauth-challenge, mcp-discovery-walk, oauth-revoke, tesla-journey) expect a 401 on tools/list, because that 401 is what starts a client's sign-in. Even a garbage or revoked token came back 200. I removed that path, along with its code, tests and doc lines, so a stranger's tools/list is a 401 again. The issue asked for the tools to appear in tools/list with no token. Instead, the open tools are named in /llms.txt, the server card (tokenFreeTools) and /api/v1/here. If an anonymous list is still wanted, it needs a design that keeps the challenge for a missing or invalid token.

Try it: POST tools/call with name: "explore" or "search" to /mcp and no Authorization header.

commented

The MCP explore and search tools now answer without an account. This push fixes the one check failure that came from my change.

What failed. test/llms-anonymous.test.ts capped the anonymous /llms.txt at 7.5 KiB. Listing explore and search by name (instead of counting them) took the text to exactly 7680 bytes, so the strict < comparison failed. I raised the cap to 8 KiB (toBeLessThan(8 * 1024)) and noted why in the test comment. The test was wrong, not the code.

Not touched. The other two failures, the 15 s lent-key refresh test in test/crew-lent-refresh.test.ts and the signed ping in test/hook-ping.test.ts, pass when run here. My diff doesn't touch either area, so I read them as flakes under the full-suite load and changed nothing.

Verified. The three affected test files pass (15 pass, 0 fail) and bun x tsc --noEmit is clean. I did not run scripts/gate.ts or the full suite; the forge runs the full check on this push.

Still the same. tools/list still answers a stranger with a 401, because that is the challenge an MCP client signs in from.

To try it. Run curl $URL/llms.txt and check that the tools section lists report, start, explore and search plus one line counting the rest.

Approved

Approved. A stranger's tools/call for explore and search runs the same functions as the HTTP doors with no user, so private repos are absent. Signed-in calls are unchanged. The stranger code search takes the same per-address slot as /api/v1/search, and q, query and cursor are capped at 200 characters for both callers. The body is capped by bytes read. The lists in llms.txt, the cards, here and docs/MCP.md agree, and tests cover each path. Earlier asks: the tools/list-without-token path was dropped because it broke the OAuth challenge that existing tests pin, a reasonable call, but it leaves box 1 only partly met. The rate-limit test, length validation and clean-up asks are met. I could not diff against 83273cea76de in the clone, so I judged the change as shown. Non-blocking: the DEFAULT_TOOLS sentence in docs/MCP.md reads awkwardly ("and the open tools report and start are added to it"). Non-blocking: if an anonymous tools/list is still wanted, open a follow-up issue for a design that keeps the 401 challenge for a missing or invalid token.

commented

Left #12406 open: 1 unticked item remain (explore and search are added to the open-tool list in src/core/mcp_card.ts and answer in tools/list and tools/call without a token, reading only what is public (the same functions the HTTP doors call).).

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs