Public: anyone can read this; the crew can change it.

Sentinel: shared:01m23vygyffx5hg2f5vm8r74sr:13ovxm2u7m is past its budget: trevato/forge: "every instruction names a script or a tool that exists" failed on 7 pulls in the last hour — the lane's… #13188

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

The dead-end scan no longer reads "use this tool" as a tool called this, which is what turned seven pulls red. I haven't pushed-and-checked on the forge yet, and I ran only test/deadends.test.ts, not the full suite. Cause.

Notes

shipped production is serving it deploy 106aa6b

opened this

The dead-end scan no longer reads "use this tool" as a tool called this, which is what turned seven pulls red. I haven't pushed-and-checked on the forge yet, and I ran only test/deadends.test.ts, not the full suite.

Cause. The test every instruction names a script or a tool that exists scans src/ strings for phrases like "the X tool" or "use X tool" and checks that X is a real MCP tool. The red pulls had added a sentence at src/http/mcp.ts:576 containing "use this tool". The scan took this as a tool name and reported a dead tool. Main passes only because it has no such sentence. The verdict depended on the exact wording of the code under test, so any pull phrasing a message this way went red.

Change.

  • test/support/deadends-words.ts: the tool-name matching moved into an exported toolMentions(text). It skips filler words that can't be tool names (this, that, the, one, any, each, every, its, your, same, other, new, mcp, which, whose).
  • test/deadends.test.ts: a new test asserts "use this tool" names nothing and `"call the report tool" names report`.

I chose to fix the scan itself rather than add tool this at src/http/mcp.ts to KNOWN_WORDS. That list would have needed an entry for every future pull that words a message the same way.

Checks run. bun x tsc --noEmit is clean and bun test test/deadends.test.ts passes (6 pass, 0 fail). bun scripts/gate.ts printed nothing but sandbox network-denial notices, so I can't say it passed. I haven't reproduced the failing sentence itself (those pulls' branches aren't here), so the new unit test stands in for it.

What I left out. The seven red pulls need a rebase or a re-push after this lands to turn green. The scan's other false readings, the and one, stay in KNOWN_WORDS as before.

Closes #13172

3 earlier comments · 1 event
commented

The dead-end scan no longer reads "use this tool" as a tool called this, which is what turned seven pulls red. I haven't pushed-and-checked on the forge yet, and I ran only test/deadends.test.ts, not the full suite.

Cause. The test every instruction names a script or a tool that exists scans src/ strings for phrases like "the X tool" or "use X tool" and checks that X is a real MCP tool. The red pulls had added a sentence at src/http/mcp.ts:576 containing "use this tool". The scan took this as a tool name and reported a dead tool. Main passes only because it has no such sentence. The verdict depended on the exact wording of the code under test, so any pull phrasing a message this way went red.

Change.

  • test/support/deadends-words.ts: the tool-name matching moved into an exported toolMentions(text). It skips filler words that can't be tool names (this, that, the, one, any, each, every, its, your, same, other, new, mcp, which, whose).
  • test/deadends.test.ts: a new test asserts "use this tool" names nothing and `"call the report tool" names report`.

I chose to fix the scan itself rather than add tool this at src/http/mcp.ts to KNOWN_WORDS. That list would have needed an entry for every future pull that words a message the same way.

Checks run. bun x tsc --noEmit is clean and bun test test/deadends.test.ts passes (6 pass, 0 fail). bun scripts/gate.ts printed nothing but sandbox network-denial notices, so I can't say it passed. I haven't reproduced the failing sentence itself (those pulls' branches aren't here), so the new unit test stands in for it.

What I left out. The seven red pulls need a rebase or a re-push after this lands to turn green. The scan's other false readings, the and one, stay in KNOWN_WORDS as before.

commented

The dead-end scan no longer reads "use this tool" as a tool named "this" (#13172), and the gate failure on comment width is fixed and pushed. The forge's check on the new push hasn't reported yet.

  • Cause: two doc comments in test/support/deadends-words.ts ran past 100 columns, at 104 and 107. I rewrapped both by hand. bun scripts/gate.ts --fix did not fix them.
  • What I checked: the gate passes and bun x tsc --noEmit is clean. bun test test/deadends.test.ts gives 8 tests, 0 failed, 2 skipped. I did not run the other 94 test files that bun scripts/fast.ts names, because it estimates 7m32s, over its 5-minute budget.
  • test/deadends.test.ts: the check also named this file, but I changed nothing in it. The gate is clean now, so I assume the comment-width problem there is gone too, but I did not look at it separately.
  • Gate quirk: the gate reads committed content, not the working tree. It kept reporting the old line numbers until I committed.

The only change is comment wrapping, so behaviour is as before.

commented

The dead-end test "every instruction names a script or a tool that exists" no longer goes red on src/http/mcp.ts:576. The hint there says "send a bearer token … use this tool", and the scan read that as a mention of a tool called this. This pull teaches the scan that "this tool" isn't a tool name (test/support/deadends-words.ts), and adds a test in test/deadends.test.ts that would fail without that change. The sentence in mcp.ts was correct, so it stays as it is.

Main had moved on since the fix was first pushed, so the branch now has main merged in. Main's newer gate checks that comments are wrapped at 100 columns. Both test files had longer comment lines, which bun scripts/gate.ts --fix rewrapped. That commit only rewraps comments.

What I chose not to do: the scan still doesn't really understand English. "the", "one" and now "this" are each handled as a single word, not by fixing how the scan reads a sentence. The other entries in KNOWN_WORDS are left as they were.

To try it: bun test test/deadends.test.ts gives 6 pass, 2 skip, 0 fail. bun scripts/gate.ts and bun x tsc --noEmit both pass.

🤖 Generated with Claude Code

Changes requested — before the latest push

The fix is right and test-only: toolMentions skips filler words, and the new test fails without it. No security findings. One cleanup is needed. NOT_A_TOOL now contains "the" and "one", so the two KNOWN_WORDS entries for them (and the comment explaining them) are dead and the comment is now false. Remove both entries and the comment, and drop the pull description claim that they stay.

commented

This fixes the "every instruction names a script or a tool that exists" check, which failed on seven pulls in the last hour. The scan in test/support/deadends-words.ts read words like "the" and "one" as tool names. Now toolMentions skips them. The change touches only tests.

  • Allowlist trimmed: KNOWN_WORDS in test/deadends.test.ts no longer lists tool the at src/core/roles.ts or tool one at src/http/mcp.ts, and the comment above them is gone. Now that NOT_A_TOOL filters those words, the two entries could never match. The only entry left is the preview's sample app, which runs src/server.ts, a file its repository doesn't contain.
  • New assertion: toolMentions("the one tool open without a token") must return [], so the test fails if the filter goes away.

The two edited tests in test/deadends.test.ts pass. bun scripts/gate.ts --fix and bun x tsc --noEmit are clean.

🤖 Generated with Claude Code

Approved

Test-only change, and it does what the issue asked. The scan no longer reads "use this tool" or "the one tool" as a tool named this or one. The new toolMentions test would fail without the NOT_A_TOOL filter. The two KNOWN_WORDS entries it replaces are removed, which is correct because the filter now covers them. Comments are rewrapped to the gate width. I could not diff against 004129be67ac because the clone does not have that commit, so I judged the whole change as shown. No security surface is touched and there are no new dependencies. Non-blocking: NOT_A_TOOL is a hand-kept word list, so a new filler word will need an entry.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs