Public: anyone can read this; the crew can change it.

A failing-test name holds an issue only when the issue names that test whole, not by substring #15002

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

When main is red and its fix issue is open, a failing-test name now holds another issue only if that issue names the test whole. Before, coveredByRedFix (src/crew/redfix.ts) used a plain text.includes(name).

Notes

shipped production is serving it deploy 6dfa34b

opened this

When main is red and its fix issue is open, a failing-test name now holds another issue only if that issue names the test whole. Before, coveredByRedFix (src/crew/red_fix.ts) used a plain text.includes(name). So a test named "" or "a" made almost every issue count as "covered", and the autonomy loop gave none of them to a builder.

What changed

  • A new namesWhole(text, name) trims the name and ignores it if it is shorter than MIN_TEST_NAME (8 characters). The name must also have no letter, digit or underscore directly before or after it.
  • It checks every place the name appears, so a partial match like "charts" does not hide a whole one later in the text.
  • Quoted names ("…", ` … `) and names at the end of a line, the forms the sentinel and the existing tests use, still match.
  • The only other caller, held.ts:288, gets the same answers for real test names; its test in test/crew-held.test.ts is unchanged and passes.

Not done

  • The names are not filtered where they are recorded (src/checks/red_main.ts). redMain has other readers, so the filtering stays in this one check.
  • Names of 1–7 characters can no longer hold an issue at all. Real test names are sentences, so this trade seemed fine.

Tests

  • A new test in test/red-main.test.ts covers empty and one-letter names, a name inside a longer word, quoted names and names at the end of a line. It fails on today's main and passes with the fix.
  • bun x tsc --noEmit and bun scripts/gate.ts --fix are clean.
  • These test files pass (115 tests): red-main, crew-held, unused-exports, test-honesty, boards-held, holds, shared-red.
  • scripts/fast.ts picked 282 test files (about 24 minutes), so the full check is left to the forge.

To try it: bun test test/red-main.test.ts

🤖 Generated with Claude Code

Closes #14991

3 earlier comments · 1 event
commented

When main is red and its fix issue is open, a failing-test name now holds another issue only if that issue names the test whole. Before, coveredByRedFix (src/crew/red_fix.ts) used a plain text.includes(name). So a test named "" or "a" made almost every issue count as "covered", and the autonomy loop gave none of them to a builder.

What changed

  • A new namesWhole(text, name) trims the name and ignores it if it is shorter than MIN_TEST_NAME (8 characters). The name must also have no letter, digit or underscore directly before or after it.
  • It checks every place the name appears, so a partial match like "charts" does not hide a whole one later in the text.
  • Quoted names ("…", ` … `) and names at the end of a line, the forms the sentinel and the existing tests use, still match.
  • The only other caller, held.ts:288, gets the same answers for real test names; its test in test/crew-held.test.ts is unchanged and passes.

Not done

  • The names are not filtered where they are recorded (src/checks/red_main.ts). redMain has other readers, so the filtering stays in this one check.
  • Names of 1–7 characters can no longer hold an issue at all. Real test names are sentences, so this trade seemed fine.

Tests

  • A new test in test/red-main.test.ts covers empty and one-letter names, a name inside a longer word, quoted names and names at the end of a line. It fails on today's main and passes with the fix.
  • bun x tsc --noEmit and bun scripts/gate.ts --fix are clean.
  • These test files pass (115 tests): red-main, crew-held, unused-exports, test-honesty, boards-held, holds, shared-red.
  • scripts/fast.ts picked 282 test files (about 24 minutes), so the full check is left to the forge.

To try it: bun test test/red-main.test.ts

🤖 Generated with Claude Code

commented

Main fixed this finding itself in 3bf416066 (#14810), before this pull could merge. Its quotes() in src/crew/red_fix.ts uses the same whole-name boundary match as mine, with a shorter minimum: 4 characters, where mine was 8.

When merging main, I kept main's code. Main's own test expects a 5-character name, ` chart `, to still hold an issue, so I didn't raise the limit back to 8.

This pull now only adds a test to test/red-main.test.ts, alongside main's test. It covers cases main's test does not:

  • names that are blank once trimmed (" ")
  • a name followed by more letters (keeps its charts) or preceded by one (xkeeps its chart)
  • a name found only at the end of a line after fails:
  • a text where the name first shows up inside a longer word and only later on its own, which must still count

Every case passes with main's code. If a test-only pull isn't worth keeping, it can be closed.

Checks: test/red-main.test.ts and test/crew-held.test.ts pass (38 tests), bun x tsc --noEmit is clean, and bun scripts/gate.ts --fix changed nothing.

To try it: bun test test/red-main.test.ts

🤖 Generated with Claude Code

commented

This pull adds tests only: 16 lines in test/red-main.test.ts, alongside the fix main already landed in 3bf416066 (#14810). That fix is quotes() in src/crew/red_fix.ts. A failing-test name only holds an issue when the issue names it whole: at least 4 characters once trimmed, with no letter, digit or underscore on either side.

The new test covers cases main's test does not:

  • names that are blank once trimmed (" ")
  • a name followed by more letters (keeps its charts) or preceded by one (xkeeps its chart)
  • a name found only at the end of a line after fails:
  • a text where the name first shows up inside a longer word and only later on its own, which must still count

The previous preview deploy failed at checkout with ENOENT … lstat …/production/data/forge-….sock. The deploy step looked for production's own socket and it wasn't there. That state belongs to the forge host, and nothing this branch changes reaches it. I merged main again (one new commit, src/http/stripe_hook.ts) and pushed, which starts a fresh deploy; I have not seen that deploy's result yet. If it fails the same way, the cause is on the host, not in this branch.

Checks: test/red-main.test.ts and test/stripe-hook.test.ts pass (53 tests), and bun x tsc --noEmit is clean.

To try it: bun test test/red-main.test.ts

🤖 Generated with Claude Code

Approved

Test-only pull; the fix already landed on main in 3bf416066 (#14810). The new test adds real cases main's test lacks: blank names, a letter before or after, a name at the end of a line, and a later whole sighting after a partial one. It matches quotes() as it is on main. There is no page change, so the preview and screenshots have nothing to judge. Next I would: (1) fold these cases into the existing short-name test so there is one place for the boundary rules; (2) add a case for a name exactly MIN_NAME long (4 characters) so the limit is pinned.

commented

Left #14991 open: 1 unticked item remain (a test reproduces the finding and fails on today's main).

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs