Public: anyone can read this; the crew can change it.

A full-suite report names a failing test under the file that ran it, never under the last file printed #9526

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

The full-suite report now puts each failing test under the file that ran it. Before, bun's closing summary (`2 tests failed:` followed by every failure again) handed all of those failures to whichever file it printed last. That is how #9424 came to name `test/spawn-cell-e2e.test.ts`. I ran bun 1.3.14 and saw the summary shape for real. When bun detects an agent it doesn't print the summary at all,…

shipped production is serving it deploy a5a2512

opened this

The full-suite report now puts each failing test under the file that ran it. Before, bun's closing summary (2 tests failed: followed by every failure again) handed all of those failures to whichever file it printed last. That is how #9424 came to name test/spawn-cell-e2e.test.ts. I ran bun 1.3.14 and saw the summary shape for real. When bun detects an agent it doesn't print the summary at all, which is why the bug hid in local runs.

What changed

  • isSummaryLine in src/checks/bun-test.ts matches the summary lines: N tests failed:, N tests skipped:, N tests todo:, the count lines ( 2 fail) and Ran N tests across M files.. It replaces the narrower pattern that module already used for nameUnnamed.
  • Three readers stop giving (fail) lines to a file once the summary starts, until the next file header. That header can come from another shard's output in the same log. A (fail) line that can't be placed is dropped:

- failedTests (src/checks/flake.ts): the failing test names. - failedFiles (src/checks/verify.ts): the files reportRed files issues for. This is the fix for "no issue for a file whose own tests all passed". - failingFiles (scripts/shards.ts): it had the same bug, so the gate re-ran the last green file alone for no reason.

Tests (each fails on the code before this change)

  • test/full-suite-report.test.ts uses a log in bun's real shape: a red file, another red file, a green file printed last, then the summary repeating both failures.

- failedTests and failedFiles return each failure once, under its own file. - A (fail) line after the summary with no file is dropped. - A second shard's report starts its files again. - Taken through redDetail and reportRed, the log files issues for the two red files only, not the green one.

  • test/shards.test.ts: failingFiles doesn't name the green file printed last.

Checks run: bun scripts/gate.ts and bun x tsc --noEmit pass. scripts/fast.ts exited 2 (diff too broad to name tests), so I ran every test file that imports the changed modules. That's 14 files, 326 tests, all passing, including the flake ones the queue's flake detection relies on.

Not done

  • Issues already filed under the wrong file, like #9424, stay as they are; closing them is a person's call.
  • I couldn't keep a plan on the issue: no keep_plan tool was available in this run.

🤖 Generated with Claude Code

Closes #9523

commented

The full-suite report now puts each failing test under the file that ran it. Before, bun's closing summary (2 tests failed: followed by every failure again) handed all of those failures to whichever file it printed last. That is how #9424 came to name test/spawn-cell-e2e.test.ts. I ran bun 1.3.14 and saw the summary shape for real. When bun detects an agent it doesn't print the summary at all, which is why the bug hid in local runs.

What changed

  • isSummaryLine in src/checks/bun-test.ts matches the summary lines: N tests failed:, N tests skipped:, N tests todo:, the count lines ( 2 fail) and Ran N tests across M files.. It replaces the narrower pattern that module already used for nameUnnamed.
  • Three readers stop giving (fail) lines to a file once the summary starts, until the next file header. That header can come from another shard's output in the same log. A (fail) line that can't be placed is dropped:

- failedTests (src/checks/flake.ts): the failing test names. - failedFiles (src/checks/verify.ts): the files reportRed files issues for. This is the fix for "no issue for a file whose own tests all passed". - failingFiles (scripts/shards.ts): it had the same bug, so the gate re-ran the last green file alone for no reason.

Tests (each fails on the code before this change)

  • test/full-suite-report.test.ts uses a log in bun's real shape: a red file, another red file, a green file printed last, then the summary repeating both failures.

- failedTests and failedFiles return each failure once, under its own file. - A (fail) line after the summary with no file is dropped. - A second shard's report starts its files again. - Taken through redDetail and reportRed, the log files issues for the two red files only, not the green one.

  • test/shards.test.ts: failingFiles doesn't name the green file printed last.

Checks run: bun scripts/gate.ts and bun x tsc --noEmit pass. scripts/fast.ts exited 2 (diff too broad to name tests), so I ran every test file that imports the changed modules. That's 14 files, 326 tests, all passing, including the flake ones the queue's flake detection relies on.

Not done

  • Issues already filed under the wrong file, like #9424, stay as they are; closing them is a person's call.
  • I couldn't keep a plan on the issue: no keep_plan tool was available in this run.

🤖 Generated with Claude Code

commented

A full-suite report now files each failing test under the file that actually ran it. Before, a failure could be filed under whichever file printed last. Fixes #9523.

The cause: failedTests, failedFiles and failingFiles gave every (fail) line to the last file header they had seen. bun ends a run with a summary (N tests failed:) that repeats every failure, so those repeats were filed under the last file printed. All three now stop reading at that summary, using one shared isSummaryLine in src/checks/bun-test.ts. The callers are src/checks/flake.ts, src/checks/verify.ts and scripts/shards.ts.

What to look at:

  • isSummaryLine and the three places that now stop at the summary.
  • test/full-suite-report.test.ts and test/shards.test.ts: they feed in a real-shaped bun log where the summary follows a different file's header, and check that each failure is filed under its own file.

On the failed preview deploy: the log shows the app running and then shutting down on a SIGTERM sent by the forge, not crashing, and the preview answered 200 on / and /healthz afterwards. This change doesn't touch how the app starts. I merged current main into the branch and pushed to get a fresh deploy.

Before pushing, bun scripts/gate.ts, bun x tsc --noEmit and the two covering test files (79 tests) all passed. bun run src/main.ts starts and stops cleanly. My health check against the local copy got no answer because I probed the wrong port, so the local start is confirmed only by its log; the preview is the real check.

To try it: bun test test/full-suite-report.test.ts test/shards.test.ts.

🤖 Generated with Claude Code

commented

A full-suite report now files each failing test under the file that actually ran it. Before, a failure could be filed under whichever file printed last. Fixes #9523.

The cause: failedTests, failedFiles and failingFiles gave every (fail) line to the last file header they had seen. bun ends a run with a summary (N tests failed:) that repeats every failure, so those repeats were filed under the last file printed. All three now stop reading at that summary, using one shared isSummaryLine in src/checks/bun-test.ts. The callers are src/checks/flake.ts, src/checks/verify.ts and scripts/shards.ts.

What to look at:

  • isSummaryLine and the three places that now stop at the summary.
  • test/full-suite-report.test.ts and test/shards.test.ts: they feed in a real-shaped bun log where the summary follows a different file's header, and check that each failure is filed under its own file.

The last check failed for reasons outside this change:

  • every module under src/ is imported by another (test/unreached.test.ts): the branch's copy of main included src/perf/load_verdict.ts and src/perf/phase_profile.ts, which only tests imported. Newer main fixes this, and merging it turned the test green.
  • The other ten failures (the queue and steer through the doors and ETag tests, planner declined, tui-people-verify, and the healthz-during-boot timing test in test/boot-beside-peer.test.ts): none of them touch the report parser, and all of them pass here.

Before pushing, bun scripts/gate.ts, bun x tsc --noEmit and those seven test files (103 tests) all passed after the merge.

To try it: bun test test/full-suite-report.test.ts test/shards.test.ts.

🤖 Generated with Claude Code

Approved

Approved. The three readers (failedTests, failedFiles, failingFiles) now drop a (fail) line once bun's closing summary starts, until the next header. The shared isSummaryLine replaces the narrower pattern. The tests use a real-shaped log and go through reportRed. This is pipeline work and nothing else. It touches no UI, so there is nothing to see in the preview. Next, two things: (1) remove the doubled blank line before the pairs each failure test in test/full-suite-report.test.ts, and wrap the long SUMMARY regex line in bun-test.ts; (2) move isSummaryLine next to the other bun-test helpers, so the HOOK_TIMEOUT and UNNAMED_FAIL constants are not split by it.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs