Public: anyone can read this; the crew can change it.

test/shards.test.ts runs in under 1 s again: it took 0.3 s on 2026-10-21 and 10.4 s on main today #8315

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

`test/shards.test.ts` now runs 64 tests in 0.30 s, down from 82 tests in 10.4 s, and its slowest test takes 43 ms. The 19 tests that spawn processes moved word for word to a new `test/shards-run.test.ts`.

shipped production is serving it deploy 2fa0c24

opened this

test/shards.test.ts now runs 64 tests in 0.30 s, down from 82 tests in 10.4 s, and its slowest test takes 43 ms. The 19 tests that spawn processes moved word for word to a new test/shards-run.test.ts.

What made it slow. Every test over 100 ms spawned the runner, bun scripts/test-shards.ts, which in turn spawns bun test, sometimes twice when a shard is retried. That is two levels of child processes, plus git commits in scratch repositories and one deliberate 1.5 s sleep in the live-output test. The test that spawns bun scripts/gate.ts cost 0.37 s on its own. Importing scripts/gate.ts and walking the tree were not the cost. The 0.3 s recording was wrong from the start: at the commit that recorded it (02f97d2e) the file already took 4.6 s alone. Pulls #7618, #7817 and #8248 later added more end-to-end tests, which brought it to 10.4 s.

What moved. These blocks are now in test/shards-run.test.ts, with every assertion unchanged:

  • "the shard's output"
  • "the gate audit before any shard"
  • the one test in "the gate audit alone, first in the check" that spawns scripts/gate.ts
  • "a failing file is run once more alone"
  • "every shard runs to the end, and a crashed one once more"
  • "a fixture suite in six shards (#7617)"

These tests check a process runner by running it, so they can't each finish in 100 ms without proving less. The new file runs in about 12.5 s and is in the gate without being listed anywhere. test/shards.test.ts keeps the tests that run in-process. It matters that it stays fast because CLAUDE.md has every crew run use it as a cross file. A new test there, "this file spawns nothing", fails if a spawning test is added back.

Other changes:

  • scripts/test-times.json records test/shards.test.ts at 0.3 (measured again) and adds test/shards-run.test.ts at 12.5.
  • The comments at the top of the four test/fixtures/shard-*.test.ts files now name the new file.

No assertion was wrong, so none was changed. I did not shorten the 1.5 s sleep in the live-output test: making the gap smaller would make the streaming check less reliable on a busy machine.

Checks run: bun scripts/gate.ts and bun x tsc --noEmit pass. Both shard test files pass, as do the cross files CLAUDE.md names and test/crew-brief-gate.test.ts.

To try it: bun test test/shards.test.ts should report about 0.3 s, and bun test test/shards-run.test.ts covers the runner end to end.

🤖 Generated with Claude Code

Closes #8310

commented

test/shards.test.ts now runs 64 tests in 0.30 s, down from 82 tests in 10.4 s, and its slowest test takes 43 ms. The 19 tests that spawn processes moved word for word to a new test/shards-run.test.ts.

What made it slow. Every test over 100 ms spawned the runner, bun scripts/test-shards.ts, which in turn spawns bun test, sometimes twice when a shard is retried. That is two levels of child processes, plus git commits in scratch repositories and one deliberate 1.5 s sleep in the live-output test. The test that spawns bun scripts/gate.ts cost 0.37 s on its own. Importing scripts/gate.ts and walking the tree were not the cost. The 0.3 s recording was wrong from the start: at the commit that recorded it (02f97d2e) the file already took 4.6 s alone. Pulls #7618, #7817 and #8248 later added more end-to-end tests, which brought it to 10.4 s.

What moved. These blocks are now in test/shards-run.test.ts, with every assertion unchanged:

  • "the shard's output"
  • "the gate audit before any shard"
  • the one test in "the gate audit alone, first in the check" that spawns scripts/gate.ts
  • "a failing file is run once more alone"
  • "every shard runs to the end, and a crashed one once more"
  • "a fixture suite in six shards (#7617)"

These tests check a process runner by running it, so they can't each finish in 100 ms without proving less. The new file runs in about 12.5 s and is in the gate without being listed anywhere. test/shards.test.ts keeps the tests that run in-process. It matters that it stays fast because CLAUDE.md has every crew run use it as a cross file. A new test there, "this file spawns nothing", fails if a spawning test is added back.

Other changes:

  • scripts/test-times.json records test/shards.test.ts at 0.3 (measured again) and adds test/shards-run.test.ts at 12.5.
  • The comments at the top of the four test/fixtures/shard-*.test.ts files now name the new file.

No assertion was wrong, so none was changed. I did not shorten the 1.5 s sleep in the live-output test: making the gap smaller would make the streaming check less reliable on a busy machine.

Checks run: bun scripts/gate.ts and bun x tsc --noEmit pass. Both shard test files pass, as do the cross files CLAUDE.md names and test/crew-brief-gate.test.ts.

To try it: bun test test/shards.test.ts should report about 0.3 s, and bun test test/shards-run.test.ts covers the runner end to end.

🤖 Generated with Claude Code

Approved

Does what the issue asked: test/shards.test.ts is back to in-process tests only, the 19 spawning tests moved verbatim to shards-run.test.ts, the times table is updated, and a guard test keeps spawns from creeping back. No code path, input, or door changes, so the security and both-doors items do not apply. Moved tests keep their assertions. Next, in order: (1) shards-run.test.ts is 12.5 s in one file, against #1242 (a time budget per file). Split it by concept (live output, audit, retry/crash, fixture suite) so shards balance and none is a long pole. (2) The guard in shards.test.ts only catches a literal Bun.spawn. Make it also catch spawnSync and node:child_process, and remove the doubled blank line before describe("this file").

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs