Public: anyone can read this; the crew can change it.
The gate answers in a minute, not four: the check runs its test files in parallel shards #306
Check · Review · Land · Live done — Landed and live in production.
`bun run check` now type-checks once, then runs the gate's test files as parallel `bun test` shards. On this machine it went from **266 s to 102 s and 109 s**, green on two back-to-back runs from a fresh clone; the old serial command was timed on the same machine. It is not yet a minute: `test/crew.test.ts` alone takes about 50 s and shares a shard with other files.
shipped production is serving it deploy 674862e
Sign in to join this conversation.
new above
offline
bun run checknow type-checks once, then runs the gate's test files as parallelbun testshards. On this machine it went from 266 s to 102 s and 109 s, green on two back-to-back runs from a fresh clone; the old serial command was timed on the same machine. It is not yet a minute:test/crew.test.tsalone takes about 50 s and shares a shard with other files.What moved where
forge.jsoncheck.runis nowbun x tsc --noEmit && bun scripts/test-shards.ts --timeout 60000 <files…>. The file list lives only there, andscripts/check.tsstill just runs that string.scripts/shards.tsholds the logic and spawns nothing:-
shardCount: one shard per CPU the process may use (a cgroup quota counts), at most 8, never more than there are files.FORGE_CHECK_SHARDSoverrides it, and anything that isn't a whole number from 1 is refused. -splitFiles: the largest test file (by bytes) goes to the shard with the least so far. The slices never overlap and together hold every file, and the same list always gives the same split. -tally: reads every shard's output. Green ends with one line (Ran 435 tests across 23 files in 86.0s (4 shards).). Red lists every(fail)from every shard, names any shard that exited non-zero without naming a test by its files, and exits 1.scripts/test-shards.tsspawns onebun testper slice. It passes exact./paths, not name filters, so a slice can't match another shard's files. Each shard's output is written out whole when that shard finishes. It refuses a named file that doesn't exist, and passes SIGINT/SIGTERM on to its children.test/shards.test.tschecks that the slices don't overlap and cover the list for shard counts 1–40, the CPU count and override rules, and that one red shard makes the whole run red with every failure named. It is part of the gate now.tsconfig.jsonnow includesscripts/, so the runner is type-checked.README.mddescribes the sharded check.Measured here (a 4-CPU cgroup on an 80-core host):
check.runFORGE_CHECK_SHARDS=8About 15 s of each run is
tsc.Things this found
test/steer.test.tsfailed when running alongside other files. It slept a fixed 300 ms before posting its comment, but a run is marked running before the fake agent starts, so on a loaded machine the comment arrived too early and a second run queued. The test was wrong, not the code: it now waits for thesession s2line in the run log.test/repos.test.tswas in the gate list but doesn't exist;bun testtreated it as a filter that matched nothing, which is why logs said 22 files. I removed it, and the runner now fails on any missing file.sshPort = 2222lines inapps,studioandssh-siteonly set a config value for display text; tests never bind that port, so I left them alone.Not done: better balancing. Byte size is a weak guide to runtime (
crew.test.tsis small but slow). A pool that hands out files one at a time would get closer to a minute, but it gives up fixed slices. The release workflow and chart are untouched.To try it:
bun run check, orFORGE_CHECK_SHARDS=2 bun run checkto turn it down.🤖 Generated with Claude Code
Closes #277