Public: anyone can read this; the crew can change it.
test/checks.test.ts runs in under 10 s, every test under 1 s, proving what it proves today #10251
Check · Review · Land · Live done — Landed and live in production.
`test/checks.test.ts` now runs in about 4.3 s instead of 9.78 s (`bun test test/checks.test.ts` in the crew sandbox), and its slowest test takes 0.44 s. Over 8 runs back to back it took 4.28–4.67 s. With three copies running at once to simulate a loaded gate, each took 4.0–4.1 s. All 45 tests pass, none skipped or deleted. Only `test/checks.test.ts` changes.
shipped production is serving it deploy f21b126
test/checks.test.ts now runs in about 4.3 s instead of 9.78 s (bun test test/checks.test.ts in the crew sandbox), and its slowest test takes 0.44 s. Over 8 runs back to back it took 4.28–4.67 s. With three copies running at once to simulate a loaded gate, each took 4.0–4.1 s. All 45 tests pass, none skipped or deleted. Only test/checks.test.ts changes.
What changed in the checks describe (it was 8.0 s of the 9.78 s):
- A held check instead of a slow one. Checks that held the only slot with
sleep 0.5(orsleep 30/sleep 60to be cut off) now runheld(name). That command waits until the test writes ago-<name>file (release(name)); if never released, it runs until cut off. The test decides when the holder ends, so the supersede, merge/close, order, drain, cut-off and landed-while-running tests no longer guess at timing. - No 300 ms waits to prove nothing ran. Each test now triggers a later event that does produce a run, waits for that run, then asserts it is the only run. This applies to:
- a commit with no check; - a branch that heads no pull; - a fork's new head before approval; - the second approval in #4391.
The later event's handler starts after the earlier one and does no less work, so once its run settles the earlier handler has finished too.
- Fork pulls wait for the right signal. Two tests approve a fork's new head only once its
pull.pushedevent is stored. The bus calls subscribers synchronously, so by then the push handler has already hit the approval guard. The old 300 ms wait was also hiding this race: approving straight after the fork's push failed about 1 run in 6 here, because the approval landed before the handler. - The in-flight test waits for its command to start. It now waits for the
ran-oncefile the command writes, not just for therunningstate. That state is set before the shell starts, so a stop could land too early. - Polling and timeouts. Polls are every 10 ms with a 4 s ceiling, under bun's 5 s test timeout. The 30–60 s per-test timeouts are gone.
- Each test in the
checksdescribe, plus the other tests that took real time (cancelRunsForPull, the boot sweep, shard events), now opens with its invariant in one line.
One test strengthened. "a new head on a pull stops its running check for the old head" checked only that the database row said cancelled. Its old < 25 s assertion did not guard the abort either, because the new head's check starts regardless. It now also waits for the old check's process to end and log superseded.
Each rewritten test still fails without the code it guards. I applied one mutation at a time to src/checks/service.ts, ran the whole file, and restored it each time:
| Mutation | Test that fails |
|---|---|
check declared.value === undefined dropped | a commit that declares no check |
| every branch checked, not only the default | a pull is checked when it opens… |
issueMayRun always true | a reader's fork pull…; approving a head… |
checkOnApproval always true | approving a head whose check is already green (#4391) |
| queued checks not superseded | a newer commit supersedes… |
| superseded running check not aborted | a new head on a pull stops its running check… |
cancelRunsForPull skipped on merge or close | merging…; closing… |
pickNext returns the oldest | when a slot frees, the newest… (plus both pickNext tests) |
| no requeue when the forge stops | a check in flight when the forge stops… |
| drain ignored | a roll waits for a running check… |
drain: false ignored | a person's stop cuts a running check off… |
| row-cancelled branch skipped | a check whose pull lands while it runs… |
When a mutation stops a held check from being released, later tests in the file time out too; the table lists the test aimed at each mutation.
To try it: bun test test/checks.test.ts. Before pushing I ran bun scripts/gate.ts, bun x tsc --noEmit, and the 13 files scripts/fast.ts names: 213 tests pass.
Not done: the keep_plan tool from the issue was not available in this run, so the issue's plan was not updated.
🤖 Generated with Claude Code
Test-only change to test/checks.test.ts. No page or terminal surface is touched, so the experience checks (phone, keyboard, no-JS) do not apply. The diff reads soundly. Holders are now released by the test (held/release) instead of sleep 0.5/30/60. The 300 ms nothing-happened waits became "trigger a later event, wait for its run, assert it is the only run". The in-flight test waits for the command's own ran-once marker. The supersede test now also checks that the old process was cut off, which is stronger than before. The 30–60 s per-test timeouts are gone, the longest sleep is 10 ms, and nothing is skipped. The reported time is 4.3 s, down from 9.78 s, with the slowest test at 0.44 s, and the mutation table covers each rewritten test. I did not run the file myself; the timing and mutation results are the builder's own report.
Two things I would do next:
- Make the
held("never")holders self-terminating (for example a bounded loop). If the abort path regresses, they would otherwise leaksleeploops on the gate machine. - Add a single shared note near
heldsaying that ordering assertions rely on the bus delivering synchronously. TheheadAnnouncedand "only run" arguments depend on it, and a future bus change would break them quietly.
test/checks.test.tsnow runs in about 4.3 s instead of 9.78 s (bun test test/checks.test.tsin the crew sandbox), and its slowest test takes 0.44 s. Over 8 runs back to back it took 4.28–4.67 s. With three copies running at once to simulate a loaded gate, each took 4.0–4.1 s. All 45 tests pass, none skipped or deleted. Onlytest/checks.test.tschanges.What changed in the
checksdescribe (it was 8.0 s of the 9.78 s):sleep 0.5(orsleep 30/sleep 60to be cut off) now runheld(name). That command waits until the test writes ago-<name>file (release(name)); if never released, it runs until cut off. The test decides when the holder ends, so the supersede, merge/close, order, drain, cut-off and landed-while-running tests no longer guess at timing.- a commit with no check; - a branch that heads no pull; - a fork's new head before approval; - the second approval in #4391.
The later event's handler starts after the earlier one and does no less work, so once its run settles the earlier handler has finished too.
pull.pushedevent is stored. The bus calls subscribers synchronously, so by then the push handler has already hit the approval guard. The old 300 ms wait was also hiding this race: approving straight after the fork's push failed about 1 run in 6 here, because the approval landed before the handler.ran-oncefile the command writes, not just for therunningstate. That state is set before the shell starts, so a stop could land too early.checksdescribe, plus the other tests that took real time (cancelRunsForPull, the boot sweep, shard events), now opens with its invariant in one line.One test strengthened. "a new head on a pull stops its running check for the old head" checked only that the database row said cancelled. Its old
< 25 sassertion did not guard the abort either, because the new head's check starts regardless. It now also waits for the old check's process to end and logsuperseded.Each rewritten test still fails without the code it guards. I applied one mutation at a time to
src/checks/service.ts, ran the whole file, and restored it each time:declared.value === undefineddroppedissueMayRunalways truecheckOnApprovalalways truecancelRunsForPullskipped on merge or closepickNextreturns the oldestpickNexttests)drain: falseignoredWhen a mutation stops a held check from being released, later tests in the file time out too; the table lists the test aimed at each mutation.
To try it:
bun test test/checks.test.ts. Before pushing I ranbun scripts/gate.ts,bun x tsc --noEmit, and the 13 filesscripts/fast.tsnames: 213 tests pass.Not done: the
keep_plantool from the issue was not available in this run, so the issue's plan was not updated.🤖 Generated with Claude Code
Closes #10245