Every test fast: a budget per test and per file enforced by the gate, the full suite under five minutes #1242

The owner: "we need to have a high perf bar too. that is a part of quality… it needs to be strong and fast."

Today a full check takes ~20 minutes alone and 45 minutes beside another: tsc ~1.5 min, then ~2,100 tests in 179 files run one shard at a time; 116 tests hold 92% of the time because they boot a server per test, run real git, launch chromium, or sleep. The two-tier gate (#1198, upstream #30) runs fewer tests per push; this issue makes every test fast.

Done looks like:

• [ ] A budget, enforced by the gate: no test file over 20 s and no single test over 2 s on the gate box, unless the file is named in a short SLOW list in scripts/gate.ts with an issue number beside it. Over budget is red, like a failing assertion.
• [ ] scripts/test-times.json is written on every gate run (per file and per test), committed by the release, and the check summary names the five slowest tests and their share of the run.
• [ ] The 116 slow tests are brought under budget by shape, not by deletion: one server per file (not per test), a fake clock instead of sleep, an in-memory SQLite where the disk is not the point, chromium only in the looks lane (nightly, not the gate), git fixtures prepared once per file.
• [ ] Target and proof: the full suite under 5 minutes on the gate box, recorded in test-times.json; the fast tier under 60 seconds for a one-file change.

Evidence: check summaries today ("check passed in 1209s", "2731s" when two ran); the time analysis from 2026-10-12 (116 tests = 92%).

The owner, 2026-10-13: "if there are 116 slow tests, can't we just chop them and rewrite? … it should be understood that we could go FAST and that should be our target. sometimes you need to prune the tree for it to grow better."

So the plan is chop and rewrite, not trim:

• Keep the invariant each slow test proves (write it as one line at the top of the new test); throw the test away and write the fastest test that proves that line. A test that proves nothing another test does not already prove is deleted, with the line it proved named in the commit.
• Shape rules for the rewrite: one forge and one server per file, created in beforeAll; no Bun.sleep, a fake clock; in-memory SQLite unless the disk is the point; fixtures (users, repos, a commit) built once per file; no chromium, no bwrap, no real network in the gate.
• Slow by nature (restore drill, sandbox boot, looks with chromium, the 50-push hook storm): a nightly lane, scripts/gate.ts list NIGHTLY, run by a workflow, red files an issue.
• Targets, enforced: the full suite under 3 minutes on the gate box; the fast tier under 30 seconds for a one-file change; no test over 1 s, no file over 10 s without a SLOW entry and an issue.
• Order of work: the 116 by their share of time, biggest first; scripts/test-times.json tells which.

The runtime is not the limit; the shape is.

Research brief (a read-only scout, 2026-10-13), for #1242, #1243, #1279 and #1198. Bun docs at v1.4.3; check bun test --help on the forge's own Bun before relying on a flag.

Bun-native, worth replacing our own machinery with:

• bun test --parallel[=N] runs test files across worker processes (default one per core, implies --isolate); BUN_TEST_WORKER_ID is set per worker for ports and databases. On the 2-core gate box: --parallel=2 for the fast files only; the slow files with at most 2 workers. Docs: https://bun.com/docs/test/parallel
• --shard=i/n --timings=<file> --update-timings balances shards by recorded duration, not file count: commit the timings file like scripts/test-times.json and the shards finish together.
• --randomize, --seed=<n> (seed implies randomize), --rerun-each N, --bail[=N], --retry N (never in the gate). The flake hunt (#1243) is --randomize --seed=$RANDOM --rerun-each 3 nightly, logging the seed. Docs: https://bun.com/docs/cli/test
• --coverage-reporter=lcov: coverage is per run, so the impact map (#1279) is built by running one test file at a time with lcov and recording the source files each one executes; --parallel merges coverage, so build the map serially (nightly is fine).
• --changed / --only-changed appear in neither doc page; do not plan on them.

Shape recipes (general practice, not sourced): one server per file in beforeAll; fake timers; in-memory SQLite where the disk is not the point; longest-first ordering comes free with the timings file; a changed forge.json, package.json, scripts/**, lockfile or kernel file runs the full tier.

The X skill the owner remembers was not found by name. The nearest public ones (ci-parallel-shard-optimizer: per-test runtimes → rebalanced shards; flaky-test-detector: reruns and flags non-determinism) do nothing the list above does not. If the owner recalls the author or name, compare.

Measured in the first real fast check after 0.7.69 (#1280): of the first 88 files run one-per-process, 55 finished under 5 s (startup and module load, not tests), while four files took 44 s, 66 s, 71 s and 91 s and set the pace (shards 59, 13, 46, 58 in run 01m4v94pk641ne5hzw5cntvwr1). Two levers, both yours: the long files are the ones to chop or give a SLOW entry with a reason; and the short files should not each pay a bun start, so group them into one process or run shards in parallel where memory allows (the lane has one slot; the Actions runner OOMed at more than one, so the grouping has to keep peak RSS to one suite).

Parked by the crew: the check failed ("the gate's list > every test/*.test.ts is in exactly one list, and every entry exists") and the decider saw no point in another try (pull #1269 closed). Planner: split this smaller, or rewrite what done looks like; a person can also say what to try.

#1247 (test/apps-adversarial.test.ts) is deferred behind the Later gate, so this no longer waits for it. That file stays in SLOW until it is picked up.