Public: anyone can read this; the crew can change it.

Two-tier gate: a fast tier judges every push in five minutes, the full gate runs once on land #1198

shipped production is serving it opened this has it
opened this

Every pull waits 16–20 minutes for the full gate (~2,100 tests, 170 files; 116 tests hold 92% of the time), and the forge runs one check at a time, so 17 open pulls mean the last one is judged five hours after it was pushed. Upstream, every merge to main runs the same full gate again on one serial runner before a release can even build: a one-line release takes 2–4 hours from merge to live. The gate is right; its shape is wrong.

Done looks like:

  • [ ] Two tiers, declared in forge.json under check: fast (the files that touch what the pull changed, found from the pull's diff by import graph and test-name match, plus every test under 1 s; budget 5 minutes) and full (everything, as today).
  • [ ] A push to a pull runs fast; the pull's checks line says "fast tier: green in 3m40s · full gate runs on land".
  • [ ] Landing runs full once on the merge with its base (#1133) and the pull lands only when it is green; a red full gate on land reopens the pull with the red tests named and the fast tier learns them (they join that pull's fast set).
  • [ ] scripts/test-shards.ts --tier fast --diff <base>..<head> is what both the forge's checks and .forgejo/workflows/check.yml run, so the runner's PR gate is the fast tier too; the tag build and main's full gate stay as they are.
  • [ ] Tests in test/check-tiers.test.ts: the fast set for a diff touching src/snapshot/store.ts includes test/snapshot-store.test.ts and excludes test/web-phone.test.ts; a red full gate on land reopens the pull and names the tests; the checks line words.

Target: pull pushed → judged in under 5 minutes; merge → live in under 1 hour.

commented

Evidence for the shape of this: at 03:58Z the owner's steward set checks.concurrency to 2 (node had 4 GB free). At 04:02:29Z the forge pod was OOMKilled at its 5 GiB limit with two checks, four crew runs and a snapshot inside it; every run was interrupted and re-queued, the snapshot in flight was lost. Back to 1 at 04:06Z. A full check is a whole bun test of ~2,100 tests inside the forge's own memory; two of them do not fit beside the crew. So the fast tier is also the memory fix: a five-minute tier on the pull's own files is small enough to run two or three at once, and the full gate runs once, alone, on land. If a check ever needs to run beside another full one, it needs its own pod with its own limit, not a share of the forge's.

commented

Linked under P0 Trust (#586). This issue waits for #1180, the check.fast key in forge.json, which is already queued for the same builder and keeps this pull smaller. #1181 is closed into this one. The full gate on land relies on #1178 (check on the merge with its base). If #1178 hasn't landed when you reach that step, check the head as today and say so in the pull.

commented

Bun-native flags that bear on this (from the research brief on #1242): --parallel[=N] with BUN_TEST_WORKER_ID, --shard=i/n --timings=<file> --update-timings balanced by recorded duration, --randomize --seed --rerun-each N for the flake hunt, --coverage-reporter=lcov one file at a time for the impact map. Prefer them to our own sharding where they fit.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs