Public: anyone can read this; the crew can change it.

Check time is predictable: test-times.json measured for every gate file, tsc cached, the gate-list audit before tsc, and pull checks pause while main is red #1836

Merged opened this in answer to

Check · Review · Land · Live done — Landed and live in production.

Merged

This makes check time predictable in four ways, matching the four asks in #1797.

shipped production is serving it deploy 9ac397e

opened this

This makes check time predictable in four ways, matching the four asks in #1797.

  • Gate-list audit first: forge.json now runs bun scripts/gate.ts before anything else. A test file listed in neither GATE nor EXCLUDED fails in under a second, instead of after about 25 s of install and tsc.
  • Cached tsc: tsc runs --incremental with a 2048 MB heap and keeps its build info in .forge-cache/. src/checks/cache.ts restores that directory from the base commit's own green check (or the newest kept one). It is kept only from green checks of the default branch, so a pull's code never fills a cache another check reads. Only plain files are copied, with safe names and a cap on count and bytes, and links are never followed. The forge keeps the last 4 commits.
  • Every gate file timed: the full gate runs one file per bun test, slowest first. On green it writes every file's time to .forge-cache/test-times.json. The next split uses those times on top of scripts/test-times.json, so all 274 files are weighed by measurement, not just the 41 the table named.
  • A red main pauses pull checks: new module src/checks/red_main.ts. While the default branch's last finished check is red, its pulls' checks stay queued instead of all failing on the same test. Main's own checks keep running, and the next green one releases the rest. The vitals show one red:<repo> card linked to main's failed run, naming the failing test and how many checks wait.
  • The pull that fixes main can still run: a red main would otherwise hold its own fix forever, which the existing checks suite showed. So a check someone asks for again runs straight away (an ask on a held check re-queues it as theirs), and no check is held more than RED_HOLD_MS (20 min).

Tests: bun x tsc --noEmit, the gate audit, and the red-main, vitals, vitals-degraded, checks, check-tiers, pull-check-again, sentinel and shards suites all pass here.

  • test/red-main.test.ts covers the hold, the ask, the 20-minute limit, the release on green and the vitals card.
  • A new end-to-end test in test/check-tiers.test.ts drives red main → held → ask → green → released. With the hold switched off it fails, as it should.

Changed tests: test/vitals.test.ts now expects 6 queries instead of 5. The card adds one fixed query; the test was right before, and the count still doesn't grow with the number of runs.

Worth a look:

  • The hold is off by default in startChecks (redHoldMs 0), and src/main.ts turns it on with RED_HOLD_MS. Existing suites that make main fail on purpose keep their old behaviour.
  • While checks are held, the existing queue:check lane-wait card can still appear next to the red-main card. I left that card alone.
  • The committed scripts/test-times.json isn't rewritten by a check; bun run check:record still refreshes it.

To try it: push a failing check to main, open a pull, and watch its check wait with one card on the vitals. Then use "check again" on the pull, or make main green.

🤖 Generated with Claude Code

Closes #1797

commented

Check time is predictable: the gate-list audit runs first, tsc is cached, every gate file is timed, and a red main pauses pull checks (#1797).

  • Gate-list audit first: forge.json now runs bun scripts/gate.ts before anything else. A test file listed in neither GATE nor EXCLUDED fails in under a second, instead of after about 25 s of install and tsc.
  • Cached tsc: tsc runs --incremental with a 2048 MB heap and keeps its build info in .forge-cache/. src/checks/cache.ts restores that directory from the base commit's own green check (or the newest kept one). It is kept only from green checks of the default branch, so a pull's code never fills a cache another check reads. Only plain files are copied, with safe names and a cap on count and bytes, and links are never followed. The forge keeps the last 4 commits.
  • Every gate file timed: the full gate runs one file per bun test, slowest first. On green it writes every file's time to .forge-cache/test-times.json. The next split uses those times on top of scripts/test-times.json, so all 274 files are weighed by measurement, not just the 41 the table named.
  • A red main pauses pull checks: new module src/checks/red_main.ts. While the default branch's last finished check is red, its pulls' checks stay queued instead of all failing on the same test. Main's own checks keep running, and the next green one releases the rest. The vitals show one red:<repo> card linked to main's failed run, naming the failing test and how many checks wait.
  • The pull that fixes main can still run: a red main would otherwise hold its own fix forever, which the existing checks suite showed. So a check someone asks for again runs straight away (an ask on a held check re-queues it as theirs), and no check is held more than RED_HOLD_MS (20 min).
  • Merge with main: main had added drainMs (#1796) in the same two places this pull adds redHoldMs. Both options are kept, in ChecksOptions and in the startChecks call in src/main.ts.

Tests: after the merge, bun x tsc --noEmit, the gate audit, and the red-main, vitals, checks, check-tiers and pull-check-again suites pass.

  • test/red-main.test.ts covers the hold, the ask, the 20-minute limit, the release on green and the vitals card.
  • A new end-to-end test in test/check-tiers.test.ts drives red main → held → ask → green → released. With the hold switched off it fails, as it should.

Known failure: test/crew-interrupted.test.ts fails one test ("a young lock that blocks checkout is recovered by a fresh clone", error name_taken). It fails the same way on a clean checkout of main, so this pull didn't cause it.

Changed tests: test/vitals.test.ts now expects 6 queries instead of 5. The card adds one fixed query; the test was right before, and the count still doesn't grow with the number of runs.

Worth a look:

  • The hold is off by default in startChecks (redHoldMs 0), and src/main.ts turns it on with RED_HOLD_MS. Existing suites that make main fail on purpose keep their old behaviour.
  • While checks are held, the existing queue:check lane-wait card can still appear next to the red-main card. I left that card alone.
  • The committed scripts/test-times.json isn't rewritten by a check; bun run check:record still refreshes it.

To try it: push a failing check to main, open a pull, and watch its check wait with one card on the vitals. Then use "check again" on the pull, or make main green.

🤖 Generated with Claude Code

commented

Check time is predictable: the gate-list audit runs first, tsc is cached, every gate file is timed, and a red main pauses pull checks (#1797).

  • Gate-list audit first: forge.json now runs bun scripts/gate.ts before anything else. A test file listed in neither GATE nor EXCLUDED fails in under a second, instead of after about 25 s of install and tsc.
  • Cached tsc: tsc runs --incremental with a 2048 MB heap and keeps its build info in .forge-cache/. src/checks/cache.ts restores that directory from the base commit's own green check (or the newest kept one). It is kept only from green checks of the default branch, so a pull's code never fills a cache another check reads. Only plain files are copied, with safe names and a cap on count and bytes, and links are never followed. The forge keeps the last 4 commits.
  • Every gate file timed: main's full gate (#1833) already runs one file per bun test under coverage. This pull orders those runs slowest first, and on green writes every file's time to .forge-cache/test-times.json. The next split uses those times on top of scripts/test-times.json, so all 274 files are weighed by measurement, not just the 41 the table named.
  • A red main pauses pull checks: new module src/checks/red_main.ts. While the default branch's last finished check is red, its pulls' checks stay queued instead of all failing on the same test. Main's own checks keep running, and the next green one releases the rest. The vitals show one red:<repo> card linked to main's failed run, naming the failing test and how many checks wait.
  • The pull that fixes main can still run: a red main would otherwise hold its own fix forever, which the existing checks suite showed. So a check someone asks for again runs straight away (an ask on a held check re-queues it as theirs), and no check is held more than RED_HOLD_MS (20 min).
  • Merges with main:

- drainMs (#1796) sits beside redHoldMs in ChecksOptions and in src/main.ts. - #1833's impact map (.forge-impact.json) is seeded and kept next to the .forge-cache/ restore and keep in src/checks/service.ts. The two never touch the same files. - The README keeps both paragraphs.

Tests: after the latest merge, bun x tsc --noEmit, the gate audit, and the shards, check-tiers, red-main, vitals, checks and pull-check-again suites pass.

  • test/red-main.test.ts covers the hold, the ask, the 20-minute limit, the release on green and the vitals card.
  • A new end-to-end test in test/check-tiers.test.ts drives red main → held → ask → green → released. With the hold switched off it fails, as it should.

Changed tests:

  • test/vitals.test.ts now expects 6 queries instead of 5. The card adds one fixed query; the test was right before, and the count still doesn't grow with the number of runs.
  • The new end-to-end test in test/check-tiers.test.ts gets an explicit 90 s budget. After the merge it runs past the default 5 s because it drives four checks.

Known failure: test/crew-interrupted.test.ts fails one test ("a young lock that blocks checkout is recovered by a fresh clone", error name_taken). It fails the same way on a clean checkout of main, so this pull didn't cause it.

Worth a look:

  • The hold is off by default in startChecks (redHoldMs 0), and src/main.ts turns it on with RED_HOLD_MS. Existing suites that make main fail on purpose keep their old behaviour.
  • While checks are held, the existing queue:check lane-wait card can still appear next to the red-main card. I left that card alone.
  • The committed scripts/test-times.json isn't rewritten by a check; bun run check:record still refreshes it.

To try it: push a failing check to main, open a pull, and watch its check wait with one card on the vitals. Then use "check again" on the pull, or make main green.

🤖 Generated with Claude Code

Approved

Approved. This is check infrastructure with no page of its own, so there is no phone or keyboard surface to fail. I judged it from the diff, not from the preview.

What I checked:

  • The cache is written only by green default-branch checks.
  • Cache keys are validated as commit ids, and file names, counts and bytes are bounded.
  • Symlinks are never followed, and tests cover each of these.
  • The red-main hold is bounded twice: a person's ask runs, and no check is held past 20 minutes.
  • The red-main query count is accounted for in vitals.test.ts, and the end-to-end test covers hold, ask, green and release.
  • There are no new dependencies, and the README describes both changes.

Two things I would do next:

  1. Fix the grammar of the card sentence in redSentence (src/checks/red_main.ts). For one waiting check it reads "1 pull check wait for it to pass", and the test pins that wording. Use "waits" for the singular and update test/red-main.test.ts.
  2. Direction pillar 0 says no red noise. The red:<repo> card is a degraded vitals card, so I would make it a calm line on the runs or pulls page, or fold it into the existing queue:check card, so the owner sees one notice instead of two. Also replace the fixed Bun.sleep(500) in the red-main end-to-end test with a poll for the queued state, to avoid flakiness on a slow machine.
Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs