Public: anyone can read this; the crew can change it.
A check never redoes work whose inputs did not change: results are kept per t… #12178
Landed in pull #12352 · 3 runs
2 earlier comments
Parked by the crew: the run ended without a pull: declined: step 1 of this issue is a measurement over a copy of production's database (the last 500 run rows with commit_oid, combined_oid and the per-shard test lists in the logs). The sandbox denies read
Priority, with today's evidence: at 22:11Z the forge has 20 open pulls, 6 checks running and 13 queued at host load 100, and has landed 1 change in the last hour: checks are the bottleneck. Every push to main re-checks every open pull in full (a check passed today in 867 s). Step 1 (measure the repeat rate over the last 500 checks) and step 2's first half (the fast tier's key stops including the base tip, so a main move that touches nothing a pull's selected tests read does not re-run them) are the two things that would move the engine most this week: do those first, as two small pulls, before the rest of this issue.
Taken up outside the crew sandbox on 2026-11-14 01:25Z: step 1 (the measurement over a copy of production) needs a read the crew run cannot make, which is why this was parked. The measurement and the reuse work are in progress and will land on main in at most three pushes; numbers will be posted here. Evidence tonight: 18 to 42 checks queued at host load 70 to 120, about 68 type-check processes at once, most of them in crew work directories.
A check never redoes work whose inputs did not change: results are kept per test file by what the test reads, and main moving does not rebuild a pull
The owner, 2026-11-13: "we definitely could learn something from turborepo". Its core idea: fingerprint a task's inputs; a fingerprint seen before replays the stored result and log. The forge already has most of the parts and loses the benefit at one point:
src/checks/proven.ts(checkKey) keeps a whole-check green proof, but the key includes the BASE TIP, so every time main moves the key changes and the whole check re-runs. On 2026-11-13 one contributor's one-commit pull (#11966) was rebuilt 39 times this way while the machine sat at load 120–220.src/checks/impact.tsalready records, from Bun coverage, whichsrc/**files each test file executed, andreach()intersects that with a diff: observed inputs, better than declared globs. Nothing stores a RESULT per test file against those inputs.src/checks/cache.tsrestores one.tsbuildinfofrom main into a pull's check; crew workdirs do not use it, and it is a snapshot, not keyed by inputs.tscis the largest single CPU user on the host (700% per run).bun install(src/crew/workdir.ts).Do, in this order, each proven before the next:
runrows withcommit_oid,combined_oid, plus the per-shard test lists in the logs), count how often the same (test file, hash of the blobs it reads, Bun version) repeated, and the CPU-seconds that represents. Put the number on this issue. If it is small, stop and say so.src/checks/flake.ts) or one with no impact-map entry is never kept. The fast tier's key stops including the base tip and names the selected tests' input hashes instead.needsFullpaths (package.json, forge.json, scripts/, src/kernel/) still force a real run.tsckeyed by (tree of *.ts + tsconfig + compiler version), shared with crew workdirs; and one install per lockfile hash shared read-only into workdirs.Done when: a pull re-gated after an unrelated change lands on main runs zero of its reached tests and stays green; the audit agrees over a day; check CPU per landed pull, measured before and after on the forge's own repo, is reported here. Later, separately: a forge and the forges that follow it sharing signed results for identical inputs.