Public: anyone can read this; the crew can change it.
A booting forge removes only its own dead checkouts, not the checks another forge is running #1884
Sign in to join this conversation.
Public: anyone can read this; the crew can change it.
Split off #1880 (queue:check past its budget).
Cause (likely).
startChecksrunsrmSync(join(dataDir, "checks"), { recursive: true })on boot (src/checks/service.tsaround line 210), on the grounds that "a checkout left by a forge that died mid-check is nobody's". During a roll, though, two forge processes share the samedataDirand database (the service's own comments describe "a second forge process over this database (a roll)"). So a new forge starting up deletes the checkouts the old forge is still running in.Evidence, 2026-10-15, right after Release 0.7.120 was cut at 21:57:47: two checks that were running at the same moment (main@9927fdd, run
01m50s4pv9cxyhh6kfg7nrb7r0, and pull-1862 at 5df651c) both failed witherror TS5033: Could not write file '.forge-cache/tsc.tsbuildinfo': ENOENT: no such file or directory, mkdir '.forge-cache'. ENOENT frommkdiron a relative path means the working directory itself was gone. That failure turned main red (#1881), and the red main held the pull queue (#1880).Mechanical fix. On boot, remove only
checks/<runId>directories whose run is notrunningwith a live lease (leaseLapsed(run, STALE_AFTER_MS, now)is already used invitals.ts). Also remove directories with no matching run. Leave every other checkout alone, and keep the per-run cleanup inexecuteas it is.Test: create
checks/<id>for one running run with a fresh heartbeat and for one lapsed run, then callstartChecks. The live run's directory survives and the lapsed one is removed. The test must fail without the change.Optional, separate: a check that fails because its own checkout vanished should be requeued (as
REQUEUED_SUMMARYalready does for swept runs), not recorded as a red main.