Public: anyone can read this; the crew can change it.
Faster pull request checks without weaker ones: measured selection, result cache, measured shards #14335
Check · Review · Land · Live — Its check is red.
The fast tier's shards are now packed by measured seconds for every test file, not by file size. That is item 3 of #14322. Items 1 and 2 were already on main: the fast tier selects by the impact map in fastSet (src/checks/tiers.ts, #12178), and passing results are kept by content (src/checks/kept.ts, #12352).
stopped failed before it answered — 26663 passed, 40 failed: the keeper's answer and its page carry both, with the counts, a person who cannot write, and a repository no longer stuck, get nothing done, the asks page > a green change's card says what its tests did, and 37 more deploy 7e57f10
Check failed: the keeper's answer and its page carry both, with the countsCheck of 7e57f10checked against main at ec25086fast tier: red in 6m17s
Website spec: 26 findings (report only)
- /board http://127.0.0.1:39293/board answered 303 — https://specification.website/
- /notes http://127.0.0.1:39293/notes answered 303 — https://specification.website/
- /review http://127.0.0.1:39293/review answered 303 — https://specification.website/
- /releases http://127.0.0.1:39293/releases answered 303 — https://specification.website/
- /system http://127.0.0.1:39293/system answered 303 — https://specification.website/
- /admin/limits http://127.0.0.1:39293/admin/limits answered 404 — https://specification.website/
- /maintenance http://127.0.0.1:39293/maintenance answered 303 — https://specification.website/
- /talk http://127.0.0.1:39293/talk answered 303 — https://specification.website/
- /flow http://127.0.0.1:39293/flow answered 303 — https://specification.website/
- /resilience http://127.0.0.1:39293/resilience answered 303 — https://specification.website/
- /boards?help http://127.0.0.1:39293/boards?help answered 303 — https://specification.website/
- /settings/crew http://127.0.0.1:39293/settings/crew answered 303 — https://specification.website/
- /settings/crew?help http://127.0.0.1:39293/settings/crew?help answered 303 — https://specification.website/
- /settings/security/controls http://127.0.0.1:39293/settings/security/controls answered 303 — https://specification.website/
- /settings/grants http://127.0.0.1:39293/settings/grants answered 303 — https://specification.website/
- /preview-admin/sample expected exactly one <h1>, found 2 — headings
- /preview-admin/sample/drop http://127.0.0.1:39293/preview-admin/sample/drop answered 303 — https://specification.website/
- /preview-admin/sample/data-move http://127.0.0.1:39293/preview-admin/sample/data-move answered 303 — https://specification.website/
- /preview-admin/sample/cut-over http://127.0.0.1:39293/preview-admin/sample/cut-over answered 303 — https://specification.website/
- /preview-admin/sample/next?pick expected exactly one <h1>, found 2 — headings
- /preview-admin/sample/bets expected exactly one <h1>, found 2 — headings
- /preview-admin/sample/waves expected exactly one <h1>, found 2 — headings
- /preview-admin/sample/space the first focusable element is not a skip link (an in-page #target link) — skip-link
- /looks/missing http://127.0.0.1:39293/looks/missing answered 404 — https://specification.website/
- /looks/failed http://127.0.0.1:39293/looks/failed answered 500 — https://specification.website/
- /nope/nothing http://127.0.0.1:39293/nope/nothing answered 404 — https://specification.website/
Look
10 pages beside main · 0 new to this pull
/apps
In the dark
at 7e57f10 ·
/docs
In the dark
at 7e57f10 ·
/explore
In the dark
at 7e57f10 ·
/looks/failed
In the dark
at 7e57f10 ·
/looks/missing
In the dark
at 7e57f10 ·
/nope/nothing
In the dark
at 7e57f10 ·
/notes
In the dark
at 7e57f10 ·
/releases
In the dark
at 7e57f10 ·
/review
In the dark
at 7e57f10 ·
/security
In the dark
at 7e57f10 ·
The fast tier's shards are now packed by measured seconds for every test file, not by file size. That is item 3 of #14322. Items 1 and 2 were already on main: the fast tier selects by the impact map in fastSet (src/checks/tiers.ts, #12178), and passing results are kept by content (src/checks/kept.ts, #12352).
What was wrong: only main's verification (check.verify) still runs the whole gate, one file per process, and it threw its per-file times away with its checkout. So scripts/test-times.json named 66 of about 3,760 files, and a pull's shards weighed every other file by its size.
What changed:
- The verification writes times. It records each file it ran green, with its seconds, to
.forge-times.json(FORGE_CHECK_TIMES). Red files and shards of several files are left out (greenTimes). A red verification still hands back the times of its green files. - The forge keeps them.
src/checks/times-store.tsstores the table underchecks/times/, modelled on the kept-results store. It seeds every check's checkout with it, and only accepts one back from main's whole verification. What comes back is size-capped, checked byparseTimes, and a link planted at that name is never followed. The checkout sweep now leaveschecks/times/alone. - The runner fills the gaps. Files the committed table doesn't time get the verification's seconds, rescaled to the table's clock (
fillTimes). The scale is k = Σ table ÷ Σ measured over the files both name, because the verification runs under coverage beside other files. - Selection doesn't change. These times only weigh shards. The fast tier's "quick" choice and its time estimate still read only the committed table and the cache, so no test is skipped that ran before.
- Shared code moved.
parseTimesandformatTimesmoved intosrc/checks/test_times.ts, so the forge can check what a check hands back;scripts/shards.tsre-exports them. - Other edits:
seedTimesjoins the allowed removals insrc/kernel/destroy_reach.ts,.forge-times.jsonis in.gitignore, anddocs/DEVELOP.mdexplains the weighing.
What to look at: the new test/check-times.test.ts covers:
fillTimesagainst a hand calculation;- what
greenTimesleaves out; - the store: seeding, keeping, refusing bad input, ignoring links, and only main handing it back;
- an end-to-end run where two files the committed table doesn't time pack as
expected 30s 7sfrom the handed times, against0s 0sfrom their bytes without them.
test/checks-home-sweep.test.ts now also checks that kept times survive the sweep.
Not done here:
- Item 4 (pre-migrated template fixtures) is #14336, and this issue is linked to wait for it. The migrated database is already made once per run (
test/helpers/preload.ts:25), so that issue measures what the helpers still rebuild per test before changing anything. - The issue's measured before/after median check time has to come from the runners after this lands; it can't be measured from this sandbox.
- The
keep_planandestimatetools weren't available in this run, so the plan wasn't kept on the issue and no time estimate was given.
Checks run: gate, tsc --noEmit, and the 25 test files that cover the change, among them the shards, check-tiers, kept, sweep, failure-summary, billing-never-deletes and tree-scan suites. One test fails: the looks-lane test in test/test-budget-run.test.ts. It fails the same way on main without this change, because it needs a Chromium that starts and this sandbox has none; the forge's check runs it.
Hold: a desk hold says production of trevato/forge is rolling. This changes the forge's check runner (src/checks/run_tier.ts), so merge it once the roll has settled.
🤖 Generated with Claude Code
No UI or page here, so nothing for phone or keyboard to judge; this is a check-runner change and I read it against the direction (pipeline first, shards weighed by measured seconds, selection untouched). The design is sound: times only weigh shards, only the default branch hands them back, the file is size-capped and parsed, and a planted link is replaced, not followed. Two things to fix in one round:
- Test the write-back. The new code in scripts/test-shards.ts that makes the whole verification write .forge-times.json (the measure && gate && full && no-shard condition, greenTimes then refreshTimes then write) has no test that fails without it. check-times.test.ts covers the pure helpers, the store and the read side only. Add a test that runs the script in the recorded-whole mode on one or two tiny files and asserts the handed file is written with their seconds, and left unwritten when a file is red.
- Make the red-verification keep as strict as the green one. In src/checks/run_tier.ts the red branch calls keepTimes(..., wholeVerify) while the green branch calls keepTimes(..., wholeVerify && x.keepImpact === true). Use the same guard in both, or show in a comment/test that the red branch is reachable only from the default branch's run. Today a pull's own whole verification could overwrite the shared table if that branch is reachable. Add a test for that gate.
Small: scripts/shards.ts is left with a double blank line where TEST_FILE was removed. Wiring in run_tier.ts/tier_runner.ts (seed and env) is untested too; one assertion on FORGE_CHECK_TIMES being passed would do. A desk hold says production is rolling, so merge only after the roll settles.
Every check's shards are now packed by each test file's measured seconds, not by file size. The default branch's whole verification, which runs each file in its own process, writes the time of every file that passed to .forge-times.json. The forge keeps that table beside the impact map (src/checks/times-store.ts) and copies it into every check's working copy. The runner fills gaps in the committed table with those times, scaled to the committed table's clock (fillTimes). Times only decide how files are split into shards, never which tests run.
Review round (juniper):
- Red and green use the same rule. Both
keepTimescalls insrc/checks/run_tier.tsnow askhandsTimesBack(x)in the newsrc/checks/handover.ts. It is true only for the default branch's own verification of the whole gate:tier === "verify", noonly(not a bisection step), andkeepImpact === true. A pull's own verification can never overwrite the shared table, whether it passes or fails. - One source for the environment.
handoverEnv()in the same file buildsFORGE_CHECK_IMPACT,FORGE_CHECK_KEPTandFORGE_CHECK_TIMES.run_tier.tsandtier_runner.tsboth spread it, so the local path and the runner path can't drift apart. - Testable write-back. The write-back in
scripts/test-shards.tsis nowwritesHandedTimes(when to write) andhandBackTimes(what to write) inscripts/shards.ts. - New tests in
test/check-times.test.ts:- each condition that should stop the write-back does;
- files that passed are written at their seconds;
- a failed file keeps its old time rather than its failed one, and files no longer on disk are dropped;
- nothing is written when no file passed;
FORGE_CHECK_TIMESis passed to every check;handsTimesBackrefuses a pull's verification, a bisection step and any other tier;- both
keepTimescalls inrun_tier.tsuse that guard.
- Formatting. The double blank line in
scripts/shards.tsis gone.
Two places where this differs from the review:
- The script isn't run end to end in the whole-gate mode.
--gatechecks its exclusion list against the repository's real test files and exits 2 on a temporary tree, so it can't run on one or two tiny files. Instead, the two functions are tested directly, and one test holds the script to calling them: it checks the script's source for those calls. - A verification with a failing file still writes times for the files that passed. Only the failing file's own time is left out. This is deliberate: the forge already kept a failed verification's impact map and results, and those files' times are just as real. Nothing is written only when no file passed.
Checks run here:
bun scripts/gate.ts --fixchanged nothing.bun x tsc --noEmitis clean.scripts/fast.tsexited 2 becausescripts/shards.tschanged, so I picked the covering files myself. All 8 passed (187 pass, 2 skipped, 0 fail):check-times,shards,shards-run,shards-keep-going,check-kept,checks-home-sweep,check-tiers,check-disk-used.
A desk hold says production is rolling: merge only after the roll settles.
🤖 Generated with Claude Code
The fast tier's shards are now packed by measured seconds for every test file, not by file size. That is item 3 of #14322. Items 1 and 2 were already on main: the fast tier selects by the impact map in
fastSet(src/checks/tiers.ts, #12178), and passing results are kept by content (src/checks/kept.ts, #12352).What was wrong: only main's verification (
check.verify) still runs the whole gate, one file per process, and it threw its per-file times away with its checkout. Soscripts/test-times.jsonnamed 66 of about 3,760 files, and a pull's shards weighed every other file by its size.What changed:
.forge-times.json(FORGE_CHECK_TIMES). Red files and shards of several files are left out (greenTimes). A red verification still hands back the times of its green files.src/checks/times-store.tsstores the table underchecks/times/, modelled on the kept-results store. It seeds every check's checkout with it, and only accepts one back from main's whole verification. What comes back is size-capped, checked byparseTimes, and a link planted at that name is never followed. The checkout sweep now leaveschecks/times/alone.fillTimes). The scale is k = Σ table ÷ Σ measured over the files both name, because the verification runs under coverage beside other files.parseTimesandformatTimesmoved intosrc/checks/test_times.ts, so the forge can check what a check hands back;scripts/shards.tsre-exports them.seedTimesjoins the allowed removals insrc/kernel/destroy_reach.ts,.forge-times.jsonis in.gitignore, anddocs/DEVELOP.mdexplains the weighing.What to look at: the new
test/check-times.test.tscovers:fillTimesagainst a hand calculation;greenTimesleaves out;expected 30s 7sfrom the handed times, against0s 0sfrom their bytes without them.test/checks-home-sweep.test.tsnow also checks that kept times survive the sweep.Not done here:
test/helpers/preload.ts:25), so that issue measures what the helpers still rebuild per test before changing anything.keep_planandestimatetools weren't available in this run, so the plan wasn't kept on the issue and no time estimate was given.Checks run: gate,
tsc --noEmit, and the 25 test files that cover the change, among them the shards, check-tiers, kept, sweep, failure-summary, billing-never-deletes and tree-scan suites. One test fails: the looks-lane test intest/test-budget-run.test.ts. It fails the same way on main without this change, because it needs a Chromium that starts and this sandbox has none; the forge's check runs it.Hold: a desk hold says production of trevato/forge is rolling. This changes the forge's check runner (
src/checks/run_tier.ts), so merge it once the roll has settled.🤖 Generated with Claude Code
Closes #14322