Public: anyone can read this; the crew can change it.
scripts/test-budget.ts names each test file over 20 s and each test over 2 s from plain times, with no test run #12142
This adds scripts/test-budget.ts and its test, the first half of the per-test budget (#1242). The module only decides what is over budget; it runs no tests and reads no files. Nothing calls it yet: scripts/test-shards.ts and scripts/gate.ts are untouched, so the gate's output is unchanged.
stopped parked by the crew
3 earlier comments · 1 event
This adds scripts/test-budget.ts and its test, the first half of the per-test budget (#1242). The module only decides what is over budget; it runs no tests and reads no files. Nothing calls it yet: scripts/test-shards.ts and scripts/gate.ts are untouched, so the gate's output is unchanged. The diff is these two files, on top of main.
The module:
breaches({files, tests}, slow = SLOW): takes file times as{path: seconds}, the same shape asscripts/test-times.json, and test times as{file, name, seconds}[]. It names each file over 20 s and each test over 2 s. A time exactly at the limit is within budget. It returns:over: the breaches that count, files first, slowest first.aside: breaches in aslowfile, each carrying its{issue, reason}.slowest: the five slowest tests, each with its share of the run.totalSeconds: the sum of the file times, or of the test times when no file is timed.
- Bad times are refused: a time that is NaN, infinite, negative or not a number throws a
RangeError, so a broken table can never pass as under budget. SLOWis empty: a file should join the list with the issue that rewrites it, and no such issues exist yet, so I didn't invent any.budgetBinds(tier, changed, file): true only on thefasttier, and only whenfileis a.test.tsamong the pull's changed files.summaryLines(result): returns the five slowest (seconds, share in %, file > test), thenover the budget: N, SLOW aside: M, then each breach asfile > test: 3.1 s over 2 s. Set-aside breaches come last and are tagged(SLOW, #issue: reason).
Tests: test/test-budget-run.test.ts uses fixture times only and passes (11 tests). It covers:
- a run under budget passes, including the exact 20 s and 2 s limits;
- a file over 20 s and a test over 2 s are named in the result and in the summary lines;
- a
SLOWfile is set aside with its issue; - the slowest five and their shares match a hand calculation (4/20 = 20%, 3/20 = 15%, …), with a separate case for when no file is timed;
- bad times are refused;
budgetBinds: a full-tier breach does not bind, and neither does a fast-tier breach in an unchanged file or in a changed file that isn't a test.
Checks: bun scripts/gate.ts and bun x tsc --noEmit pass. scripts/fast.ts exits 2 because a script changed, so I chose the covering files myself: the new test, test/unreached.test.ts and test/unused-exports.test.ts. One test in test/unreached.test.ts fails because nothing under src/ imports src/crew/week_burn.ts; it fails the same way on main without this change.
Not done here: wiring the budget into scripts/test-shards.ts and the check summary, and writing per-test times into test-times.json. Those are the second half of #1242.
To try it: bun test test/test-budget-run.test.ts
🤖 Generated with Claude Code
This adds scripts/test-budget.ts and its test, the first half of the per-test budget (#1242). The module only decides what is over budget; it runs no tests and reads no files. Nothing calls it yet: scripts/test-shards.ts and scripts/gate.ts are untouched, so the gate's output is unchanged. The diff is these two files, on top of main.
The module:
breaches({files, tests}, slow = SLOW): takes file times as{path: seconds}, the same shape asscripts/test-times.json, and test times as{file, name, seconds}[]. It names each file over 20 s and each test over 2 s. A time exactly at the limit is within budget. It returns:over: the breaches that count, files first, slowest first.aside: breaches in aslowfile, each carrying its{issue, reason}.slowest: the five slowest tests, each with its share of the run.totalSeconds: the sum of the file times, or of the test times when no file is timed.
- Bad times are refused: a time that is NaN, infinite, negative or not a number throws a
RangeError, so a broken table can never pass as under budget. SLOWis empty: a file should join the list with the issue that rewrites it, and no such issues exist yet, so I didn't invent any.budgetBinds(tier, changed, file): true only on thefasttier, and only whenfileis a.test.tsamong the pull's changed files.summaryLines(result): returns the five slowest (seconds, share in %, file > test), thenover the budget: N, SLOW aside: M, then each breach asfile > test: 3.1 s over 2 s. Set-aside breaches come last and are tagged(SLOW, #issue: reason).
Tests: test/test-budget-run.test.ts uses fixture times only and passes (11 tests). It covers:
- a run under budget passes, including the exact 20 s and 2 s limits;
- a file over 20 s and a test over 2 s are named in the result and in the summary lines;
- a
SLOWfile is set aside with its issue; - the slowest five and their shares match a hand calculation (4/20 = 20%, 3/20 = 15%, …), with a separate case for when no file is timed;
- bad times are refused;
budgetBinds: a full-tier breach does not bind, and neither does a fast-tier breach in an unchanged file or in a changed file that isn't a test.
Fix for the failed check: a comment in the test was 103 columns and broke the gate's 100-column rule; it is now two lines. The rule only checks lines a pull adds, so the gate passes locally only when run with FORGE_CHECK_BASE set to the merge base. With that set, bun scripts/gate.ts passes now, and so do bun x tsc --noEmit and the new test.
Pre-existing failure: test/unreached.test.ts fails because nothing under src/ imports src/crew/week_burn.ts. It fails the same way on main without this change.
Not done here: wiring the budget into scripts/test-shards.ts and the check summary, and writing per-test times into test-times.json. Those are the second half of #1242.
To try it: bun test test/test-budget-run.test.ts
🤖 Generated with Claude Code
This pull adds scripts/test-budget.ts, which reads plain test times and lists each test file over 20 s and each test over 2 s. Files that are already known to be slow go in a separate SLOW list. The check only fails on the fast tier, and only when the change touches a test file. It also writes out the five slowest tests and their share of the total. test/test-budget-run.test.ts covers this against times that can be checked by hand.
The 50 failures in the last check don't come from this branch. They are already on main at e56c7c8a3, which this branch has merged in. The diff against main is only the two files above: neither touches src/, and the new test only works on in-memory data. To confirm, I ran a sample of the failing tests on a separate checkout of origin/main, and they fail there the same way:
test/unreached.test.ts:src/crew/week_burn.ts(fromce7846af2) is not imported by anything.test/reserved-names.test.ts:brokeris a top-level page that is not in the list of reserved names.test/board-grid-style.test.ts: the.bgrid/.bp-*rules and.chart-box.is-tableare no longer in the stylesheet that gets served.
These come from different commits that landed on main today, not one shared regression. I left them out of this pull: they cover unrelated parts of the app, and their fixes belong in their own pulls. Nothing new is pushed. Once main is green again, re-running the check should pass with no change here.
On this branch, bun scripts/gate.ts, bun x tsc --noEmit and bun test test/test-budget-run.test.ts all pass (11 tests).
To try it: bun scripts/test-budget.ts, giving it a file of per-test times; the header comment in the script describes the input format.
🤖 Generated with Claude Code
Parked by the crew: the check stayed red through 2 automatic tries
This adds
scripts/test-budget.tsand its test, the first half of the per-test budget (#1242). The module only decides what is over budget; it runs no tests and reads no files. Nothing calls it yet:scripts/test-shards.tsandscripts/gate.tsare untouched, so the gate's output is unchanged. The diff is these two files, on top of main.The module:
breaches({files, tests}, slow = SLOW): takes file times as{path: seconds}, the same shape asscripts/test-times.json, and test times as{file, name, seconds}[]. It names each file over 20 s and each test over 2 s. A time exactly at the limit is within budget. It returns:over: the breaches that count, files first, slowest first.aside: breaches in aslowfile, each carrying its{issue, reason}.slowest: the five slowest tests, each with its share of the run.totalSeconds: the sum of the file times, or of the test times when no file is timed.RangeError, so a broken table can never pass as under budget.SLOWis empty: a file should join the list with the issue that rewrites it, and no such issues exist yet, so I didn't invent any.budgetBinds(tier, changed, file): true only on thefasttier, and only whenfileis a.test.tsamong the pull's changed files.summaryLines(result): returns the five slowest (seconds, share in %, file > test), thenover the budget: N, SLOW aside: M, then each breach asfile > test: 3.1 s over 2 s. Set-aside breaches come last and are tagged(SLOW, #issue: reason).Tests:
test/test-budget-run.test.tsuses fixture times only and passes (11 tests). It covers:SLOWfile is set aside with its issue;budgetBinds: a full-tier breach does not bind, and neither does a fast-tier breach in an unchanged file or in a changed file that isn't a test.Checks:
bun scripts/gate.tsandbun x tsc --noEmitpass.scripts/fast.tsexits 2 because a script changed, so I chose the covering files myself: the new test,test/unreached.test.tsandtest/unused-exports.test.ts. One test intest/unreached.test.tsfails because nothing undersrc/importssrc/crew/week_burn.ts; it fails the same way on main without this change.Not done here: wiring the budget into
scripts/test-shards.tsand the check summary, and writing per-test times intotest-times.json. Those are the second half of #1242.To try it:
bun test test/test-budget-run.test.ts🤖 Generated with Claude Code
Closes #12136