No wall-clock speed asserts in tests: one budget helper that is load-proof, and the gate refuses raw "under N ms" expects #4431

Speed assertions in tests now go through one helper, expectBudget, and a new lint in the flake test refuses a raw "under N ms" title or bound. Items 1–3 are done. The new lint and the migrated timing suites pass (238 tests in flake.test.ts, the *-timing files and explore-rank). I did not run the type check, because bun x tsc couldn't find the bun types here. I also did not run the 20-times-beside-6-busy-cells test; the first push will be its first run in the forge's check.

What changed

• Helper: test/helpers/budget.ts has expectBudget(measuredMs, { ms, against }). Unless FORGE_TEST_QUIET=1, it allows max(200, against × 10), so load that slows the page also slows the baseline. When FORGE_TEST_QUIET=1, it asserts the absolute ms. baselineMs(url) measures the best of five /healthz requests in the same run to use as against.
• Migrated tests: the six "answers under N ms" files (signin, settings, verify, asked, cloud-org-apps, repo-pages, app-pages timing tests) and explore-rank now use the helper. Titles now read "answers within its budget".
• Phase checks: the phase-gap assertions in settings-timing and cloud-org-apps-timing also use the helper, because they flake under load too.
• Lint: rawBudgets in scripts/flake.ts, enforced in test/flake.test.ts, fails on a title saying "under N ms". It also fails on a Server-Timing or elapsed-time toBeLessThan under 200 ms outside the helper, and its message points at expectBudget. CLAUDE.md documents the rule.

Left out

• Hang guards: bounds of 200 ms or more (for example expect(elapsed).toBeLessThan(10_000) in session-settle and agents) stay as plain expects. They catch hangs, not slowness, and the existing floor rule already allows them.
• Nightly lane: the perf lane that sets FORGE_TEST_QUIET=1 is not created. It needs a workflow, which I did not add.
• Item 4: p95 budgets on /boards from production Server-Timing are not done.
• Other tests: non-Server-Timing tests with generous time bounds are untouched.

Try it: bun test test/flake.test.ts test/signin-timing.test.ts. To see the absolute check, run FORGE_TEST_QUIET=1 bun test test/*-timing.test.ts.

Closes #4429

Speed assertions in tests now go through one helper, expectBudget, and a new lint in the flake test refuses a raw "under N ms" title or bound. Items 1–3 are done. The new lint and the migrated timing suites pass (238 tests in flake.test.ts, the *-timing files and explore-rank). I did not run the type check, because bun x tsc couldn't find the bun types here. I also did not run the 20-times-beside-6-busy-cells test; the first push will be its first run in the forge's check.

What changed

• Helper: test/helpers/budget.ts has expectBudget(measuredMs, { ms, against }). Unless FORGE_TEST_QUIET=1, it allows max(200, against × 10), so load that slows the page also slows the baseline. When FORGE_TEST_QUIET=1, it asserts the absolute ms. baselineMs(url) measures the best of five /healthz requests in the same run to use as against.
• Migrated tests: the six "answers under N ms" files (signin, settings, verify, asked, cloud-org-apps, repo-pages, app-pages timing tests) and explore-rank now use the helper. Titles now read "answers within its budget".
• Phase checks: the phase-gap assertions in settings-timing and cloud-org-apps-timing also use the helper, because they flake under load too.
• Lint: rawBudgets in scripts/flake.ts, enforced in test/flake.test.ts, fails on a title saying "under N ms". It also fails on a Server-Timing or elapsed-time toBeLessThan under 200 ms outside the helper, and its message points at expectBudget. CLAUDE.md documents the rule.

Left out

• Hang guards: bounds of 200 ms or more (for example expect(elapsed).toBeLessThan(10_000) in session-settle and agents) stay as plain expects. They catch hangs, not slowness, and the existing floor rule already allows them.
• Nightly lane: the perf lane that sets FORGE_TEST_QUIET=1 is not created. It needs a workflow, which I did not add.
• Item 4: p95 budgets on /boards from production Server-Timing are not done.
• Other tests: non-Server-Timing tests with generous time bounds are untouched.

Try it: bun test test/flake.test.ts test/signin-timing.test.ts. To see the absolute check, run FORGE_TEST_QUIET=1 bun test test/*-timing.test.ts.

The branch head doesn't match your description of it, so I made no new changes and pushed nothing. The head is 8cfdc882 on scout/4429-no-wall-clock-speed-asserts-in-tests-one, and origin has the same commit.

• The budget work is in the head. git diff main...HEAD shows 13 files:

- test/helpers/budget.ts exists in the tree. - rawBudgets is in scripts/flake.ts. - The migrated timing tests and the test/flake.test.ts cases are there. - The CLAUDE.md note is there.

• The .live-toggle removal is not in it. Nothing under src/web/style/content.ts or docs/DESIGN.md is touched, so that cleanup is in a different branch or pull.
• Type check. I ran bun install and then bun x tsc --noEmit, and it came back clean. My earlier run had failed only because node_modules was missing the bun types. bun test test/flake.test.ts and the migrated *-timing files plus explore-rank also passed earlier.

Please re-read the diff on the current head of #4431. If you saw the .live-toggle change there, tell me which commit it came from, because I can't find it.

I made no changes and pushed nothing. I can't reproduce the mismatch you describe.

• The branch has the issue's work. Head 8cfdc882 on scout/4429-no-wall-clock-speed-asserts-in-tests-one, which origin also has, changes 13 files against main:

- The new test/helpers/budget.ts holds expectBudget and baselineMs. - rawBudgets is in scripts/flake.ts, with its tests in test/flake.test.ts. - The nine timing and rank tests are migrated, and the CLAUDE.md note is added.

• .live-toggle is not in my diff. The diff touches neither src/web/style/content.ts nor docs/DESIGN.md, and .live-toggle is still there in my tree. The diff you reviewed was probably a different head or pull.
• Type check. bun x tsc --noEmit is clean on this head. I ran it after bun install, since my first run had no bun types.
• Tests. The flake lint, the six migrated *-timing suites and explore-rank passed locally. I never ran them 20 times beside 6 busy cells, and the forge's check on the push is the first full run.

Please check which head #4431 currently points at. If it isn't 8cfdc882, that's the cause. Main has also moved since I branched (now 7d8c6a35), so a rebase may be needed.

Handed to a person by the crew: two follow-up runs on the review's changes ended without a push, so no more start here until you say what to try. (The forge posts this under the reviewer's name; the reviewer did not write it.)

This pull adds expectBudget, a load-proof budget helper, and makes the flake test refuse raw "under N ms" expects. The new triage message on #4429 needs no code change: it only says the issue has no pillar yet and waits behind the Later gate. Naming a pillar is for a person to do.

• Helper: test/helpers/budget.ts adds expectBudget. It compares a timing against a baseline measured in the same run, so machine load affects both sides equally. A fixed millisecond limit applies only when FORGE_TEST_QUIET=1.
• Gate: scripts/flake.ts flags raw wall-clock "under N ms" expects, and test/flake.test.ts covers that check.
• Callers: the eight timing tests that had raw asserts now go through the helper:

- app-pages - asked - cloud-org-apps - explore-rank - repo-pages - settings - signin - thread-render

• Docs: CLAUDE.md records the rule.

I did not touch timing asserts outside those tests. I re-ran the changed suites (48 pass, 0 fail) and did not run the full check; the forge runs that on the push.

To try it, run bun test test/flake.test.ts test/signin-timing.test.ts.

🤖 Generated with Claude Code

I merged main into the branch and pushed it. The merge now applies cleanly. The flake test and the two timing tests pass, and bun x tsc --noEmit is clean. I didn't run the rest of bun test; the forge's check runs the whole suite on this push.

• Conflict: it was in test/app-pages-timing.test.ts. I kept main's change of judging the best of five requests, and the total is now bounded with expectBudget, so it is relative to a same-run baseline instead of a raw 50 ms.
• New raw timing assert: the flake test caught one more from main, in test/settings-repo-timing.test.ts (from #4318). I changed it to expectBudget with a baseline too.
• Slip on the first push: I pushed the merge before I had read the flake test's failure. The second commit fixes it.

Speed checks in tests now go through one helper, expectBudget in test/helpers/budget.ts, and test/flake.test.ts refuses a raw "under N ms" assert or title. The budget is relative to a same-run /healthz baseline, and the absolute ms is asserted only when FORGE_TEST_QUIET=1. Tests pass. After the last commit I re-ran only test/flake.test.ts and test/settings-timing.test.ts, plus the type check, and all were clean. The forge runs the full check on the push.

Changes for your review:

• Limit with teeth: outside the quiet lane the limit is now min(ms * 4, max(ms, against * 10)). A 50 ms budget can never go above 200 ms, so a page that regresses from 5 ms to 150 ms fails. The 200 ms floor is gone.
• No quiet lane yet: I did not add one. Nothing sets FORGE_TEST_QUIET, and CLAUDE.md and the helper's header now say that absolute budgets are not enforced anywhere. They no longer mention a nightly lane.
• Phase-gap checks: the checks in test/cloud-org-apps-timing.test.ts and test/settings-timing.test.ts are plain expects again, marked as accounting rather than speed. The settings one needed a gap variable so the speed gate doesn't flag it.
• test/thread-render-timing.test.ts: the title now says what bounds it. The bounds are the markdown memo hit and miss counters, plus the existing loose 250 ms guard.
• test/explore-rank.test.ts: I dropped the expectBudget call, since it was redundant with the next line (warm × 5 < cold). The absolute 15 ms is asserted only when quiet() is true. That check does not run anywhere yet.
• New test/budget.test.ts: it covers budgetLimit and expectBudget: quiet versus loaded, the baseline factor, the 4× cap, and that only "1" counts as quiet. It also checks that 150 ms beside a 5 ms baseline fails.

I pushed twice before the flake test was fully green, once with the gate still failing on the settings line. The last commit fixes it.