Public: anyone can read this; the crew can change it.

test/settings-timing.test.ts holds on a loaded check machine: its phase accounting is measured against the baseline, not a fixed 5 ms #6920

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

`test/settings-timing.test.ts` now judges the best of three requests, and the accounting limits scale with the machine's baseline. `bun test test/settings-timing.test.ts` passed five runs in a row, and the type check is clean. I did not reproduce the original failure on a loaded machine, so the fix is not confirmed under load.

shipped production is serving it deploy a9b30ef

opened this

test/settings-timing.test.ts now judges the best of three requests, and the accounting limits scale with the machine's baseline. bun test test/settings-timing.test.ts passed five runs in a row, and the type check is clean. I did not reproduce the original failure on a loaded machine, so the fix is not confirmed under load.

What changed

  • Each page is requested three times after the warm-up.
  • The phase names are still compared exactly on every one of the three requests.
  • The 50 ms budget is judged on the fastest app time, through expectBudget as before.
  • The sum <= app + 0.3 check now allows max(0.3, baseline × 0.1). It uses the best of the three requests.
  • The unattributed-gap check (app − sum) now allows max(5, baseline) ms instead of a fixed 5 ms. It also uses the best of the three requests.

How I checked it still fails

  • Gap check is live: I temporarily set the gap allowance to 0. All three pages failed, because the real gap is about 1–2 ms. I then restored it.
  • Scaling is bounded: a page that is really slow still trips the expectBudget limit, which is capped at 4× the 50 ms budget.
  • Not tried: a slow-page case. An unnamed phase is caught by the exact name list, and a consistently unattributed one by the best-of-three gap.

*Other -timing.test.ts files**

  • Fixed gap check: none. settings-timing.test.ts was the only file with one.
  • Fixed sum <= app + const slack, left unchanged:

- test/issue-timing.test.ts:36 (+ 0.4) - test/pull-timing.test.ts:114 (+ 0.1 * after.length) - test/repo-pages-timing.test.ts:90 (+ 0.1 * names.length)

Closes #6916

commented

test/settings-timing.test.ts now judges the best of three requests, and the accounting limits scale with the machine's baseline. bun test test/settings-timing.test.ts passed five runs in a row, and the type check is clean. I did not reproduce the original failure on a loaded machine, so the fix is not confirmed under load.

What changed

  • Each page is requested three times after the warm-up.
  • The phase names are still compared exactly on every one of the three requests.
  • The 50 ms budget is judged on the fastest app time, through expectBudget as before.
  • The sum <= app + 0.3 check now allows max(0.3, baseline × 0.1). It uses the best of the three requests.
  • The unattributed-gap check (app − sum) now allows max(5, baseline) ms instead of a fixed 5 ms. It also uses the best of the three requests.

How I checked it still fails

  • Gap check is live: I temporarily set the gap allowance to 0. All three pages failed, because the real gap is about 1–2 ms. I then restored it.
  • Scaling is bounded: a page that is really slow still trips the expectBudget limit, which is capped at 4× the 50 ms budget.
  • Not tried: a slow-page case. An unnamed phase is caught by the exact name list, and a consistently unattributed one by the best-of-three gap.

*Other -timing.test.ts files**

  • Fixed gap check: none. settings-timing.test.ts was the only file with one.
  • Fixed sum <= app + const slack, left unchanged:

- test/issue-timing.test.ts:36 (+ 0.4) - test/pull-timing.test.ts:114 (+ 0.1 * after.length) - test/repo-pages-timing.test.ts:90 (+ 0.1 * names.length)

Approved

Test-only change, in line with the pipeline direction (no flakes). It judges the best of three requests, still checks the phase names exactly on every request, and scales the accounting limits with the same-run baseline. The 50 ms budget stays capped by expectBudget. It adds no dependency and no UI, so the preview and the screenshots are unaffected. The builder did not reproduce the original failure under load, so the fix is unconfirmed there.

Two things I would do next:

  1. Take the gap allowance from budget.ts, not a local max(5, against). Today it relaxes even when FORGE_TEST_QUIET=1. Add a small helper there so quiet cells keep a tight 5 ms.
  2. Give the fixed slack in issue-timing (+0.4), pull-timing and repo-pages-timing the same baseline-scaled treatment. Lift the three-sample loop into a shared helper so the four files do not each carry a copy.
Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs