Sentinel: shared:01m23vygyffx5hg2f5vm8r74sr:1p9qyksd9g is past its budget: trevato/forge: "the page with 150 repositories costs a handful of statements, not one per repository" failed on 3 pulls… #9300

This pull makes the 150-repositories test's verdict depend on the statement count alone, not on how slow the check machine is. I couldn't reproduce the failure here, so the cause is inferred, not confirmed: the test passes locally in about 3 s.

What changed: test/new-forge-pick-150.test.ts only. The page's cost is measured as a count of SQL statements, so only the clock could make it flaky.

• The three tests had Bun's default 5 s timeout. The first one fetches the 150-repository page once, then makes five measured fetches, each preceded by a wait for the database to go quiet. That is the part most likely to run past 5 s under load.
• I set the default test timeout to 120 s.
• I raised the beforeAll timeout, which creates the 150 repositories, from 120 s to 300 s.
• The max: 20 bound on statements is unchanged, so a real N+1 on the page still fails the test.

Not done: I didn't change test/helpers/statements.ts or src/. I haven't seen the failed runs' logs, so if the check still fails after this, the log will show which step it was.

Checks: bun x tsc --noEmit is clean and the test file passes locally. I didn't run scripts/gate.ts or the wider suite; the forge runs the full check on push.

To confirm it worked: after a green check on this pull, shared:01m23vygyffx5hg2f5vm8r74sr:1p9qyksd9g should leave the degraded list.

Closes #9299

This pull makes the 150-repositories test's verdict depend on the statement count alone, not on how slow the check machine is. I couldn't reproduce the failure here, so the cause is inferred, not confirmed: the test passes locally in about 3 s.

What changed: test/new-forge-pick-150.test.ts only. The page's cost is measured as a count of SQL statements, so only the clock could make it flaky.

• The three tests had Bun's default 5 s timeout. The first one fetches the 150-repository page once, then makes five measured fetches, each preceded by a wait for the database to go quiet. That is the part most likely to run past 5 s under load.
• I set the default test timeout to 120 s.
• I raised the beforeAll timeout, which creates the 150 repositories, from 120 s to 300 s.
• The max: 20 bound on statements is unchanged, so a real N+1 on the page still fails the test.

Not done: I didn't change test/helpers/statements.ts or src/. I haven't seen the failed runs' logs, so if the check still fails after this, the log will show which step it was.

Checks: bun x tsc --noEmit is clean and the test file passes locally. I didn't run scripts/gate.ts or the wider suite; the forge runs the full check on push.

To confirm it worked: after a green check on this pull, shared:01m23vygyffx5hg2f5vm8r74sr:1p9qyksd9g should leave the degraded list.

Juniper was right on both points. The timeout widening is reverted, and the pull now fixes a hang in the shared statement counter. I inferred that cause from the logs and could reproduce the hang only with an artificial frozen clock; I never saw the clock frozen in a failed run.

What the logs showed. I read the three failed runs: 01m6gfbec6… (#9297), 01m6gfbea2… (#9293) and 01m6gbqh4m… (#9253). In all three the failure is the test timing out at the 60 s default, not expectWork max 20. The test takes about 3 s when it runs alone. The same runs show other tests that count statements through test/helpers/statements.ts hanging for 60 s each, such as GET /settings/keys and /settings/tokens. The check log says the test "fails only beside others", so the shard's other test files matter. A 60 s hang cannot be fixed by raising a timeout, so I removed my 120 s / 300 s change.

Likely cause. quiet() in test/helpers/statements.ts waits for the database to go idle using Date.now(). Test files in a shard share one process, and several files freeze or mock the wall clock (setSystemTime, Date.now spies). If one of those leaves the clock stopped, quiet() never sees time pass, its 2 s give-up never fires, and the test hangs until the 60 s limit.

What changed.

• test/helpers/statements.ts now reads performance.now(), which a frozen wall clock doesn't affect. max: 20 and the five measured runs are unchanged.
• The new test/statements-frozen-clock.test.ts freezes the wall clock and calls quiet and statements. It times out at 10 s without the fix and passes with it.

Checks. bun x tsc --noEmit is clean. The new test and new-forge-pick-150 pass locally. I didn't run scripts/gate.ts or the full suite; the forge runs the check on this push.

Not done. I didn't find which earlier file leaves the clock frozen. The helper is robust either way, but if the check still hangs, the next place to look is test/backup-truth.test.ts and test/quotas-storage.test.ts, which mock Date.now directly. The #1242 per-test time budget is untouched.

The flaky "page with 150 repositories" test timed out at 60 s on #9297, #9293 and #9253; it never failed the expectWork max 20. Each of those runs also had other statement-counting tests, like GET /settings/keys and /settings/tokens, hanging for exactly 60 s. Run alone, the test takes about 3 s, and the check log says it "fails only beside others".

Cause, inferred from the logs: quiet() in test/helpers/statements.ts used Date.now() both to see when the database had gone idle and for its 2 s give-up. Test files in a shard share one process, and some of them freeze or mock the wall clock (setSystemTime, or replacing Date.now). With the clock stopped, neither condition is ever met and the test waits out its whole timeout. I haven't identified which neighbouring file leaves the clock stopped; test/backup-truth.test.ts and test/quotas-storage.test.ts replace Date.now directly and are the first places to look.

Fix:

• quiet() now reads performance.now(), which a frozen wall clock doesn't touch. It reads it once per turn of the loop.
• The five measured runs and max: 20 are unchanged.
• The new test/statements-frozen-clock.test.ts freezes the wall clock with setSystemTime and calls quiet() and statements(). It times out after 10 s without the fix and passes with it.
• test/new-forge-pick-150.test.ts is not part of this pull. An earlier commit raised its timeouts; that is undone, since a longer timeout only lets a real hang use more of the check.

The type check passes, and the new test and test/new-forge-pick-150.test.ts pass locally. I didn't run scripts/gate.ts or the full suite here; the forge runs the full check on this push. To verify it: the new test should pass, and shared:01m23vygyffx5hg2f5vm8r74sr:1p9qyksd9g should leave the degraded list once pulls stop timing out on it.

🤖 Generated with Claude Code