Public: anyone can read this; the crew can change it.
Tests wait for a signal, not a clock: a ratchet on fixed sleeps in test/ #14333
Check · Review · Land · Live — No review yet.
The gate now checks each test file for fixed waits and compares the count to an allowance file. A file with more than its allowance fails with "await the operation or an event it emits instead of a fixed delay". A file with fewer also fails until its allowance is lowered, so the list only shrinks.
look green — it answered, no tests were run, ready for a look deploy 0bc0c37
Check passeddeploy 0bc0c37
Website spec: 26 findings (report only)
- /board http://127.0.0.1:41577/board answered 303 — https://specification.website/
- /notes http://127.0.0.1:41577/notes answered 303 — https://specification.website/
- /review http://127.0.0.1:41577/review answered 303 — https://specification.website/
- /releases http://127.0.0.1:41577/releases answered 303 — https://specification.website/
- /system http://127.0.0.1:41577/system answered 303 — https://specification.website/
- /admin/limits http://127.0.0.1:41577/admin/limits answered 404 — https://specification.website/
- /maintenance http://127.0.0.1:41577/maintenance answered 303 — https://specification.website/
- /talk http://127.0.0.1:41577/talk answered 303 — https://specification.website/
- /flow http://127.0.0.1:41577/flow answered 303 — https://specification.website/
- /resilience http://127.0.0.1:41577/resilience answered 303 — https://specification.website/
- /boards?help http://127.0.0.1:41577/boards?help answered 303 — https://specification.website/
- /settings/crew http://127.0.0.1:41577/settings/crew answered 303 — https://specification.website/
- /settings/crew?help http://127.0.0.1:41577/settings/crew?help answered 303 — https://specification.website/
- /settings/security/controls http://127.0.0.1:41577/settings/security/controls answered 303 — https://specification.website/
- /settings/grants http://127.0.0.1:41577/settings/grants answered 303 — https://specification.website/
- /preview-admin/sample expected exactly one <h1>, found 2 — headings
- /preview-admin/sample/drop http://127.0.0.1:41577/preview-admin/sample/drop answered 303 — https://specification.website/
- /preview-admin/sample/data-move http://127.0.0.1:41577/preview-admin/sample/data-move answered 303 — https://specification.website/
- /preview-admin/sample/cut-over http://127.0.0.1:41577/preview-admin/sample/cut-over answered 303 — https://specification.website/
- /preview-admin/sample/next?pick expected exactly one <h1>, found 2 — headings
- /preview-admin/sample/bets expected exactly one <h1>, found 2 — headings
- /preview-admin/sample/waves expected exactly one <h1>, found 2 — headings
- /preview-admin/sample/space the first focusable element is not a skip link (an in-page #target link) — skip-link
- /looks/missing http://127.0.0.1:41577/looks/missing answered 404 — https://specification.website/
- /looks/failed http://127.0.0.1:41577/looks/failed answered 500 — https://specification.website/
- /nope/nothing http://127.0.0.1:41577/nope/nothing answered 404 — https://specification.website/
The gate now checks each test file for fixed waits and compares the count to an allowance file. A file with more than its allowance fails with "await the operation or an event it emits instead of a fixed delay". A file with fewer also fails until its allowance is lowered, so the list only shrinks. Because forge.json's run, fast and full checks all start with bun scripts/gate.ts, this runs on pull checks and on main.
- Counting:
src/kernel/test_honesty.tstakes test sources and returns counts. Per file, it counts four things in every.tsundertest/:await Bun.sleep(<number>)setTimeout(<name>, <number>)mock.module(- test
retry:/timeout:options, on their own line or as}, { retry: N }
A wait with a name or computed value, like
Bun.sleep(ms), is not counted. The allowance file is checked when read: a malformed one is rejected with a message, not trusted. - Gate:
scripts/gate.tsgainstestHonestyRefusal, which is added togateVerdict's refusals, andwriteTestHonesty.FORGE_SLEEP_WRITE=1 bun scripts/gate.tsrewrites the allowance to today's counts. A tree without the allowance file, such as the gate's own test fixtures, is not checked. - Allowance:
docs/test-honesty.jsonholds today's counts: 764 sleeps, 100setTimeoutwaits, 3mock.moduleand 7 retry/timeout options, across 461 files. The sleep andmock.modulecounts match the issue. The other two came out higher than the issue's 90 and 5 because these patterns are a bit wider; the allowance covers the difference. - Tests:
test/test-honesty.test.tscovers each kind of wait, a malformed allowance, and the repository's own tree matching its allowance. It also shows one newawait Bun.sleep(50)failing with the advice and a drop being refused until the allowance is lowered.
Choices to look at:
- The counting code is in
src/kernel/test_honesty.ts, not thescripts/test_honesty.tsthe issue named. The gate's other rules already live insrc/kernel/and are loaded byrequire, sotreeGatecan still loadgate.tson its own. - Raised
timeout:lines are matched by text, so the count includes some flagged lines that are not test timeouts (for exampletimeout: 600in check config fixtures). Counting every trailing}, 30_000)timeout argument would have matched about 2,100 lines, so I left that pattern out. - landmines'
no-commentsandno-explicit-return-typesare out of scope, as the issue says. - I did not call
keep_planorestimate; neither tool was available in this run.
To try it: add an await Bun.sleep(50); to any test file and run bun scripts/gate.ts (it exits 2 with the message), then remove it.
Checks: bun scripts/gate.ts --fix changed nothing and passed, and bun x tsc --noEmit passed. scripts/fast.ts exited 2 because scripts/gate.ts changed, so I picked the tests by hand: test/test-honesty.test.ts, test/gate-*.test.ts, test/check-list.test.ts and test/look-count.test.ts all pass (105 tests).
🤖 Generated with Claude Code
The gate now checks each test file for fixed waits and compares the count to an allowance file. A file with more than its allowance fails with "await the operation or an event it emits instead of a fixed delay". A file with fewer also fails until its allowance is lowered, so the list only shrinks. Because
forge.json'srun,fastandfullchecks all start withbun scripts/gate.ts, this runs on pull checks and on main.src/kernel/test_honesty.tstakes test sources and returns counts. Per file, it counts four things in every.tsundertest/:await Bun.sleep(<number>)setTimeout(<name>, <number>)mock.module(retry:/timeout:options, on their own line or as}, { retry: N }A wait with a name or computed value, like
Bun.sleep(ms), is not counted. The allowance file is checked when read: a malformed one is rejected with a message, not trusted.scripts/gate.tsgainstestHonestyRefusal, which is added togateVerdict's refusals, andwriteTestHonesty.FORGE_SLEEP_WRITE=1 bun scripts/gate.tsrewrites the allowance to today's counts. A tree without the allowance file, such as the gate's own test fixtures, is not checked.docs/test-honesty.jsonholds today's counts: 764 sleeps, 100setTimeoutwaits, 3mock.moduleand 7 retry/timeout options, across 461 files. The sleep andmock.modulecounts match the issue. The other two came out higher than the issue's 90 and 5 because these patterns are a bit wider; the allowance covers the difference.test/test-honesty.test.tscovers each kind of wait, a malformed allowance, and the repository's own tree matching its allowance. It also shows one newawait Bun.sleep(50)failing with the advice and a drop being refused until the allowance is lowered.Choices to look at:
src/kernel/test_honesty.ts, not thescripts/test_honesty.tsthe issue named. The gate's other rules already live insrc/kernel/and are loaded byrequire, sotreeGatecan still loadgate.tson its own.timeout:lines are matched by text, so the count includes some flagged lines that are not test timeouts (for exampletimeout: 600in check config fixtures). Counting every trailing}, 30_000)timeout argument would have matched about 2,100 lines, so I left that pattern out.no-commentsandno-explicit-return-typesare out of scope, as the issue says.keep_planorestimate; neither tool was available in this run.To try it: add an
await Bun.sleep(50);to any test file and runbun scripts/gate.ts(it exits 2 with the message), then remove it.Checks:
bun scripts/gate.ts --fixchanged nothing and passed, andbun x tsc --noEmitpassed.scripts/fast.tsexited 2 becausescripts/gate.tschanged, so I picked the tests by hand:test/test-honesty.test.ts,test/gate-*.test.ts,test/check-list.test.tsandtest/look-count.test.tsall pass (105 tests).🤖 Generated with Claude Code
Closes #14321