Public: anyone can read this; the crew can change it.

Automate a review finding as a lint or test: Two things to fix, first one first. #9270

Parked opened this
Closed without merging

This adds a check for one of the repeated review findings, a fixed sleep in a test. It doesn't cover the other findings in those reviews, and the issue's four comments don't point to a single rule. They are all the opening line of a two-finding review, and the four pulls' actual findings differ.

Notes

stopped parked by the crew deploy 1b17e47

opened this

This adds a check for one of the repeated review findings, a fixed sleep in a test. It doesn't cover the other findings in those reviews, and the issue's four comments don't point to a single rule. They are all the opening line of a two-finding review, and the four pulls' actual findings differ. I picked the one that can be checked mechanically: #7080 asked to poll with a deadline instead of "Fixed 300 ms sleep is a flake source on a loaded runner". The new test and check-list.test.ts pass, and tsc --noEmit and scripts/gate.ts are clean. I did not run the full suite.

What changed

  • scripts/review-rules.ts: rule fixed-sleep. A test file may not call sleep or Bun.sleep with a literal of 250 ms or more. The failure reads review rule fixed-sleep: test/x.test.ts:12 sleeps a fixed 250 ms or more (…); poll for the condition with a deadline instead (#7080).
  • Existing offenders: 48 test files already sleep that long, so they are pinned in BASELINE at their current count. The count may fall but not rise, so a new fixed sleep fails in the pull that adds it.
  • test/review-rules.test.ts: the tests are in the gate automatically. They check:
    • the tree has no sleeps beyond the baseline;
    • a planted sleep is named with the rule and its line;
    • short sleeps and underscore literals are handled;
    • every baseline entry names a file that exists.

What I left out

  • Other findings: #7547 and #8101 asked for the same wording or rule to be shared between two places (the terminal and web screens, keyName and addKey). #9253's comment is an automated note about an unticked item. None of these can be checked without many false hits.
  • Wall-clock margins: I did not add a rule for asserting a tight wall-clock margin, the other half of #7080's feedback.
  • Non-literal sleeps: sleeps built from a variable or a setTimeout are not caught.
  • Baseline burn-down: I did not clean up the existing 48 files.

To try it: bun test test/review-rules.test.ts. Add await Bun.sleep(300) to any test file and the same command fails and names the rule.

🤖 Generated with Claude Code

Closes #9268

3 earlier comments
commented

This adds a check for one of the repeated review findings, a fixed sleep in a test. It doesn't cover the other findings in those reviews, and the issue's four comments don't point to a single rule. They are all the opening line of a two-finding review, and the four pulls' actual findings differ. I picked the one that can be checked mechanically: #7080 asked to poll with a deadline instead of "Fixed 300 ms sleep is a flake source on a loaded runner". The new test and check-list.test.ts pass, and tsc --noEmit and scripts/gate.ts are clean. I did not run the full suite.

What changed

  • scripts/review-rules.ts: rule fixed-sleep. A test file may not call sleep or Bun.sleep with a literal of 250 ms or more. The failure reads review rule fixed-sleep: test/x.test.ts:12 sleeps a fixed 250 ms or more (…); poll for the condition with a deadline instead (#7080).
  • Existing offenders: 48 test files already sleep that long, so they are pinned in BASELINE at their current count. The count may fall but not rise, so a new fixed sleep fails in the pull that adds it.
  • test/review-rules.test.ts: the tests are in the gate automatically. They check:
    • the tree has no sleeps beyond the baseline;
    • a planted sleep is named with the rule and its line;
    • short sleeps and underscore literals are handled;
    • every baseline entry names a file that exists.

What I left out

  • Other findings: #7547 and #8101 asked for the same wording or rule to be shared between two places (the terminal and web screens, keyName and addKey). #9253's comment is an automated note about an unticked item. None of these can be checked without many false hits.
  • Wall-clock margins: I did not add a rule for asserting a tight wall-clock margin, the other half of #7080's feedback.
  • Non-literal sleeps: sleeps built from a variable or a setTimeout are not caught.
  • Baseline burn-down: I did not clean up the existing 48 files.

To try it: bun test test/review-rules.test.ts. Add await Bun.sleep(300) to any test file and the same command fails and names the rule.

🤖 Generated with Claude Code

commented

I fixed the two hollow-check failures and pushed. I did not fix the other two failures from the check.

What I fixed: scripts/hollow.ts was red on main as well as on this pull. scripts/hollow-allow.ts still listed viewBuilds and forgetFoundryDoor as unreached, but main now reaches both. I removed those two entries. That clears both hollow failures (bun scripts/hollow.ts says so and exits 0 and carries no hollow work). test/hollow.test.ts, test/hollow-cli.test.ts, tsc --noEmit and scripts/gate.ts pass after the change.

What I left alone:

  • "the numbers add up" in test/returns.test.ts: it passes when I run it here, so I don't have a cause for it. My change doesn't touch it.
  • "second look at home as the owner costs a bounded number of statements": I did not run it here. My change doesn't touch it.

I'd expect both to be timing-sensitive and unrelated to this pull. If the next check still fails them, the log will show whether that's so.

commented

I fixed the one failure I could reproduce and pushed. I did not fix the other two.

  • every module under src/ is imported by another, or allowed: src/arcade/cover.ts came in on main with #9283, and nothing under src/ imports it. I added it to scripts/unreached-allow.json. That is a stopgap: cover.ts still needs a real caller, which is outside this issue. test/unreached.test.ts, tsc --noEmit and scripts/gate.ts pass.
  • "another forge's live runs > the wait says so once…" and "running-forge attacks > hostile ?page values…": I couldn't reproduce either. test/loop-stall.test.ts and test/hardening.test.ts pass here (the hardening run also skipped 2 tests), and nothing in this pull touches that code. They look like load flakes in the full run. If they fail again on this push, the log should show whether they're timing.
commented

Parked by the crew: the check stayed red through 2 automatic tries

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs