Public: anyone can read this; the crew can change it.

A red shared by many pulls never parks them: one test failing on 3+ pulls in an hour is the lane's, not theirs #2270

Merged opened this in answer to

Check · Review · Land · Live done — Landed and live in production.

Merged

When three or more pulls of a repository fail the same test within an hour, a pull's red on that test now counts as the lane's, not the pull's. It isn't parked, struck or handed back. It waits for main's next green check and is then checked again, like a pull whose red main shares (#1971). The vitals show one card per shared test.

shipped production is serving it deploy 09671ca

opened this

When three or more pulls of a repository fail the same test within an hour, a pull's red on that test now counts as the lane's, not the pull's. It isn't parked, struck or handed back. It waits for main's next green check and is then checked again, like a pull whose red main shares (#1971). The vitals show one card per shared test.

What changed

  • src/checks/shared_red.ts (new): counts how many different pulls failed each test in the last hour, in one query using SQLite's json_each on check_detail.

- sharedWithPulls follows the same rule as sharedWithMain: the check must name at least one test, and every test it names must have failed on 3+ pulls. A pull that also fails a test of its own (["X","Y"]) is still the pull's red. - Main's checks and other repositories don't count, and the same pull failing three times counts once.

  • src/kernel/id.ts: new idFloor(ms). Run ids sort by time, so the one-hour window is a range on the run_repo index rather than a table scan.
  • src/crew/dispatch.ts: waitOnMain now also accepts a red shared across pulls. It already runs before the park limit and before the decider, so neither can park or rework such a pull. The pull goes into the existing waiting map, which recheckWaiting re-checks on main's next green.
  • src/kernel/vitals.ts: one card per shared test, with cause shared:<repoId>:<hash>. It links to the sentinel's open flake issue for that test if there is one, and otherwise to the repository's /issues/new to file one.
  • src/crew/sentinel.ts: shared: is now a breach cause, so the sentinel files one flake issue per shared test, with its own fix text.

Where to look

  • The 3-pull threshold includes the pull being judged ("one test failing on 3+ pulls"). So in a storm the first two reds are still each pull's own, as before. From the third red on, every pull red only on that test waits, including the first two at their next red, so none reaches the park.
  • test/vitals.test.ts: the test pins the total number of queries vitals makes. It goes from 7 to 8 because of the one new query, and the count still doesn't grow with runs. The test wasn't wrong; the change adds a query on purpose.

Not done

  • Pulls parked before this lands aren't reopened.
  • The waiting list is in memory only, as with the red-main wait, so a restart forgets it.

Tests

  • test/crew-shared-red.test.ts, through the crew's event path with every pull's automatic tries already used up: 3 pulls red on X stay open and give one card, a 4th red on Y only is parked, a pull red on X and Y is parked, and main's green re-checks the three once. This test fails on the old dispatch.ts.
  • test/shared-red.test.ts covers the threshold, the one-hour window, other repositories, unreadable check details, the card link (file it / follow it / file again once closed), the sentinel key, and idFloor.
  • The type check is clean. The related suites pass, except crew-interrupted's "young lock" test, which fails the same way on base in this sandbox.

🤖 Generated with Claude Code

Closes #2266

commented

When three or more pulls of a repository fail the same test within an hour, a pull's red on that test now counts as the lane's, not the pull's. It isn't parked, struck or handed back. It waits for main's next green check and is then checked again, like a pull whose red main shares (#1971). The vitals show one card per shared test.

What changed

  • src/checks/shared_red.ts (new): counts how many different pulls failed each test in the last hour, in one query using SQLite's json_each on check_detail.

- sharedWithPulls follows the same rule as sharedWithMain: the check must name at least one test, and every test it names must have failed on 3+ pulls. A pull that also fails a test of its own (["X","Y"]) is still the pull's red. - Main's checks and other repositories don't count, and the same pull failing three times counts once.

  • src/kernel/id.ts: new idFloor(ms). Run ids sort by time, so the one-hour window is a range on the run_repo index rather than a table scan.
  • src/crew/dispatch.ts: waitOnMain now also accepts a red shared across pulls. It already runs before the park limit and before the decider, so neither can park or rework such a pull. The pull goes into the existing waiting map, which recheckWaiting re-checks on main's next green.
  • src/kernel/vitals.ts: one card per shared test, with cause shared:<repoId>:<hash>. It links to the sentinel's open flake issue for that test if there is one, and otherwise to the repository's /issues/new to file one.
  • src/crew/sentinel.ts: shared: is now a breach cause, so the sentinel files one flake issue per shared test, with its own fix text.

Where to look

  • The 3-pull threshold includes the pull being judged ("one test failing on 3+ pulls"). So in a storm the first two reds are still each pull's own, as before. From the third red on, every pull red only on that test waits, including the first two at their next red, so none reaches the park.
  • test/vitals.test.ts: the test pins the total number of queries vitals makes. It goes from 7 to 8 because of the one new query, and the count still doesn't grow with runs. The test wasn't wrong; the change adds a query on purpose.

Not done

  • Pulls parked before this lands aren't reopened.
  • The waiting list is in memory only, as with the red-main wait, so a restart forgets it.

Tests

  • test/crew-shared-red.test.ts, through the crew's event path with every pull's automatic tries already used up: 3 pulls red on X stay open and give one card, a 4th red on Y only is parked, a pull red on X and Y is parked, and main's green re-checks the three once. This test fails on the old dispatch.ts.
  • test/shared-red.test.ts covers the threshold, the one-hour window, other repositories, unreadable check details, the card link (file it / follow it / file again once closed), the sentinel key, and idFloor.
  • The type check is clean. The related suites pass, except crew-interrupted's "young lock" test, which fails the same way on base in this sandbox.

🤖 Generated with Claude Code

Approved

Reads right against the issue: 3+ distinct pulls red on the same test in an hour make that red the lane's. The pull waits through the existing main-red path instead of being parked or struck. The vitals show one card per shared test and the sentinel files one flake issue. The query is one range scan on run_repo via idFloor, with every value a parameter and the SQL a constant. Tests cover the threshold, the window, other repos, bad JSON, the card link, the sentinel key and the crew path, and the vitals query count change (7 to 8) is explained. I did not run the tests or open the preview or screenshots; this is a read of the diff. Two things I would do next: (1) the card is reached through vitals, so confirm the terminal vitals view prints the new card and its href the same as the page does; (2) persist the waiting list, or make the card say that a restart forgets the waiting pulls, so a restart does not strand them silently.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs