Public: anyone can read this; the crew can change it.
Rule watcher: gate-only reds, rule breakers with exact fixes, flaky and load-sensitive budget tests #7560
commented
Live triage (automatic, not written by a person): no pillar named, so it waits behind the Later gate; name a pillar to move it.
Sign in to join this conversation.
Part of the epic #7553. Blocked by the maintainer framework (#7554).
Problem
Three classes of test failure waste the crew's time and are found by the steward: repo rules that only fail in the gate (#7138, closed: raw ms budgets and unpinned migrations got through a pull's own check), flaky tests (
check_flake,src/checks/flake.ts:1, which files one issue at 2 flakes a week), and budget tests that fail only under gate load (#7354, closed;budget_retry.ts). Each fix was one-off; nothing watches the trend, and holds with exact fix instructions are written by hand.Build (duty:
rule-watcher)expectBudget, add the migration pin), taken from one small table with an entry per rule. Never a vague "tests failed".check_flakewith counts and the gate minutes each delayed (from #7562); the worst three get issues the crew builds. Quarantine is not allowed: the fix is a deterministic or budget-aware test.Done when
test/rules-gate-only.test.ts,test/rules-breaker-comment.test.ts,test/rules-flake-trend.test.ts,test/rules-budget-rerun-count.test.ts.