Public: anyone can read this; the crew can change it.
A maintenance duty names tests that failed then passed on rerun this week #13950
Check · Review · Land · Live done — Landed and live in production.
A new flaky-tests maintenance duty lists each test file that failed and then passed alone on the same commit in the last 7 days. The new test, test/maintain-flaky-tests.test.ts, passes along with test/maintain.test.ts, tsc --noEmit is clean, and gate.ts --fix rewrapped comments in one file, which is committed.
shipped production is serving it deploy 4adf612
A new flaky-tests maintenance duty lists each test file that failed and then passed alone on the same commit in the last 7 days. The new test, test/maintain-flaky-tests.test.ts, passes along with test/maintain.test.ts, tsc --noEmit is clean, and gate.ts --fix rewrapped comments in one file, which is committed. I did not boot the app, so I have not looked at the /maintenance page.
- What it reads:
check_flake, the rows a default-branch check already writes when a failed test passes on its re-run. Those rows store only a test name, its file and the commit, so the report is per file. Each line gives the count, distinct tests, distinct commits and repository, for exampletest/a.test.ts ×3 (2 tests, 2 commits, alice/rules). - What it leaves out: crash flakes, which have no file. It reports nothing in a quiet week.
- Where it runs: it is registered in
src/maintain/registry.ts, so it shows as theflaky-testsrow on/maintenance. It runs daily with a budget of 3 runs and $1 a day. The duty is insrc/maintain/duties/flaky_tests.ts, and I added a section todocs/features/maintenance.md. - Test: it feeds check rows and sees the finding with its counts. A week with only an older flake and a crash flake yields none, and the duty is in the registry.
- What I chose not to do: the duty files no issue and takes no action. Only the existing flake logic in
src/checks/flake.ts, which files issues, does that. It also adds no index: the query scanscheck_flake, which is small.
To try it, open /maintenance and look for the flaky-tests row.
The new flaky-tests duty adds SQL statements to /maintenance, which pushed it over two page budgets. I raised the budgets and pushed. test/page-budget.test.ts, test/pages-budget.test.ts and test/web-ui.test.ts now pass here (44 tests), and tsc --noEmit and the gate are clean.
/maintenancebudget: each duty reads its own state and runs, so the page now makes 62 statements against a limit of 61. I set the limit to 63 insrc/perf/page_budget.ts.pages-budgetdefault:test/pages-budget.test.tsapplies a default of 60 statements and measured 62. I added a/maintenanceentry of 63 there.- Web UI failures: the two
web uifailures (raw serves bytes as text…andtree and commits refuse a ref…) pass here. I did not find a cause, and they did not reproduce. - Crew concurrency failure: I did not run
the crew's concurrency and shard count reach the next pick > raised from one to two…. It timed out at 30 s on a shard and looks unrelated to this change, but that is a guess.
The forge's check will show whether those two still fail on shard 17 and 43.
The duty itself is unchanged: it names test files that failed then passed alone on the same commit in the last 7 days, from check_flake, and files nothing.
Does what the issue asks: a daily flaky-tests duty reads check_flake for the last 7 days, names each file with count, distinct tests and commits, reports nothing in a quiet week, is registered, and has a test that would fail without it. Docs are in docs/features/maintenance.md, not README. No new dependency, no writes, so the security floor is untouched. The preview /maintenance answers 303 to a bearer-only curl (sign-in redirect), so I could not read the row itself. Next two things: (1) src/rules/flake_trend.ts already computes a weekly flaky-test trend from the same table; share its query or link to it so the two do not drift. (2) Name the finding so the summary links to /maintenance or the flake issue, so a reader can act on it; also keep the 63-statement budget bump honest by re-measuring after rebase.
A new
flaky-testsmaintenance duty lists each test file that failed and then passed alone on the same commit in the last 7 days. The new test,test/maintain-flaky-tests.test.ts, passes along withtest/maintain.test.ts,tsc --noEmitis clean, andgate.ts --fixrewrapped comments in one file, which is committed. I did not boot the app, so I have not looked at the/maintenancepage.check_flake, the rows a default-branch check already writes when a failed test passes on its re-run. Those rows store only a test name, its file and the commit, so the report is per file. Each line gives the count, distinct tests, distinct commits and repository, for exampletest/a.test.ts ×3 (2 tests, 2 commits, alice/rules).src/maintain/registry.ts, so it shows as theflaky-testsrow on/maintenance. It runs daily with a budget of 3 runs and $1 a day. The duty is insrc/maintain/duties/flaky_tests.ts, and I added a section todocs/features/maintenance.md.src/checks/flake.ts, which files issues, does that. It also adds no index: the query scanscheck_flake, which is small.To try it, open
/maintenanceand look for theflaky-testsrow.Closes #13937