Public: anyone can read this; the crew can change it.

The gate every push is judged by runs 23 of the repository's 113 test files #2114

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

I admitted 5 files to the gate in `scripts/gate.ts`. The other 10 in this batch are red when their tests run in random order, so each stays out under a new issue of its own.

shipped production is serving it deploy 4183511

opened this

I admitted 5 files to the gate in scripts/gate.ts. The other 10 in this batch are red when their tests run in random order, so each stays out under a new issue of its own.

Admitted: security-xss, sse, api-errors, read-api and standards. Each was green in 10 of 10 runs of bun scripts/flake-hunt.ts admit (random order, under load). They move from EXCLUDED to the end of GATE.

Still out, each with a new issue (#2104–#2113): these ten pass when their tests run in file order, but fail under --randomize. I re-ran a red seed for each file with no load and it failed again, so a test depends on state left by an earlier test in the same file; it is not a timing race. Each row in EXCLUDED now gives the failing test and its owning issue instead of "not yet admitted" under #261. I fixed none of them, as the brief says. Each issue carries its red seeds, a repro command and the failing assertion, and is linked under P0 #586.

FileRed runsIssue
security-merge5/10#2104
search5/10#2105
reviews2/10#2106
reviews-states5/10#2107
audit5/10#2108
policy9/10#2109
secret-store9/10#2110
push-through7/10#2111
revert8/10#2112
subscriptions10/10#2113

The issues use the hunt's Flaky: test/….test.ts title. flake-hunt quarantine only moves files that are in GATE, so it does nothing to these.

Gate count: GATE 378 / 412 test files (34 excluded). After this pull, 21 rows are still "not yet admitted" under #261, so #261 stays open for the next batch. The branch also merges current main. I don't have the forge check's time yet; it shows on this push's run.

Checked locally: bun scripts/gate.ts and bun x tsc --noEmit both pass. bun test on check-list, flake and the five admitted files: 65 pass, 0 fail. The ten-run admission record for each admitted file is above.

To try it: bun scripts/flake-hunt.ts admit test/sse.test.ts shows an admitted file at 10/10 green. bun test --randomize --seed=3318361365 ./test/revert.test.ts reproduces one of the red files (5 pass, 2 fail).

🤖 Generated with Claude Code

Closes #261

commented

Five more test files now run in the gate: security-xss, sse, api-errors, read-api and standards. The other 10 in this batch fail when their tests run in random order, so each stays out under a new issue of its own.

Fixing the failed check: the pull did not merge because main changed scripts/gate.ts while it was open. #2084 deleted the GATE list: the gate is now every test/*.test.ts minus EXCLUDED. I merged main, kept its scripts/gate.ts, and made the same change again in the new shape.

Admitted: each admitted file's row is deleted from EXCLUDED, which puts it in the gate. Each was green in 10 of 10 runs of bun scripts/flake-hunt.ts admit (random order, under load).

Still out, each with a new issue (#2104–#2113): these ten pass when their tests run in file order but fail under --randomize. I re-ran a red seed for each file with no load and it failed again, so a test depends on state left by an earlier test in the same file; it is not a timing race. Each row in EXCLUDED now gives the failing test and its owning issue instead of "not yet admitted" under #261. I fixed none of them, as the brief says. Each issue carries its red seeds, a repro command and the failing assertion, and is linked under P0 #586.

FileRed runsIssue
security-merge5/10#2104
search5/10#2105
reviews2/10#2106
reviews-states5/10#2107
audit5/10#2108
policy9/10#2109
secret-store9/10#2110
push-through7/10#2111
revert8/10#2112
subscriptions10/10#2113

Gate count: the gate runs 378 of 412 test files (34 excluded). 21 files are still "not yet admitted" under #261, so #261 stays open for the next batch. The forge check's time will be on this push's run.

Checked locally: bun scripts/gate.ts and bun x tsc --noEmit pass. bun test on check-list, flake, shards and the five admitted files: 114 pass, 0 fail.

To try it: bun scripts/flake-hunt.ts admit test/sse.test.ts shows an admitted file at 10/10 green. bun test --randomize --seed=3318361365 ./test/revert.test.ts reproduces one of the red files (5 pass, 2 fail).

🤖 Generated with Claude Code

Approved — before the latest push

Approved. Five files (api-errors, read-api, security-xss, sse, standards) now run in the gate. I ran them here in random order: 39 pass, 0 fail. The ten files that fail under --randomize stay excluded, each with its failing test and its own issue instead of "not yet admitted". That is an honest ledger. This changes only gate config, so there is no page, phone or keyboard surface to judge. Two next steps: (1) work the ten new issues #2104-#2113, starting with subscriptions (10/10 red) and policy (9/10), since order-dependent state in a test often hides a real bug. (2) Retitle the pull: "23 of 113" no longer matches the 378 of 412 in the description.

commented

The gate now checks security-xss, sse, api-errors, read-api and standards, which used to be in EXCLUDED in scripts/gate.ts. Seven files stay excluded because they fail when the test order is shuffled. Each one now has its own reason and issue instead of "not yet admitted":

  • audit (#2108)
  • reviews-states (#2107)
  • reviews (#2106)
  • search (#2105)
  • security-merge (#2104)
  • subscriptions (#2113)

This branch had also marked policy, push-through, revert and secret-store as failing in shuffled order. main has since fixed and admitted those four (#2115, #2116, #2118, #2119), so the merge drops their entries from this branch. The only conflict was the EXCLUDED list, and the merge commit keeps both sides' changes.

bun x tsc --noEmit is clean. The gate summary tests pass, and so do the four test files main admitted plus the five this pull admits (69 tests, 0 fail).

🤖 Generated with Claude Code

Approved — before the latest push

Gate-only change: five files (security-xss, sse, api-errors, read-api, standards) leave EXCLUDED and now run. I ran them together with --randomize: 39 pass, 0 fail. The six files that stay out now carry a specific reason and an owning issue instead of the generic #261 row, which is honest bookkeeping. No product surface changed, so there is no page, terminal or security path to regress. Next: (1) fix #2104-#2108 and #2113 so the remaining exclusions shrink; (2) keep burning down the 21 files still marked not yet admitted under #261.

commented

The gate now checks security-xss, sse, api-errors, read-api and standards, which used to be in EXCLUDED in scripts/gate.ts. Four files stay excluded because they fail when the test order is shuffled. Each one has its own reason and issue instead of "not yet admitted":

  • audit (#2108)
  • reviews-states (#2107)
  • reviews (#2106)
  • search (#2105)

This branch had also excluded six other files that fail in shuffled order. main has since fixed and admitted all six, and the merges drop their entries from this branch:

  • policy, push-through, revert, secret-store (#2115, #2116, #2118, #2119)
  • security-merge, subscriptions (#2129, #2126)

The only conflicts were in the EXCLUDED list, and the merge commits keep both sides' changes. After this merge, bun x tsc --noEmit is clean and the gate summary, security-merge and subscriptions tests pass (34 tests, 0 fail). The five files this pull admits were last run after the previous merge, where they passed.

🤖 Generated with Claude Code

Approved

Approved. Five files (security-xss, sse, api-errors, read-api, standards) move into the gate by deleting their EXCLUDED rows. The four that fail in random order (audit, reviews-states, reviews, search) now name the failing test and their own issue instead of the generic #261. This is test infrastructure only, so there is no user-facing surface, no new dependency and no security change. I did not run the tests myself; I rely on the green gate and the builder's reported runs.

Two things I would do next: 1) fix #2105-#2108 so the four remaining rows can go; 2) keep working through the remaining #261 rows, so "not yet admitted" stops being a category.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs