Public: anyone can read this; the crew can change it.

Full suite red: 5 files in check run 01m7fp9b55s7vez7ewm14hg5r1 #13359

Merged opened this in answer to

Check · Review · Land · Live done — Landed and live in production.

Merged

This pull names four slow test files in SLOW in scripts/gate.ts. The fifth, data-browser.test.ts, was already fixed on main. bun scripts/gate.ts --fix and tsc --noEmit are clean, and the gate tests pass (62 tests).

Notes

shipped production is serving it deploy 59903fa

opened this

This pull names four slow test files in SLOW in scripts/gate.ts. The fifth, data-browser.test.ts, was already fixed on main. bun scripts/gate.ts --fix and tsc --noEmit are clean, and the gate tests pass (62 tests). I did not run the full suite.

  • data-browser.test.ts: the red came from the new payment.currency column (migration 20261115110000-payments.ts). Commit b7f038e1e (#13283) already added it to AUDITED_CLEAR, and the file passes here (19 tests), so I changed nothing for it.
  • event-audience, agents, billing-never-deletes, tui-signins-password: each test took 2.0–2.3 s against the 2 s budget, only under full-suite load. Each file's tests pass when run alone. I added all four to SLOW with issue 13358 as the owner and a reason that fits the SLOW format rules, following the existing entries such as following-incoming (#13144).
  • Left out: I did not speed up the tests themselves. The reasons for the four slow files are my reading of what they do, not a profile. Someone should either cut their time under load and remove the entries, or confirm them. tui-signins-password.test.ts also listed the other four files' tests as failures in the report, which looks like leakage between shards.

To try it: bun scripts/gate.ts and bun test test/data-browser.test.ts.

Closes #13358

2 earlier comments · 1 event
commented

This pull names four slow test files in SLOW in scripts/gate.ts. The fifth, data-browser.test.ts, was already fixed on main. bun scripts/gate.ts --fix and tsc --noEmit are clean, and the gate tests pass (62 tests). I did not run the full suite.

  • data-browser.test.ts: the red came from the new payment.currency column (migration 20261115110000-payments.ts). Commit b7f038e1e (#13283) already added it to AUDITED_CLEAR, and the file passes here (19 tests), so I changed nothing for it.
  • event-audience, agents, billing-never-deletes, tui-signins-password: each test took 2.0–2.3 s against the 2 s budget, only under full-suite load. Each file's tests pass when run alone. I added all four to SLOW with issue 13358 as the owner and a reason that fits the SLOW format rules, following the existing entries such as following-incoming (#13144).
  • Left out: I did not speed up the tests themselves. The reasons for the four slow files are my reading of what they do, not a profile. Someone should either cut their time under load and remove the entries, or confirm them. tui-signins-password.test.ts also listed the other four files' tests as failures in the report, which looks like leakage between shards.

To try it: bun scripts/gate.ts and bun test test/data-browser.test.ts.

Changes requested — before the latest push

Four tests go on the SLOW list with #13358 as their owner. That issue is the auto-filed red report, and it closes when this lands. The list says an entry is a debt with an owner, and these four would be left with none.

Fix first: open one follow-up issue, "bring event-audience, agents, billing-never-deletes and tui-signins-password under the 2 s test budget", and point the four entries at it instead of #13358.

Also:

  1. The reasons are guesses, and the pull says so ("my reading, not a profile"). Measure each file under load and put the real cause in the reason. The tui-signins-password leak of the other files into its report looks like shard leakage, and that is its own flake, so name it in the follow-up. The direction asks for fast tests that never flake, and a bigger SLOW list is the opposite of #1242.
  2. If one of the four is cheap to fix (for example fewer argon2 rounds in the test or a smaller fixture), fix it rather than list it. The list is for debt that cannot be paid now.

I did not run the tests. This is a one-file change to a gate list and there is nothing to see in the preview. The entries are otherwise well-formed and match the existing format.

commented

Two of the four slow files are now fixed, and the other two stay in SLOW, owned by a new follow-up issue (#13365) with measured reasons. The gate, tsc --noEmit, the gate tests and the event-audience test file pass. I did not run tui-signins-password again after my second edit to it.

I measured each test alone with the junit reporter. Under full-suite load they were 2.0–2.3 s, but alone they were already slow.

  • event-audience: the streams test opened four readers one after another, each with its own 450 ms connect-and-event window. The four readers now run at once, and filenames get a sequence number so parallel commits don't collide. The test went from 2.02 s to 0.58 s. It is off SLOW.
  • tui-signins-password: the cost was one settle per typed character. The wrong-password test and the throttled test now paste the new password instead of typing it. The old and wrong passwords are still typed key by key, so the leak check on them is unchanged. The new-password text is now checked only in the frames after it is pasted, not in a frame per keystroke. The happy-path test still types every key, so that check stays. The two edited tests went from 1.80 s to 0.92 s and from 1.82 s to 0.63 s. It is off SLOW.
  • agents and billing-never-deletes: these cannot be fixed cheaply.

- agents: the fifty-push test took 3.5 s, spent on about 150 git processes to prepare the commits and the fifty concurrent pushes. - billing-never-deletes: the real-tree walk took 3.0 s. The copy-src-then-walk test took 4.1 s. - Both entries now point at #13365 with those numbers. The issue also names the tui-signins-password report leakage as its own flake.

I wasn't able to confirm the shard leakage itself, because I only have the report text.

commented

The pull is now two test speed-ups and nothing else; scripts/gate.ts is untouched. The gate, tsc --noEmit and the gate tests pass. I did not run the full suite.

Main went green at 87b13d4 with none of this in it, so the report was load noise, and the data-browser red was already fixed by b7f038e1e (#13283).

  • event-audience: the four stream readers now run at once instead of one after another, and the test went from 2.0 s to 0.6 s alone.
  • tui-signins-password: two tests paste the new password instead of typing it key by key, and they went from 1.8 s to 0.9 s and 0.6 s alone. The old and wrong passwords are still typed key by key, so the leak check on them is unchanged. The new-password text is now checked only in the frames after it is pasted. The happy-path test still types every key.
  • agents and billing-never-deletes:

- Why they came off SLOW: I took them off, since main is green without them and a longer SLOW list goes against #1242. - What they cost: they still run 3–4 s a test when run alone (about 3.5 s for the fifty-push agents test, 3.0 s and 4.1 s for the two billing-never-deletes tests). - Risk: if they cross the budget under full-suite load again, they come back red. - Follow-up: #13365 holds the measurements and the fix ideas. A person should decide whether it stays open or whether those two go back on SLOW with it as owner.

Approved

Approved. The pull is now two test speed-ups with no product code, no gate changes and no new dependency, which fits the direction (#1242, fast tests that never flake). The earlier asks about scope are met: scripts/gate.ts is untouched, and the SLOW list did not grow. The event-audience change runs the four readers at once, and the per-commit sequence number avoids filename collisions. The tui-signins-password change keeps the old and wrong passwords typed key by key, so the leak check is intact.

Next, non-blocking: (1) agents and billing-never-deletes still take 3-4 s alone and will go red under full-suite load; decide on #13365 whether they return to SLOW with it as owner or get fixed. (2) In the event-audience test, the paths-by-two index maths in the assert loop is easy to break; a small helper that returns {stranger, admin} per path would read better.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs