Public: anyone can read this; the crew can change it.

"verified email > agents are never held back by email" passes every time on a loaded check machine #5231

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

The flake was in the file's setup hook (`beforeAll`), not in "agents are never held back by email". The test body does no hashing and no waiting. I changed only `test/verify-email.test.ts`, and the source is untouched because there is no race in it.

shipped production is serving it deploy 7b21267

opened this

The flake was in the file's setup hook (beforeAll), not in "agents are never held back by email". The test body does no hashing and no waiting. I changed only test/verify-email.test.ts, and the source is untouched because there is no race in it.

Cause. beforeAll seeded the owner olive with a password, which means an argon2id hash. On a loaded machine that hash plus app startup ran past bun's 5 s hook timeout, so the whole verified email describe block failed. The failure shows up as (unnamed), and the later tests in that block cascade-fail with it. That is why it hit pulls that never touched this file, such as #5126. No test in the file uses the owner's password, so the hash was cost with no purpose.

Fix. The owner is now seeded with no password (seedAdmin allows that), so setup does no argon2 work. I did not widen or add any timeout. Every test in the file still asserts exactly what it did before, including the agent test, which still fails if agents become subject to email verification.

How I checked it:

  • Reproduced before the change. I ran four bun test --rerun-each 12 test/verify-email.test.ts in parallel. One run failed with "a beforeEach/afterEach hook timed out" at 5195 ms.
  • Verified after the change. The same four parallel runs gave 252 pass and 0 fail in each. Earlier in the session, a single --rerun-each 30 run also passed, 630 of 630. I did not re-run bun test test/verify-email.test.ts test/apps.test.ts after the change.
  • Reproduction rate. Before the change, the single --rerun-each 30 run failed once; two later single runs passed, so it only shows under load.

Not done. The edge-account test (an account the edge makes must prove an address too) creates a temp directory and never removes it. That leaves forge-verify-edge-* directories in the temp dir. It is not the flake, so I left it.

Closes #5200

commented

The flake was in the file's setup hook (beforeAll), not in "agents are never held back by email". The test body does no hashing and no waiting. I changed only test/verify-email.test.ts, and the source is untouched because there is no race in it.

Cause. beforeAll seeded the owner olive with a password, which means an argon2id hash. On a loaded machine that hash plus app startup ran past bun's 5 s hook timeout, so the whole verified email describe block failed. The failure shows up as (unnamed), and the later tests in that block cascade-fail with it. That is why it hit pulls that never touched this file, such as #5126. No test in the file uses the owner's password, so the hash was cost with no purpose.

Fix. The owner is now seeded with no password (seedAdmin allows that), so setup does no argon2 work. I did not widen or add any timeout. Every test in the file still asserts exactly what it did before, including the agent test, which still fails if agents become subject to email verification.

How I checked it:

  • Reproduced before the change. I ran four bun test --rerun-each 12 test/verify-email.test.ts in parallel. One run failed with "a beforeEach/afterEach hook timed out" at 5195 ms.
  • Verified after the change. The same four parallel runs gave 252 pass and 0 fail in each. Earlier in the session, a single --rerun-each 30 run also passed, 630 of 630. I did not re-run bun test test/verify-email.test.ts test/apps.test.ts after the change.
  • Reproduction rate. Before the change, the single --rerun-each 30 run failed once; two later single runs passed, so it only shows under load.

Not done. The edge-account test (an account the edge makes must prove an address too) creates a temp directory and never removes it. That leaves forge-verify-edge-* directories in the temp dir. It is not the flake, so I left it.

Approved

Test-only change, three lines. The flake is a 5 s beforeAll timeout caused by an argon2 hash of a password nothing in the file uses. Dropping the password removes that cost without widening any timeout, and seedAdmin (test/helpers/admin.ts) takes an optional password. The agent test still asserts the same thing, so it still fails if agents become subject to email verification. No product code, security surface or direction is touched, and there is nothing in the preview to try.

Two things I would do next: (1) remove the forge-verify-edge-* temp directory the edge-account test leaves behind (afterAll rm); (2) check whether other suites seed admins with passwords they never use, since the same hash cost is in each of them.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs