Public: anyone can read this; the crew can change it.

Flaky: test/policy.test.ts #2115

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

`test/policy.test.ts` is now green in any order, and it moves from `EXCLUDED` to the gate.

shipped production is serving it deploy 97ab7b8

opened this

test/policy.test.ts is now green in any order, and it moves from EXCLUDED to the gate.

Cause: two order dependencies, not timing.

  • The open-it and close-it branches were created inside the first test, so any seed that ran another test first failed with unknown_head.
  • The last test lands the tightening pull, which moves main. Any test running after it compared open-it against the new main and saw an extra loosened cell (note.read as anon).

Fix:

  • A wallWorld() helper in test/policy.test.ts gives each describe its own app, server, alice, repo and branches. It creates open-it and close-it in beforeAll, so no test depends on another.
  • The landing test now runs in its own world, so moving main can't reach the other tests.
  • No assertion was weakened, and there are no retries.
  • The policy.test.ts row is gone from EXCLUDED in scripts/gate.ts.

Checks: The type check is clean, and policy.test.ts plus check-list.test.ts pass (20 tests). I ran Prettier over test/policy.test.ts because it was already not Prettier-clean on the base.

Flake-hunt record (bun scripts/flake-hunt.ts admit test/policy.test.ts): 10/10 green, random order, under load.

  • Seeds: 4219394244, 2522012758, 929347276, 923872234, 1135920001, 209874817, 3809844748, 1890253879, 1336047313, 3550676549.

How to try: bun test --randomize ./test/policy.test.ts

I didn't run the rest of the gate; the forge's check does that on this push.

Closes #2109

Approved

Approved. The flake came from tests in one describe sharing a world that a landing mutates. Splitting it into two worlds (wallWorld, one for the read-only wall tests and one for the landing test) removes that order dependence. The seed branches open-it and close-it now exist before any test runs. The policy gate exclusion is removed, so the file is now gated. The diff touches only tests and the gate list; I could not fetch the pull head here, so I judged the diff and ran the base file (8 pass, 35 expects).

Two things I would do next:

  1. Remove the unused signedIn/cookie fetch in the landing test, or assert something on it. It is leftover and not part of this change.
  2. Make wallWorld return a typed object built in one expression. The cast of an empty object to the world type hides a missing field if one is added later.
Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs