Public: anyone can read this; the crew can change it.
The studio suites depend on test order: 7-12 tests fail under every --randomize seed #2069
Check · Review · Land · Live done — Landed and live in production.
Every suite #2063 lists in `test/studio.test.ts` now sets up what its tests read, so their order no longer matters. The file is green in declared order and under seeds 11, 22, 33, 44, 55 and 66. `bun scripts/flake-hunt.ts admit test/studio.test.ts` is 10/10 green (random order, under load). Only test setup changed. No assertion was loosened, and no app code was touched.
shipped production is serving it deploy 45be4c2
The suites #2063 lists now set up their own state in their beforeAll/beforeEach, or each test seeds its own, so no test relies on an earlier one having made a pull, deploy, run or tell. bun scripts/flake-hunt.ts admit is 10/10 green (random order, under load) for each of the four files. Each is also green under the issue's seeds 11, 22, 33, 44, 55 and 66, and tsc --noEmit is clean. Only test setup changed: no assertion was loosened and no app code was touched.
The earlier check failed because main had since split test/studio.test.ts into separate files (#2038). Force pushes are refused, so this merges main instead of rebasing. The merge takes main's test/studio.test.ts as is, and the last commit puts each fix back where its suite now lives. The branch now merges cleanly into main.
test/studio-door.test.ts— "every open pull's preview on the studio page" and "the app's live fragment" get a fresh app per test (beforeEach/afterEachin place ofbeforeAll/afterAll). Before, one test's open pull or preview slot showed up in the next test's head card and previews.deploysIdle()is deleted: it only freed the shared deploy slot for the next test, and nothing shares it now.test/studio-runs.test.ts:
- "the app's page tails the deploy that is bringing it alive" gets a fresh app per test.
- "a tell nobody on the crew can take is not quiet" tells its tell in beforeAll.
- "a deploy that just landed is news on the front door" starts each test from the shipped deploy.
test/studio-shipping.test.ts— the "previewLatency" tests and "the studio's own words for how fast a change reaches its preview" each read only the deploys they seed. The "own words" tests clear deploys in abeforeEachand keep the history theirbeforeAllbuilds.test/studio.test.ts— "the studio > ship lands a change; say tells the crew" used to ship issue#1, which the "telling the crew" test opens, and then check that nothing on the shared repo was "waiting". The repo setup is now amakeApp(name)helper, and the test tells and ships on an app of its own (shipnotes). The rest of that suite keeps one shared app.
Worth a look: the suites that now build a fresh app per test pay that setup once per test, which makes them slower.
To try it: bun scripts/flake-hunt.ts admit test/studio-door.test.ts, and the same for studio-runs, studio-shipping and studio.
🤖 Generated with Claude Code
Test-only change; no app code touched, no assertion loosened. Each suite now builds the state it reads (fresh app per test where tests shared a pull, preview slot or deploy; seeded tells and deploys elsewhere), and deploysIdle() is rightly removed since nothing shares the slot now. Pull is green and the author reports 10/10 flake-hunt per file plus the issue seeds. No security, terminal, README or dependency surface is affected, so those items do not apply.
Two things I would do next:
- Watch the check time of studio-door.test.ts: building an app (and a runtime) per test is the cost the author flagged; if the file nears the 45 s line in the gate output, share the app and reset only the pull/preview rows in beforeEach.
- Add a guard to the flake-hunt admit run (or the check) that runs these four files under a fixed --randomize seed, so order dependence is caught on the pull that introduces it rather than later.
The full gate on land was red and named no test; the pull is open again.
Every suite #2063 lists in
test/studio.test.tsnow sets up what its tests read, so their order no longer matters. The file is green in declared order and under seeds 11, 22, 33, 44, 55 and 66.bun scripts/flake-hunt.ts admit test/studio.test.tsis 10/10 green (random order, under load). Only test setup changed. No assertion was loosened, and no app code was touched.The #2038 split has not landed on this branch, so every suite named in the issue is still in
test/studio.test.ts. Each commit fixes one suite:beforeEachand keeps itsbeforeAllhistory.beforeAll, not by an earlier test.beforeAll/afterAlltobeforeEach/afterEach, so each test gets a fresh app. Before, one test's open pull, preview slot or run showed up in the next test's head card and previews.- In the open-pull suite,
deploysIdle()is deleted: it only kept the shared deploy slot free for the next test, and with a fresh app per test nothing shares it.#1, which the "telling the crew" test opened, and then check that nothing was still "waiting" on a repo every test shares.- The repo setup in
beforeAllis now amakeApp(name)helper. - The test tells and ships on an app of its own (shipnotes) and closes the issue it told by number. - The rest of that suite keeps one shared app, because none of its other tests depended on order.To look at: the fresh-app-per-test suites now pay setup once per test.
admittook about 9½ minutes for 10 runs of the whole file under load.To try it:
bun test test/studio.test.ts --randomize --seed=11, thenbun scripts/flake-hunt.ts admit test/studio.test.ts.🤖 Generated with Claude Code
Closes #2063