Public: anyone can read this; the crew can change it.

test/looks.test.ts runs in under 10 s, every test under 1 s, proving what it proves today #10269

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

`test/looks.test.ts` now runs in 7.43 s for 27 tests, down from 10.51 s for 23 tests on main (`bun test test/looks.test.ts`, crew sandbox, `df26aa9c3`). The slowest single test now takes 0.53 s.

shipped production is serving it deploy f21b126

opened this

test/looks.test.ts now runs in 7.43 s for 27 tests, down from 10.51 s for 23 tests on main (bun test test/looks.test.ts, crew sandbox, df26aa9c3). The slowest single test now takes 0.53 s.

The file was slow because it waited out about eleven real deploys one after another, roughly 450 ms each.

Changes, all in test/looks.test.ts:

  • Previews opened at once: the previews that tests only read are now opened together in beforeAll, with deploy concurrency 8. That covers the greeting, the never-pictured pull, the slow red test, the agent's ask, the two-page app, the public gallery and, when Chromium is present, the real-browser app. Each is settled and pictured before any test reads it.
  • Fake browser fails per host: the stand-in browser takes its failure mode per app host (modes) instead of from one global flag, so deploys that overlap can't pick up each other's failure. Pictures are counted per environment (callsAt).
  • First test split: the old 2 s first test is now four tests, each with one invariant and at most one deploy: production's three looks; the preview's looks and vitals; the same head not pictured again; a newer head replacing the older one. The "could not be taken" test is also split from "never had a picture". That accounts for the 23 → 27 tests.
  • Invariant comments: every rewritten test opens with a one-line // Invariant: comment.
  • Shorter waits: until polls every 10 ms instead of 100 ms. No sleep over 10 ms remains.
  • Nothing dropped: no test is deleted or skipped. The three Chromium captures still use a real browser behind their skipIf(chromium === undefined) guard, and none needs a SLOW entry in scripts/gate.ts.

Same three failures before and after: the three real-Chromium captures fail in this sandbox (Chromium exits 133). They fail the same way on main's version of the file, which the issue attributes to the sandbox's egress proxy. Every other test passes: 20 of 23 on main, 24 of 27 here. The forge's check runs those three for real.

The tests still fail without the code they guard. I broke src/looks/service.ts one way at a time, ran the tests, then restored the file:

  • Dropping deleteLooks for older heads fails "a newer head is pictured, and the older head's pictures go", plus four tests that depend on it.
  • Renaming the "no browser" skip fails "with no browser the check's summary says so".
  • Dropping deleteLooks in forget fails "a closed pull's pictures go with its preview".
  • Dropping the queue's replacement of a waiting look fails neither main's nor this version of "two events for one environment…". The superseded commit is apparently dropped somewhere else as well, so this test's strength is unchanged; the gap is older than this change and is not addressed here.

I did not mutation-test every rewritten test one by one. Some tests depend on the ones before them in the file, as on main, so they only show their failure in a full-file run.

bun scripts/gate.ts and bun x tsc --noEmit both pass. I couldn't call the issue's keep_plan tool because this session doesn't have it.

To try it: bun test test/looks.test.ts

🤖 Generated with Claude Code

Closes #10258

commented

test/looks.test.ts now runs in 7.43 s for 27 tests, down from 10.51 s for 23 tests on main (bun test test/looks.test.ts, crew sandbox, df26aa9c3). The slowest single test now takes 0.53 s.

The file was slow because it waited out about eleven real deploys one after another, roughly 450 ms each.

Changes, all in test/looks.test.ts:

  • Previews opened at once: the previews that tests only read are now opened together in beforeAll, with deploy concurrency 8. That covers the greeting, the never-pictured pull, the slow red test, the agent's ask, the two-page app, the public gallery and, when Chromium is present, the real-browser app. Each is settled and pictured before any test reads it.
  • Fake browser fails per host: the stand-in browser takes its failure mode per app host (modes) instead of from one global flag, so deploys that overlap can't pick up each other's failure. Pictures are counted per environment (callsAt).
  • First test split: the old 2 s first test is now four tests, each with one invariant and at most one deploy: production's three looks; the preview's looks and vitals; the same head not pictured again; a newer head replacing the older one. The "could not be taken" test is also split from "never had a picture". That accounts for the 23 → 27 tests.
  • Invariant comments: every rewritten test opens with a one-line // Invariant: comment.
  • Shorter waits: until polls every 10 ms instead of 100 ms. No sleep over 10 ms remains.
  • Nothing dropped: no test is deleted or skipped. The three Chromium captures still use a real browser behind their skipIf(chromium === undefined) guard, and none needs a SLOW entry in scripts/gate.ts.

Same three failures before and after: the three real-Chromium captures fail in this sandbox (Chromium exits 133). They fail the same way on main's version of the file, which the issue attributes to the sandbox's egress proxy. Every other test passes: 20 of 23 on main, 24 of 27 here. The forge's check runs those three for real.

The tests still fail without the code they guard. I broke src/looks/service.ts one way at a time, ran the tests, then restored the file:

  • Dropping deleteLooks for older heads fails "a newer head is pictured, and the older head's pictures go", plus four tests that depend on it.
  • Renaming the "no browser" skip fails "with no browser the check's summary says so".
  • Dropping deleteLooks in forget fails "a closed pull's pictures go with its preview".
  • Dropping the queue's replacement of a waiting look fails neither main's nor this version of "two events for one environment…". The superseded commit is apparently dropped somewhere else as well, so this test's strength is unchanged; the gap is older than this change and is not addressed here.

I did not mutation-test every rewritten test one by one. Some tests depend on the ones before them in the file, as on main, so they only show their failure in a full-file run.

bun scripts/gate.ts and bun x tsc --noEmit both pass. I couldn't call the issue's keep_plan tool because this session doesn't have it.

To try it: bun test test/looks.test.ts

🤖 Generated with Claude Code

Approved

Test-only change; no product code, no dependency, no README edit. Previews are opened once in beforeAll and deploy concurrently, the fake browser fails per host, and the slow first test is split into single-invariant tests that each open with a one-line Invariant comment. The Chromium tests keep their skipIf guard. Reported 7.43 s (from 10.51 s), slowest test 0.53 s, and the pull is green. Non-blocking nits: the const opened = { value: x } adapters are shims that could be dropped, and the split tests share pull/first set by an earlier test, so one failure cascades. The two events test staying weak is older than this change.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs