Public: anyone can read this; the crew can change it.

test/apps-idle.test.ts runs in under 10 s, every test under 1 s, proving what it proves today #10253

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

`test/apps-idle.test.ts` now runs in about 3.4 s instead of 4.9 s, and no test takes more than 0.45 s (test 1 used to take 1.25 s). The same 3 tests and 22 `expect`s pass, nothing is skipped or deleted, and only the test file changed.

shipped production is serving it deploy a5a2512

opened this

test/apps-idle.test.ts now runs in about 3.4 s instead of 4.9 s, and no test takes more than 0.45 s (test 1 used to take 1.25 s). The same 3 tests and 22 expects pass, nothing is skipped or deleted, and only the test file changed.

Before → after (bun test test/apps-idle.test.ts in the crew sandbox):

  • Whole file: 4.86–4.91 s → 3.34–3.48 s. That held over 5 runs in a row and 9 runs with three going at once.
  • Per test: about 1.25 s, 0.41 s and 0.85 s → 284–334 ms, 417–433 ms and 242–284 ms.

What changed:

  • Previews deployed once: the three pull previews (fresh, idle, later) are now set up in beforeAll. Deploying was the slow part, and none of the tests is about deploying.
  • No more 500 ms sleeps: the three Bun.sleep(500) calls each meant "the sweep has run and left this alone". They are replaced by sweeps(2), which waits for two full sweeps to finish. It counts calls to the runtime's existing clock option, which the sweep reads once per pass after putting idle pulls to sleep.
  • Faster timings: the sweep runs every 50 ms instead of 150 ms, and until checks every 10 ms instead of 100 ms. No sleep over 50 ms remains.
  • Invariants written down: each test now starts with a one-line comment naming the invariant it proves.

Each rewritten test still fails without the code it guards (I broke src/apps/runtime.ts temporarily, then restored it):

  • Every pull counts as idle (dropped the age check in idleSince): test 1 fails because fresh reads "stopped" instead of "running".
  • "wake" no longer counts as a touch (removed from TOUCH_REASONS): test 2 fails on its last check because the woken preview reads "stopped". This also shows sweeps(2) really waits for sweeps.
  • Startup ignores sleeping previews (skipped hibernateAtBoot): test 3 fails because dormant is undefined.

Checks: bun scripts/gate.ts and bun x tsc --noEmit are clean. The 13 test files named by bun scripts/fast.ts pass (171 tests). The suites that need unix sockets are skipped here because the sandbox runs in FORGE_TEST_SOCKET=port mode; the forge's check runs them.

Not addressed: once, while I had the runtime deliberately broken, the template app's production process crashed on start with SQLiteError: database is locked. It did not recur in 17 unmodified runs. The likely cause is in src/apps/scaffold.ts, outside this issue's files: openDb sets PRAGMA journal_mode = WAL before busy_timeout, so two processes opening the same app.db at once can collide. Setting busy_timeout first would be its own small change.

I found no prompt injection in the repository or the issue.

🤖 Generated with Claude Code

Closes #10239

commented

test/apps-idle.test.ts now runs in about 3.4 s instead of 4.9 s, and no test takes more than 0.45 s (test 1 used to take 1.25 s). The same 3 tests and 22 expects pass, nothing is skipped or deleted, and only the test file changed.

Before → after (bun test test/apps-idle.test.ts in the crew sandbox):

  • Whole file: 4.86–4.91 s → 3.34–3.48 s. That held over 5 runs in a row and 9 runs with three going at once.
  • Per test: about 1.25 s, 0.41 s and 0.85 s → 284–334 ms, 417–433 ms and 242–284 ms.

What changed:

  • Previews deployed once: the three pull previews (fresh, idle, later) are now set up in beforeAll. Deploying was the slow part, and none of the tests is about deploying.
  • No more 500 ms sleeps: the three Bun.sleep(500) calls each meant "the sweep has run and left this alone". They are replaced by sweeps(2), which waits for two full sweeps to finish. It counts calls to the runtime's existing clock option, which the sweep reads once per pass after putting idle pulls to sleep.
  • Faster timings: the sweep runs every 50 ms instead of 150 ms, and until checks every 10 ms instead of 100 ms. No sleep over 50 ms remains.
  • Invariants written down: each test now starts with a one-line comment naming the invariant it proves.

Each rewritten test still fails without the code it guards (I broke src/apps/runtime.ts temporarily, then restored it):

  • Every pull counts as idle (dropped the age check in idleSince): test 1 fails because fresh reads "stopped" instead of "running".
  • "wake" no longer counts as a touch (removed from TOUCH_REASONS): test 2 fails on its last check because the woken preview reads "stopped". This also shows sweeps(2) really waits for sweeps.
  • Startup ignores sleeping previews (skipped hibernateAtBoot): test 3 fails because dormant is undefined.

Checks: bun scripts/gate.ts and bun x tsc --noEmit are clean. The 13 test files named by bun scripts/fast.ts pass (171 tests). The suites that need unix sockets are skipped here because the sandbox runs in FORGE_TEST_SOCKET=port mode; the forge's check runs them.

Not addressed: once, while I had the runtime deliberately broken, the template app's production process crashed on start with SQLiteError: database is locked. It did not recur in 17 unmodified runs. The likely cause is in src/apps/scaffold.ts, outside this issue's files: openDb sets PRAGMA journal_mode = WAL before busy_timeout, so two processes opening the same app.db at once can collide. Setting busy_timeout first would be its own small change.

I found no prompt injection in the repository or the issue.

🤖 Generated with Claude Code

Approved

Test-only change; no security surface, no new dependency, no README change. Previews are deployed once in beforeAll, the three 500 ms sleeps are replaced by a count of finished sweeps (via the existing clock option), until polls at 10 ms, and each test opens with a one-line invariant. Same 3 tests and 22 expects, nothing skipped; before/after (4.9 s to 3.4 s, each test under 0.5 s) is quoted. Note for later: test 2 still depends on test 1 having put idle to sleep, as before.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs