test/apps-adversarial-test-step.test.ts runs under 10 s #3581

test/apps-adversarial-test-step.test.ts now finishes in about 1.7 s here, down from 9.5 s; scripts/test-times.json had it at 33.5 s. The two tests take 0.08 s and 0.35 s. It passed ten runs in a row, and the type check passes. Both tests are kept and still prove what they proved before. No production code changed.

What changed

• One app per file. The tests used to each create an app and wait for its production deploy. Now one app (steps) is built and deployed once in beforeAll, and each test opens its own pull against it. A small pullWith(branch, manifest) helper commits the forge.json and opens the pull.
• No polling, no sleeps. settled() and the "is the preview up" wait used to poll every 50 ms with Bun.sleep. They now listen on app.bus, using the run.* and app.serving events the runtime already sends, with a setTimeout limit.
• The closed-pull test no longer sleeps 6.5 s. It used to wait past the test step's sleep 5 and check that a marker file never appeared. Now the preview's test step sleeps 30 s. After the merge, the test waits for the preview run to finish as failed with summary test: test cancelled. The runtime only reports a step's outcome after its process has exited, and the run can't settle within the test's 20 s limit unless the close killed the step. So a finished run means the shell that would write the marker is gone, and checking the marker straight away is sound. The test also checks that the preview run is still running at the moment of the close.
• Production skips the slow step. Only the preview's test step sleeps (if [ "$FORGE_APP_ENV" = preview ]). Before, merging made production run the same 5 s step, and an env-keyed marker told the two apart. Now production's run finishes at once.

Checking it still proves the kill: I commented out state.deploying?.abort.abort() in teardown in src/apps/runtime.ts and the closed-pull test failed, then restored it.

Worth a look

• I put 1.8 s in scripts/test-times.json, which is my local time here, where apps run without containment. The vxrail runs apps in cells, which is slower: the old 9.5 s here was 33.5 s there. The check's own recording should replace this number.
• Each test now has a 20 s limit, the same as the bus wait. Bun's default of 5 s is too tight for a preview deploy in a cell.
• Not done: the install-markers in $TMPDIR may not be writable from inside a cell, so the "never ran" checks in the failed-install test may pass without proving anything on the vxrail. That was already true before this change, and fixing it is outside this issue.

To try it: bun test test/apps-adversarial-test-step.test.ts

🤖 Generated with Claude Code

Closes #3567

test/apps-adversarial-test-step.test.ts now finishes in about 1.7 s here, down from 9.5 s; scripts/test-times.json had it at 33.5 s. The two tests take 0.08 s and 0.35 s. It passed ten runs in a row, and the type check passes. Both tests are kept and still prove what they proved before. No production code changed.

What changed

• One app per file. The tests used to each create an app and wait for its production deploy. Now one app (steps) is built and deployed once in beforeAll, and each test opens its own pull against it. A small pullWith(branch, manifest) helper commits the forge.json and opens the pull.
• No polling, no sleeps. settled() and the "is the preview up" wait used to poll every 50 ms with Bun.sleep. They now listen on app.bus, using the run.* and app.serving events the runtime already sends, with a setTimeout limit.
• The closed-pull test no longer sleeps 6.5 s. It used to wait past the test step's sleep 5 and check that a marker file never appeared. Now the preview's test step sleeps 30 s. After the merge, the test waits for the preview run to finish as failed with summary test: test cancelled. The runtime only reports a step's outcome after its process has exited, and the run can't settle within the test's 20 s limit unless the close killed the step. So a finished run means the shell that would write the marker is gone, and checking the marker straight away is sound. The test also checks that the preview run is still running at the moment of the close.
• Production skips the slow step. Only the preview's test step sleeps (if [ "$FORGE_APP_ENV" = preview ]). Before, merging made production run the same 5 s step, and an env-keyed marker told the two apart. Now production's run finishes at once.

Checking it still proves the kill: I commented out state.deploying?.abort.abort() in teardown in src/apps/runtime.ts and the closed-pull test failed, then restored it.

Worth a look

• I put 1.8 s in scripts/test-times.json, which is my local time here, where apps run without containment. The vxrail runs apps in cells, which is slower: the old 9.5 s here was 33.5 s there. The check's own recording should replace this number.
• Each test now has a 20 s limit, the same as the bus wait. Bun's default of 5 s is too tight for a preview deploy in a cell.
• Not done: the install-markers in $TMPDIR may not be writable from inside a cell, so the "never ran" checks in the failed-install test may pass without proving anything on the vxrail. That was already true before this change, and fixing it is outside this issue.

To try it: bun test test/apps-adversarial-test-step.test.ts

🤖 Generated with Claude Code

test/apps-adversarial-test-step.test.ts now runs as one app, waits on bus events instead of sleeping, and runs in about 1.8s instead of 33.5s (#3567). No behaviour changed: the suite asserts the same outcomes as before.

main is merged in. The only conflict was in scripts/test-times.json, which stores how long each test file takes so scripts/test-shards.ts can split the files into shards. Both sides had rewritten one slow suite and recorded its new time. The merge keeps main's 3s for test/apps-adversarial-teardown.test.ts and this branch's 1.8s for the test-step suite. Without that, the shard planner would weigh the two files with each other's times.

After the merge, bun x tsc --noEmit is clean and the teardown and test-step suites pass together (4 tests). To try it: bun test test/apps-adversarial-test-step.test.ts.

🤖 Generated with Claude Code