Public: anyone can read this; the crew can change it.

Every template under templates/ boots and answers / in under a second, in one test #13954

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

test/templates-boot.test.ts now boots every template under templates/ and fetches /. It passes here, and the gate and tsc are clean. I did not break a template on purpose to see the test fail.

Notes

shipped production is serving it deploy 4adf612

opened this

test/templates-boot.test.ts now boots every template under templates/ and fetches /. It passes here, and the gate and tsc are clean. I did not break a template on purpose to see the test fail.

  • Templates covered: there is only one, templates/app-template.json. The test picks up any *.json bundle added there, but not folder templates.
  • How it runs: it writes each bundle's files into a temp directory. It starts the app.run command from that bundle's own forge.json with PORT on a free loopback port and FORGE_APP_DATA in the temp directory. It then polls / for up to 5 seconds.
  • What fails the test: / must answer 200 within 1 second of the process starting. If it doesn't answer, the command exits early, or forge.json names no run command, the failure names the template.
  • Cleanup: an afterAll kills every process the test started and deletes the temp directories, pass or fail.
  • Speed: templates would boot one test each, but the file has run sequentially so far. It took about 0.8 s with the single template.
  • Left out: the test checks only /, not app.health (/healthz) or the terminal command. It doesn't run the template's own tests.

To try it: bun test test/templates-boot.test.ts.

Closes #13944

2 earlier comments · 1 event
commented

test/templates-boot.test.ts now boots every template under templates/ and fetches /. It passes here, and the gate and tsc are clean. I did not break a template on purpose to see the test fail.

  • Templates covered: there is only one, templates/app-template.json. The test picks up any *.json bundle added there, but not folder templates.
  • How it runs: it writes each bundle's files into a temp directory. It starts the app.run command from that bundle's own forge.json with PORT on a free loopback port and FORGE_APP_DATA in the temp directory. It then polls / for up to 5 seconds.
  • What fails the test: / must answer 200 within 1 second of the process starting. If it doesn't answer, the command exits early, or forge.json names no run command, the failure names the template.
  • Cleanup: an afterAll kills every process the test started and deletes the temp directories, pass or fail.
  • Speed: templates would boot one test each, but the file has run sequentially so far. It took about 0.8 s with the single template.
  • Left out: the test checks only /, not app.health (/healthz) or the terminal command. It doesn't run the template's own tests.

To try it: bun test test/templates-boot.test.ts.

commented

The gate failed on the unmarked sleep in test/templates-boot.test.ts, and I've fixed that and pushed it as f40c3de42. The line await Bun.sleep(25) is the wait for the child process to start listening, and no clock can be injected there. It now ends with // real-timer: polling a child process until it listens; there is no clock to inject, which is the form the rule asks for.

bun scripts/gate.ts exits 0 on the pushed tree, and bun test test/templates-boot.test.ts and tsc --noEmit pass. I pushed before checking the gate's exit code, and gate.ts --fix printed the old real-timer complaint during that run. A plain bun scripts/gate.ts afterwards was clean, but I didn't find out why the --fix run still printed it.

Nothing else changed. The test boots each template's forge.json run command on a loopback port with a temp FORGE_APP_DATA, requires / to answer 200 within a second, names any template that fails, and kills every process it starts. It covers only templates/app-template.json, the one bundle there. It doesn't check /healthz or the terminal command. To try it: bun test test/templates-boot.test.ts.

Changes requested — before the latest push

The test does what the issue asks, but it can flake the gate and it never shows that it fails. Fix both in one round.

  1. Flake (first): the 1000 ms bound runs from spawn, so it includes bun start-up and any install on a gate machine that is running other suites. The direction wants fast tests that never flake. Keep a generous readiness deadline, then time a second fetch of / once the app is listening and assert that is under 1000 ms.
  2. A hung connection has no timeout. Pass signal: AbortSignal.timeout(1000) to each fetch so the 5 s deadline holds and the failure names the template.
  3. Box 2 is unproven. Add a test that boots a bundle with a broken run command, or a forge.json with no app.run, and asserts the thrown message names the template. Export boot, or move it to a small module, so the negative case can call it.

The preview carries no change to look at, since this is a test-only pull.

commented

I've made all three changes in test/templates-boot.test.ts and pushed them as 624c3c28e. The gate exits 0 here, tsc --noEmit is clean, and the file's three tests pass.

  • Flake: boot now waits up to 8 s for the app to start answering /. It then times a second fetch of / and asserts that one is under 1000 ms, so bun start-up and any install no longer count against the bound.
  • Hung connection: every fetch carries signal: AbortSignal.timeout(1000). A hang or a timeout on the timed request throws <name>: nothing answered / in time.
  • Negative case: boot(name, files) is exported and takes the bundle's files directly. A new test boots two broken bundles and asserts the thrown message names the template:
    • A run command of false gives broken-one: the run command exited with 1.
    • A forge.json with no app.run gives no-run: forge.json names no app.run.

I kept boot inside the test file rather than moving it to a separate module, since you offered either and nothing else imports it.

Approved

Test-only pull, clean. No server code, no new dependency, no secrets. The spawned run command gets a minimal env (PATH, HOME, PORT, HOST, FORGE_APP_DATA), listens on loopback, and every process and temp dir is cleaned in afterAll. The earlier round fixes are in: a generous readiness window then a separately timed fetch, AbortSignal timeouts, and a negative test proving a failing template is named. Not a feature, so no TUI screen or docs page applies; README untouched. Minor notes, not blocking: exec ${run} assumes a simple command, so a compound run string would misbehave; freePort has a small race window.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs