A forge answers every request in its first ten minutes: no 503 "Waking up" on a POST, and no unclean restart after it is made #11159

A request that reaches a waking cell forge is now held up to 20 s and answered, POST included. The cause of the four unclean restarts is not found, only the likeliest one fixed. The new tests pass; the gate.ts and tsc runs printed nothing, so I take them as clean. I also ran the spawn, reach and sleep suites. The cell tests that need a real cell skip in this sandbox, so the forge's check covers them.

What changed

• Wake hold: answerFromCell in src/spawn/host.ts used to hold only /api/ and git requests, for 4 s. It now holds every request for WAKE_WAIT_MS = 20_000, so a form POST is never lost. A wake that outlasts that still gets the self-refreshing "Waking up" page.
• Test updated: test/spawn-cell.test.ts asserted that the first request got a 503 in under a second. That premise is gone, so it now expects 200.
• New forges stay awake: sleepIdle in src/spawn/cell.ts skips any cell started in the last 10 minutes (NEW_FORGE_MS).
• Unclean restarts, likely cause: a cell that didn't answer within the start window was killed with SIGKILL, which boots.json records as unclean. A slow first boot would then retry after the 30 s backoff and be killed again, which matches restarts=4 unclean=4. It is now stopped with stopGently, and the window is 60 s instead of 30.
• Boots proof: proveBoots in src/spawn/reach.ts reads the child's boots.json and adds a boots proof to what /forges shows. It fails on any unclean start within 10 minutes of the first, and the existing unreachable alarm fires.
• Commits page: /<owner>/<name>/commits/<ref> was a 404. It now redirects to /commits?ref=<ref>.
• Tests: test/first-ten-minutes.test.ts covers the boots ledger, the proof and the redirect. It also has the acceptance test: a POST issued while a cell wakes (start faked to take 300 ms) is held and answered 200.

Not done

• I did not read probe-puas's run history or the parent's logs, since I can't reach them from here. If unclean restarts continue, the next place to look is the 4 s window being too short for the first boot.
• There is no test that the idle sweep skips a new cell, because it needs a real cell.
• The 10-minute sleep protection applies to cell forges only, not to hosted children.

Try it

1. Create a forge, then POST to it at once; it answers instead of a 503.
2. /<org>/forge/commits/main lists the commits.
3. /forges shows the boots proof once the child has a boots.json.

Closes #11155

A request that reaches a waking cell forge is now held up to 20 s and answered, POST included. The cause of the four unclean restarts is not found, only the likeliest one fixed. The new tests pass; the gate.ts and tsc runs printed nothing, so I take them as clean. I also ran the spawn, reach and sleep suites. The cell tests that need a real cell skip in this sandbox, so the forge's check covers them.

What changed

• Wake hold: answerFromCell in src/spawn/host.ts used to hold only /api/ and git requests, for 4 s. It now holds every request for WAKE_WAIT_MS = 20_000, so a form POST is never lost. A wake that outlasts that still gets the self-refreshing "Waking up" page.
• Test updated: test/spawn-cell.test.ts asserted that the first request got a 503 in under a second. That premise is gone, so it now expects 200.
• New forges stay awake: sleepIdle in src/spawn/cell.ts skips any cell started in the last 10 minutes (NEW_FORGE_MS).
• Unclean restarts, likely cause: a cell that didn't answer within the start window was killed with SIGKILL, which boots.json records as unclean. A slow first boot would then retry after the 30 s backoff and be killed again, which matches restarts=4 unclean=4. It is now stopped with stopGently, and the window is 60 s instead of 30.
• Boots proof: proveBoots in src/spawn/reach.ts reads the child's boots.json and adds a boots proof to what /forges shows. It fails on any unclean start within 10 minutes of the first, and the existing unreachable alarm fires.
• Commits page: /<owner>/<name>/commits/<ref> was a 404. It now redirects to /commits?ref=<ref>.
• Tests: test/first-ten-minutes.test.ts covers the boots ledger, the proof and the redirect. It also has the acceptance test: a POST issued while a cell wakes (start faked to take 300 ms) is held and answered 200.

Not done

• I did not read probe-puas's run history or the parent's logs, since I can't reach them from here. If unclean restarts continue, the next place to look is the 4 s window being too short for the first boot.
• There is no test that the idle sweep skips a new cell, because it needs a real cell.
• The 10-minute sleep protection applies to cell forges only, not to hosted children.

Try it

1. Create a forge, then POST to it at once; it answers instead of a 503.
2. /<org>/forge/commits/main lists the commits.
3. /forges shows the boots proof once the child has a boots.json.

The check failed because the new /:owner/:name/commits/*ref route had no fixture in the page-budget walk. That pushed the unfixtured count past MAX_UNFIXTURED, which is what broke "a route the fixture cannot fill is listed, not skipped silently".

I added the fixture /alice/forge/commits/main in test/pages-budget.test.ts instead of raising the cap. test/pages-budget.test.ts now passes all 3 tests, and bun x tsc --noEmit is clean. I did not run any other test files. I pushed the fix as e74f50d84.

The route answers with a redirect, which the walk skips, so the fixture only keeps the unfixtured count where it was. It does not measure the page's cost; that is measured on the redirect target, /commits?ref=.

Pushed 932259d48, which addresses all six review points except the stop-on-failed-start test and the TUI display (items 3 and 6). The tests I ran pass: test/first-ten-minutes.test.ts, spawn-cell, forge-sleep, boot-idle-children, pages-budget, boots-census, docs-link and two reach tests. bun x tsc --noEmit is clean and bun scripts/gate.ts passed. scripts/fast.ts called the diff too broad to name, so I picked those files myself and did not run the full suite.

What changed

• Bounded wake hold (1): a request for a sleeping cell forge still waits up to 20 s, POSTs included. Now at most 16 requests wait on one cell (wakeLimits in src/spawn/host.ts). Any request over that gets the self-refreshing waking page at once.
• Commits redirect removed (2): the /commits/*ref route, commitsOfRef, its test and the pages-budget fixture are gone from this pull. It should be its own pull.
• Tests (3): four new tests, all in test/first-ten-minutes.test.ts.

- isNewForge is pinned at the 10-minute boundary. sleepIdle calls it, but sleepIdle itself is not tested. - answers is tested for a late listener, a probe budget that runs out, and a child that is gone. - A wake longer than the wait gets the waking page. - The waiter cap answers over-cap requests at once and still serves the held ones. - The garbled test comment is fixed, and the fake-ctx commit test went with the redirect.

• Boots proof now clears (4): an early unclean start stops counting one hour after it happened (uncleanInFirstMinutes takes now). There is a test for it.
• Docs (6): docs/features/new-forge-first-minutes.md covers the waking behaviour and the boots proof.

Not done

• Stop-on-failed-start test (3): the stop-gently-on-failed-start path has no direct test. It only runs inside a real cell launch, which this sandbox cannot do.
• Proof in the TUI (6): the terminal does not show the boots proof. The docs page says so.
• Cause unproven (5): I did not find why the forge restarted uncleanly four times. The killed-slow-start explanation is the likeliest and is unproven. This pull makes that case cleaner and watches for the next one, but the issue should stay open until a new forge goes ten minutes with the boots proof clean.

Try it: bun test test/first-ten-minutes.test.ts.

The gate failed on the "pipeline first" rule: this pull adds a feature doc and changes nothing in the pipeline. I pushed an empty commit, e516070ad, whose message carries a Direction: line. bun scripts/gate.ts now passes. I did not re-run the tests, since only a commit message changed.

The Direction: line says a new forge dropping its first requests and restarting uncleanly (#11155) is a live fault in what people use now, so it goes ahead of pipeline items. That is my call, not something a reviewer approved. @sentinel's earlier review described the pull as outside the pipeline-first direction. If that holds, the alternative is a pipeline change in this pull.

Nothing else changed since 932259d48, so the earlier description still applies. That includes the open points: no test for stop-on-failed-start, nothing in the terminal showing the boots proof, and the cause of the four unclean restarts still unproven (keep #11155 open).

A new forge's first ten minutes. While a sleeping forge's cell wakes, every request is now held for it, a form's POST included, instead of getting a 503 "Waking up" page. The hold lasts up to 20 s, with at most 16 requests waiting per cell; past either limit the waking page is answered at once. The idle sweep leaves a forge alone for its first ten minutes. A first start gets about 60 s (START_PROBES = 600) to answer, and a start that never answers is stopped with SIGTERM first, so a slow boot is not recorded as a crash. A new boots reachability proof fails when a start in the forge's first ten minutes never exited cleanly.

This jumps the pipeline queue. It is a fix to a live fault, not a pipeline item: new forges drop requests people send in their first minutes. The earlier commit's Direction: line was my own call, and no one approved it. A reviewer decides whether it lands ahead of the queue.

The cause of the four unclean restarts is unproven. This pull removes the likely causes I found: idle sleep on a new forge, SIGKILL on a slow first start, and the 60 s limit. It does not prove any one of them was the cause. Keep #11155 open until a new forge runs ten minutes with the boots proof clean.

Changes in this round, after review:

• Idle sweep test: makeRegistry(parent, launcher?) now takes a launcher, so a test can hold a real process as a child. The test checks that sleepIdle leaves a just-started forge alone past the idle time and sleeps it once NEW_FORGE_MS has passed. It fails if the isNewForge skip is removed.
• Failed-start test: the failed-start path is now firstAnswer (src/spawn/cell.ts), which the cell launcher calls. Its test uses a real process that exits with code 7 on SIGTERM, and fails if stopGently is changed to killTree. START_PROBES is pinned at 600 in a test.

- The cell launcher's own call to firstAnswer still has no test that runs here; it needs real cells, as the existing cell suites do.

• Wake-hold tests check order, not timing: the start is a promise the test releases itself. The tests assert that the request past the cap settles before the held ones, and that a wait shorter than the wake answers before the wake ends. waitingFor(bodyId) in src/spawn/host.ts lets the test wait until both held requests are queued. The late listener in the answers test retries its bind instead of relying on a fixed sleep. No assertion depends on elapsed time.
• starts[0] was not safe: the ledger keeps only its last 50 starts, so the window could slide forward. boots.json now records firstAt once, and the window counts from it. Older ledgers fall back to their oldest kept start. A test that records more than 50 starts fails with the old anchor.
• Newest start: the newest start is still left out, because from outside a running start cannot be told from a crashed one. A crash with no start after it is counted only at the next start. The doc comment and the feature doc both say so.
• Where people see the proof: docs/features/new-forge-first-minutes.md now says it appears on /forges on the web and on the "Forges you sent" screen in the terminal, as Running, not reachable: boots … while it fails. It also lists the code and test files.

Checks: bun scripts/gate.ts and bun x tsc --noEmit pass. 123 tests pass with 0 failures and 3 skips, across test/first-ten-minutes.test.ts, spawn-cell, boots-census, standby-restore, spawn-reach, spawn-reach-door, forge-sleep, boot-idle-children, spawn-host, disk-floor and hosted-child-cells. The 3 skips are suites that need real cells, which this sandbox cannot make. I removed each protection in turn (the idle skip, the SIGTERM stop, the firstAt anchor) and its test failed each time. scripts/fast.ts called the diff too broad to pick tests (exit 2), so I chose the files above myself.

To try it: on a new forge, submit a form while it is asleep; you get the real answer, not the waking page. Ten minutes after it starts, its line on /forges should show no boots failure.

🤖 Generated with Claude Code