A Small forge (1 CPU, 2 GiB) builds a first app: today the host stops it for… #11976

This pull covers only fault 3 of #11975, in part. A child forge that the host stops for going over a limit now restarts by itself, and its pages say what happened. The interrupted run is not queued again yet, and faults 1, 2 and 4 are not touched.

What changed

• New src/spawn/stopped.ts records each limit stop of a cell child: when, why, and how many times in the last hour.
• src/spawn/cell.ts records the stop where it already logs "child forge stopped".
• The restart goes through the existing due-wake path in src/spawn/host.ts (wakeDue), so the awake ceiling and the disk floor still apply. If the host is full, the restart retries every 60 s, so the forge is not left stopped for good.
• Backoff before the restart is 5 s, doubling for each stop in the hour, up to 5 min.
• While the forge is down, the wake gate answers: "This forge ran out of memory and was restarted at HH:MM UTC; it is coming back by itself." This replaces "the forge is full", which now only means a full host. A forge stopped more than once in the hour also gets a line saying a larger plan would fit it.
• test/spawn-stopped.test.ts covers the backoff, the restart firing, and the sentence. The tsc check, the gate and the existing spawn, sleep and cell tests pass. scripts/fast.ts named about 2,000 test files, too many to run, so I ran those spawn, sleep and cell suites instead. The unix-socket cell tests are skipped in my sandbox; the forge's check runs them.
• docs/features/child-stopped.md describes the behaviour.

What I did not do (the pull should not close #11975)

• The interrupted run is not queued again, and the sentence does not say so. The issue's sentence ends "the build is queued again"; mine does not.
• Page wording. The sentence appears only on the wake gate's "Not now" page and its text answer. I did not change the other pages.
• Fault 2: nothing slows or refuses processes in a child near its memory limit, no idle app cells are reclaimed, and the stop still takes the whole child rather than its largest process.
• Owner's view: an owner of a child that keeps hitting its limit has no view of it.
• Fault 1: a first ask on a claim-link forge still goes to <name>/forge.
• Fault 4: the real-cell Small-plan test is not written.
• Unconfirmed cause of the 429: I could not read production, so I did not confirm why the 429 lasted ten minutes. My reading of the code is that, after the stop, the wake gate found the host at its FORGE_CHILD_AWAKE_MAX of 8 and answered with the full-host sentence.

To try it: bun test test/spawn-stopped.test.ts. To see it live, run a cell child over its memory limit, then request its address; you should get the sentence, and then the forge should come back after the backoff.

🤖 Generated with Claude Code

Closes #11975

This pull covers only fault 3 of #11975, in part. A child forge that the host stops for going over a limit now restarts by itself, and its pages say what happened. The interrupted run is not queued again yet, and faults 1, 2 and 4 are not touched.

What changed

• New src/spawn/stopped.ts records each limit stop of a cell child: when, why, and how many times in the last hour.
• src/spawn/cell.ts records the stop where it already logs "child forge stopped".
• The restart goes through the existing due-wake path in src/spawn/host.ts (wakeDue), so the awake ceiling and the disk floor still apply. If the host is full, the restart retries every 60 s, so the forge is not left stopped for good.
• Backoff before the restart is 5 s, doubling for each stop in the hour, up to 5 min.
• While the forge is down, the wake gate answers: "This forge ran out of memory and was restarted at HH:MM UTC; it is coming back by itself." This replaces "the forge is full", which now only means a full host. A forge stopped more than once in the hour also gets a line saying a larger plan would fit it.
• test/spawn-stopped.test.ts covers the backoff, the restart firing, and the sentence. The tsc check, the gate and the existing spawn, sleep and cell tests pass. scripts/fast.ts named about 2,000 test files, too many to run, so I ran those spawn, sleep and cell suites instead. The unix-socket cell tests are skipped in my sandbox; the forge's check runs them.
• docs/features/child-stopped.md describes the behaviour.

What I did not do (the pull should not close #11975)

• The interrupted run is not queued again, and the sentence does not say so. The issue's sentence ends "the build is queued again"; mine does not.
• Page wording. The sentence appears only on the wake gate's "Not now" page and its text answer. I did not change the other pages.
• Fault 2: nothing slows or refuses processes in a child near its memory limit, no idle app cells are reclaimed, and the stop still takes the whole child rather than its largest process.
• Owner's view: an owner of a child that keeps hitting its limit has no view of it.
• Fault 1: a first ask on a claim-link forge still goes to <name>/forge.
• Fault 4: the real-cell Small-plan test is not written.
• Unconfirmed cause of the 429: I could not read production, so I did not confirm why the 429 lasted ten minutes. My reading of the code is that, after the stop, the wake gate found the host at its FORGE_CHILD_AWAKE_MAX of 8 and answered with the full-host sentence.

To try it: bun test test/spawn-stopped.test.ts. To see it live, run a cell child over its memory limit, then request its address; you should get the sentence, and then the forge should come back after the backoff.

🤖 Generated with Claude Code

This pull covers only part of fault 3 of #11975. A child forge that the host stops for going over a limit now restarts by itself, and its address says what happened. The interrupted run is not queued again, and faults 1, 2 and 4 are not touched.

What changed

• src/spawn/stopped.ts is new. It records each limit stop of a cell child (when, why, how many times in the hour) and arms a restart. The wait is 5 s, doubling for each stop in the hour, up to 5 min.
• The restart goes through the existing due-wake path in src/spawn/host.ts, so the awake ceiling and the disk floor still apply. If the host is full, the restart retries every 60 s.
• While the forge is down, the wake gate answers "This forge ran out of memory and was restarted at HH:MM UTC; it is coming back by itself." This replaces "the forge is full", which now only means a full host. A forge stopped more than once in the hour also gets a line saying a larger plan would fit.
• src/spawn/cell.ts records the stop where it already logs "child forge stopped".
• test/spawn-stopped.test.ts covers the backoff, the restart firing, and the sentence. docs/features/child-stopped.md describes the behaviour.
• Check fix: the test no longer sleeps on a real timer; it waits for the restart callback. Two comments are rewrapped to 100 columns. A commit now carries a Direction: line, which the "pipeline first" gate rule requires. bun scripts/gate.ts, tsc and the new test pass locally; the forge's check re-runs on this push.

What I did not do (the pull should not close #11975)

• The interrupted run is not queued again, and the sentence does not say so. The issue's sentence ends "the build is queued again"; mine does not.
• Page wording. The sentence appears only on the wake gate's "Not now" page and its text answer. I did not change the other pages.
• Fault 2: nothing slows or refuses processes in a child near its memory limit, no idle app cells are reclaimed, and the stop still takes the whole child rather than its largest process.
• Owner's view: an owner of a child that keeps hitting its limit has no view of it.
• Fault 1: a first ask on a claim-link forge still goes to <name>/forge.
• Fault 4: the real-cell Small-plan test is not written.
• Unconfirmed cause of the 429: I could not read production. My reading of the code is that the wake gate found the host at its FORGE_CHILD_AWAKE_MAX of 8 after the stop and answered with the full-host sentence.

To try it: run bun test test/spawn-stopped.test.ts. To see it live, run a cell child over its memory limit, then request its address. You should get the sentence, and the forge should come back after the backoff.

🤖 Generated with Claude Code

This round answers all six points except the direction question, which I can't settle. bun scripts/gate.ts and bun x tsc --noEmit are clean, and test/spawn-stopped.test.ts, test/boot-idle-children.test.ts, test/billing-lapsed-sleep.test.ts and the eight test files that use clientRows/forgeLines pass. bun scripts/fast.ts found the diff too broad to name tests from (2008 of 2990 files), so I picked those myself.

What changed

• Wording: the page now says "…and was stopped at HH:MM UTC; the host starts it again by itself." It no longer says "restarted".
• Stale stops: a stop expires after an hour. It is also dropped when the forge wakes, including when the wake is granted but the forge isn't open yet. It is dropped too when the restart is skipped (lapsed, refused or the ledger not done), so that case no longer promises a retry.
• Clean-up: stopped.ts keeps one state object per parent and the host passes the restarter in, so there is no module-level restarter and nothing leaks between tests. forgetStops is now cancelRestarts.
• Terminal: the forges list in src/tui/screens/forges.ts shows the stop sentence for a forge that is asleep and was stopped. It comes from a new optional stopped field on ClientRow.
• Tests: test/boot-idle-children.test.ts now has two wake-gate tests.

- The first checks that a stopped forge at the awake ceiling gets the stop sentence, not the full-host sentence. It then checks that a granted wake clears the stop, so the full-host sentence is back afterwards, and that the terminal row carries it. - The second checks that closeHostedChildren forgets stops. - test/spawn-stopped.test.ts covers the one-hour expiry (in lastStop and in the restart count), a skipped restart dropping the stop, and separate state per parent.

Not done

• The noteStop call in cell.ts on a breach has no test. It sits in the real-cell launch path, which this sandbox can't run, and no existing test drives that path with a fake launcher.
• Direction: I can't tell from the repository which pipeline item this serves. If it serves none of #6534, #6611, #5276, #3316, #6217, #6428, #1242 or #1073, hold it until they land.

To try it: boot the app, stop a cell child over its memory limit, then request its address while the host is at its awake ceiling. It is not reachable on the preview because a real stop doesn't happen there.

🤖 Generated with Claude Code

Gate is clean now. The earlier red output came from the gate reading the committed files before the commit landed.

The check failed on three gate rules, all fixed in fdb153969:

• Comment wrapping: the comments at src/spawn/stopped.ts lines 10 and 77 are rewrapped under 100 columns.
• Real timer in a test: test/spawn-stopped.test.ts:53 no longer sleeps. It waits one setImmediate so the restart's answer settles before the assertion.

bun scripts/gate.ts and bun x tsc --noEmit are clean, and test/spawn-stopped.test.ts passes (5 tests). I did not rerun the other test files, because this round only rewrapped comments and swapped the test's wait.

I pushed before checking the gate. It was red at that moment, but the commit landed first and the gate then passed.

The changes from the earlier round are unchanged. The noteStop call in cell.ts on a breach is still untested, since it sits in the real-cell launch path the sandbox can't run. The direction question (which pipeline item this serves) still needs a person to answer.

🤖 Generated with Claude Code