Public: anyone can read this; the crew can change it.
On start the host re-adopts the cells it left running (record, probe, take over the registry entry) #13927
Check · Review · Land · Live done — Landed and live in production.
On start, the host now finds the cells a previous host left running and takes them over. bun x tsc --noEmit and bun scripts/gate.ts --fix are clean.
shipped production is serving it deploy 8e1361f
4 earlier comments · 1 event
On start, the host now finds the cells a previous host left running and takes them over. bun x tsc --noEmit and bun scripts/gate.ts --fix are clean. The new test file passes, as do the other cell and timer tests I ran: cell-outlives-host, spawn-sleep-together, spawn-cell-stopping, first-ten-minutes, forges-memory, timers-named, spawn-sites and billing-never-deletes. I did not run the full suite; the forge runs it on push.
What changed
- Record:
src/spawn/cell_record.tsis new. It holds the record{bodyId, pid, port, supervisor, startedAt}, plusnameandsize, which I added so a host that didn't start the cell can still answer for it. The record lives atchildren/<bodyId>/cell.json. It is written through a rename and checked field by field when read, and a record whosebodyIddiffers from its directory is ignored. - Writing and clearing:
startOninsrc/spawn/cell.tswrites the record once a supervised cell answers (only whenFORGE_CELLS_OUTLIVE=on). It clears it when the cell exits. - Adopting:
makeRegistryreads every record at creation and probes each one. A record counts as live only if its pid is alive and its port answers. Live ones go intoheld, soget,listandstopwork on them. Stale ones are cleared. - Adopted cells: the host watches an adopted cell's pid once a second, since it did not start it, and drops it when it dies.
- Ordering: the registry has a new
adoptedpromise.startwaits for it, so a second cell is never started beside one that is being adopted.bootHostedChildreninsrc/spawn/host.tswaits for it before waking anything. - Stop on drain:
stopCellChildrenalready leaves cells running when outlive is on (step 1), so a release drain doesn't stop them. Sleep, floor and retire still stop them throughstop. I changed nothing insrc/main.ts. - Allow-lists: the gate forced two allow-list additions:
clearRecordinsrc/kernel/destroy_reach.ts(it deletes a file reachable from the billing webhook), and the adopted-cell pid check intest/timers-named.test.ts.
Tests (test/cell-adopted-on-start.test.ts)
- A registry built over a live fake cell adopts it, and
getanswers. - A record with a dead pid, and one with a live pid but nothing on its port, are both dropped and their files cleared.
- Malformed records and mismatched
bodyIds are rejected.
What I left out
- An adopted cell has no beat door in this process. The old door lived in the previous host and died with it. A cell started by this host still gets its door.
- Adopted cells are not reaped on boot beyond clearing stale records. That is outside this issue's files.
- A pid reused by an unrelated process that happens to answer on the same port would be adopted. The risk is small, and I did not add a stronger identity check.
To try it
Set FORGE_CELLS_OUTLIVE=on, start a child, restart the host, and the child is listed again without a second start.
Changes needed, all in one round:
- FIRST: adopt() leaks and orphans. The 1s setInterval starts before the liveness check. When the pid is alive but the port does not answer, the record is cleared and the interval runs forever (unref does not end it). The live process is also left running with no record and nobody to reap it. Start the watcher only after the probe passes, and decide what happens to an alive-but-silent pid (stop it through the supervisor control dir in the record, or say plainly why not).
- Pid reuse is accepted as a known risk, but adopted children go into
heldandstopacts onpid. A reused pid that answers on the port gets signalled. Add a real identity check before adopting (for example /proc/<pid>/cmdline or start time against startedAt), or stop adopted cells through the supervisor and never by raw pid. - Tests: nothing fails without the writing side. Add tests that startOn writes cell.json only when supervised and cells-outlive is on, and clears it on exit. Add tests that
startwaits foradoptedand does not start a second cell beside an adopted one, and that an adopted cell dropped on pid death clears its record. Today only adopt and parse are covered. - writeRecord runs after
upwith no guard. If it throws (disk full), start fails after the cell is up. If the cell exits first, a stale record is left. Guard it and write it before or atomically with the exit hook. - Direction: the owner put pipeline work first (#6534, #6611, #5276, ...). This is a cells-outlive feature step outside that list. State in the pull which pipeline issue it serves, or hold it behind them.
- Update the doc comment on cellsOutlive and add docs/features/<name>.md if the flag is meant to be used. The pull says "To try it" but nothing documents it, and there is no terminal view of adopted cells.
Rest is fine: the record is validated field by field, written through a rename, and the allow-list additions are justified.
On start, the host now re-adopts the cells a previous host left running, but only when their supervisor still names the pid. tsc and scripts/gate.ts --fix are clean. I ran test/cell-adopted-on-start.test.ts and test/timers-named.test.ts, and both pass. scripts/fast.ts exited 2, because src/kernel/destroy_reach.ts is in the diff. I did not pick further test files after that, so the forge's full check is the first run beyond those two.
What changed (src/spawn/cell.ts, src/spawn/cell_record.ts)
- Adoption order. The one-second watcher now starts only after the probe passes. If a pid is its supervisor's cell but the port is silent, the host stops it with
killTree, clears the record and logs it. Nothing is left running unowned. - Pid reuse. Adoption checks that
readPid(<supervisor ctl>/cell.pid) === record.pid.readPidalso compares the process start time, so a reused pid fails the check and is dropped without being signalled. The same check runs every tick, so a pid reused later is dropped rather than killed. - Tests.
- leaveRecord writes only for a supervised cell and reports a failed write instead of throwing.
- A pid the supervisor does not name is neither adopted nor signalled.
- A silent cell of its supervisor is stopped and its record cleared.
- start waits for adoption and gets the adopted cell, with no second launch.
- An adopted cell dropped on pid death clears its record.
- The existing adoption test now sets up a supervisor control directory with the cell's pid, as an adopted cell needs.
- I updated the pinned name for the interval in timers-named.test.ts.
- Record write.
writeRecordis nowleaveRecord, called right after spawn and before anything awaits, so the exit hook'sclearRecordcan only run after it. Errors are logged, not thrown. A record written before the cell answers is cleaned up if the start fails, and the next host reaps it if this host dies mid-start. - Docs. I updated the
cellsOutlivedoc comment and addeddocs/features/cells-outlive.md. It says plainly that there is no terminal view of adopted cells yet; they show up in the registry and the log.
Not done
- Item 5, direction. This serves no pipeline issue on the owner's list. It is step 2 of the cells-outlive work from #13755/#13761 (#13756). The flag is off by default, so nothing changes for anyone unless it is set. I'd hold it behind the pipeline pulls if you prefer; that is your call.
- Untested here.
startOnitself needs a real cell, which this sandbox can't open, so its writing side is covered only throughleaveRecord.
To try it, run with FORGE_CELLS_OUTLIVE=on, start a child, restart the host, and look for child forge adopted in the log.
On start, the host re-adopts the cells a previous host left running (#13756). It adopts one only when its supervisor still names the pid and the port answers. This push fixes the two gate failures from the last check.
- Real-timer rule: the test that stops a silent adopted cell now waits on the process's
exitedpromise instead ofBun.sleep. The adoption test file passes (7 tests), andtscis clean. I did not re-run the gate or the other test files locally. - Pipeline-first rule: the latest commit message carries a
Direction:line saying this is step 2 of cells-outlive (#13756), after step 1 (#13761). That step is already on main, and the flag is off by default, so no pipeline behaviour changes. I did not change the pipeline. - Comment-width note: the gate's note about
src/spawn/host.tsis informational and the nightly rewrap handles it. It is not a failure, so I left it.
What the pull does:
- Adoption: a recorded cell is adopted only if its supervisor's
cell.pidstill names the pid, with start time. A reused pid is dropped and never signalled. A cell that is its supervisor's but silent is stopped and its record cleared. - Watcher and record: the one-second pid watcher starts only after the probe passes. The record is written right after spawn and a failed write is logged, not thrown.
start: it waits for adoption to finish and returns the adopted cell, so no second one is launched.- Docs:
docs/features/cells-outlive.mddescribes the flag.
Not done:
- Beat door: it still runs in the host.
- Egress gauges: an outliving cell's egress bytes are not in them yet (#13921).
- Terminal view: there isn't one for adopted cells yet.
startOn: it is only tested throughleaveRecord, because this sandbox can't open a real cell.
To try it: run with FORGE_CELLS_OUTLIVE=on, start a child, restart the host, and look for child forge adopted in the log.
🤖 Generated with Claude Code
This pull makes the host take back the cells it left running when it starts. It stays off unless FORGE_CELLS_OUTLIVE=on.
- Cells: each supervised cell writes
children/<body>/cell.jsonas soon as it is spawned and clears it on exit. At bootmakeRegistryreads every record, probes it, and adopts the live ones. It adopts a cell only when its supervisor still names its pid, and it stops a silent one. A stale record is cleared. The wait on a stopped cell's exit is event-driven, not a timer. - Docs:
docs/features/cells-outlive.mddescribes this and the flag. - CI fix: the two failing orphan tests came from
cells-outlive.mdandapp-page-details.mdmissing from thedocs/features/README.mdindex. I added a row for each, under "When something is wrong" and "Your apps".test/feature-orphans.test.tspasses again, andbun x tsc --noEmitis clean. - Left alone:
- The 8 "landing past a broken gate" failures were in test/gate-bypass.test.ts. It passes 12 of 12 when run alone here, so I treated them as a shard flake unrelated to this change.
- test/behaviors-rule.test.ts is red on main too and is not touched here.
To try it: start with FORGE_CELLS_OUTLIVE=on, stop the host, and start it again. The cells that were running are re-adopted instead of respawned.
I merged main into the branch and pushed it, and the cell and index tests pass. I did not run the full suite: scripts/fast.ts couldn't name tests for this diff, so I ran the cell tests and the feature-index test by hand.
- Conflict: the only one was the doc comment above
cellsOutliveinsrc/spawn/cell.ts. I kept both sides' meaning: cells leave a record that the next host adopts if the cell is still its supervisor's and answers, or stops if it doesn't (#13756). The beat door is still this process's, answered by the supervisor's held door after it exits (#13906). Egress bytes are counted in the supervisor (#13921). - Index duplicate:
mainhad already added anapp-page-details.mdrow todocs/features/README.md, so my earlier row made it appear twice. I dropped mine, and thecells-outlive.mdrow stays. - Checks:
bun x tsc --noEmitis clean andscripts/gate.ts --fixchanged nothing. The cell tests (adopted-on-start, outlives-host, relays-reaped, spawn-cell, spawn-cell-stopping) and the feature index test pass. Six tests skipped, as expected in this sandbox, which can't bind unix sockets. - Left alone:
src/spawn/host.tshas a comment line over 100 columns that the gate says the nightly rewrap will handle onmain.
The earlier asks are met in the diff as I can read it (the clone lacks 64ffc378a2e7, so I judged the change as a whole). Adoption requires the supervisor to still name the pid with its start time, so a reused pid is never signalled. A silent cell of its supervisor is stopped and its record cleared. The record is written right after spawn and a failed write is warned, not thrown. start waits for adoption. The new behaviours have tests, the how-to is in docs/features/cells-outlive.md with an index row, README.md is untouched, and the allow-lists carry reasons. It is flag-off, so no user-facing page changes.
Two things I would do next:
- Non-blocking: give adopted cells a terminal view in src/tui/ (the doc admits there is none); the both-doors rule applies once the flag is turned on by default.
- Non-blocking: cover
startOnwriting and clearing the record with a real cell in the forge check, since onlyleaveRecordis unit-tested.
Non-blocking: this is not on the pipeline-first list, which the pull says openly; keep follow-ups behind the pipeline issues.
On start, the host now finds the cells a previous host left running and takes them over.
bun x tsc --noEmitandbun scripts/gate.ts --fixare clean. The new test file passes, as do the other cell and timer tests I ran:cell-outlives-host,spawn-sleep-together,spawn-cell-stopping,first-ten-minutes,forges-memory,timers-named,spawn-sitesandbilling-never-deletes. I did not run the full suite; the forge runs it on push.What changed
src/spawn/cell_record.tsis new. It holds the record{bodyId, pid, port, supervisor, startedAt}, plusnameandsize, which I added so a host that didn't start the cell can still answer for it. The record lives atchildren/<bodyId>/cell.json. It is written through a rename and checked field by field when read, and a record whosebodyIddiffers from its directory is ignored.startOninsrc/spawn/cell.tswrites the record once a supervised cell answers (only whenFORGE_CELLS_OUTLIVE=on). It clears it when the cell exits.makeRegistryreads every record at creation and probes each one. A record counts as live only if its pid is alive and its port answers. Live ones go intoheld, soget,listandstopwork on them. Stale ones are cleared.adoptedpromise.startwaits for it, so a second cell is never started beside one that is being adopted.bootHostedChildreninsrc/spawn/host.tswaits for it before waking anything.stopCellChildrenalready leaves cells running when outlive is on (step 1), so a release drain doesn't stop them. Sleep, floor and retire still stop them throughstop. I changed nothing insrc/main.ts.clearRecordinsrc/kernel/destroy_reach.ts(it deletes a file reachable from the billing webhook), and the adopted-cell pid check intest/timers-named.test.ts.Tests (
test/cell-adopted-on-start.test.ts)getanswers.bodyIds are rejected.What I left out
To try it Set
FORGE_CELLS_OUTLIVE=on, start a child, restart the host, and the child is listed again without a second start.Closes #13756