Public: anyone can read this; the crew can change it.
A cell child cannot steer its adoption: cell records live outside the child's directory and adoption checks pid start time and the supervisor's port #15237
Check · Review · Land · Live done — Landed and live in production.
mason/forge:mason/15235-a-cell-child-cannot-steer-its-adoption-c → main 2 commits fork
8 files
changed +284
−103
docs/features/cells-outlive.md +5 −2
| @@ -6,12 +6,15 @@ row: Cells that outlive the host, and are taken back on start | ||
| 6 | 6 | |
| 7 | 7 | On when the host boots (#13750), so a production release leaves a hosted forge's checks and runs running; set `FORGE_CELLS_OUTLIVE=off` to tie cells to the host again. With it on, a child's cell runs its proxy and relays in a supervisor of its own and is not tied to this process: stopping the host leaves the cell running (#13755). |
| 8 | 8 | |
| 9 | **On start the host takes its cells back (#13756).** Each supervised cell writes `children/<body>/cell.json` (pid, port, supervisor control directory, start time, name, size) as soon as it is spawned, and clears it when it exits. At boot the host reads every record and, for each: | |
| 9 | **On start the host takes its cells back (#13756).** For each supervised cell the host writes a record, `<data>/cell-records/<body>.json` (pid, port, supervisor control directory, start time, name, size; `spawn/cell_record.ts`), as soon as it is spawned, and clears it when it exits. That directory is the parent's and is bound into no cell, and a record path takes only a safe body id, so a child cannot write or name a record (#15235). A `cell.json` an older host left in a child's own directory is removed at boot without being read (`dropLegacyRecords`), and the log says so for each. At boot the host reads every record and, for each: | |
| 10 | 10 | |
| 11 | - the pid must still be the cell its supervisor names (`cell.pid` in the control directory, pid and start time), so a reused pid is never adopted or signalled; | |
| 11 | - the pid must still be the cell its supervisor names: `cell.pid` in the control directory must carry the pid and its start time, and that start time must be the running process's (`readStartedPid`). A bare pid, or a reused one, is never adopted or signalled; | |
| 12 | - the port must be the one the supervisor relays the cell's way in on (`config.json` `listen`, `readListen`). A record naming any other port is dropped without that port being asked anything; | |
| 12 | 13 | - the port must answer. A cell that is its supervisor's but silent is stopped and its record cleared, since nothing else would reap it; |
| 13 | 14 | - otherwise the cell is held in the registry as if this host had started it. A start for the same body waits for adoption to finish and finds the adopted cell, never a second one. The record is cleared when the cell goes. |
| 14 | 15 | |
| 15 | 16 | Not yet: the beat door still runs in the host, egress bytes of an outliving cell are not in the host's gauges (#13921). |
| 16 | 17 | |
| 17 | 18 | **What the host did is shown.** For this host's life it keeps each cell it adopted or dropped at boot (`spawn/adopted.ts`: name, pid, when, why). /system lists them under "Child forges across the restart" ("kept running across the restart", "stopped: did not answer"), and nothing when there were none; the MCP `system` tool (`kind: adopted`) and the terminal's System screen read the same rows. |
| 19 | ||
| 20 | **Upgrading from a host that kept records in the child's directory.** The first boot after #15235 finds no record in `cell-records/`, so cells the older host left running are neither adopted nor signalled: each is started afresh when next asked, and the old cell ends with its supervisor. Every restart after that adopts the cells it recorded. |
src/kernel/destroy_reach.ts +5 −1
| @@ -95,7 +95,11 @@ export const ALLOWED_REMOVALS: readonly { unit: string; why: string }[] = [ | ||
| 95 | 95 | { unit: "src/apps/contain.ts:openCell", why: "clears the cell's socket directory before the cell opens" }, |
| 96 | 96 | { unit: "src/spawn/supervisor.ts:startSupervisor", why: "removes the cell's own control directory (a fresh temp directory it made) when the host closes the cell" }, |
| 97 | 97 | { unit: "src/spawn/supervisor.ts:runSupervisor", why: "removes the cell's own socket directory and, when no host is left, its control directory: both fresh temp directories made for that cell" }, |
| 98 | { unit: "src/spawn/cell_record.ts:clearRecord", why: "removes the one cell.json a running cell left in its own child directory, never the directory or anything else in it" }, | |
| 98 | { unit: "src/spawn/cell_record.ts:clearRecord", why: "removes the one record file of a cell, `<data>/cell-records/<body>.json` for a checked body id, never the directory or anything else in it" }, | |
| 99 | { | |
| 100 | unit: "src/spawn/cell_record.ts:dropLegacyRecords", | |
| 101 | why: "removes the one cell.json an older host left in each child's directory, a file the child could write and so never read, never the directory or anything else in it", | |
| 102 | }, | |
| 99 | 103 | { unit: "src/apps/cgroup.ts:removeDir", why: "removes an empty cgroup directory" }, |
| 100 | 104 | { unit: "src/actions/spawn.ts:unmake", why: "removes the directory and ledger of a child whose first attempt failed before it was ever finished, and only that call's own (never a finished or resumed one): nothing of anyone's work is in it (#13135)" }, |
| 101 | 105 | { unit: "src/spawn/own_code.ts:checkout", why: "removes a half-made export of the forge's own code, keyed by commit" }, |
src/spawn/cell.ts +19 −12
| @@ -27,9 +27,9 @@ import { childModelHosts, childPaymentHosts, childRegistryHosts } from "../core/ | ||
| 27 | 27 | import { liveChildDisk, type ChildDisk } from "./disk_mount.ts"; |
| 28 | 28 | import { oneAtATime, sourcesFor, startFailure, startFirstThatWorks, useTree, type CodeSource } from "./own_code.ts"; |
| 29 | 29 | import { childGate, type ChildGate } from "./child_reach.ts"; |
| 30 | import { files, readPid } from "./supervisor.ts"; | |
| 30 | import { files, readListen, readStartedPid } from "./supervisor.ts"; | |
| 31 | 31 | import { adoptionRows, type Adoption, type AdoptionRow } from "./adopted.ts"; |
| 32 | import { clearRecord, leaveRecord, readRecords, type CellRecord } from "./cell_record.ts"; | |
| 32 | import { clearRecord, dropLegacyRecords, leaveRecord, readRecords, type CellRecord } from "./cell_record.ts"; | |
| 33 | 33 | |
| 34 | 34 | /** |
| 35 | 35 | * Whether a child's cell outlives this process (`FORGE_CELLS_OUTLIVE=on`; #13755): its proxy and |
| @@ -462,13 +462,18 @@ export function makeRegistry(parent: App, launcher?: Launcher): CellChildren { | ||
| 462 | 462 | * A cell a previous host left running: its pid is watched, since this process did not start it. |
| 463 | 463 | */ |
| 464 | 464 | const boot: Adoption[] = []; |
| 465 | const adopt = async (record: CellRecord, dir: string): Promise<void> => { | |
| 465 | const dataDir = parent.config.dataDir; | |
| 466 | const adopt = async (record: CellRecord): Promise<void> => { | |
| 467 | const drop = (): void => clearRecord(dataDir, record.bodyId); | |
| 466 | 468 | // The pid alone can be another process's by now: it is the cell only when its supervisor |
| 467 | // still names it (pid and start time, `readPid`). A pid that fails this is never signalled. | |
| 468 | const isCell = (): boolean => alive(record.pid) && readPid(files(record.supervisor).cell) === record.pid; | |
| 469 | if (!isCell()) { | |
| 469 | // still names it with the start time that process has (`readStartedPid`). A pid that fails | |
| 470 | // this is never signalled. | |
| 471 | const isCell = (): boolean => alive(record.pid) && readStartedPid(files(record.supervisor).cell) === record.pid; | |
| 472 | // The port is the supervisor's word too (`config.json`), never the record's alone: a record | |
| 473 | // naming another port is dropped before that port is asked anything. | |
| 474 | if (!isCell() || readListen(record.supervisor) !== record.port) { | |
| 470 | 475 | boot.push({ name: record.name, pid: record.pid, at: Date.now(), outcome: "dropped", why: "not-its-cell" }); |
| 471 | clearRecord(dir); | |
| 476 | drop(); | |
| 472 | 477 | return; |
| 473 | 478 | } |
| 474 | 479 | if (!(await answers(record.port, Promise.resolve(), 3))) { |
| @@ -476,7 +481,7 @@ export function makeRegistry(parent: App, launcher?: Launcher): CellChildren { | ||
| 476 | 481 | if (isCell()) killTree(record.pid); |
| 477 | 482 | boot.push({ name: record.name, pid: record.pid, at: Date.now(), outcome: "dropped", why: "silent" }); |
| 478 | 483 | parent.log.warn("child forge left by a previous host did not answer; stopped", { name: record.name, pid: record.pid }); |
| 479 | clearRecord(dir); | |
| 484 | drop(); | |
| 480 | 485 | return; |
| 481 | 486 | } |
| 482 | 487 | // Watched only now it is held: the timer ends when the cell is no longer the one recorded. |
| @@ -495,14 +500,16 @@ export function makeRegistry(parent: App, launcher?: Launcher): CellChildren { | ||
| 495 | 500 | boot.push({ name: record.name, pid: record.pid, at: Date.now(), outcome: "kept", why: null }); |
| 496 | 501 | parent.log.info("child forge adopted", { name: record.name, pid: record.pid }); |
| 497 | 502 | void gone.then(() => { |
| 498 | clearRecord(dir); | |
| 503 | drop(); | |
| 499 | 504 | if (held.get(record.bodyId) === child) { |
| 500 | 505 | held.delete(record.bodyId); |
| 501 | 506 | startedAt.delete(record.bodyId); |
| 502 | 507 | } |
| 503 | 508 | }); |
| 504 | 509 | }; |
| 505 | const adopted = Promise.all(readRecords(join(parent.config.dataDir, "children")).map(({ record, dir }) => adopt(record, dir))).then(() => {}); | |
| 510 | // What an older host left in a child's own directory is the child's to write: never read. | |
| 511 | dropLegacyRecords(dataDir, (bodyId) => parent.log.warn("child forge's own cell.json removed unread", { bodyId })); | |
| 512 | const adopted = Promise.all(readRecords(dataDir).map(adopt)).then(() => {}); | |
| 506 | 513 | let adoptionOver = false; |
| 507 | 514 | void adopted.finally(() => (adoptionOver = true)).catch(() => {}); |
| 508 | 515 | |
| @@ -547,7 +554,7 @@ export function makeRegistry(parent: App, launcher?: Launcher): CellChildren { | ||
| 547 | 554 | noteStop(parent, input.bodyId, input.name, why); |
| 548 | 555 | } |
| 549 | 556 | cell.close(); |
| 550 | clearRecord(dir); | |
| 557 | clearRecord(dataDir, input.bodyId); | |
| 551 | 558 | door.close(); |
| 552 | 559 | release(); |
| 553 | 560 | if (why !== undefined) recent.push(why); |
| @@ -556,7 +563,7 @@ export function makeRegistry(parent: App, launcher?: Launcher): CellChildren { | ||
| 556 | 563 | cell.watch(proc.pid, () => killTree(proc.pid)); |
| 557 | 564 | // Written before anything awaits, so the exit hook above (which clears it) can only come |
| 558 | 565 | // after; a host that dies before the cell answers leaves a record the next one reaps. |
| 559 | leaveRecord(dir, cell.supervised, { bodyId: input.bodyId, pid: proc.pid, port, startedAt: Date.now(), name: input.name, size: input.size }, (why) => | |
| 566 | leaveRecord(dataDir, cell.supervised, { bodyId: input.bodyId, pid: proc.pid, port, startedAt: Date.now(), name: input.name, size: input.size }, (why) => | |
| 560 | 567 | parent.log.warn("child forge record not written", { name: input.name, why }), |
| 561 | 568 | ); |
| 562 | 569 | const child: Held = { ...input, port, limits: spec.limits, pid: proc.pid, exited, cell, proc, door }; |
src/spawn/cell_record.ts +77 −22
| @@ -1,15 +1,25 @@ | ||
| 1 | 1 | /** |
| 2 | 2 | * What a cell that outlives its host leaves for the next host to find (#13756): one small file |
| 3 | * under the child's directory saying which process is the cell, where it answers and which | |
| 4 | * supervisor holds its proxy. Plain data in, plain data out; `cell.ts` writes it when a cell | |
| 5 | * answers and reads it at boot to adopt the live ones. | |
| 3 | * saying which process is the cell, where it answers and which supervisor holds its proxy. Plain | |
| 4 | * data in, plain data out; `cell.ts` writes it when a cell answers and reads it at boot to adopt | |
| 5 | * the live ones. | |
| 6 | * | |
| 7 | * The records live in `<data>/cell-records/<body>.json` (#15235): a directory the parent owns and | |
| 8 | * binds into no cell. They used to sit in the child's own directory, which its cell writes, so a | |
| 9 | * child could name any port and pid for its parent to adopt or signal; `dropLegacyRecords` | |
| 10 | * removes those, unread. | |
| 6 | 11 | */ |
| 7 | 12 | |
| 8 | import { mkdirSync, readdirSync, readFileSync, renameSync, rmSync, writeFileSync } from "node:fs"; | |
| 13 | import { lstatSync, mkdirSync, readdirSync, readFileSync, renameSync, rmSync, writeFileSync } from "node:fs"; | |
| 9 | 14 | import { join } from "node:path"; |
| 10 | 15 | import { SIZES, type Size } from "./sizes.ts"; |
| 16 | import { validBodyId } from "./disk_mount.ts"; | |
| 11 | 17 | |
| 12 | export const RECORD_FILE = "cell.json"; | |
| 18 | /** The directory under the parent's data the records live in, one file a body. */ | |
| 19 | export const RECORDS_DIR = "cell-records"; | |
| 20 | ||
| 21 | /** The file an older host left in a child's own directory, which the child could write. */ | |
| 22 | export const LEGACY_RECORD_FILE = "cell.json"; | |
| 13 | 23 | |
| 14 | 24 | export interface CellRecord { |
| 15 | 25 | readonly bodyId: string; |
| @@ -36,12 +46,18 @@ export function parseRecord(raw: unknown): CellRecord | undefined { | ||
| 36 | 46 | return { bodyId, pid, port, supervisor, startedAt, name, size: size as Size }; |
| 37 | 47 | } |
| 38 | 48 | |
| 39 | export const recordPath = (childDir: string): string => join(childDir, RECORD_FILE); | |
| 49 | export const recordsDir = (dataDir: string): string => join(dataDir, RECORDS_DIR); | |
| 50 | ||
| 51 | /** The record's path for `bodyId`; a body id that could leave the directory is refused (throws). */ | |
| 52 | export function recordPath(dataDir: string, bodyId: string): string { | |
| 53 | if (!validBodyId(bodyId)) throw new Error(`not a body id: ${JSON.stringify(bodyId.slice(0, 80))}`); | |
| 54 | return join(recordsDir(dataDir), `${bodyId}.json`); | |
| 55 | } | |
| 40 | 56 | |
| 41 | 57 | /** Write the record whole, through a rename, so a reader never sees half of one. */ |
| 42 | export function writeRecord(childDir: string, record: CellRecord): void { | |
| 43 | mkdirSync(childDir, { recursive: true }); | |
| 44 | const to = recordPath(childDir); | |
| 58 | export function writeRecord(dataDir: string, record: CellRecord): void { | |
| 59 | const to = recordPath(dataDir, record.bodyId); | |
| 60 | mkdirSync(recordsDir(dataDir), { recursive: true, mode: 0o700 }); | |
| 45 | 61 | const tmp = `${to}.tmp`; |
| 46 | 62 | writeFileSync(tmp, JSON.stringify(record)); |
| 47 | 63 | renameSync(tmp, to); |
| @@ -51,36 +67,75 @@ export function writeRecord(childDir: string, record: CellRecord): void { | ||
| 51 | 67 | * Leave the record for a cell, only when it is supervised (so it can outlive its host). A write |
| 52 | 68 | * that fails (disk full) is told to `warn`, never thrown: the cell is up either way. |
| 53 | 69 | */ |
| 54 | export function leaveRecord(childDir: string, supervised: { readonly ctl: string } | undefined, record: Omit<CellRecord, "supervisor">, warn: (why: string) => void): void { | |
| 70 | export function leaveRecord(dataDir: string, supervised: { readonly ctl: string } | undefined, record: Omit<CellRecord, "supervisor">, warn: (why: string) => void): void { | |
| 55 | 71 | if (supervised === undefined) return; |
| 56 | 72 | try { |
| 57 | writeRecord(childDir, { ...record, supervisor: supervised.ctl }); | |
| 73 | writeRecord(dataDir, { ...record, supervisor: supervised.ctl }); | |
| 58 | 74 | } catch (error) { |
| 59 | 75 | warn(String(error).slice(0, 200)); |
| 60 | 76 | } |
| 61 | 77 | } |
| 62 | 78 | |
| 63 | export function clearRecord(childDir: string): void { | |
| 64 | rmSync(recordPath(childDir), { force: true }); | |
| 79 | /** Remove the one record of `bodyId`; nothing for a body id that is not one. */ | |
| 80 | export function clearRecord(dataDir: string, bodyId: string): void { | |
| 81 | if (!validBodyId(bodyId)) return; | |
| 82 | rmSync(recordPath(dataDir, bodyId), { force: true }); | |
| 65 | 83 | } |
| 66 | 84 | |
| 67 | /** Every record under `childrenDir` (one directory a body); unreadable ones are skipped. */ | |
| 68 | export function readRecords(childrenDir: string): { record: CellRecord; dir: string }[] { | |
| 85 | /** The record of `bodyId`, when there is one that parses and names that body. */ | |
| 86 | export function readRecord(dataDir: string, bodyId: string): CellRecord | undefined { | |
| 87 | if (!validBodyId(bodyId)) return undefined; | |
| 88 | try { | |
| 89 | const record = parseRecord(JSON.parse(readFileSync(recordPath(dataDir, bodyId), "utf8"))); | |
| 90 | return record?.bodyId === bodyId ? record : undefined; | |
| 91 | } catch { | |
| 92 | return undefined; // no record, or not JSON | |
| 93 | } | |
| 94 | } | |
| 95 | ||
| 96 | /** Every record in the parent's records directory; unreadable ones are skipped. */ | |
| 97 | export function readRecords(dataDir: string): CellRecord[] { | |
| 98 | let names: string[]; | |
| 99 | try { | |
| 100 | names = readdirSync(recordsDir(dataDir)); | |
| 101 | } catch { | |
| 102 | return []; | |
| 103 | } | |
| 104 | const found: CellRecord[] = []; | |
| 105 | for (const name of names) { | |
| 106 | if (!name.endsWith(".json")) continue; | |
| 107 | const record = readRecord(dataDir, name.slice(0, -".json".length)); | |
| 108 | if (record !== undefined) found.push(record); | |
| 109 | } | |
| 110 | return found; | |
| 111 | } | |
| 112 | ||
| 113 | /** | |
| 114 | * Remove every `cell.json` an older host left in a child's own directory | |
| 115 | * (`<data>/children/<body>`), without reading it: the child could have written it. Each removal is | |
| 116 | * told to `removed`. A path that is a directory (or anything rm without recursion refuses) is left | |
| 117 | * alone. | |
| 118 | */ | |
| 119 | export function dropLegacyRecords(dataDir: string, removed: (bodyId: string) => void): void { | |
| 120 | const childrenDir = join(dataDir, "children"); | |
| 69 | 121 | let bodies: string[]; |
| 70 | 122 | try { |
| 71 | 123 | bodies = readdirSync(childrenDir); |
| 72 | 124 | } catch { |
| 73 | return []; | |
| 125 | return; | |
| 74 | 126 | } |
| 75 | const found: { record: CellRecord; dir: string }[] = []; | |
| 76 | 127 | for (const body of bodies) { |
| 77 | const dir = join(childrenDir, body); | |
| 128 | const path = join(childrenDir, body, LEGACY_RECORD_FILE); | |
| 78 | 129 | try { |
| 79 | const record = parseRecord(JSON.parse(readFileSync(recordPath(dir), "utf8"))); | |
| 80 | if (record !== undefined && record.bodyId === body) found.push({ record, dir }); | |
| 130 | lstatSync(path); | |
| 81 | 131 | } catch { |
| 82 | // no record, or not JSON | |
| 132 | continue; // none here | |
| 133 | } | |
| 134 | try { | |
| 135 | rmSync(path); | |
| 136 | removed(body); | |
| 137 | } catch { | |
| 138 | // not a file this can remove | |
| 83 | 139 | } |
| 84 | 140 | } |
| 85 | return found; | |
| 86 | 141 | } |
src/spawn/reach.ts +4 −10
| @@ -28,7 +28,7 @@ import type { FiredAlarm } from "../flow/alarms.ts"; | ||
| 28 | 28 | import { childPublicUrl, parentOrigin } from "./placement.ts"; |
| 29 | 29 | import { uncleanInFirstMinutes } from "../kernel/boots.ts"; |
| 30 | 30 | import { lastStop, stoppedMark } from "./stopped.ts"; |
| 31 | import { parseRecord, recordPath } from "./cell_record.ts"; | |
| 31 | import { readRecord } from "./cell_record.ts"; | |
| 32 | 32 | import { during } from "../perf/jobs.ts"; |
| 33 | 33 | |
| 34 | 34 | export const APP_HEADER = "x-forge-app"; |
| @@ -321,14 +321,8 @@ export async function settleChild(app: App, name: string): Promise<void> { | ||
| 321 | 321 | } |
| 322 | 322 | } |
| 323 | 323 | |
| 324 | /** When the cell record in a child's directory says its cell started, if it has one. */ | |
| 325 | function startedOf(dir: string): number | undefined { | |
| 326 | try { | |
| 327 | return parseRecord(JSON.parse(readFileSync(recordPath(dir), "utf8")))?.startedAt; | |
| 328 | } catch { | |
| 329 | return undefined; | |
| 330 | } | |
| 331 | } | |
| 324 | /** When the child's cell record (`cell_record.ts`) says its cell started, if it has one. */ | |
| 325 | const startedOf = (app: App, bodyId: string): number | undefined => readRecord(app.config.dataDir, bodyId)?.startedAt; | |
| 332 | 326 | |
| 333 | 327 | /** |
| 334 | 328 | * Whether the host stopped or (re)started this child's cell within the settle window: its cell |
| @@ -338,7 +332,7 @@ function startedOf(dir: string): number | undefined { | ||
| 338 | 332 | export function restartedRecently(app: App, child: Pick<Child, "bodyId">, now = Date.now()): boolean { |
| 339 | 333 | const window = optionsOf(app).settleMs ?? SETTLE_MS; |
| 340 | 334 | const dir = childDirOf(app, child.bodyId); |
| 341 | const times = [lastStop(app, child.bodyId, now)?.at, stoppedMark(dir)?.at, startedOf(dir)]; | |
| 335 | const times = [lastStop(app, child.bodyId, now)?.at, stoppedMark(dir)?.at, startedOf(app, child.bodyId)]; | |
| 342 | 336 | return times.some((at) => at !== undefined && now - at >= 0 && now - at < window); |
| 343 | 337 | } |
| 344 | 338 |
src/spawn/supervisor.ts +26
| @@ -191,6 +191,32 @@ export function readPid(path: string): number | undefined { | ||
| 191 | 191 | } |
| 192 | 192 | } |
| 193 | 193 | |
| 194 | /** | |
| 195 | * The pid in a pid file only when it carries its start time and that is still the process's | |
| 196 | * (#15235): stricter than `readPid`, for adopting a cell, where a bare pid could be anyone's. | |
| 197 | */ | |
| 198 | export function readStartedPid(path: string): number | undefined { | |
| 199 | try { | |
| 200 | const [a, start, more] = readFileSync(path, "utf8").trim().split(":"); | |
| 201 | const n = Number(a); | |
| 202 | if (!Number.isInteger(n) || n <= 0 || start === undefined || start === "" || more !== undefined) return undefined; | |
| 203 | return startTime(n) === start ? n : undefined; | |
| 204 | } catch { | |
| 205 | return undefined; | |
| 206 | } | |
| 207 | } | |
| 208 | ||
| 209 | /** | |
| 210 | * The port the supervisor in `ctl` relays the cell's way in on (`config.json` `listen`), if any. | |
| 211 | */ | |
| 212 | export function readListen(ctl: string): number | undefined { | |
| 213 | try { | |
| 214 | return parseConfig(readFileSync(files(ctl).config, "utf8"))?.listen; | |
| 215 | } catch { | |
| 216 | return undefined; | |
| 217 | } | |
| 218 | } | |
| 219 | ||
| 194 | 220 | /** The entries of `allowed.json`, only ever strings; what is not is ignored. */ |
| 195 | 221 | export function readAllowed(path: string): string[] { |
| 196 | 222 | try { |
test/cell-adopted-on-start.test.ts +145 −50
| @@ -1,12 +1,12 @@ | ||
| 1 | 1 | import { expect, test } from "bun:test"; |
| 2 | 2 | import type { Ctx } from "../src/http/context.ts"; |
| 3 | import { existsSync, mkdirSync, mkdtempSync, writeFileSync } from "node:fs"; | |
| 3 | import { existsSync, mkdirSync, mkdtempSync, renameSync, writeFileSync } from "node:fs"; | |
| 4 | 4 | import { tmpdir } from "node:os"; |
| 5 | 5 | import { join } from "node:path"; |
| 6 | 6 | import { createApp } from "../src/app.ts"; |
| 7 | 7 | import { makeRegistry } from "../src/spawn/cell.ts"; |
| 8 | import { files } from "../src/spawn/supervisor.ts"; | |
| 9 | import { clearRecord, leaveRecord, parseRecord, readRecords, recordPath, writeRecord, type CellRecord } from "../src/spawn/cell_record.ts"; | |
| 8 | import { files, pidRecord } from "../src/spawn/supervisor.ts"; | |
| 9 | import { clearRecord, leaveRecord, parseRecord, readRecord, readRecords, recordPath, writeRecord, type CellRecord } from "../src/spawn/cell_record.ts"; | |
| 10 | 10 | |
| 11 | 11 | const recordFor = (over: Partial<CellRecord>): CellRecord => ({ bodyId: "cell-a", pid: process.pid, port: 1, supervisor: "/x", startedAt: 1, name: "a", size: "small", ...over }); |
| 12 | 12 | |
| @@ -15,14 +15,24 @@ const setup = (): { app: ReturnType<typeof createApp>; dir: string; childDir: (i | ||
| 15 | 15 | return { app: createApp({ dataDir: dir, port: 0 }), dir, childDir: (id) => join(dir, "children", id) }; |
| 16 | 16 | }; |
| 17 | 17 | |
| 18 | /** | |
| 19 | * A supervisor's control directory as `startSupervisor` leaves it: `cell.pid` with the start time | |
| 20 | * (`pidRecord`), and the port it relays the cell's way in on in `config.json`. | |
| 21 | */ | |
| 22 | const supervised = (ctl: string, pid: number, listen?: number): void => { | |
| 23 | mkdirSync(ctl, { recursive: true }); | |
| 24 | writeFileSync(files(ctl).cell, pidRecord(pid)); | |
| 25 | const config = { sockets: join(ctl, "s"), hosts: [], report: false, loops: [], origins: [], ...(listen === undefined ? {} : { listen }) }; | |
| 26 | writeFileSync(files(ctl).config, JSON.stringify(config)); | |
| 27 | }; | |
| 28 | ||
| 18 | 29 | test("a registry built over a live cell adopts it, and get answers", async () => { |
| 19 | const { app, childDir, dir } = setup(); | |
| 30 | const { app, dir } = setup(); | |
| 20 | 31 | const server = Bun.serve({ port: 0, hostname: "127.0.0.1", fetch: () => new Response("ok") }); |
| 21 | 32 | try { |
| 22 | 33 | const ctl = join(dir, "ctl"); |
| 23 | mkdirSync(ctl, { recursive: true }); | |
| 24 | writeFileSync(files(ctl).cell, String(process.pid)); | |
| 25 | writeRecord(childDir("cell-a"), recordFor({ port: server.port as number, supervisor: ctl })); | |
| 34 | supervised(ctl, process.pid, server.port as number); | |
| 35 | writeRecord(dir, recordFor({ port: server.port as number, supervisor: ctl })); | |
| 26 | 36 | const cells = makeRegistry(app); |
| 27 | 37 | await cells.adopted; |
| 28 | 38 | const child = cells.get("cell-a"); |
| @@ -36,23 +46,23 @@ test("a registry built over a live cell adopts it, and get answers", async () => | ||
| 36 | 46 | }); |
| 37 | 47 | |
| 38 | 48 | test("a record whose cell is gone is dropped and nothing is adopted", async () => { |
| 39 | const { app, childDir } = setup(); | |
| 49 | const { app, dir } = setup(); | |
| 40 | 50 | const dead = Bun.spawn(["true"]); |
| 41 | 51 | await dead.exited; |
| 42 | 52 | const server = Bun.serve({ port: 0, hostname: "127.0.0.1", fetch: () => new Response("ok") }); |
| 43 | 53 | const port = server.port as number; |
| 44 | 54 | try { |
| 45 | writeRecord(childDir("cell-dead"), recordFor({ bodyId: "cell-dead", pid: dead.pid, port })); | |
| 55 | writeRecord(dir, recordFor({ bodyId: "cell-dead", pid: dead.pid, port })); | |
| 46 | 56 | // Alive pid, but nothing answers on the port. |
| 47 | 57 | const closed = Bun.serve({ port: 0, hostname: "127.0.0.1", fetch: () => new Response("") }); |
| 48 | 58 | const closedPort = closed.port as number; |
| 49 | 59 | closed.stop(true); |
| 50 | writeRecord(childDir("cell-quiet"), recordFor({ bodyId: "cell-quiet", port: closedPort })); | |
| 60 | writeRecord(dir, recordFor({ bodyId: "cell-quiet", port: closedPort })); | |
| 51 | 61 | const cells = makeRegistry(app); |
| 52 | 62 | await cells.adopted; |
| 53 | 63 | expect(cells.list()).toEqual([]); |
| 54 | expect(existsSync(recordPath(childDir("cell-dead")))).toBe(false); | |
| 55 | expect(existsSync(recordPath(childDir("cell-quiet")))).toBe(false); | |
| 64 | expect(existsSync(recordPath(dir, "cell-dead"))).toBe(false); | |
| 65 | expect(existsSync(recordPath(dir, "cell-quiet"))).toBe(false); | |
| 56 | 66 | } finally { |
| 57 | 67 | server.stop(true); |
| 58 | 68 | app.close(); |
| @@ -65,45 +75,55 @@ test("a record that is not one is never trusted", () => { | ||
| 65 | 75 | expect(parseRecord({ ...recordFor({}), size: "huge" })).toBeUndefined(); |
| 66 | 76 | expect(parseRecord({ ...recordFor({}), pid: "1" })).toBeUndefined(); |
| 67 | 77 | expect(parseRecord(null)).toBeUndefined(); |
| 68 | const { dir, childDir } = setup(); | |
| 69 | mkdirSync(childDir("cell-x"), { recursive: true }); | |
| 70 | writeRecord(childDir("cell-x"), recordFor({ bodyId: "cell-other" })); | |
| 71 | expect(readRecords(join(dir, "children"))).toEqual([]); | |
| 78 | const { dir } = setup(); | |
| 79 | // A record filed under another body's name is not that body's. | |
| 80 | writeRecord(dir, recordFor({ bodyId: "cell-other" })); | |
| 81 | renameSync(recordPath(dir, "cell-other"), recordPath(dir, "cell-x")); | |
| 82 | expect(readRecords(dir)).toEqual([]); | |
| 83 | expect(readRecord(dir, "cell-x")).toBeUndefined(); | |
| 84 | }); | |
| 85 | ||
| 86 | test("a record path takes only a safe body id, and the records sit outside every child's directory", () => { | |
| 87 | const { dir } = setup(); | |
| 88 | expect(recordPath(dir, "cell-a")).toBe(join(dir, "cell-records", "cell-a.json")); | |
| 89 | for (const bad of ["../children/cell-a/x", "..", ".hidden", "a/b", "", "cell a"]) { | |
| 90 | expect(() => recordPath(dir, bad)).toThrow(); | |
| 91 | expect(readRecord(dir, bad)).toBeUndefined(); | |
| 92 | expect(() => clearRecord(dir, bad)).not.toThrow(); | |
| 93 | } | |
| 94 | expect(() => writeRecord(dir, recordFor({ bodyId: "../../escape" }))).toThrow(); | |
| 95 | expect(existsSync(join(dir, "..", "escape.json"))).toBe(false); | |
| 72 | 96 | }); |
| 73 | 97 | |
| 74 | 98 | test("leaveRecord writes only for a supervised cell, and a failed write is told, not thrown", () => { |
| 75 | const { childDir } = setup(); | |
| 99 | const { dir } = setup(); | |
| 76 | 100 | const base = { bodyId: "cell-w", pid: 5, port: 6, startedAt: 7, name: "w", size: "small" as const }; |
| 77 | leaveRecord(childDir("cell-w"), undefined, base, () => {}); | |
| 78 | expect(existsSync(recordPath(childDir("cell-w")))).toBe(false); | |
| 79 | leaveRecord(childDir("cell-w"), { ctl: "/ctl" }, base, () => {}); | |
| 80 | expect(readRecords(join(childDir("cell-w"), ".."))[0]?.record.supervisor).toBe("/ctl"); | |
| 81 | clearRecord(childDir("cell-w")); | |
| 82 | expect(existsSync(recordPath(childDir("cell-w")))).toBe(false); | |
| 101 | leaveRecord(dir, undefined, base, () => {}); | |
| 102 | expect(existsSync(recordPath(dir, "cell-w"))).toBe(false); | |
| 103 | leaveRecord(dir, { ctl: "/ctl" }, base, () => {}); | |
| 104 | expect(readRecords(dir)[0]?.supervisor).toBe("/ctl"); | |
| 105 | clearRecord(dir, "cell-w"); | |
| 106 | expect(existsSync(recordPath(dir, "cell-w"))).toBe(false); | |
| 83 | 107 | const told: string[] = []; |
| 84 | 108 | // a file where the directory should be: the write fails |
| 85 | writeFileSync(join(childDir("cell-w"), "..", "blocked"), "x"); | |
| 86 | leaveRecord(join(childDir("cell-w"), "..", "blocked", "x"), { ctl: "/ctl" }, base, (why) => told.push(why)); | |
| 109 | const blocked = mkdtempSync(join(tmpdir(), "adopt-blocked-")); | |
| 110 | writeFileSync(join(blocked, "cell-records"), "x"); | |
| 111 | leaveRecord(blocked, { ctl: "/ctl" }, base, (why) => told.push(why)); | |
| 87 | 112 | expect(told.length).toBe(1); |
| 88 | 113 | }); |
| 89 | 114 | |
| 90 | const supervised = (ctl: string, pid: number): void => { | |
| 91 | mkdirSync(ctl, { recursive: true }); | |
| 92 | writeFileSync(files(ctl).cell, String(pid)); | |
| 93 | }; | |
| 94 | ||
| 95 | 115 | test("an alive pid its supervisor does not name is not adopted, and not signalled", async () => { |
| 96 | const { app, childDir, dir } = setup(); | |
| 116 | const { app, dir } = setup(); | |
| 97 | 117 | const server = Bun.serve({ port: 0, hostname: "127.0.0.1", fetch: () => new Response("ok") }); |
| 98 | 118 | const other = Bun.spawn(["sleep", "30"]); |
| 99 | 119 | try { |
| 100 | 120 | const ctl = join(dir, "ctl"); |
| 101 | supervised(ctl, other.pid + 1); | |
| 102 | writeRecord(childDir("cell-a"), recordFor({ pid: other.pid, port: server.port as number, supervisor: ctl })); | |
| 121 | supervised(ctl, other.pid + 1, server.port as number); | |
| 122 | writeRecord(dir, recordFor({ pid: other.pid, port: server.port as number, supervisor: ctl })); | |
| 103 | 123 | const cells = makeRegistry(app); |
| 104 | 124 | await cells.adopted; |
| 105 | 125 | expect(cells.list()).toEqual([]); |
| 106 | expect(existsSync(recordPath(childDir("cell-a")))).toBe(false); | |
| 126 | expect(existsSync(recordPath(dir, "cell-a"))).toBe(false); | |
| 107 | 127 | expect(other.killed).toBe(false); |
| 108 | 128 | expect(process.kill(other.pid, 0)).toBe(true); |
| 109 | 129 | } finally { |
| @@ -114,19 +134,19 @@ test("an alive pid its supervisor does not name is not adopted, and not signalle | ||
| 114 | 134 | }); |
| 115 | 135 | |
| 116 | 136 | test("its supervisor's cell that does not answer is stopped, and the record cleared", async () => { |
| 117 | const { app, childDir, dir } = setup(); | |
| 137 | const { app, dir } = setup(); | |
| 118 | 138 | const quiet = Bun.spawn(["sleep", "30"]); |
| 119 | 139 | const closed = Bun.serve({ port: 0, hostname: "127.0.0.1", fetch: () => new Response("") }); |
| 120 | 140 | const port = closed.port as number; |
| 121 | 141 | closed.stop(true); |
| 122 | 142 | try { |
| 123 | 143 | const ctl = join(dir, "ctl"); |
| 124 | supervised(ctl, quiet.pid); | |
| 125 | writeRecord(childDir("cell-a"), recordFor({ pid: quiet.pid, port, supervisor: ctl })); | |
| 144 | supervised(ctl, quiet.pid, port); | |
| 145 | writeRecord(dir, recordFor({ pid: quiet.pid, port, supervisor: ctl })); | |
| 126 | 146 | const cells = makeRegistry(app); |
| 127 | 147 | await cells.adopted; |
| 128 | 148 | expect(cells.list()).toEqual([]); |
| 129 | expect(existsSync(recordPath(childDir("cell-a")))).toBe(false); | |
| 149 | expect(existsSync(recordPath(dir, "cell-a"))).toBe(false); | |
| 130 | 150 | await quiet.exited; |
| 131 | 151 | expect(quiet.signalCode !== null || quiet.exitCode !== null).toBe(true); |
| 132 | 152 | } finally { |
| @@ -136,13 +156,13 @@ test("its supervisor's cell that does not answer is stopped, and the record clea | ||
| 136 | 156 | }); |
| 137 | 157 | |
| 138 | 158 | test("an adopted cell dropped on its death clears its record, and start waits for adoption", async () => { |
| 139 | const { app, childDir, dir } = setup(); | |
| 159 | const { app, dir } = setup(); | |
| 140 | 160 | const server = Bun.serve({ port: 0, hostname: "127.0.0.1", fetch: () => new Response("ok") }); |
| 141 | 161 | const proc = Bun.spawn(["sleep", "30"]); |
| 142 | 162 | try { |
| 143 | 163 | const ctl = join(dir, "ctl"); |
| 144 | supervised(ctl, proc.pid); | |
| 145 | writeRecord(childDir("cell-a"), recordFor({ pid: proc.pid, port: server.port as number, supervisor: ctl })); | |
| 164 | supervised(ctl, proc.pid, server.port as number); | |
| 165 | writeRecord(dir, recordFor({ pid: proc.pid, port: server.port as number, supervisor: ctl })); | |
| 146 | 166 | let launched = 0; |
| 147 | 167 | const cells = makeRegistry(app, async () => { |
| 148 | 168 | launched++; |
| @@ -154,7 +174,7 @@ test("an adopted cell dropped on its death clears its record, and start waits fo | ||
| 154 | 174 | expect(launched).toBe(0); |
| 155 | 175 | proc.kill(); |
| 156 | 176 | await started?.exited; |
| 157 | expect(existsSync(recordPath(childDir("cell-a")))).toBe(false); | |
| 177 | expect(existsSync(recordPath(dir, "cell-a"))).toBe(false); | |
| 158 | 178 | expect(cells.get("cell-a")).toBeUndefined(); |
| 159 | 179 | } finally { |
| 160 | 180 | proc.kill(); |
| @@ -167,7 +187,7 @@ test("the rows say which cells were kept and which stopped, the page and the ter | ||
| 167 | 187 | const { adoptedLines } = await import("../src/tui/screens/system.ts"); |
| 168 | 188 | const { adoptedSection } = await import("../src/web/pages/system.ts"); |
| 169 | 189 | const { adoptionsOf } = await import("../src/spawn/cell.ts"); |
| 170 | const { app, childDir, dir } = setup(); | |
| 190 | const { app, dir } = setup(); | |
| 171 | 191 | const server = Bun.serve({ port: 0, hostname: "127.0.0.1", fetch: () => new Response("ok") }); |
| 172 | 192 | const quiet = Bun.spawn(["sleep", "30"]); |
| 173 | 193 | const closed = Bun.serve({ port: 0, hostname: "127.0.0.1", fetch: () => new Response("") }); |
| @@ -175,11 +195,11 @@ test("the rows say which cells were kept and which stopped, the page and the ter | ||
| 175 | 195 | closed.stop(true); |
| 176 | 196 | try { |
| 177 | 197 | const ctl = join(dir, "ctl"); |
| 178 | supervised(ctl, process.pid); | |
| 179 | writeRecord(childDir("cell-a"), recordFor({ port: server.port as number, supervisor: ctl })); | |
| 198 | supervised(ctl, process.pid, server.port as number); | |
| 199 | writeRecord(dir, recordFor({ port: server.port as number, supervisor: ctl })); | |
| 180 | 200 | const ctlQuiet = join(dir, "ctl-quiet"); |
| 181 | supervised(ctlQuiet, quiet.pid); | |
| 182 | writeRecord(childDir("cell-q"), recordFor({ bodyId: "cell-q", name: "q", pid: quiet.pid, port: closedPort, supervisor: ctlQuiet })); | |
| 201 | supervised(ctlQuiet, quiet.pid, closedPort); | |
| 202 | writeRecord(dir, recordFor({ bodyId: "cell-q", name: "q", pid: quiet.pid, port: closedPort, supervisor: ctlQuiet })); | |
| 183 | 203 | const cells = makeRegistry(app); |
| 184 | 204 | await cells.adopted; |
| 185 | 205 | const rows = cells.adoptions(); |
| @@ -206,12 +226,12 @@ test("the MCP system tool's adopted read gives an admin the rows and refuses any | ||
| 206 | 226 | const { tool } = await import("../src/http/mcp/tools/system.ts"); |
| 207 | 227 | const { cellChildren } = await import("../src/spawn/cell.ts"); |
| 208 | 228 | const { createUser } = await import("../src/core/users.ts"); |
| 209 | const { app, childDir, dir } = setup(); | |
| 229 | const { app, dir } = setup(); | |
| 210 | 230 | const server = Bun.serve({ port: 0, hostname: "127.0.0.1", fetch: () => new Response("ok") }); |
| 211 | 231 | try { |
| 212 | 232 | const ctl = join(dir, "ctl"); |
| 213 | supervised(ctl, process.pid); | |
| 214 | writeRecord(childDir("cell-a"), recordFor({ port: server.port as number, supervisor: ctl })); | |
| 233 | supervised(ctl, process.pid, server.port as number); | |
| 234 | writeRecord(dir, recordFor({ port: server.port as number, supervisor: ctl })); | |
| 215 | 235 | await cellChildren(app).adopted; |
| 216 | 236 | const person = async (username: string, isAdmin: boolean) => { |
| 217 | 237 | const r = await createUser(app.db, { origin: "seed", trusted: true, username, email: `${username}@forge.local`, password: "a-good-passphrase", isAdmin }); |
| @@ -230,3 +250,78 @@ test("the MCP system tool's adopted read gives an admin the rows and refuses any | ||
| 230 | 250 | app.close(); |
| 231 | 251 | } |
| 232 | 252 | }); |
| 253 | ||
| 254 | /** A server standing in for something else on this host: it counts what it is asked. */ | |
| 255 | const victimServer = (): { port: number; asked: () => number; stop: () => void } => { | |
| 256 | let n = 0; | |
| 257 | const server = Bun.serve({ port: 0, hostname: "127.0.0.1", fetch: () => (n++, new Response("victim")) }); | |
| 258 | return { port: server.port as number, asked: () => n, stop: () => server.stop(true) }; | |
| 259 | }; | |
| 260 | ||
| 261 | test("a cell.json a child forged in its own directory is removed unread, and the server it names is never asked (#15235)", async () => { | |
| 262 | const { app, childDir } = setup(); | |
| 263 | const victim = victimServer(); | |
| 264 | try { | |
| 265 | // Everything here is what a child's cell can write: its own directory, a control directory | |
| 266 | // inside it naming a live pid with its true start time, and the victim's port. | |
| 267 | const ctl = join(childDir("cell-a"), "fake-ctl"); | |
| 268 | supervised(ctl, process.pid, victim.port); | |
| 269 | writeFileSync(join(childDir("cell-a"), "cell.json"), JSON.stringify(recordFor({ port: victim.port, supervisor: ctl }))); | |
| 270 | const cells = makeRegistry(app); | |
| 271 | await cells.adopted; | |
| 272 | expect(cells.list()).toEqual([]); | |
| 273 | expect(existsSync(join(childDir("cell-a"), "cell.json"))).toBe(false); | |
| 274 | expect(victim.asked()).toBe(0); | |
| 275 | } finally { | |
| 276 | victim.stop(); | |
| 277 | app.close(); | |
| 278 | } | |
| 279 | }); | |
| 280 | ||
| 281 | test("a record whose port is not its supervisor's is dropped, and that port is never asked (#15235)", async () => { | |
| 282 | const { app, dir } = setup(); | |
| 283 | const victim = victimServer(); | |
| 284 | const cellSide = Bun.serve({ port: 0, hostname: "127.0.0.1", fetch: () => new Response("ok") }); | |
| 285 | try { | |
| 286 | const ctl = join(dir, "ctl"); | |
| 287 | supervised(ctl, process.pid, cellSide.port as number); | |
| 288 | writeRecord(dir, recordFor({ port: victim.port, supervisor: ctl })); | |
| 289 | const cells = makeRegistry(app); | |
| 290 | await cells.adopted; | |
| 291 | expect(cells.list()).toEqual([]); | |
| 292 | expect(cells.adoptions().map((r) => r.outcome)).toEqual(["dropped"]); | |
| 293 | expect(existsSync(recordPath(dir, "cell-a"))).toBe(false); | |
| 294 | expect(victim.asked()).toBe(0); | |
| 295 | } finally { | |
| 296 | cellSide.stop(true); | |
| 297 | victim.stop(); | |
| 298 | app.close(); | |
| 299 | } | |
| 300 | }); | |
| 301 | ||
| 302 | test("a cell.pid with no start time is no cell's: the process it names is never signalled (#15235)", async () => { | |
| 303 | const { app, dir } = setup(); | |
| 304 | // `sleep` runs under `sh`, which says its pid and exits with its status: 143 for the SIGTERM | |
| 305 | // this test sends, 137 had adoption killed the tree (`killTree`, SIGKILL). | |
| 306 | const victim = Bun.spawn(["sh", "-c", "sleep 30 & echo $!; wait $!; exit $?"], { stdout: "pipe" }); | |
| 307 | const said = await victim.stdout.getReader().read(); | |
| 308 | const sleeper = Number(new TextDecoder().decode(said.value).trim()); | |
| 309 | const closed = Bun.serve({ port: 0, hostname: "127.0.0.1", fetch: () => new Response("") }); | |
| 310 | const silentPort = closed.port as number; | |
| 311 | closed.stop(true); | |
| 312 | try { | |
| 313 | const ctl = join(dir, "ctl"); | |
| 314 | supervised(ctl, victim.pid, silentPort); | |
| 315 | // A bare pid: what an older host wrote, or anyone who knows a pid and not its start. | |
| 316 | writeFileSync(files(ctl).cell, String(victim.pid)); | |
| 317 | writeRecord(dir, recordFor({ pid: victim.pid, port: silentPort, supervisor: ctl })); | |
| 318 | const cells = makeRegistry(app); | |
| 319 | await cells.adopted; | |
| 320 | expect(cells.adoptions().map((r) => r.outcome)).toEqual(["dropped"]); | |
| 321 | process.kill(sleeper, "SIGTERM"); | |
| 322 | expect(await victim.exited).toBe(143); | |
| 323 | } finally { | |
| 324 | victim.kill(); | |
| 325 | app.close(); | |
| 326 | } | |
| 327 | }); |
test/spawn-reach.test.ts +3 −6
| @@ -13,6 +13,7 @@ import { listEvents } from "../src/core/events.ts"; | ||
| 13 | 13 | import { clientRows, runningWords, useChildFetch } from "../src/spawn/clients.ts"; |
| 14 | 14 | import { Database } from "bun:sqlite"; |
| 15 | 15 | import { childDirOf } from "../src/actions/spawn.ts"; |
| 16 | import { clearRecord, writeRecord } from "../src/spawn/cell_record.ts"; | |
| 16 | 17 | import { appUrlOf, slugsOf, proveApp, proveHome, reachAlarms, reachOf, reachWords, sweepReach, useReach } from "../src/spawn/reach.ts"; |
| 17 | 18 | import { noteStop, cancelRestarts, lastStop, markStopped } from "../src/spawn/stopped.ts"; |
| 18 | 19 | import { quietLogs } from "./helpers/globals.ts"; |
| @@ -171,17 +172,13 @@ describe("a forge watches its children's addresses", () => { | ||
| 171 | 172 | const reset = () => { |
| 172 | 173 | world.set("https://ok-app--kid--forge.test/", () => new Response("ok", { headers: { "x-forge-app": "ok-app" } })); |
| 173 | 174 | rmSync(join(dir, "reach"), { recursive: true, force: true }); |
| 174 | rmSync(childDirOf(parent, "cell-k") + "/cell.json", { force: true }); | |
| 175 | clearRecord(dir, "cell-k"); | |
| 175 | 176 | rmSync(childDirOf(parent, "cell-k") + "/stopped.json", { force: true }); |
| 176 | 177 | cancelRestarts(parent); |
| 177 | 178 | }; |
| 178 | 179 | const raised = () => events("forge.child_unreachable").length; |
| 179 | 180 | |
| 180 | const startCell = (at: number) => | |
| 181 | writeFileSync( | |
| 182 | join(childDirOf(parent, "cell-k"), "cell.json"), | |
| 183 | JSON.stringify({ bodyId: "cell-k", pid: 1, port: 1, supervisor: "x", startedAt: at, name: "kid", size: "small" }), | |
| 184 | ); | |
| 181 | const startCell = (at: number) => writeRecord(dir, { bodyId: "cell-k", pid: 1, port: 1, supervisor: "x", startedAt: at, name: "kid", size: "small" }); | |
| 185 | 182 | const answers = () => world.set("https://deal-clock--kid--forge.test/", () => new Response("deal", { headers: { "x-forge-app": "deal-clock" } })); |
| 186 | 183 | const keptQuietly = async () => { |
| 187 | 184 | const before = raised(); |
new above
offline