Public: anyone can read this; the crew can change it.
A child forge can run its own fork: the owner switches from the parent's code to their forge repository's main, and back #11182
Check · Review · Land · Live done — Landed and live in production.
scout/forge:scout/11144-a-child-forge-can-run-its-own-fork-the-o → main 8 commits fork
11 files
changed +891
−51
docs/THREATS.md +13
| @@ -186,6 +186,19 @@ How each part proves itself to the next, today. | ||
| 186 | 186 | Gaps: edge → pod and runner → forge ride a shared secret, not a per-call |
| 187 | 187 | signature; child → parent has no credential. |
| 188 | 188 | |
| 189 | ## A child forge running its own code (#11144) | |
| 190 | ||
| 191 | A cell child's owner may switch it from the parent's code to its own `<org>/forge` main (`spawn/own_code.ts`). The code is then the child's author's, so treat it as hostile code in a cell, the same as any app. | |
| 192 | ||
| 193 | What it cannot reach: the cell's memory, CPU, disk and I/O limits, its sandbox and its egress list are the same as with the parent's code, and nothing in the child's repository is read to make them. The parent's data, home and temporary directory are hidden; only the parent's code and the one checkout (a tree, with no `.git`) are revealed, read-only. Its `forge.json` is not run by the parent: the parent refuses a repository or tree over 256 MB or 50,000 files and any symbolic link in the tree, checks that `src/main.ts` parses and `forge.json` is JSON, and never executes the child's code outside the cell. What the parent records about a failed start is fixed wording; no text from the child's files reaches the rollback row or the `forge.rolled_back` event. | |
| 194 | ||
| 195 | What it can reach: everything a child forge can reach, no more: its own data directory, the parent's heartbeat door and public address, the hosts the admin allowed for models, registry and payment. It can lie in what it serves and in its heartbeat, and can break itself: a start that fails to answer within the deadline rolls back to the last commit that came up, then to the parent's code, and the owner's `/forges` page shows a Rolled back row saying so. A commit that failed is not tried again until main moves. A child that comes up and breaks later is not rolled back; the owner switches it off. Hosted children run inside the parent's process and cannot turn this on. | |
| 196 | ||
| 197 | - The cell limits, sandbox, egress list and hidden parent paths are built by the | |
| 198 | parent from outside, never read from the child's repository, so a child's own | |
| 199 | code is held exactly like any app. | |
| 200 | Test: `test/own-code.test.ts` | |
| 201 | ||
| 189 | 202 | ## Honeypots (later) |
| 190 | 203 | |
| 191 | 204 | A `kind: "honeypot"` app, a decoy that records what touches it, is not built. |
docs/features/own-code.md added +34
| @@ -0,0 +1,34 @@ | ||
| 1 | # A forge that runs its own code | |
| 2 | ||
| 3 | A forge you sent runs this forge's code until you switch it. Once switched, it | |
| 4 | starts from its own `<org>/forge` repository's main, so changes made there are | |
| 5 | what it runs. Its limits, sandbox and egress rules are set by this forge, not | |
| 6 | by its code, and they are the same either way. | |
| 7 | ||
| 8 | ## Switch it | |
| 9 | ||
| 10 | - **Page:** open `/forges`. Under **Forge code**, press **Use its own code** | |
| 11 | on the forge's row. Press **Use this forge's code** to switch back. | |
| 12 | - **Terminal:** open the palette, choose **Forges you sent**, press `o` and | |
| 13 | pick the forge. The picker says whose code each forge runs and when it | |
| 14 | last rolled back. | |
| 15 | ||
| 16 | A running forge is put to sleep. Its next visitor wakes it on the new code. | |
| 17 | ||
| 18 | Only a forge in its own cell, with a forge repository, can be switched. A | |
| 19 | forge that shares this forge's process is told so and stays as it is. | |
| 20 | ||
| 21 | ## What a start does | |
| 22 | ||
| 23 | 1. It copies its main into this forge's data, under `own-code/<name>/<commit>`. | |
| 24 | The repository and the copy must each stay under 256 MB and 50,000 files. | |
| 25 | The copy must hold no symbolic links, a `src/main.ts` that parses and a | |
| 26 | `forge.json`. | |
| 27 | 2. It starts the cell on that copy. If the copy fails its check, or the cell | |
| 28 | does not answer in time, it tries the last commit that came up, then this | |
| 29 | forge's code. | |
| 30 | 3. Each fallback is a **Rolled back** row on `/forges` and a | |
| 31 | `forge.rolled_back` event, in fixed words. A commit that failed is not | |
| 32 | tried again until main moves. | |
| 33 | ||
| 34 | How a child running its own code is contained: `docs/THREATS.md`. |
src/core/event_audience.ts +3
| @@ -110,6 +110,9 @@ const READER_KINDS: ReadonlySet<string> = new Set([ | ||
| 110 | 110 | "forge.spawned", |
| 111 | 111 | "forge.removed", |
| 112 | 112 | "forge.stopped", |
| 113 | "forge.own_code_on", | |
| 114 | "forge.own_code_off", | |
| 115 | "forge.rolled_back", | |
| 113 | 116 | "forge.claim_reissued", |
| 114 | 117 | "forge.claim_revoked", |
| 115 | 118 | "forge.claim_restored", |
src/http/server.ts +3 −1
| @@ -272,7 +272,8 @@ import { | ||
| 272 | 272 | import { claimFragmentPage, claimOptions, claimSubmit } from "../web/pages/claim.ts"; |
| 273 | 273 | import { firstAdminOptions, firstAdminPage, firstAdminSubmit } from "../web/pages/first-admin.ts"; |
| 274 | 274 | import { familyPage } from "../web/pages/family.ts"; |
| 275 | import { forgeLinkRevoke, forgeLinkSubmit, forgeLinkUndo, forgeRemoveConfirm, forgeRemoveSubmit, forgeStopSubmit, forgesPage } from "../web/pages/forges.ts"; | |
| 275 | import { forgeLinkRevoke, forgeLinkSubmit, forgeLinkUndo, forgeRemoveConfirm, forgeRemoveSubmit, forgeStopSubmit, | |
| 276 | forgeOwnCodeSubmit, forgesPage } from "../web/pages/forges.ts"; | |
| 276 | 277 | import { forgeItSubmit } from "../web/pages/forge-it.ts"; |
| 277 | 278 | import { newForgePage, newForgeSubmit } from "../web/pages/new-forge.ts"; |
| 278 | 279 | import { askedPage } from "../web/studio/asked_page.ts"; |
| @@ -842,6 +843,7 @@ export function buildRouter(): Router<Handler> { | ||
| 842 | 843 | router.add("POST", "/forges/:name/revoke-link", forgeLinkRevoke); |
| 843 | 844 | router.add("POST", "/forges/:name/undo-link", forgeLinkUndo); |
| 844 | 845 | router.add("POST", "/forges/:name/stop", forgeStopSubmit); |
| 846 | router.add("POST", "/forges/:name/own-code", forgeOwnCodeSubmit); | |
| 845 | 847 | router.add("GET", "/forges/:name/remove", forgeRemoveConfirm); |
| 846 | 848 | router.add("POST", "/forges/:name/remove", forgeRemoveSubmit); |
| 847 | 849 | router.add("GET", "/forges/new", newForgePage); |
src/spawn/cell.ts +68 −47
| @@ -25,6 +25,7 @@ import { floorRefusal } from "./disk_floor.ts"; | ||
| 25 | 25 | import { CGROUP_DIR_ENV, SIZE_ENV } from "./usage.ts"; |
| 26 | 26 | import { childModelHosts, childPaymentHosts, childRegistryHosts } from "../core/capacity.ts"; |
| 27 | 27 | import { liveChildDisk, type ChildDisk } from "./disk_mount.ts"; |
| 28 | import { oneAtATime, sourcesFor, startFirstThatWorks, useTree, type CodeSource } from "./own_code.ts"; | |
| 28 | 29 | |
| 29 | 30 | export const CELL_PREFIX = "cell-"; |
| 30 | 31 | export const ownsBody = (bodyId: string): boolean => bodyId.startsWith(CELL_PREFIX); |
| @@ -133,6 +134,8 @@ export function cellSpecFor(input: { | ||
| 133 | 134 | * what it always was. |
| 134 | 135 | */ |
| 135 | 136 | disk?: ChildDisk; |
| 137 | /** A checked copy of the child's own forge code, when it runs that (`own_code.ts`); the parent's code is revealed either way, for its dependencies. */ | |
| 138 | ownRoot?: string; | |
| 136 | 139 | }): CellSpec { |
| 137 | 140 | return { |
| 138 | 141 | dir: input.dir, |
| @@ -144,7 +147,7 @@ export function cellSpecFor(input: { | ||
| 144 | 147 | // data and is bound back, deepest last. |
| 145 | 148 | hide: [input.parentDataDir, input.parentHome ?? process.env["HOME"] ?? "", tmpdir()].filter((path) => path !== ""), |
| 146 | 149 | // The forge's own code may live under the parent's data (a release checkout). |
| 147 | reveal: [CODE], | |
| 150 | reveal: input.ownRoot === undefined ? [CODE] : [CODE, input.ownRoot], | |
| 148 | 151 | egress: [...(input.modelHosts ?? []), ...(input.registryHosts ?? []), ...(input.paymentHosts ?? [])], |
| 149 | 152 | // A child's cell is held to this list whatever cells.egress says: report |
| 150 | 153 | // mode is for apps still declaring hosts, not for a whole forge. |
| @@ -282,6 +285,7 @@ export function launchSpecFor( | ||
| 282 | 285 | input: { readonly bodyId: string; readonly name: string; readonly size: Size }, |
| 283 | 286 | at: { readonly port: number; readonly parentPort: number }, |
| 284 | 287 | mountinfo?: string, |
| 288 | ownRoot?: string, | |
| 285 | 289 | ): CellSpec { |
| 286 | 290 | return cellSpecFor({ |
| 287 | 291 | dir: childDirOf(parent, input.bodyId), |
| @@ -297,6 +301,7 @@ export function launchSpecFor( | ||
| 297 | 301 | registryHosts: childRegistryHosts(parent.db), |
| 298 | 302 | paymentHosts: childPaymentHosts(parent.db), |
| 299 | 303 | disk: liveChildDisk(parent.config.dataDir, input.bodyId, mountinfo), |
| 304 | ...(ownRoot === undefined ? {} : { ownRoot }), | |
| 300 | 305 | }); |
| 301 | 306 | } |
| 302 | 307 | |
| @@ -315,6 +320,59 @@ export function makeRegistry(parent: App, launcher?: Launcher): CellChildren { | ||
| 315 | 320 | const busy = new Map<string, number>(); |
| 316 | 321 | |
| 317 | 322 | const failOf = (bodyId: string) => (why: string): void => void failed.set(bodyId, { at: Date.now(), why }); |
| 323 | const startOn = async (input: { bodyId: string; name: string; size: Size }, source: CodeSource): Promise<{ child: CellChild } | { why: string }> => { | |
| 324 | const recent: string[] = []; | |
| 325 | let why: string | undefined; | |
| 326 | const port = freePort(); | |
| 327 | const dir = childDirOf(parent, input.bodyId); | |
| 328 | const url = childPublicUrl(input.name, parentOrigin(parent.config)); | |
| 329 | const door = openBeatDoor(parent); | |
| 330 | const parentPort = door.port; | |
| 331 | const spec = launchSpecFor(parent, input, { port, parentPort }, undefined, source.kind === "parent" ? undefined : source.root); | |
| 332 | const heard = childLog((line) => parent.log.info("child forge", { name: input.name, line })); | |
| 333 | const log = (raw: string): void => { | |
| 334 | const line = plainLine(raw); | |
| 335 | recent.push(line); | |
| 336 | if (recent.length > 5) recent.shift(); | |
| 337 | heard(line); | |
| 338 | }; | |
| 339 | // Its tree is not pruned while this cell may read it. | |
| 340 | const release = source.kind === "parent" ? () => {} : useTree(source.root); | |
| 341 | let cell: Cell; | |
| 342 | try { | |
| 343 | cell = openCell(spec, log); | |
| 344 | } catch (error) { | |
| 345 | door.close(); | |
| 346 | release(); | |
| 347 | throw error; | |
| 348 | } | |
| 349 | const command = `exec ${JSON.stringify(process.execPath)} ${JSON.stringify(source.main)}`; | |
| 350 | const proc = Bun.spawn(cell.argv(command), { | |
| 351 | cwd: dirname(source.main), | |
| 352 | env: cell.env(cellEnv({ port, dataDir: dir, url, parentPort, size: input.size, ...(cell.cgroupDir === undefined ? {} : { cgroupDir: cell.cgroupDir }) })), | |
| 353 | stdin: "ignore", | |
| 354 | stdout: "pipe", | |
| 355 | stderr: "pipe", | |
| 356 | }); | |
| 357 | const exited = proc.exited.then((code) => { | |
| 358 | const why = cell.breach(); | |
| 359 | if (why !== undefined) parent.log.warn("child forge stopped", { name: input.name, why }); | |
| 360 | cell.close(); | |
| 361 | door.close(); | |
| 362 | release(); | |
| 363 | if (why !== undefined) recent.push(why); | |
| 364 | return code; | |
| 365 | }); | |
| 366 | cell.watch(proc.pid, () => killTree(proc.pid)); | |
| 367 | void drain(proc.stdout, log); | |
| 368 | void drain(proc.stderr, log); | |
| 369 | const child: Held = { ...input, port, limits: spec.limits, pid: proc.pid, exited, cell, proc, door }; | |
| 370 | const up = await firstAnswer(child, port, () => { | |
| 371 | why = `its cell did not come up: ${recent.slice(-3).join(" | ") || "no output"}`.slice(0, 400); | |
| 372 | parent.log.warn("child forge did not answer", { name: input.name, why }); | |
| 373 | }); | |
| 374 | return up ? { child } : { why: why ?? "its cell did not come up" }; | |
| 375 | }; | |
| 318 | 376 | const launchInCell: Launcher = async (input, fail) => { |
| 319 | 377 | // Below the disk floor no child cell starts, a sleeping one included (#10850). |
| 320 | 378 | const floor = floorRefusal(parent.config.dataDir); |
| @@ -329,53 +387,16 @@ export function makeRegistry(parent: App, launcher?: Launcher): CellChildren { | ||
| 329 | 387 | fail(cells.reason); |
| 330 | 388 | return undefined; |
| 331 | 389 | } |
| 332 | const recent: string[] = []; | |
| 333 | const port = freePort(); | |
| 334 | const dir = childDirOf(parent, input.bodyId); | |
| 335 | const url = childPublicUrl(input.name, parentOrigin(parent.config)); | |
| 336 | const door = openBeatDoor(parent); | |
| 337 | const parentPort = door.port; | |
| 338 | const spec = launchSpecFor(parent, input, { port, parentPort }); | |
| 339 | const heard = childLog((line) => parent.log.info("child forge", { name: input.name, line })); | |
| 340 | const log = (raw: string): void => { | |
| 341 | const line = plainLine(raw); | |
| 342 | recent.push(line); | |
| 343 | if (recent.length > 5) recent.shift(); | |
| 344 | heard(line); | |
| 345 | }; | |
| 346 | let cell: Cell; | |
| 347 | try { | |
| 348 | cell = openCell(spec, log); | |
| 349 | } catch (error) { | |
| 350 | door.close(); | |
| 351 | throw error; | |
| 390 | // The parent decides what the cell runs: its own code, the last that came up, or the parent's (`own_code.ts`). | |
| 391 | const started = await oneAtATime(parent, input.name, async () => { | |
| 392 | const sources = await sourcesFor(parent, { name: input.name, childDir: childDirOf(parent, input.bodyId) }, { main: MAIN, root: CODE }); | |
| 393 | return startFirstThatWorks(parent, input.name, sources, (source) => startOn(input, source)); | |
| 394 | }); | |
| 395 | if ("why" in started) { | |
| 396 | fail(started.why); | |
| 397 | return undefined; | |
| 352 | 398 | } |
| 353 | const command = `exec ${JSON.stringify(process.execPath)} ${JSON.stringify(MAIN)}`; | |
| 354 | const proc = Bun.spawn(cell.argv(command), { | |
| 355 | cwd: dirname(MAIN), | |
| 356 | env: cell.env(cellEnv({ port, dataDir: dir, url, parentPort, size: input.size, ...(cell.cgroupDir === undefined ? {} : { cgroupDir: cell.cgroupDir }) })), | |
| 357 | stdin: "ignore", | |
| 358 | stdout: "pipe", | |
| 359 | stderr: "pipe", | |
| 360 | }); | |
| 361 | const exited = proc.exited.then((code) => { | |
| 362 | const why = cell.breach(); | |
| 363 | if (why !== undefined) parent.log.warn("child forge stopped", { name: input.name, why }); | |
| 364 | cell.close(); | |
| 365 | door.close(); | |
| 366 | if (why !== undefined) recent.push(why); | |
| 367 | return code; | |
| 368 | }); | |
| 369 | cell.watch(proc.pid, () => killTree(proc.pid)); | |
| 370 | void drain(proc.stdout, log); | |
| 371 | void drain(proc.stderr, log); | |
| 372 | const child: Held = { ...input, port, limits: spec.limits, pid: proc.pid, exited, cell, proc, door }; | |
| 373 | const up = await firstAnswer(child, port, () => { | |
| 374 | const why = `its cell did not come up: ${recent.slice(-3).join(" | ") || "no output"}`.slice(0, 400); | |
| 375 | parent.log.warn("child forge did not answer", { name: input.name, why }); | |
| 376 | fail(why); | |
| 377 | }); | |
| 378 | return up ? child : undefined; | |
| 399 | return started.child; | |
| 379 | 400 | }; |
| 380 | 401 | const launch = async (input: { bodyId: string; name: string; size: Size }): Promise<CellChild | undefined> => { |
| 381 | 402 | const child = await (launcher ?? launchInCell)(input, failOf(input.bodyId)); |
src/spawn/manage.ts +14
| @@ -13,6 +13,7 @@ import { restoreClaim, revokeClaim, spawnedOrg } from "./claim.ts"; | ||
| 13 | 13 | import { forgetHeard } from "./clients.ts"; |
| 14 | 14 | import { sleepChild } from "./host.ts"; |
| 15 | 15 | import { CHILD_NAME } from "./placement.ts"; |
| 16 | import { switchOwnCode } from "./own_code.ts"; | |
| 16 | 17 | |
| 17 | 18 | export type Managed = { readonly ok: true; readonly says: string } | { readonly ok: false; readonly says: string }; |
| 18 | 19 | |
| @@ -69,3 +70,16 @@ export async function changeLink( | ||
| 69 | 70 | says: act === "revoke" ? `The link for ${name} has ended. You can undo this for a minute.` : `The link for ${name} is open again.`, |
| 70 | 71 | }; |
| 71 | 72 | } |
| 73 | ||
| 74 | /** | |
| 75 | * Switch a child between its parent's code and its own forge repository's | |
| 76 | * main (#11144). A running child is put to sleep so its next visitor wakes it | |
| 77 | * on the new code; a hosted one is refused, with the reason. | |
| 78 | */ | |
| 79 | export async function ownCode(app: App, name: string, on: boolean): Promise<Managed> { | |
| 80 | const child = await find(app, name); | |
| 81 | if (child === undefined) return { ok: false, says: `no forge named ${name}` }; | |
| 82 | const switched = switchOwnCode(app, { name, placement: child.placement, childDir: childDirOf(app, child.bodyId), on }); | |
| 83 | if (switched.ok) await sleepChild(app, name); | |
| 84 | return switched; | |
| 85 | } |
src/spawn/own_code.ts added +357
| @@ -0,0 +1,357 @@ | ||
| 1 | /** | |
| 2 | * A child forge that runs its own code (#11144). A cell child runs the | |
| 3 | * parent's code by default; with this on, the owner has it start from a | |
| 4 | * checked copy of its own `<org>/forge` main instead. One small file per | |
| 5 | * child under the parent's data (`own-code/<name>.json`) holds the switch, | |
| 6 | * the last commit that came up, and the last rollback. The parent, never the | |
| 7 | * child, decides what the cell runs, so the cell's limits, sandbox and egress | |
| 8 | * are built by the same `cellSpecFor` either way. | |
| 9 | * | |
| 10 | * A start is tried in order: the repository's main, the last commit that came | |
| 11 | * up, the parent's code. Each that fails its check or its health deadline is | |
| 12 | * written down as a rollback, and the owner's `/forges` page says so. | |
| 13 | */ | |
| 14 | ||
| 15 | import { existsSync, lstatSync, mkdirSync, readdirSync, readFileSync, renameSync, rmSync, symlinkSync, writeFileSync } from "node:fs"; | |
| 16 | import { join } from "node:path"; | |
| 17 | import type { App } from "../app.ts"; | |
| 18 | import { runGit } from "../git/env.ts"; | |
| 19 | import type { Managed } from "./manage.ts"; | |
| 20 | import { CHILD_NAME, type PlacementKind } from "./placement.ts"; | |
| 21 | ||
| 22 | export interface Rollback { | |
| 23 | readonly at: string; | |
| 24 | /** The commit that failed (short), or empty when there was none. */ | |
| 25 | readonly commit: string; | |
| 26 | readonly why: string; | |
| 27 | /** Where the child ran instead. */ | |
| 28 | readonly to: "last-good" | "parent"; | |
| 29 | } | |
| 30 | ||
| 31 | export interface OwnCode { | |
| 32 | readonly on: boolean; | |
| 33 | /** The last commit of its own code that came up healthy. */ | |
| 34 | readonly good?: string; | |
| 35 | readonly rolledBack?: Rollback; | |
| 36 | /** A commit of its own that failed: not checked out or reported again until main moves. */ | |
| 37 | readonly failed?: string; | |
| 38 | } | |
| 39 | ||
| 40 | /** Code a cell can run: where its entry file is, and the directory it must be able to read. */ | |
| 41 | export interface CodeSource { | |
| 42 | readonly kind: "own" | "last-good" | "parent"; | |
| 43 | readonly commit?: string; | |
| 44 | readonly main: string; | |
| 45 | readonly root: string; | |
| 46 | } | |
| 47 | ||
| 48 | const dirOf = (app: App): string => join(app.config.dataDir, "own-code"); | |
| 49 | const fileOf = (app: App, name: string): string => join(dirOf(app), `${name}.json`); | |
| 50 | ||
| 51 | export function readOwnCode(app: App, name: string): OwnCode { | |
| 52 | if (!CHILD_NAME.test(name)) return { on: false }; | |
| 53 | try { | |
| 54 | const raw = JSON.parse(readFileSync(fileOf(app, name), "utf8")) as Record<string, unknown>; | |
| 55 | const rb = raw["rolledBack"] as Record<string, unknown> | undefined; | |
| 56 | return { | |
| 57 | on: raw["on"] === true, | |
| 58 | ...(typeof raw["good"] === "string" && /^[0-9a-f]{40}$/.test(raw["good"]) ? { good: raw["good"] } : {}), | |
| 59 | ...(typeof raw["failed"] === "string" && /^[0-9a-f]{40}$/.test(raw["failed"]) ? { failed: raw["failed"] } : {}), | |
| 60 | ...(rb !== undefined && typeof rb["at"] === "string" && typeof rb["why"] === "string" | |
| 61 | ? { | |
| 62 | rolledBack: { | |
| 63 | at: rb["at"], | |
| 64 | commit: typeof rb["commit"] === "string" ? rb["commit"] : "", | |
| 65 | why: rb["why"], | |
| 66 | to: rb["to"] === "last-good" ? "last-good" : "parent", | |
| 67 | }, | |
| 68 | } | |
| 69 | : {}), | |
| 70 | }; | |
| 71 | } catch { | |
| 72 | return { on: false }; | |
| 73 | } | |
| 74 | } | |
| 75 | ||
| 76 | function write(app: App, name: string, state: OwnCode): void { | |
| 77 | mkdirSync(dirOf(app), { recursive: true }); | |
| 78 | const path = fileOf(app, name); | |
| 79 | writeFileSync(`${path}.tmp`, JSON.stringify(state)); | |
| 80 | renameSync(`${path}.tmp`, path); | |
| 81 | } | |
| 82 | ||
| 83 | /** The child's own forge repository, found on disk without opening its database: `<childDir>/repos/<org>/forge.git`. */ | |
| 84 | export function ownRepoOf(childDir: string): string | undefined { | |
| 85 | const root = join(childDir, "repos"); | |
| 86 | try { | |
| 87 | for (const org of readdirSync(root).sort()) { | |
| 88 | const dir = join(root, org, "forge.git"); | |
| 89 | if (existsSync(join(dir, "HEAD"))) return dir; | |
| 90 | } | |
| 91 | } catch { | |
| 92 | // no repositories yet | |
| 93 | } | |
| 94 | return undefined; | |
| 95 | } | |
| 96 | ||
| 97 | /** | |
| 98 | * Turn the switch on or off for a named child. Hosted children cannot turn it | |
| 99 | * on, and are told why. `placement` and `childDir` come from the ledger. | |
| 100 | */ | |
| 101 | export function switchOwnCode( | |
| 102 | app: App, | |
| 103 | input: { readonly name: string; readonly placement: PlacementKind; readonly childDir: string; readonly on: boolean }, | |
| 104 | ): Managed { | |
| 105 | const { name, on } = input; | |
| 106 | if (!CHILD_NAME.test(name)) return { ok: false, says: `no forge named ${name}` }; | |
| 107 | const was = readOwnCode(app, name); | |
| 108 | if (on) { | |
| 109 | if (input.placement !== "cell") | |
| 110 | return { | |
| 111 | ok: false, | |
| 112 | says: "This forge runs inside its parent's process, so it can only run the parent's code. A forge in a cell of its own can run its own.", | |
| 113 | }; | |
| 114 | if (ownRepoOf(input.childDir) === undefined) return { ok: false, says: "This forge has no forge repository of its own yet." }; | |
| 115 | write(app, name, { ...was, on: true }); | |
| 116 | } else { | |
| 117 | rmSync(join(dirOf(app), name), { recursive: true, force: true }); | |
| 118 | write(app, name, { on: false }); | |
| 119 | } | |
| 120 | app.bus.append({ kind: on ? "forge.own_code_on" : "forge.own_code_off", payload: { name } }); | |
| 121 | return { ok: true, says: on ? `${name} will run its own forge code from its next start` : `${name} will run its parent's code from its next start` }; | |
| 122 | } | |
| 123 | ||
| 124 | /** The most a child's repository, or its checked-out tree, may hold before the parent copies it. */ | |
| 125 | export const MAX_CODE_BYTES = 256 * 1024 * 1024; | |
| 126 | export const MAX_CODE_FILES = 50_000; | |
| 127 | ||
| 128 | /** | |
| 129 | * Walk a directory without following links: "link" when anything in it is a | |
| 130 | * symlink (the child's files are read by the parent, so a link could name a | |
| 131 | * parent file), "big" past the caps, else undefined. Stops at the first. | |
| 132 | */ | |
| 133 | export function walkTree(dir: string, caps = { bytes: MAX_CODE_BYTES, files: MAX_CODE_FILES }): "link" | "big" | undefined { | |
| 134 | let bytes = 0; | |
| 135 | let files = 0; | |
| 136 | const walk = (at: string): "link" | "big" | undefined => { | |
| 137 | for (const entry of readdirSync(at, { withFileTypes: true })) { | |
| 138 | if (entry.isSymbolicLink()) return "link"; | |
| 139 | const path = join(at, entry.name); | |
| 140 | if (++files > caps.files) return "big"; | |
| 141 | if (entry.isDirectory()) { | |
| 142 | const found = walk(path); | |
| 143 | if (found !== undefined) return found; | |
| 144 | } else if (entry.isFile()) { | |
| 145 | bytes += lstatSync(path).size; | |
| 146 | if (bytes > caps.bytes) return "big"; | |
| 147 | } | |
| 148 | } | |
| 149 | return undefined; | |
| 150 | }; | |
| 151 | return walk(dir); | |
| 152 | } | |
| 153 | ||
| 154 | /** | |
| 155 | * What the tree must hold before a cell is started on it: no symlinks, an | |
| 156 | * entry that parses and a readable forge.json. The answers are fixed words; | |
| 157 | * nothing the child's files say is passed on. | |
| 158 | */ | |
| 159 | export function checkTree(root: string): string | undefined { | |
| 160 | try { | |
| 161 | const found = walkTree(root); | |
| 162 | if (found === "link") return "it contains a symbolic link"; | |
| 163 | if (found === "big") return "it is too large to run"; | |
| 164 | } catch { | |
| 165 | return "its files could not be read"; | |
| 166 | } | |
| 167 | const main = join(root, "src", "main.ts"); | |
| 168 | const regular = (path: string): boolean => { | |
| 169 | try { | |
| 170 | return lstatSync(path).isFile(); | |
| 171 | } catch { | |
| 172 | return false; | |
| 173 | } | |
| 174 | }; | |
| 175 | if (!regular(main)) return "it has no src/main.ts"; | |
| 176 | try { | |
| 177 | new Bun.Transpiler({ loader: "ts" }).transformSync(readFileSync(main, "utf8")); | |
| 178 | } catch { | |
| 179 | return "src/main.ts does not parse"; | |
| 180 | } | |
| 181 | if (!regular(join(root, "forge.json"))) return "forge.json is missing"; | |
| 182 | try { | |
| 183 | const manifest = JSON.parse(readFileSync(join(root, "forge.json"), "utf8")) as unknown; | |
| 184 | if (typeof manifest !== "object" || manifest === null) return "forge.json is not an object"; | |
| 185 | } catch { | |
| 186 | return "forge.json is not JSON"; | |
| 187 | } | |
| 188 | return undefined; | |
| 189 | } | |
| 190 | ||
| 191 | const SHA = /^[0-9a-f]{40}$/; | |
| 192 | ||
| 193 | /** The commit its own repository's main is at, or undefined. */ | |
| 194 | export async function tipOf(repo: string): Promise<string | undefined> { | |
| 195 | const out = await runGit(["rev-parse", "--verify", "HEAD^{commit}"], { cwd: repo, timeoutMs: 10_000 }); | |
| 196 | return out.ok && SHA.test(out.value) ? out.value : undefined; | |
| 197 | } | |
| 198 | ||
| 199 | /** A checked-out copy of `commit` under `own-code/<name>/<commit>`; the failure in words when it cannot be made. */ | |
| 200 | export async function checkout(app: App, name: string, repo: string, commit: string, parentModules?: string): Promise<{ ok: true; root: string } | { ok: false; why: string }> { | |
| 201 | if (!SHA.test(commit)) return { ok: false, why: "not a commit" }; | |
| 202 | const root = join(dirOf(app), name, commit); | |
| 203 | if (existsSync(join(root, ".checked"))) return { ok: true, root }; | |
| 204 | // The child controls its repository's size, and the copy lands in the parent's data: measure before copying. | |
| 205 | try { | |
| 206 | if (walkTree(repo) === "big") return { ok: false, why: "its repository is too large to copy" }; | |
| 207 | } catch { | |
| 208 | return { ok: false, why: "its repository could not be read" }; | |
| 209 | } | |
| 210 | rmSync(root, { recursive: true, force: true }); | |
| 211 | mkdirSync(join(dirOf(app), name), { recursive: true }); | |
| 212 | const cloned = await runGit(["clone", "--quiet", "--no-hardlinks", "--no-checkout", repo, root], { timeoutMs: 120_000 }); | |
| 213 | if (!cloned.ok) return { ok: false, why: "its code could not be copied" }; | |
| 214 | const got = await runGit(["checkout", "--quiet", "--detach", commit], { cwd: root, timeoutMs: 120_000 }); | |
| 215 | if (!got.ok) return { ok: false, why: "its code could not be checked out" }; | |
| 216 | // Nothing of git goes into the cell: only the tree. | |
| 217 | rmSync(join(root, ".git"), { recursive: true, force: true }); | |
| 218 | const bad = checkTree(root); | |
| 219 | if (bad !== undefined) { | |
| 220 | rmSync(root, { recursive: true, force: true }); | |
| 221 | return { ok: false, why: bad }; | |
| 222 | } | |
| 223 | if (parentModules !== undefined && existsSync(parentModules) && !existsSync(join(root, "node_modules"))) | |
| 224 | symlinkSync(parentModules, join(root, "node_modules")); | |
| 225 | writeFileSync(join(root, ".checked"), commit); | |
| 226 | return { ok: true, root }; | |
| 227 | } | |
| 228 | ||
| 229 | /** `full` is the failed commit in full, remembered so a wake does not try it, or report it, again. */ | |
| 230 | export function recordRollback(app: App, name: string, rolled: Rollback, full?: string): void { | |
| 231 | write(app, name, { ...readOwnCode(app, name), rolledBack: rolled, ...(full !== undefined && SHA.test(full) ? { failed: full } : {}) }); | |
| 232 | app.bus.append({ kind: "forge.rolled_back", payload: { name, commit: rolled.commit, to: rolled.to, why: rolled.why } }); | |
| 233 | } | |
| 234 | ||
| 235 | /** Trees a running or starting cell reads from, counted: pruning never removes one. */ | |
| 236 | const inUse = new Map<string, number>(); | |
| 237 | ||
| 238 | /** Mark a tree in use until the returned function is called. */ | |
| 239 | export function useTree(root: string): () => void { | |
| 240 | inUse.set(root, (inUse.get(root) ?? 0) + 1); | |
| 241 | let done = false; | |
| 242 | return () => { | |
| 243 | if (done) return; | |
| 244 | done = true; | |
| 245 | const left = (inUse.get(root) ?? 1) - 1; | |
| 246 | if (left <= 0) inUse.delete(root); | |
| 247 | else inUse.set(root, left); | |
| 248 | }; | |
| 249 | } | |
| 250 | ||
| 251 | export function recordGood(app: App, name: string, commit: string): void { | |
| 252 | const was = readOwnCode(app, name); | |
| 253 | if (was.good === commit) return; | |
| 254 | write(app, name, { ...was, good: commit }); | |
| 255 | // The running commit, the one before it, and any tree a cell still reads are kept. | |
| 256 | const base = join(dirOf(app), name); | |
| 257 | try { | |
| 258 | for (const dir of readdirSync(base)) | |
| 259 | if (dir !== commit && dir !== was.good && !inUse.has(join(base, dir))) rmSync(join(base, dir), { recursive: true, force: true }); | |
| 260 | } catch { | |
| 261 | // nothing to prune | |
| 262 | } | |
| 263 | } | |
| 264 | ||
| 265 | /** | |
| 266 | * The sources a start tries, best first, the parent's last. Checkouts that | |
| 267 | * fail their check are rolled back here, before any cell is made. | |
| 268 | */ | |
| 269 | export async function sourcesFor( | |
| 270 | app: App, | |
| 271 | input: { readonly name: string; readonly childDir: string }, | |
| 272 | parent: { readonly main: string; readonly root: string }, | |
| 273 | ): Promise<CodeSource[]> { | |
| 274 | const fallback: CodeSource = { kind: "parent", ...parent }; | |
| 275 | const state = readOwnCode(app, input.name); | |
| 276 | const repo = ownRepoOf(input.childDir); | |
| 277 | if (!state.on || repo === undefined) return [fallback]; | |
| 278 | const modules = join(parent.root, "node_modules"); | |
| 279 | const out: CodeSource[] = []; | |
| 280 | const tip = await tipOf(repo); | |
| 281 | let broke: { commit: string; why: string } | undefined; | |
| 282 | if (tip === undefined) broke = { commit: "", why: "its main has no commits" }; | |
| 283 | else if (tip === state.failed) { | |
| 284 | // Already failed and reported: wait for main to move. | |
| 285 | } else { | |
| 286 | const made = await checkout(app, input.name, repo, tip, modules); | |
| 287 | if (made.ok) out.push({ kind: "own", commit: tip, main: join(made.root, "src", "main.ts"), root: made.root }); | |
| 288 | else broke = { commit: tip, why: made.why }; | |
| 289 | } | |
| 290 | const good = state.good; | |
| 291 | if (good !== undefined && good !== tip) { | |
| 292 | const goodTree = join(dirOf(app), input.name, good); | |
| 293 | if (existsSync(join(goodTree, ".checked"))) out.push({ kind: "last-good", commit: good, main: join(goodTree, "src", "main.ts"), root: goodTree }); | |
| 294 | } | |
| 295 | if (broke !== undefined) | |
| 296 | recordRollback( | |
| 297 | app, | |
| 298 | input.name, | |
| 299 | { at: new Date().toISOString(), commit: broke.commit.slice(0, 12), why: broke.why, to: out[0]?.kind === "last-good" ? "last-good" : "parent" }, | |
| 300 | broke.commit, | |
| 301 | ); | |
| 302 | return [...out, fallback]; | |
| 303 | } | |
| 304 | ||
| 305 | /** | |
| 306 | * Try each source until one starts. `attempt` starts a cell on a source and | |
| 307 | * says why not on failure. Every source that failed before the one that | |
| 308 | * worked is a rollback. `up` hears the one that worked. | |
| 309 | */ | |
| 310 | export async function startFirstThatWorks<T>( | |
| 311 | app: App, | |
| 312 | name: string, | |
| 313 | sources: readonly CodeSource[], | |
| 314 | attempt: (source: CodeSource) => Promise<{ child: T } | { why: string }>, | |
| 315 | ): Promise<{ child: T; source: CodeSource } | { why: string }> { | |
| 316 | let last = "no code to start"; | |
| 317 | for (let i = 0; i < sources.length; i++) { | |
| 318 | const source = sources[i]; | |
| 319 | if (source === undefined) break; | |
| 320 | let result: { child: T } | { why: string }; | |
| 321 | try { | |
| 322 | result = await attempt(source); | |
| 323 | } catch (error) { | |
| 324 | // A cell that cannot even be made is a failed attempt like any other: the next source is tried. | |
| 325 | app.log.warn("child forge cell could not be made", { name, kind: source.kind, error: String(error) }); | |
| 326 | result = { why: "its cell could not be made" }; | |
| 327 | } | |
| 328 | if ("child" in result) { | |
| 329 | if (source.kind !== "parent" && source.commit !== undefined) recordGood(app, name, source.commit); | |
| 330 | return { child: result.child, source }; | |
| 331 | } | |
| 332 | last = result.why; | |
| 333 | const next = sources[i + 1]; | |
| 334 | if (next !== undefined && source.kind !== "parent") | |
| 335 | recordRollback( | |
| 336 | app, | |
| 337 | name, | |
| 338 | { at: new Date().toISOString(), commit: (source.commit ?? "").slice(0, 12), why: result.why, to: next.kind === "last-good" ? "last-good" : "parent" }, | |
| 339 | source.kind === "own" ? source.commit : undefined, | |
| 340 | ); | |
| 341 | } | |
| 342 | return { why: last }; | |
| 343 | } | |
| 344 | ||
| 345 | /** Starts of one forge run one at a time, so two wakes never check out or prune the same directory together. */ | |
| 346 | const queues = new Map<string, Promise<unknown>>(); | |
| 347 | export function oneAtATime<T>(app: App, name: string, run: () => Promise<T>): Promise<T> { | |
| 348 | const key = `${app.config.dataDir}\0${name}`; | |
| 349 | const before = queues.get(key) ?? Promise.resolve(); | |
| 350 | const mine = before.catch(() => {}).then(run); | |
| 351 | const tail = mine.catch(() => {}); | |
| 352 | queues.set(key, tail); | |
| 353 | void tail.then(() => { | |
| 354 | if (queues.get(key) === tail) queues.delete(key); | |
| 355 | }); | |
| 356 | return mine; | |
| 357 | } |
src/tui/screens/forges.ts +35 −2
| @@ -14,7 +14,9 @@ import { drawFacts, runAsync } from "./common.ts"; | ||
| 14 | 14 | import type { PaletteEntry } from "./entries.ts"; |
| 15 | 15 | import { clientRows } from "../../spawn/clients.ts"; |
| 16 | 16 | import { diskWords } from "../../spawn/disk_mount.ts"; |
| 17 | import { changeLink, removeForge, stopForge } from "../../spawn/manage.ts"; | |
| 17 | import { changeLink, ownCode, removeForge, stopForge } from "../../spawn/manage.ts"; | |
| 18 | import { readOwnCode } from "../../spawn/own_code.ts"; | |
| 19 | import type { App } from "../../app.ts"; | |
| 18 | 20 | import { undoable, type LinkInfo } from "../../spawn/claim.ts"; |
| 19 | 21 | import { confirmScreen, pickerScreen } from "./common.ts"; |
| 20 | 22 | import type { Session } from "../session.ts"; |
| @@ -50,6 +52,18 @@ export function fitLines(lines: [string, string][], room: number): [string, stri | ||
| 50 | 52 | return kept; |
| 51 | 53 | } |
| 52 | 54 | |
| 55 | /** The `o` picker's rows, as /forges's Forge code rows say them: whose code each runs, and its last rollback. */ | |
| 56 | export function ownCodePicks(app: App, rows: readonly ClientRow[]): { id: string; label: string; hint: string }[] { | |
| 57 | return rows | |
| 58 | .filter((r) => r.made) | |
| 59 | .map((r) => { | |
| 60 | const own = readOwnCode(app, r.name); | |
| 61 | const rb = own.rolledBack; | |
| 62 | const back = rb === undefined ? "" : ` · rolled back${rb.commit === "" ? "" : ` from ${rb.commit}`}: ${rb.why}`; | |
| 63 | return { id: r.name, label: r.name, hint: `${own.on ? "its own code" : "this forge's code"}${back}` }; | |
| 64 | }); | |
| 65 | } | |
| 66 | ||
| 53 | 67 | export function forgesScreen(ctx: Ctx, rows: readonly ClientRow[], session?: Session): Screen { |
| 54 | 68 | const lines = forgeLines(rows); |
| 55 | 69 | // Stop and Remove are the page's buttons, through the same actions (`spawn/manage.ts`). |
| @@ -76,9 +90,27 @@ export function forgesScreen(ctx: Ctx, rows: readonly ClientRow[], session?: Ses | ||
| 76 | 90 | }), |
| 77 | 91 | ); |
| 78 | 92 | }; |
| 93 | // "Use its own code" and back: the page's switch, through the same action (`ownCode`), an admin's. | |
| 94 | const switchCode = (): void => { | |
| 95 | const user = session?.user; | |
| 96 | if (session === undefined || user?.isAdmin !== true) return; | |
| 97 | const items = ownCodePicks(session.app, rows); | |
| 98 | if (items.length === 0) return ctx.toast("No forge to switch"); | |
| 99 | ctx.push( | |
| 100 | pickerScreen(ctx, screen, { | |
| 101 | title: "Switch whose code which forge runs?", | |
| 102 | items, | |
| 103 | onPick: (ids) => { | |
| 104 | const name = ids[0]; | |
| 105 | if (name === undefined) return; | |
| 106 | void ownCode(session.app, name, !readOwnCode(session.app, name).on).then((done) => ctx.toast(done.says, done.ok ? "success" : "danger")); | |
| 107 | }, | |
| 108 | }), | |
| 109 | ); | |
| 110 | }; | |
| 79 | 111 | const screen: Screen = { |
| 80 | 112 | title: "Forges you sent", |
| 81 | keys: [binding(["s"], "stop one (sleep now)"), binding(["x"], "remove one"), binding(["l"], "end a link"), binding(["u"], "undo an ended link"), binding(["esc"], "close")], | |
| 113 | keys: [binding(["s"], "stop one (sleep now)"), binding(["x"], "remove one"), binding(["l"], "end a link"), binding(["u"], "undo an ended link"), binding(["o"], "own code or this forge's"), binding(["esc"], "close")], | |
| 82 | 114 | render(area, s) { |
| 83 | 115 | const c = ctx.theme.c; |
| 84 | 116 | const w = Math.min(100, Math.max(30, area.w - 6)); |
| @@ -94,6 +126,7 @@ export function forgesScreen(ctx: Ctx, rows: readonly ClientRow[], session?: Ses | ||
| 94 | 126 | choose("Stop which forge?", (name) => void stopForge(session.app, name).then((done) => ctx.toast(done.says, done.ok ? "success" : "danger"))); |
| 95 | 127 | else if (e.key === "l") changeOne("revoke"); |
| 96 | 128 | else if (e.key === "u") changeOne("undo"); |
| 129 | else if (e.key === "o") switchCode(); | |
| 97 | 130 | else if (e.key === "x" && session !== undefined) |
| 98 | 131 | choose("Remove which forge?", (name) => |
| 99 | 132 | ctx.push( |
src/web/pages/forges.ts +61 −1
| @@ -20,7 +20,8 @@ import { noRoom } from "../../spawn/door.ts"; | ||
| 20 | 20 | import { sizeWords, useWords } from "../../spawn/usage.ts"; |
| 21 | 21 | import { diskWords } from "../../spawn/disk_mount.ts"; |
| 22 | 22 | import { listChildren } from "../../actions/spawn.ts"; |
| 23 | import { removeForge, stopForge, type Managed } from "../../spawn/manage.ts"; | |
| 23 | import { ownCode, removeForge, stopForge, type Managed } from "../../spawn/manage.ts"; | |
| 24 | import { readOwnCode } from "../../spawn/own_code.ts"; | |
| 24 | 25 | import { claimLinkBody } from "./new-forge.ts"; |
| 25 | 26 | |
| 26 | 27 | |
| @@ -177,6 +178,7 @@ export async function forgesPage(ctx: Ctx): Promise<Response> { | ||
| 177 | 178 | ${rows.map((r) => forgeRow(r, Date.now()))} |
| 178 | 179 | </ul> |
| 179 | 180 | </section>`} |
| 181 | ${await ownCodeSection(ctx)} | |
| 180 | 182 | <p class="mt"><a href="/fleet/graph">Fleet graph</a></p> |
| 181 | 183 | </div>`, |
| 182 | 184 | }); |
| @@ -299,3 +301,61 @@ export async function forgeRemoveConfirm(ctx: Ctx): Promise<Response> { | ||
| 299 | 301 | response.headers.set("cache-control", "no-store"); |
| 300 | 302 | return response; |
| 301 | 303 | } |
| 304 | ||
| 305 | const CODE_WORDS = { | |
| 306 | on: "will run its own code from its next start.", | |
| 307 | off: "will run this forge's code from its next start.", | |
| 308 | refused: "stays on this forge's code: only a forge in its own cell, with a forge repository, can run its own.", | |
| 309 | } as const; | |
| 310 | ||
| 311 | /** What `/forges?code=…&forge=…` says after the switch: fixed words, and only for a forge in the list. */ | |
| 312 | function codeReceipt(ctx: Ctx, names: readonly string[]): Html | null { | |
| 313 | const name = ctx.url.searchParams.get("forge") ?? ""; | |
| 314 | const code = ctx.url.searchParams.get("code") ?? ""; | |
| 315 | if (!names.includes(name) || !(code in CODE_WORDS)) return null; | |
| 316 | return html`<p role="status" class="muted" data-receipt>${name} ${CODE_WORDS[code as keyof typeof CODE_WORDS]}</p>`; | |
| 317 | } | |
| 318 | ||
| 319 | /** Each forge's code, one row each: whose code it runs, the switch, and the last rollback. */ | |
| 320 | async function ownCodeSection(ctx: Ctx): Promise<Html | null> { | |
| 321 | const children = (await listChildren(ctx.app)).filter((c) => c.done); | |
| 322 | if (children.length === 0) return null; | |
| 323 | return html`<section class="settings-block" id="forge-code"> | |
| 324 | <h2>Forge code</h2> | |
| 325 | ${codeReceipt(ctx, children.map((c) => c.name))} | |
| 326 | <dl class="status-rows"> | |
| 327 | ${children.map((c) => { | |
| 328 | const own = readOwnCode(ctx.app, c.name); | |
| 329 | const rb = own.rolledBack; | |
| 330 | const state = html`${own.on ? "Its own code" : "This forge's code"}${rb === undefined | |
| 331 | ? null | |
| 332 | : html`<br /><span data-rolled-back>${badge("Rolled back", { tone: "attn" })} ${time(rb.at)}${rb.commit === "" ? null : html` from ${rb.commit}`}: ${rb.why}. Ran ${rb.to === "last-good" ? "its last good commit" : "this forge's code"}.</span>`}`; | |
| 333 | const action = | |
| 334 | c.placement === "cell" | |
| 335 | ? html`<form method="post" action="/forges/${c.name}/own-code" class="inline"> | |
| 336 | <input type="hidden" name="on" value="${own.on ? "off" : "on"}" /> | |
| 337 | ${btn(own.on ? "Use this forge's code" : "Use its own code", { size: "sm", variant: "ghost" })} | |
| 338 | </form>` | |
| 339 | : html`<span class="muted small">Shares this forge's process</span>`; | |
| 340 | return html`<div class="status-row" data-own-code="${own.on ? "on" : "off"}"> | |
| 341 | <dt>${c.name}</dt> | |
| 342 | <dd>${state}</dd> | |
| 343 | <dd class="status-act">${action}</dd> | |
| 344 | </div>`; | |
| 345 | })} | |
| 346 | </dl> | |
| 347 | </section>`; | |
| 348 | } | |
| 349 | ||
| 350 | // POST /forges/:name/own-code — the owner's switch: `on` is "on" or "off"; then back to the list, with a receipt. | |
| 351 | export async function forgeOwnCodeSubmit(ctx: Ctx): Promise<Response> { | |
| 352 | const user = ctx.auth.user; | |
| 353 | if (!user?.isAdmin) return notFoundPage(ctx); | |
| 354 | const form = await readForm(ctx, "admin"); | |
| 355 | if (form instanceof Response) return form; | |
| 356 | const want = form.get("on"); | |
| 357 | if (want !== "on" && want !== "off") return notFoundPage(ctx); | |
| 358 | const name = ctx.params["name"] ?? ""; | |
| 359 | const done = await ownCode(ctx.app, name, want === "on"); | |
| 360 | return redirect(`/forges?code=${done.ok ? want : "refused"}&forge=${encodeURIComponent(name)}#forge-code`); | |
| 361 | } |
test/own-code.test.ts added +293
| @@ -0,0 +1,293 @@ | ||
| 1 | /** | |
| 2 | * A child forge runs its own code (#11144): the switch, the checked copy, the | |
| 3 | * rollback and its card, and the way back. Cells themselves are not needed: | |
| 4 | * `startFirstThatWorks` takes the start as a function. | |
| 5 | */ | |
| 6 | import { afterAll, beforeAll, expect, test } from "bun:test"; | |
| 7 | import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, symlinkSync, writeFileSync } from "node:fs"; | |
| 8 | import { tmpdir } from "node:os"; | |
| 9 | import { join } from "node:path"; | |
| 10 | import { createApp, type App } from "../src/app.ts"; | |
| 11 | import { serve, type Servers } from "../src/http/server.ts"; | |
| 12 | import { adminCookie } from "./helpers/admin.ts"; | |
| 13 | import { quietLogs } from "./helpers/globals.ts"; | |
| 14 | import { cellSpecFor } from "../src/spawn/cell.ts"; | |
| 15 | import { clientRows } from "../src/spawn/clients.ts"; | |
| 16 | import { createForgeApp } from "../src/tui/forge.ts"; | |
| 17 | import { ownCodePicks } from "../src/tui/screens/forges.ts"; | |
| 18 | import { headlessTerminal } from "../src/tui/term/terminal.ts"; | |
| 19 | import { getUserByName } from "../src/core/users.ts"; | |
| 20 | import { settleApp } from "./helpers/tui.ts"; | |
| 21 | import { ownCode } from "../src/spawn/manage.ts"; | |
| 22 | import { checkTree, oneAtATime, readOwnCode, recordGood, sourcesFor, startFirstThatWorks, switchOwnCode, useTree, walkTree, type CodeSource } from "../src/spawn/own_code.ts"; | |
| 23 | ||
| 24 | quietLogs(); | |
| 25 | ||
| 26 | let dataDir: string; | |
| 27 | let app: App; | |
| 28 | let servers: Servers; | |
| 29 | let cookie: string; | |
| 30 | const childDir = () => join(dataDir, "children", "cell-1"); | |
| 31 | const repo = () => join(childDir(), "repos", "kid", "forge.git"); | |
| 32 | const parent = { main: "/code/src/main.ts", root: "/code" }; | |
| 33 | ||
| 34 | function git(args: string[], cwd: string): void { | |
| 35 | const r = Bun.spawnSync(["git", ...args], { cwd, env: { ...process.env, GIT_AUTHOR_NAME: "t", GIT_AUTHOR_EMAIL: "t@t", GIT_COMMITTER_NAME: "t", GIT_COMMITTER_EMAIL: "t@t" } }); | |
| 36 | if (r.exitCode !== 0) throw new Error(r.stderr.toString()); | |
| 37 | } | |
| 38 | ||
| 39 | /** Commit a tree to the child's forge repository's main. */ | |
| 40 | function commit(files: Record<string, string>): string { | |
| 41 | const work = join(dataDir, "work"); | |
| 42 | rmSync(work, { recursive: true, force: true }); | |
| 43 | mkdirSync(work, { recursive: true }); | |
| 44 | if (!existsSync(repo())) { | |
| 45 | mkdirSync(repo(), { recursive: true }); | |
| 46 | git(["init", "--bare", "-b", "main", repo()], dataDir); | |
| 47 | } | |
| 48 | git(["clone", "--quiet", repo(), work], dataDir); | |
| 49 | git(["checkout", "-B", "main"], work); | |
| 50 | for (const [path, text] of Object.entries(files)) { | |
| 51 | mkdirSync(join(work, path, ".."), { recursive: true }); | |
| 52 | rmSync(join(work, path), { force: true }); | |
| 53 | if (text.startsWith("->")) symlinkSync(text.slice(2), join(work, path)); | |
| 54 | else writeFileSync(join(work, path), text); | |
| 55 | } | |
| 56 | git(["add", "-A"], work); | |
| 57 | git(["commit", "--quiet", "-m", "change"], work); | |
| 58 | git(["push", "--quiet", "origin", "main"], work); | |
| 59 | return Bun.spawnSync(["git", "rev-parse", "HEAD"], { cwd: work }).stdout.toString().trim(); | |
| 60 | } | |
| 61 | const tree = (visible: string) => ({ "src/main.ts": `export const visible = ${JSON.stringify(visible)};\n`, "forge.json": "{}" }); | |
| 62 | ||
| 63 | beforeAll(async () => { | |
| 64 | dataDir = mkdtempSync(join(tmpdir(), "own-code-")); | |
| 65 | app = createApp({ dataDir, port: 0 }); | |
| 66 | servers = await serve(app); | |
| 67 | cookie = await adminCookie(app, { username: "alice", email: "[email protected]" }); | |
| 68 | mkdirSync(join(dataDir, "spawn"), { recursive: true }); | |
| 69 | for (const [name, placement] of [["kid", "cell"], ["inner", "hosted"]] as const) | |
| 70 | writeFileSync(join(dataDir, "spawn", `${name}.json`), JSON.stringify({ name, bodyId: name === "kid" ? "cell-1" : "h-1", done: true, placement })); | |
| 71 | }); | |
| 72 | afterAll(async () => { | |
| 73 | await servers.stop?.(); | |
| 74 | app.close(); | |
| 75 | rmSync(dataDir, { recursive: true, force: true }); | |
| 76 | }); | |
| 77 | ||
| 78 | const sources = (name = "kid") => sourcesFor(app, { name, childDir: childDir() }, parent); | |
| 79 | const ok = (s: { kind: string }) => async () => ({ child: s.kind }); | |
| 80 | /** The i-th source, or a failure that names it. */ | |
| 81 | function nth(found: readonly CodeSource[], i: number): CodeSource { | |
| 82 | const s = found[i]; | |
| 83 | if (s === undefined) throw new Error(`no source ${i} in ${found.map((f) => f.kind).join(",")}`); | |
| 84 | return s; | |
| 85 | } | |
| 86 | /** Payloads of the events of one kind, oldest first. */ | |
| 87 | const events = (kind: string): Record<string, unknown>[] => | |
| 88 | (app.db.query("SELECT payload FROM event WHERE kind = ? ORDER BY rowid").all(kind) as { payload: string }[]).map((r) => JSON.parse(r.payload) as Record<string, unknown>); | |
| 89 | ||
| 90 | test("off by default: the parent's code, and a hosted forge cannot turn it on", async () => { | |
| 91 | expect(readOwnCode(app, "kid")).toEqual({ on: false }); | |
| 92 | expect((await sources()).map((s) => s.kind)).toEqual(["parent"]); | |
| 93 | const hosted = await ownCode(app, "inner", true); | |
| 94 | expect(hosted.ok).toBe(false); | |
| 95 | expect(hosted.says).toContain("inside its parent's process"); | |
| 96 | // A cell forge with no repository of its own is told so, too. | |
| 97 | const none = await ownCode(app, "kid", true); | |
| 98 | expect(none).toEqual({ ok: false, says: "This forge has no forge repository of its own yet." }); | |
| 99 | }); | |
| 100 | ||
| 101 | test("on: it starts from a checked copy of its own main, and what it shows is its change", async () => { | |
| 102 | commit(tree("v1")); | |
| 103 | expect((await ownCode(app, "kid", true)).ok).toBe(true); | |
| 104 | expect(events("forge.own_code_on")).toEqual([{ name: "kid" }]); | |
| 105 | const found = await sources(); | |
| 106 | expect(found.map((s) => s.kind)).toEqual(["own", "parent"]); | |
| 107 | const own = nth(found, 0); | |
| 108 | expect(readFileSync(own.main, "utf8")).toContain('"v1"'); | |
| 109 | expect(existsSync(join(own.root, ".git"))).toBe(false); | |
| 110 | const started = await startFirstThatWorks(app, "kid", found, ok(own)); | |
| 111 | expect("child" in started && started.source.kind).toBe("own"); | |
| 112 | expect(readOwnCode(app, "kid").good).toBe(own.commit); | |
| 113 | expect(readOwnCode(app, "kid").rolledBack).toBeUndefined(); | |
| 114 | }); | |
| 115 | ||
| 116 | test("the cell is the same either way: its own code only adds a revealed tree", () => { | |
| 117 | const base = { dir: "/d/c", parentDataDir: "/d", parentUrl: "https://f.test", size: "small" as const, port: 4100, label: "forge:kid" }; | |
| 118 | const a = cellSpecFor(base); | |
| 119 | const b = cellSpecFor({ ...base, ownRoot: "/d/own-code/kid/abc" }); | |
| 120 | expect(JSON.parse(JSON.stringify({ ...b, reveal: a.reveal }))).toEqual(JSON.parse(JSON.stringify(a))); | |
| 121 | expect(b.reveal).toContain("/d/own-code/kid/abc"); | |
| 122 | }); | |
| 123 | ||
| 124 | test("a broken main is rolled back to the last good commit, with a card", async () => { | |
| 125 | const good = readOwnCode(app, "kid").good; | |
| 126 | expect(good).toBeDefined(); | |
| 127 | commit({ ...tree("v2"), "src/main.ts": "export const = ;;{" }); | |
| 128 | const found = await sources(); | |
| 129 | expect(found.map((s) => s.kind)).toEqual(["last-good", "parent"]); | |
| 130 | expect(nth(found, 0).commit).toBe(good); | |
| 131 | const state = readOwnCode(app, "kid"); | |
| 132 | expect(state.rolledBack?.to).toBe("last-good"); | |
| 133 | expect(state.rolledBack?.why).toContain("does not parse"); | |
| 134 | const page = await (await fetch(`${servers.url}/forges`, { headers: { cookie } })).text(); | |
| 135 | expect(page).toContain("data-rolled-back"); | |
| 136 | expect(page).toContain("its last good commit"); | |
| 137 | expect(page).toContain("Shares this forge's process"); | |
| 138 | expect(events("forge.rolled_back").at(-1)).toMatchObject({ name: "kid", to: "last-good", why: "src/main.ts does not parse" }); | |
| 139 | }); | |
| 140 | ||
| 141 | test("a failed commit is not checked out or reported again until main moves", async () => { | |
| 142 | const before = events("forge.rolled_back").length; | |
| 143 | expect((await sources()).map((s) => s.kind)).toEqual(["last-good", "parent"]); | |
| 144 | expect(events("forge.rolled_back").length).toBe(before); | |
| 145 | }); | |
| 146 | ||
| 147 | test("a symlinked main or forge.json is refused, and none of the target is repeated", async () => { | |
| 148 | const secret = join(dataDir, "secret.key"); | |
| 149 | writeFileSync(secret, "TOP-SECRET-VALUE {{"); | |
| 150 | for (const link of ["src/main.ts", "forge.json"]) { | |
| 151 | commit({ ...tree("v3"), [link]: `->${secret}` }); | |
| 152 | await sources(); | |
| 153 | const state = readOwnCode(app, "kid"); | |
| 154 | expect(state.rolledBack?.why).toBe("it contains a symbolic link"); | |
| 155 | expect(JSON.stringify(state)).not.toContain("TOP-SECRET"); | |
| 156 | } | |
| 157 | commit(tree("v4")); | |
| 158 | }); | |
| 159 | ||
| 160 | test("a main that parses but never answers falls back, then to the parent's code", async () => { | |
| 161 | commit(tree("v3")); | |
| 162 | const found = await sources(); | |
| 163 | expect(found.map((s) => s.kind)).toEqual(["own", "last-good", "parent"]); | |
| 164 | const tried: string[] = []; | |
| 165 | const started = await startFirstThatWorks(app, "kid", found, async (s) => { | |
| 166 | tried.push(s.kind); | |
| 167 | return s.kind === "parent" ? { child: "up" } : { why: "its cell did not come up: boom" }; | |
| 168 | }); | |
| 169 | expect(tried).toEqual(["own", "last-good", "parent"]); | |
| 170 | expect("child" in started && started.source.kind).toBe("parent"); | |
| 171 | const rb = readOwnCode(app, "kid").rolledBack; | |
| 172 | expect(rb?.to).toBe("parent"); | |
| 173 | expect(rb?.why).toContain("boom"); | |
| 174 | const last = events("forge.rolled_back").slice(-2); | |
| 175 | expect(last.map((e) => e["to"])).toEqual(["last-good", "parent"]); | |
| 176 | }); | |
| 177 | ||
| 178 | test("a cell that cannot be made is a failed attempt: the next source is tried", async () => { | |
| 179 | const found: CodeSource[] = [{ kind: "last-good", commit: "a".repeat(40), main: "/x/src/main.ts", root: "/x" }, { kind: "parent", ...parent }]; | |
| 180 | const started = await startFirstThatWorks(app, "kid", found, async (s) => { | |
| 181 | if (s.kind !== "parent") throw new Error("cgroup refused"); | |
| 182 | return { child: "up" }; | |
| 183 | }); | |
| 184 | expect("child" in started && started.source.kind).toBe("parent"); | |
| 185 | expect(readOwnCode(app, "kid").rolledBack?.why).toBe("its cell could not be made"); | |
| 186 | }); | |
| 187 | ||
| 188 | test("pruning after a good start keeps any tree a cell still reads", () => { | |
| 189 | const base = join(dataDir, "own-code", "kid"); | |
| 190 | const held = join(base, "b".repeat(40)); | |
| 191 | const loose = join(base, "c".repeat(40)); | |
| 192 | for (const d of [held, loose]) mkdirSync(d, { recursive: true }); | |
| 193 | const release = useTree(held); | |
| 194 | recordGood(app, "kid", "d".repeat(40)); | |
| 195 | expect(existsSync(held)).toBe(true); | |
| 196 | expect(existsSync(loose)).toBe(false); | |
| 197 | release(); | |
| 198 | recordGood(app, "kid", "e".repeat(40)); | |
| 199 | expect(existsSync(held)).toBe(false); | |
| 200 | }); | |
| 201 | ||
| 202 | test("starts of one forge run one at a time", async () => { | |
| 203 | const order: string[] = []; | |
| 204 | const slow = (tag: string, ms: number) => async () => { | |
| 205 | order.push(`${tag}+`); | |
| 206 | await Bun.sleep(ms); | |
| 207 | order.push(`${tag}-`); | |
| 208 | }; | |
| 209 | await Promise.all([oneAtATime(app, "kid", slow("a", 30)), oneAtATime(app, "kid", slow("b", 1))]); | |
| 210 | expect(order).toEqual(["a+", "a-", "b+", "b-"]); | |
| 211 | }); | |
| 212 | ||
| 213 | test("a repository or tree past the caps is too large", () => { | |
| 214 | const dir = mkdtempSync(join(tmpdir(), "own-big-")); | |
| 215 | writeFileSync(join(dir, "a"), "x".repeat(100)); | |
| 216 | expect(walkTree(dir, { bytes: 50, files: 10 })).toBe("big"); | |
| 217 | expect(walkTree(dir, { bytes: 500, files: 10 })).toBeUndefined(); | |
| 218 | rmSync(dir, { recursive: true, force: true }); | |
| 219 | }); | |
| 220 | ||
| 221 | test("a start that fails on every source says why", async () => { | |
| 222 | const found = await sources(); | |
| 223 | const started = await startFirstThatWorks(app, "kid", found, async () => ({ why: "nope" })); | |
| 224 | expect(started).toEqual({ why: "nope" }); | |
| 225 | }); | |
| 226 | ||
| 227 | test("off returns it to the parent's code and clears what was kept", async () => { | |
| 228 | expect((await ownCode(app, "kid", false)).ok).toBe(true); | |
| 229 | expect(events("forge.own_code_off")).toEqual([{ name: "kid" }]); | |
| 230 | expect(readOwnCode(app, "kid")).toEqual({ on: false }); | |
| 231 | expect((await sources()).map((s) => s.kind)).toEqual(["parent"]); | |
| 232 | expect(existsSync(join(dataDir, "own-code", "kid"))).toBe(false); | |
| 233 | }); | |
| 234 | ||
| 235 | test("the switch is the admin's, same-origin, and takes only on or off", async () => { | |
| 236 | const post = (body: string, headers: Record<string, string>) => | |
| 237 | fetch(`${servers.url}/forges/kid/own-code`, { method: "POST", redirect: "manual", headers: { "content-type": "application/x-www-form-urlencoded", ...headers }, body }); | |
| 238 | expect((await post("on=on", { cookie, origin: "https://evil.test" })).status).toBe(403); | |
| 239 | expect((await post("on=on", { origin: servers.url })).status).toBe(404); | |
| 240 | expect((await post("on=maybe", { cookie, origin: servers.url })).status).toBe(404); | |
| 241 | const done = await post("on=on", { cookie, origin: servers.url }); | |
| 242 | expect(done.status).toBe(303); | |
| 243 | expect(done.headers.get("location")).toBe("/forges?code=on&forge=kid#forge-code"); | |
| 244 | expect(readOwnCode(app, "kid").on).toBe(true); | |
| 245 | const back = await (await fetch(`${servers.url}/forges?code=on&forge=kid`, { headers: { cookie } })).text(); | |
| 246 | expect(back).toContain("kid will run its own code from its next start."); | |
| 247 | const hosted = await post("on=on", { cookie, origin: servers.url }).then(() => | |
| 248 | fetch(`${servers.url}/forges/inner/own-code`, { method: "POST", redirect: "manual", headers: { "content-type": "application/x-www-form-urlencoded", cookie, origin: servers.url }, body: "on=on" }), | |
| 249 | ); | |
| 250 | expect(hosted.headers.get("location")).toBe("/forges?code=refused&forge=inner#forge-code"); | |
| 251 | expect((await post("on=off", { cookie, origin: servers.url })).status).toBe(303); | |
| 252 | expect(readOwnCode(app, "kid").on).toBe(false); | |
| 253 | expect(switchOwnCode(app, { name: "../x", placement: "cell", childDir: childDir(), on: true }).ok).toBe(false); | |
| 254 | }); | |
| 255 | ||
| 256 | test("the terminal's o picker says whose code each runs and switches it, as the page does", async () => { | |
| 257 | const rows = await clientRows(app); | |
| 258 | expect(ownCodePicks(app, rows).find((p) => p.id === "kid")?.hint).toStartWith("this forge's code"); | |
| 259 | const terminal = headlessTerminal(140, 32); | |
| 260 | const user = getUserByName(app.db, "alice") ?? undefined; | |
| 261 | const { app: tui } = createForgeApp(app, { sshUsername: "t", remoteAddress: "127.0.0.1", user }, { terminal, exitHooks: false }); | |
| 262 | tui.start(); | |
| 263 | const key = async (k: string, ctrl = false) => { | |
| 264 | tui.dispatch({ type: "key", key: k, ctrl, alt: false, shift: false }); | |
| 265 | await settleApp(tui); | |
| 266 | }; | |
| 267 | await key("k", true); | |
| 268 | for (const ch of "Forges you sent") await key(ch); | |
| 269 | await key("enter"); | |
| 270 | await key("o"); | |
| 271 | expect(tui.screen.toText()).toContain("Switch whose code"); | |
| 272 | await key("/"); | |
| 273 | for (const ch of "kid") await key(ch); | |
| 274 | await key("enter"); | |
| 275 | expect(tui.screen.toText()).toContain("▸ kid"); | |
| 276 | await key("enter"); | |
| 277 | await Bun.sleep(50); | |
| 278 | await settleApp(tui); | |
| 279 | expect(readOwnCode(app, "kid").on).toBe(true); | |
| 280 | tui.stop(); | |
| 281 | expect((await ownCode(app, "kid", false)).ok).toBe(true); | |
| 282 | }); | |
| 283 | ||
| 284 | test("a tree without an entry or a manifest is not green", () => { | |
| 285 | const dir = mkdtempSync(join(tmpdir(), "own-tree-")); | |
| 286 | expect(checkTree(dir)).toContain("no src/main.ts"); | |
| 287 | mkdirSync(join(dir, "src")); | |
| 288 | writeFileSync(join(dir, "src", "main.ts"), "export {};"); | |
| 289 | expect(checkTree(dir)).toContain("forge.json"); | |
| 290 | writeFileSync(join(dir, "forge.json"), "{}"); | |
| 291 | expect(checkTree(dir)).toBeUndefined(); | |
| 292 | rmSync(dir, { recursive: true, force: true }); | |
| 293 | }); |
test/tui-forges.test.ts +10
| @@ -113,6 +113,16 @@ describe("forges you sent in the terminal", () => { | ||
| 113 | 113 | h.stop(); |
| 114 | 114 | }); |
| 115 | 115 | |
| 116 | test("o switches a forge's code; with no forge it says so", async () => { | |
| 117 | const h = boot("alice"); | |
| 118 | await h.palette(); | |
| 119 | await h.enter(); | |
| 120 | expect(h.text()).toContain("own code or this forge's"); | |
| 121 | await h.press("o"); | |
| 122 | expect(h.text()).toContain("No forge to switch"); | |
| 123 | h.stop(); | |
| 124 | }); | |
| 125 | ||
| 116 | 126 | test("a row says whether its link is open, ended, or can be undone; a used link says nothing", () => { |
| 117 | 127 | const line = (link: ClientRow["link"]) => forgeLines([row({ link })])[0]![1]; |
| 118 | 128 | expect(line({ state: "open", revokedAt: null })).toContain("link open"); |
new above
offline