Public: anyone can read this; the crew can change it.

Family tree across forges: a forge shows its parent and its children #1790

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

scout/forge:scout/425-family-tree-across-forges-a-forge-shows → main 6 commits fork 2 threads · 2 unresolved

  • Merged. scout/425-family-tree-across-forges-a-forge-shows was merged into main as 13573e9.

Review

Sign in to review this change.
19 files changed +824 −2
README.md +5
@@ -86,6 +86,11 @@ sees `Reduced protection: …` in one line, under the filter on the terminal's
8686 first screen and in the web footer; nobody else does, and a fully protected
8787 forge shows nothing.
8888
89 The family across forges: `:` → Family in the terminal, `/family` on the web,
90 `GET /api/v1/family` and the MCP `family` tool all give the same view. The
91 forge that sent this one is shown to anyone; the forges it sent, only to a
92 site admin (docs/SPAWN.md, "The family across forges").
93
8994 Passkeys: Settings → Passkeys lists, adds and removes them (several per
9095 account; `src/auth/webauthn.ts` verifies, no dependency, `none` attestation).
9196 The sign-in page's "Sign in with a passkey" gives the same HttpOnly session a
docs/SPAWN.md +6
@@ -48,6 +48,12 @@ Spawn makes the child's `secret.key` and SSH host key in the child's own directo
4848
4949 A spawner may give the child one credential: `modelKey`, an API-billed key (`sk-ant-api…`), with `budgetUsd`, a whole-life budget in dollars from 1 to 1000. A subscription token (`sk-ant-oat…`) is refused, because it is a person's own plan. The key never enters the plan, the dry run, the answer, the ledger or the idempotency table. In the child it is sealed with the child's own key as the account `starter` and granted to its organisation's crew. The budget is locked on that account (`lockBudget`): it is enforced where every run picks its account, whoever started the run and on whichever repository, and the child's admin cannot raise it, only add their own key. A retry with a different key is refused; a key is given once. Said plainly in every answer that carries a key (`keyNote`): the budget holds while this forge hosts the child, whose admin has no shell or database access; on a machine the child's owner controls, only a spend limit set at the provider binds the key, so use a key from a workspace with its own spend limit. The MCP `spawn` tool refuses `modelKey`, so a secret never passes through an agent's conversation. With no key the child is in a clean state: its pages say the crew has no model key yet and where an admin adds one.
5050
51 ## The family across forges (#425)
52
53 `/family` draws the forge-level tree as nested lists: the parent (from `FORGE_PARENT_URL`, or the `upstream` in a spawned child's `parent.json`), this forge, and the forges it sent, each with last seen, version, releases behind this forge, and open proposals. Anyone may see the parent; only this forge's admins see children. Bounded: 100 children, four levels.
54
55 Children report in; the parent never reaches in. Every ten minutes a spawned child POSTs `/api/v1/heartbeat` (`src/spawn/heartbeat.ts`): `{name, at, version, openProposals, children}` and nothing else, signed HMAC-SHA256 over the body in `x-forge-signature` with a per-child `beatKey` that spawn put in the parent's ledger and the child's `parent.json`. A name the parent did not make, a bad signature, a time more than ten minutes off, a replay or any extra field gets the same 404 and changes nothing. The last beat per child is kept in `<data>/beats/<name>.json`. A child silent for thirty minutes reads "quiet", never an error; one never heard from reads "has not reported yet". Children made before this change have no key and stay unheard until respawned.
56
5157 ## A client's forge in one form (#1608)
5258
5359 `/forges/new` is the whole path: a name, the owner's apps to fork, optionally one fresh app from a one-line description (`fresh`, at most 200 characters, one line), a budget with a model key; it ends in one claim link, shown once. Each picked app is a fork: its full history is copied and `parent.json` lists `apps: [{repo, upstream}]` so the child knows what it follows. A fresh app is a private repository in the child with a README, a `forge.json` and a tiny `index.ts` that answers `/` with the sentence, escaped; its name is the sentence's slug, never one of the forked names. A custom domain is not part of this: the child is reached at `<name>--<host>` and nothing else is promised. The same field is on `POST /api/v1/forges` and the MCP `spawn` (`fresh`). It is absent from the plan when unsaid, so older plan hashes are unchanged.
scripts/gate.ts +1
@@ -59,6 +59,7 @@ export const GATE: readonly string[] = [
5959 "test/health.test.ts",
6060 "test/client-address.test.ts",
6161 "test/client-list.test.ts",
62 "test/family-heartbeat.test.ts",
6263 "test/contents-api.test.ts",
6364 "test/crew-agents-only.test.ts",
6465 "test/crew-brief-audience.test.ts",
src/actions/spawn.ts +16 −2
@@ -109,6 +109,11 @@ interface Ledger {
109109 * last happened); the child keeps only its hash. Never in an answer.
110110 */
111111 statusToken?: string;
112 /**
113 * The key a child signs its heartbeats with (`spawn/heartbeat.ts`); the
114 * child holds the same one in its `parent.json`. Never in an answer.
115 */
116 beatKey?: string;
112117 }
113118
114119 const SAFE_ID = /^[A-Za-z0-9_-]{1,80}$/;
@@ -220,8 +225,9 @@ async function make(
220225 }
221226 if (ledger.done)
222227 return ok({ name: plan.name, org: plan.org, url: liveChildUrl(app, plan.name), bodyId: ledger.bodyId, existed: true });
223 if (ledger.statusToken === undefined) {
224 ledger.statusToken = secret();
228 if (ledger.statusToken === undefined || ledger.beatKey === undefined) {
229 ledger.statusToken ??= secret();
230 ledger.beatKey ??= secret();
225231 await writeLedger(app, ledger);
226232 }
227233 } else {
@@ -241,6 +247,7 @@ async function make(
241247 apps: plan.apps.map((a) => a.ref),
242248 createdAt: new Date().toISOString(),
243249 statusToken: secret(),
250 beatKey: secret(),
244251 };
245252 await writeLedger(app, ledger);
246253 }
@@ -437,6 +444,8 @@ async function copyInto(
437444 : { forgeCode: { repo: `${plan.org}/forge`, upstream: forgeCode.value } }),
438445 // Only the hash: the parent alone holds what it answers to.
439446 ...(ledger.statusToken === undefined ? {} : { statusHash: sha256(ledger.statusToken) }),
447 // The key the child signs its heartbeats with; the parent holds the same.
448 ...(ledger.beatKey === undefined ? {} : { beatKey: ledger.beatKey }),
440449 }),
441450 );
442451 return ok(mintClaim(child.db, org.id));
@@ -613,6 +622,11 @@ export async function statusTokenOf(app: App, name: string): Promise<string | un
613622 return (await readLedger(app, name))?.statusToken;
614623 }
615624
625 /** The key a child of this forge signs heartbeats with; undefined for a name this forge did not make. */
626 export async function beatKeyOf(app: App, name: string): Promise<string | undefined> {
627 return (await readLedger(app, name))?.beatKey;
628 }
629
616630 /** One child, as the ledger keeps it. */
617631 export interface Child {
618632 readonly name: string;
src/core/reserved.ts +1
@@ -66,6 +66,7 @@ export const SYSTEM_NAMES: readonly string[] = [
6666 "boards",
6767 "events",
6868 "explore",
69 "family",
6970 "fleet",
7071 "flow",
7172 "front",
src/http/api.ts +19
@@ -47,6 +47,8 @@ import { emailVerified } from "../core/verify.ts";
4747 import { readHealth } from "../actions/health.ts";
4848 import { listChildren } from "../actions/spawn.ts";
4949 import { forgeState, PARENT_HEADER } from "../spawn/clients.ts";
50 import { familyView } from "../spawn/family.ts";
51 import { receiveBeat, SIGNATURE_HEADER } from "../spawn/heartbeat.ts";
5052 import { reissueDoor, spawnDoor } from "../spawn/door.ts";
5153 import { listSecrets, putSecret, removeSecret } from "../actions/secrets.ts";
5254 import { proposeUpstream } from "../actions/upstream.ts";
@@ -4134,6 +4136,23 @@ export function apiRoutes(router: Router<Handler>): void {
41344136 return json(state);
41354137 });
41364138
4139 // A child's signed heartbeat (docs/SPAWN.md): counts and version, never
4140 // data. Every refusal is the same 404, so nothing says who is a child.
4141 router.add("POST", "/api/v1/heartbeat", async (ctx) => {
outdated

High: this trusts the declared Content-Length. A chunked request has none, so length is 0 and ctx.request.text() reads up to REQUEST_MAX on an endpoint that needs no sign-in. Use readBounded(ctx.request, 1024) from src/http/limits.ts and send too_large back as the same 404.

Good: capped by bytes read, signed, the same 404 for every refusal. Non-blocking: the paired test would pass without the cap (parseBeat already rejects >1 KiB). A multi-chunk stream that asserts an early cut-off would pin it.

4142 const bytes = await readBounded(ctx.request, 1024);
4143 if (bytes === "too_large") return apiError("not_found", 404, "no such door");
4144 const raw = new TextDecoder().decode(bytes);
4145 const kept = await receiveBeat(ctx.app, raw, ctx.request.headers.get(SIGNATURE_HEADER));
4146 return kept ? json({ ok: true }) : apiError("not_found", 404, "no such door");
4147 });
4148
4149 // The family (#425): the parent for anyone, the children for an admin.
4150 router.add("GET", "/api/v1/family", async (ctx) => {
4151 const denied = ctx.auth.user === undefined ? undefined : requireScope(ctx, "ro");
4152 if (denied) return denied;
4153 return json(await familyView(ctx.app, ctx.auth.user?.isAdmin === true));
4154 });
4155
41374156 // A forge's children (docs/SPAWN.md): an admin's, planned before made.
41384157 router.add("GET", "/api/v1/forges", async (ctx) => {
41394158 const denied = requireScope(ctx, "ro");
src/http/mcp.ts +11
@@ -40,6 +40,7 @@ import {
4040 import { holdPull, releasePull } from "../actions/holds.ts";
4141 import { openPlan, parsePlan, PLAN_MAX } from "../actions/plan.ts";
4242 import { cancelRepoRun } from "../actions/runs.ts";
43 import { familyView } from "../spawn/family.ts";
4344 import { clientRows } from "../spawn/clients.ts";
4445 import { reissueDoor, spawnDoor } from "../spawn/door.ts";
4546 import { contextNote, parseContext } from "../actions/tell_context.ts";
@@ -1157,6 +1158,16 @@ const TOOLS: readonly Tool[] = [
11571158 });
11581159 },
11591160 },
1161 {
1162 name: "family",
1163 description:
1164 "The forge's family across forges: the forge that sent this one (anyone) and, for a site admin, the forges it sent, each as last heard — reporting, quiet (silent for thirty minutes) or unheard — with version, releases behind, open proposals. The same data as /family and /api/v1/family.",
1165 inputSchema: { type: "object", properties: {} },
1166 writes: false,
1167 async run(ctx, user) {
1168 return good(await familyView(ctx.app, user.isAdmin === true));
1169 },
1170 },
11601171 {
11611172 name: "next_steps",
11621173 description:
src/http/server.ts +2
@@ -203,6 +203,7 @@ import {
203203 } from "../web/studio/actions.ts";
204204 import { everydayCreate, everydayKeep, everydaySwitch } from "../web/pages/everyday.ts";
205205 import { claimFragmentPage, claimSubmit } from "../web/pages/claim.ts";
206 import { familyPage } from "../web/pages/family.ts";
206207 import { forgeLinkSubmit, forgesPage } from "../web/pages/forges.ts";
207208 import { newForgePage, newForgeSubmit } from "../web/pages/new-forge.ts";
208209 import { askedPage } from "../web/studio/asked_page.ts";
@@ -580,6 +581,7 @@ export function buildRouter(): Router<Handler> {
580581 router.add("POST", "/claim", claimSubmit);
581582 // "+ New forge": the spawn door as a page, any signed-in person (spawn/door.ts).
582583 router.add("GET", "/forges", forgesPage);
584 router.add("GET", "/family", familyPage);
583585 router.add("POST", "/forges/:name/claim-link", forgeLinkSubmit);
584586 router.add("GET", "/forges/new", newForgePage);
585587 router.add("POST", "/forges/new", newForgeSubmit);
src/main.ts +4
@@ -12,6 +12,7 @@ import { createApp } from "./app.ts";
1212 import { reconcile } from "./core/reconcile.ts";
1313 import { syncAllPillars } from "./actions/pillars.ts";
1414 import { firstBoot } from "./spawn/first-boot.ts";
15 import { startHeartbeat } from "./spawn/heartbeat.ts";
1516 import { pruneSessions } from "./core/sessions.ts";
1617 import { runDelivery } from "./core/webhooks.ts";
1718 import { isHookName, runHookClient } from "./git/hooks.ts";
@@ -320,6 +321,8 @@ const crew = appsOn && !crewHeldOff
320321 // A spawned child's first boot: its direction, opened once for its planner
321322 // (spawn/first-boot.ts). A forge that was not spawned is untouched.
322323 await firstBoot(app);
324 // A spawned child reports its standing to its parent (spawn/heartbeat.ts).
325 const heartbeatTimer = startHeartbeat(app);
323326 // Prompts from Instagram (docs/APPS.md): once a day, for each repository
324327 // that connected an account, the top comment on its latest Reel is told to
325328 // the crew. A minute's clock; nothing leaves the box until a source exists.
@@ -465,6 +468,7 @@ function shutdown(signal: NodeJS.Signals): void {
465468 // process that replaced this one, not to this one while it finishes.
466469 drainStreams();
467470 clearInterval(deliveryTimer);
471 clearInterval(heartbeatTimer);
468472 clearInterval(syncTimer);
469473 clearInterval(staleTimer);
470474 clearInterval(leaseTimer);
src/spawn/family.ts added +59
@@ -0,0 +1,59 @@
1 /**
2 * The family as plain data (#425): one answer for the page, the JSON read,
3 * the MCP tool and the terminal. Anyone sees the parent; only a site admin
4 * sees the children, so the gate lives here and nowhere else.
5 */
6
7 import type { App } from "../app.ts";
8 import { parentForgeFor } from "../core/parent_releases.ts";
9 import { running } from "../kernel/version.ts";
10 import { childStandings, parentUpstream, type ChildStanding } from "./heartbeat.ts";
11
12 export interface FamilyView {
13 readonly parent: { readonly url: string; readonly host: string; readonly behind: number | null } | null;
14 readonly here: { readonly host: string | null; readonly version: string };
15 readonly children: readonly ChildStanding[];
16 readonly more: number;
17 }
18
19 export async function familyView(app: App, isAdmin: boolean): Promise<FamilyView> {
20 const known = parentForgeFor(app.config);
21 const url = app.config.parentUrl ?? (await parentUpstream(app));
22 const kids = isAdmin ? await childStandings(app) : { items: [], more: 0 };
23 return {
24 parent:
25 url === undefined
26 ? null
27 : { url, host: known?.host ?? new URL(url).host, behind: known?.behind ?? null },
28 here: {
29 host: app.config.url === undefined ? null : new URL(app.config.url).host,
30 version: running().version,
31 },
32 children: kids.items,
33 more: kids.more,
34 };
35 }
36
37 /** The view as plain lines, for a terminal. */
38 export function familyLines(view: FamilyView): [string, string][] {
39 const out: [string, string][] = [];
40 if (view.parent !== null) {
41 out.push(["parent", view.parent.url]);
42 if (view.parent.behind !== null)
43 out.push(["", view.parent.behind === 0 ? "this forge is up to date" : `this forge is ${view.parent.behind} behind`]);
44 } else out.push(["parent", "none"]);
45 out.push(["this forge", `${view.here.host ?? "here"} · version ${view.here.version}`]);
46 for (const c of view.children) {
47 const bits =
48 c.state === "unheard"
49 ? ["has not reported yet"]
50 : [
51 c.state,
52 c.version === null ? "" : `version ${c.version}`,
53 c.openProposals === null ? "" : `${c.openProposals} open`,
54 ].filter((b) => b !== "");
55 out.push([`child ${c.name}`, bits.join(" · ")]);
56 }
57 if (view.more > 0) out.push(["", `${view.more} more not shown`]);
58 return out;
59 }
src/spawn/heartbeat.ts added +265
@@ -0,0 +1,265 @@
1 /**
2 * The family across forges (#425): a child tells its parent how it stands,
3 * and the parent never reaches in. A heartbeat is a small signed JSON
4 * document carrying the child's name, a time, its version and two counts —
5 * open proposals and children of its own — and nothing else: no titles, no
6 * names, no data. The parent keeps the last one per child under
7 * `<data>/beats/<name>.json`.
8 *
9 * Signed with the key spawn gave both sides (`beatKey` in the parent's
10 * ledger and in the child's `parent.json`): HMAC-SHA256 over the exact body
11 * bytes. A name the parent did not make, a bad signature, a stale or
12 * replayed time, a malformed body all get the same answer and change
13 * nothing: unknown children are ignored.
14 */
15
16 import { createHmac, timingSafeEqual } from "node:crypto";
17 import { mkdir, readFile, rename, writeFile } from "node:fs/promises";
18 import { join } from "node:path";
19 import type { App } from "../app.ts";
20 import { beatKeyOf, listChildren } from "../actions/spawn.ts";
21 import releases from "../core/releases.json" with { type: "json" };
22 import { running } from "../kernel/version.ts";
23
24 export const SIGNATURE_HEADER = "x-forge-signature";
25 /** A child reports this often; one silent for three beats reads as quiet. */
26 export const BEAT_MS = 10 * 60_000;
27 export const QUIET_MS = 3 * BEAT_MS;
28 /** A heartbeat's time may differ from the parent's clock by this much. */
29 const SKEW_MS = 10 * 60_000;
30 const MAX_BODY = 1024;
31 export const MAX_CHILDREN = 100;
32 export const MAX_DEPTH = 4;
33 const NAME = /^[a-z0-9](?:[a-z0-9-]{0,37}[a-z0-9])?$/;
34 const VERSION = /^[0-9A-Za-z][0-9A-Za-z.+_-]{0,63}$/;
35 const COUNT_MAX = 1_000_000;
36
37 export interface Beat {
38 readonly name: string;
39 /** The child's clock, ms since the epoch. */
40 readonly at: number;
41 readonly version: string;
42 readonly openProposals: number;
43 readonly children: number;
44 }
45
46 export function sign(key: string, body: string): string {
47 return createHmac("sha256", key).update(body).digest("hex");
48 }
49
50 function verified(key: string, body: string, shown: string | null): boolean {
51 if (shown === null || !/^[0-9a-f]{64}$/.test(shown)) return false;
52 return timingSafeEqual(Buffer.from(sign(key, body)), Buffer.from(shown));
53 }
54
55 const count = (v: unknown): v is number =>
56 typeof v === "number" && Number.isInteger(v) && v >= 0 && v <= COUNT_MAX;
57
58 /** The beat in `raw`, or undefined when it is not exactly one. */
59 export function parseBeat(raw: string): Beat | undefined {
60 if (raw.length > MAX_BODY) return undefined;
61 let doc: unknown;
62 try {
63 doc = JSON.parse(raw);
64 } catch {
65 return undefined;
66 }
67 if (typeof doc !== "object" || doc === null || Array.isArray(doc)) return undefined;
68 const d = doc as Record<string, unknown>;
69 const keys = Object.keys(d).sort().join(",");
70 if (keys !== "at,children,name,openProposals,version") return undefined;
71 const { name, at, version, openProposals, children } = d;
72 if (typeof name !== "string" || !NAME.test(name)) return undefined;
73 if (typeof at !== "number" || !Number.isSafeInteger(at) || at <= 0) return undefined;
74 if (typeof version !== "string" || !VERSION.test(version)) return undefined;
75 if (!count(openProposals) || !count(children)) return undefined;
76 return { name, at, version, openProposals, children };
77 }
78
79 const beatPath = (app: App, name: string): string =>
80 join(app.config.dataDir, "beats", `${name}.json`);
81
82 /** What this forge heard, and when (ms), for one child. */
83 export interface Heard {
84 readonly beat: Beat;
85 readonly receivedAt: number;
86 }
87
88 async function lastHeard(app: App, name: string): Promise<Heard | undefined> {
89 try {
90 const doc = JSON.parse(await readFile(beatPath(app, name), "utf8")) as {
91 receivedAt?: unknown;
92 beat?: unknown;
93 };
94 const beat = parseBeat(JSON.stringify(doc.beat));
95 return beat !== undefined && typeof doc.receivedAt === "number"
96 ? { beat, receivedAt: doc.receivedAt }
97 : undefined;
98 } catch {
99 return undefined;
100 }
101 }
102
103 /**
104 * Take one heartbeat. True when it was kept; false for every refusal, which
105 * the caller answers alike so nothing says whether a name is a child.
106 */
107 export function receiveBeat(
108 app: App,
109 raw: string,
110 signature: string | null,
111 now = Date.now(),
112 ): Promise<boolean> {
113 // One at a time per name, so two identical beats cannot both be kept.
114 const name = parseBeat(raw)?.name ?? "";
115 const run = (turns.get(name) ?? Promise.resolve()).then(() =>
116 takeBeat(app, raw, signature, now),
117 );
118 const tail = run.then(() => undefined, () => undefined);
119 turns.set(name, tail);
120 void tail.then(() => {
121 if (turns.get(name) === tail) turns.delete(name);
122 });
123 return run;
124 }
125
126 const turns = new Map<string, Promise<void>>();
127
128 async function takeBeat(
129 app: App,
130 raw: string,
131 signature: string | null,
132 now: number,
133 ): Promise<boolean> {
134 const beat = parseBeat(raw);
135 if (beat === undefined) return false;
136 const key = await beatKeyOf(app, beat.name);
137 if (key === undefined || !verified(key, raw, signature)) return false;
138 if (Math.abs(now - beat.at) > SKEW_MS) return false;
139 const before = await lastHeard(app, beat.name);
140 if (before !== undefined && beat.at <= before.beat.at) return false;
141 await mkdir(join(app.config.dataDir, "beats"), { recursive: true });
142 const path = beatPath(app, beat.name);
143 await writeFile(`${path}.tmp`, JSON.stringify({ receivedAt: now, beat }));
144 await rename(`${path}.tmp`, path);
145 return true;
146 }
147
148 /** What a child knows of its parent from `parent.json`, when it has all of it. */
149 async function parentLink(
150 app: App,
151 ): Promise<{ name: string; upstream: string; key: string } | undefined> {
152 try {
153 const doc = (await Bun.file(join(app.config.dataDir, "parent.json")).json()) as Record<string, unknown>;
154 const { name, upstream, beatKey } = doc;
155 if (typeof name !== "string" || !NAME.test(name)) return undefined;
156 if (typeof upstream !== "string" || !/^https?:\/\/[^\s?#]+$/.test(upstream)) return undefined;
157 if (typeof beatKey !== "string" || beatKey === "") return undefined;
158 return { name, upstream: upstream.replace(/\/+$/, ""), key: beatKey };
159 } catch {
160 return undefined;
161 }
162 }
163
164 /** This forge's standing, as the beat carries it. Counts only. */
165 export function standing(app: App, name: string, at = Date.now()): Beat {
166 const proposals = app.db
167 .query<{ n: number }, []>(
168 `SELECT COUNT(*) AS n FROM pull JOIN issue ON issue.id = pull.issue_id WHERE issue.state = 'open'`,
169 )
170 .get();
171 return {
172 name,
173 at,
174 version: running().version,
175 openProposals: Math.min(proposals?.n ?? 0, COUNT_MAX),
176 children: 0,
177 };
178 }
179
180 type FetchLike = (url: string, init?: RequestInit) => Promise<Response>;
181 let lastAt = 0;
182
183 /** A spawned child's one report to its parent; false when it has no parent or the try failed. Never throws. */
184 export async function sendBeat(
185 app: App,
186 fetcher: FetchLike = fetch,
187 now = Date.now(),
188 ): Promise<boolean> {
189 const link = await parentLink(app);
190 if (link === undefined) return false;
191 const own = (await listChildren(app)).length;
192 // Strictly later than the last one sent, so a quick restart is not a replay.
193 lastAt = Math.max(now, lastAt + 1);
194 const body = JSON.stringify({
195 ...standing(app, link.name, lastAt),
196 children: Math.min(own, COUNT_MAX),
197 });
198 try {
199 const res = await fetcher(`${link.upstream}/api/v1/heartbeat`, {
200 method: "POST",
201 headers: { "content-type": "application/json", [SIGNATURE_HEADER]: sign(link.key, body) },
202 body,
203 signal: AbortSignal.timeout(5000),
204 redirect: "error",
205 });
206 return res.ok;
207 } catch {
208 return false;
209 }
210 }
211
212 /** Report now and then every BEAT_MS; returns the timer to clear. */
213 export function startHeartbeat(app: App): ReturnType<typeof setInterval> {
214 void sendBeat(app);
215 return setInterval(() => void sendBeat(app), BEAT_MS);
216 }
217
218 export type Standing = "reporting" | "quiet" | "unheard";
219
220 export interface ChildStanding {
221 readonly name: string;
222 readonly url: string;
223 readonly state: Standing;
224 /** ms since the epoch the parent last heard, or null. */
225 readonly lastSeen: number | null;
226 readonly version: string | null;
227 /** Releases the child is behind this forge, or null when its version is not one of ours. */
228 readonly behind: number | null;
229 readonly openProposals: number | null;
230 readonly ownChildren: number | null;
231 }
232
233 /** How many releases of this forge are newer than `version`; null when it is not one of them. */
234 export function releasesBehind(version: string): number | null {
235 const i = releases.releases.findIndex((r) => r.version === version);
236 return i < 0 ? null : i;
237 }
238
239 /** Each child this forge made, as last heard; bounded, and a silent one is quiet, not broken. */
240 export async function childStandings(
241 app: App,
242 now = Date.now(),
243 ): Promise<{ items: ChildStanding[]; more: number }> {
244 const all = await listChildren(app);
245 const items: ChildStanding[] = [];
246 for (const c of all.slice(0, MAX_CHILDREN)) {
247 const heard = await lastHeard(app, c.name);
248 items.push({
249 name: c.name,
250 url: c.url,
251 state: heard === undefined ? "unheard" : now - heard.receivedAt > QUIET_MS ? "quiet" : "reporting",
252 lastSeen: heard?.receivedAt ?? null,
253 version: heard?.beat.version ?? null,
254 behind: heard === undefined ? null : releasesBehind(heard.beat.version),
255 openProposals: heard?.beat.openProposals ?? null,
256 ownChildren: heard?.beat.children ?? null,
257 });
258 }
259 return { items, more: Math.max(0, all.length - MAX_CHILDREN) };
260 }
261
262 /** The address of the forge that sent this one, when it was spawned and told. */
263 export async function parentUpstream(app: App): Promise<string | undefined> {
264 return (await parentLink(app))?.upstream;
265 }
src/tui/forge.ts +13
@@ -27,7 +27,10 @@ import {
2727 errorText,
2828 promptScreen,
2929 openOnWeb,
30 runAsync,
3031 } from "./screens/common.ts";
32 import { familyScreen } from "./screens/family.ts";
33 import { familyLines, familyView } from "../spawn/family.ts";
3134 import { issueScreen } from "./screens/issue.ts";
3235 import { pullScreen } from "./screens/pull.ts";
3336 import { blobScreen } from "./screens/repo.ts";
@@ -192,6 +195,16 @@ export function forgeCommands(session: Session): (ctx: Ctx) => Command[] {
192195 hint: "the pipeline's bottleneck",
193196 run: () => ctx.toast(verdict(readPipeline(session.db)), "default"),
194197 });
198 commands.push({
199 id: "family",
200 title: "Family",
201 hint: "the forge that sent this one, and the forges it sent",
202 run: () =>
203 runAsync(ctx, async () => {
204 const view = await familyView(session.app, session.user?.isAdmin === true);
205 ctx.push(familyScreen(ctx, familyLines(view)));
206 }),
207 });
195208 commands.push({
196209 id: "quit",
197210 title: "Quit",
src/tui/screens/family.ts added +29
@@ -0,0 +1,29 @@
1 /**
2 * The family across forges (#425) in a card: the same lines as /family, from
3 * the same view, so the gate (guests see the parent, admins also the
4 * children) is `spawn/family.ts`'s and not this screen's.
5 */
6
7 import type { Ctx, Screen } from "../app/app.ts";
8 import { binding } from "../app/keys.ts";
9 import { card } from "../ui/widgets.ts";
10 import { drawFacts } from "./common.ts";
11
12 export function familyScreen(ctx: Ctx, rows: readonly (readonly [string, string])[]): Screen {
13 return {
14 title: "Family",
15 keys: [binding(["esc"], "close")],
16 render(area, s) {
17 const c = ctx.theme.c;
18 const w = Math.min(78, Math.max(30, area.w - 6));
19 const h = Math.min(area.h, rows.length + 2);
20 const r = { x: area.x + ((area.w - w) >> 1), y: area.y + ((area.h - h) >> 1), w, h };
21 const inner = card(s, r, c, { title: "Family", focused: true });
22 drawFacts(s, { x: inner.x + 1, y: inner.y, w: Math.max(0, inner.w - 1), h: inner.h }, c, rows, c.card);
23 },
24 key(e) {
25 if (e.key === "esc" || e.key === "q") ctx.pop();
26 return true;
27 },
28 };
29 }
src/web/layout.ts +1
@@ -324,6 +324,7 @@ function paletteIndex(ctx: Ctx, extra: PaletteItem[]): Html {
324324 ? [
325325 { t: "Flow", h: "/flow", c: "Go to" },
326326 { t: "Forges you sent", h: "/forges", c: "Go to" },
327 { t: "Family of forges", h: "/family", c: "Go to" },
327328 ]
328329 : []),
329330 ];
src/web/pages/family.ts added +141
@@ -0,0 +1,141 @@
1 /**
2 * `/family`: the forge-level family (#425). The forge this one was spawned
3 * from, this forge, and the forges it sent, each with how it stands as it
4 * last reported (`spawn/heartbeat.ts`). Nested lists, like the repository
5 * family tree, bounded in depth and count. Anyone may see the parent; only
6 * this forge's admins see its children. A child that stopped reporting
7 * reads "quiet": a fact about the clock, not a fault.
8 */
9
10 import type { Ctx } from "../../http/context.ts";
11 import { familyView } from "../../spawn/family.ts";
12 import {
13 MAX_DEPTH,
14 type ChildStanding,
15 } from "../../spawn/heartbeat.ts";
16 import { pageHead } from "../chrome.ts";
17 import { html, type Html } from "../html.ts";
18 import { page } from "../layout.ts";
19 import { card, time } from "../ui.ts";
20
21 interface Node {
22 readonly label: string;
23 readonly href: string | null;
24 readonly meta: readonly Html[];
25 readonly badge: Html | null;
26 readonly here?: boolean;
27 readonly children: readonly Node[];
28 }
29
30 const plural = (n: number, one: string, many = `${one}s`) =>
31 `${n} ${n === 1 ? one : many}`;
32
33 function standingNode(c: ChildStanding): Node {
34 const meta: Html[] = [];
35 if (c.state === "unheard") meta.push(html`has not reported yet`);
36 else {
37 if (c.lastSeen !== null)
38 meta.push(html`last seen ${time(new Date(c.lastSeen).toISOString())}`);
39 if (c.version !== null) meta.push(html`version ${c.version}`);
40 if (c.behind !== null)
41 meta.push(html`${c.behind === 0 ? "up to date" : `${plural(c.behind, "release")} behind`}`);
42 if (c.openProposals !== null)
43 meta.push(html`${plural(c.openProposals, "open proposal")}`);
44 if (c.ownChildren !== null && c.ownChildren > 0)
45 meta.push(html`${plural(c.ownChildren, "forge")} of its own`);
46 }
47 return {
48 label: c.name,
49 href: c.url,
50 meta,
51 badge:
52 c.state === "reporting"
53 ? html`<span class="pill" data-tone="accent">reporting</span>`
54 : c.state === "quiet"
55 ? html`<span class="pill">quiet</span>`
56 : null,
57 children: [],
outdated

An unheard child falls through to the quiet pill. docs/SPAWN.md says a child never heard from reads 'has not reported yet', not quiet. Give unheard no badge, or a neutral one, and fix the test that expects 'quiet' here.

58 };
59 }
60
61 function row(n: Node, depth: number): Html {
62 const inner = html`<span class="row-main">
63 <span class="row-title"
64 >${n.label}${n.here ? html` <span class="lineage-here">you are here</span>` : null}</span
65 >
66 ${n.meta.length === 0
67 ? null
68 : html`<span class="row-meta"
69 >${n.meta.map((m, i) => html`${i > 0 ? " · " : ""}${m}`)}</span
70 >`}
71 </span>
72 ${n.badge === null ? null : html`<span class="row-trail">${n.badge}</span>`}`;
73 return html`<li>
74 ${n.href === null
75 ? html`<div class="row">${inner}</div>`
76 : html`<a class="row" href="${n.href}" rel="noopener noreferrer">${inner}</a>`}
77 ${n.children.length === 0 || depth + 1 >= MAX_DEPTH
78 ? null
79 : html`<ul class="lineage-children">
80 ${n.children.map((c) => row(c, depth + 1))}
81 </ul>`}
82 </li>`;
83 }
84
85 // GET /family
86 export async function familyPage(ctx: Ctx): Promise<Response> {
87 const app = ctx.app;
88 const owns = ctx.auth.user?.isAdmin === true;
89 const view = await familyView(app, owns);
90 const parent = view.parent;
91 const kids = { items: view.children, more: view.more };
92
93 const here: Node = {
94 label: view.here.host ?? "this forge",
95 href: null,
96 meta: [html`version ${view.here.version}`],
97 badge: null,
98 here: true,
99 children: kids.items.map(standingNode),
100 };
101 const root: Node =
102 parent === null
103 ? here
104 : {
105 label: parent.host,
106 href: parent.url,
107 meta:
108 parent.behind === null
109 ? []
110 : [html`this forge is ${parent.behind === 0 ? "up to date" : plural(parent.behind, "release") + " behind"}`],
111 badge: null,
112 children: [here],
113 };
114
115 const alone = parent === null && kids.items.length === 0;
116 return page({
117 ctx,
118 title: "Family",
119 body: html`<div class="narrow-wide">
120 ${pageHead("Family", {
121 sub: "The forge this one came from, and the forges it sent. Children report in; nothing here reaches into one.",
122 })}
123 ${card(
124 html`<ul class="list lineage" aria-label="Forges, by who sent whom">
125 ${row(root, 0)}
126 </ul>
127 ${kids.more > 0
128 ? html`<p class="muted small card-body">
129 ${plural(kids.more, "more forge")} not shown.
130 </p>`
131 : null}
132 ${alone
133 ? html`<p class="muted small card-body">
134 This forge has no parent and has sent none${owns ? "" : " that you can see"}.
135 </p>`
136 : null}`,
137 { pad: false },
138 )}
139 </div>`,
140 });
141 }
test/family-heartbeat.test.ts added +247
@@ -0,0 +1,247 @@
1 /**
2 * The family across forges (#425): signed heartbeats carry counts and
3 * version only, unknown children and bad signatures are ignored alike, a
4 * silent child reads quiet, and only the parent's admins see children.
5 */
6
7 import { afterAll, beforeAll, describe, expect, test } from "bun:test";
8 import { mkdtemp, readFile, rm } from "node:fs/promises";
9 import { tmpdir } from "node:os";
10 import { join } from "node:path";
11 import { createRepo } from "../src/actions/repos.ts";
12 import { beatKeyOf, childDirOf } from "../src/actions/spawn.ts";
13 import { createApp, type App } from "../src/app.ts";
14 import { createUser } from "../src/core/users.ts";
15 import { commitFiles } from "../src/git/write.ts";
16 import { repoDir } from "../src/git/repos.ts";
17 import { createToken } from "../src/core/tokens.ts";
18 import { serve, type Servers } from "../src/http/server.ts";
19 import {
20 childStandings,
21 parseBeat,
22 QUIET_MS,
23 sendBeat,
24 sign,
25 SIGNATURE_HEADER,
26 } from "../src/spawn/heartbeat.ts";
27 import { familyLines, familyView } from "../src/spawn/family.ts";
28 import { firstBoot } from "../src/spawn/first-boot.ts";
29 import { quietLogs } from "./helpers/globals.ts";
30
31 quietLogs();
32
33 const PASSWORD = "a-long-family-passphrase";
34
35 describe("family heartbeats", () => {
36 let dir = "";
37 let parent: App;
38 let child: App;
39 let servers: Servers;
40 let key = "";
41 let adminCookie = "";
42 let bobCookie = "";
43
44 const post = (body: string, sig: string | null) =>
45 fetch(`${servers.url}/api/v1/heartbeat`, {
46 method: "POST",
47 headers: sig === null ? {} : { [SIGNATURE_HEADER]: sig },
48 body,
49 });
50 const beat = (o: Record<string, unknown> = {}) =>
51 JSON.stringify({
52 name: "maria",
53 at: Date.now(),
54 version: "0.7.114",
55 openProposals: 3,
56 children: 0,
57 ...o,
58 });
59 const signIn = async (username: string) => {
60 const res = await fetch(`${servers.url}/signin`, {
61 method: "POST",
62 body: new URLSearchParams({ username, password: PASSWORD }),
63 redirect: "manual",
64 });
65 return (res.headers.get("set-cookie") ?? "").split(";")[0]!;
66 };
67 const family = async (cookie?: string) =>
68 (
69 await fetch(`${servers.url}/family`, {
70 headers: cookie === undefined ? {} : { cookie },
71 })
72 ).text();
73
74 beforeAll(async () => {
75 dir = await mkdtemp(join(tmpdir(), "forge-family-"));
76 parent = createApp({ dataDir: dir, url: "https://forge.test", port: 0 });
77 const alice = await createUser(parent.db, {
78 trusted: true,
79 username: "alice",
80 email: "[email protected]",
81 password: PASSWORD,
82 isAdmin: true,
83 });
84 const bob = await createUser(parent.db, {
85 trusted: true,
86 username: "bob",
87 email: "[email protected]",
88 password: PASSWORD,
89 });
90 if (!alice.ok || !bob.ok) throw new Error("users");
91 const token = createToken(parent.db, alice.value.id, "rw", "rw").token;
92 const repo = await createRepo(parent, alice.value, { name: "bakery" });
93 if (!repo.ok) throw new Error("repo");
94 const c = await commitFiles(repoDir(parent.reposRoot, "alice", "bakery"), {
95 branch: "main",
96 message: "bakery",
97 files: [{ path: "README.md", content: "# bakery\n" }],
98 author: { name: "a", email: "[email protected]" },
99 });
100 if (!c.ok) throw new Error("commit");
101 servers = await serve(parent);
102 const made = await fetch(`${servers.url}/api/v1/forges`, {
103 method: "POST",
104 headers: { authorization: `Bearer ${token}`, "content-type": "application/json" },
105 body: JSON.stringify({ name: "maria", apps: ["alice/bakery"], dryRun: false }),
106 });
107 if (made.status !== 201) throw new Error(`spawn ${made.status}`);
108 const { bodyId } = (await made.json()) as { bodyId: string };
109 child = createApp({ dataDir: childDirOf(parent, bodyId), port: 0 });
110 await firstBoot(child);
111 key = (await beatKeyOf(parent, "maria")) ?? "";
112 adminCookie = await signIn("alice");
113 bobCookie = await signIn("bob");
114 }, 60_000);
115
116 afterAll(async () => {
117 servers.stop();
118 child.close();
119 parent.close();
120 await rm(dir, { recursive: true, force: true });
121 });
122
123 test("before any report a child reads as not heard from, not as an error", async () => {
124 const html = await family(adminCookie);
125 expect(html).toContain("maria");
126 expect(html).toContain("has not reported yet");
127 expect(html).not.toContain("quiet");
128 });
129
130 test("a heartbeat holds only name, time, version and two counts", () => {
131 expect(parseBeat(beat())).toBeDefined();
132 expect(parseBeat(beat({ title: "secret plans" }))).toBeUndefined();
133 expect(parseBeat(beat({ openProposals: -1 }))).toBeUndefined();
134 expect(parseBeat(beat({ version: "<b>" }))).toBeUndefined();
135 expect(parseBeat(beat({ name: "../x" }))).toBeUndefined();
136 expect(parseBeat("x".repeat(5000))).toBeUndefined();
137 });
138
139 test("unknown children, bad signatures and stale times are ignored alike", async () => {
140 const stranger = beat({ name: "nobody" });
141 const a = await post(stranger, sign(key, stranger));
142 const own = beat();
143 const b = await post(own, sign("not-the-key", own));
144 const c = await post(own, null);
145 const old = beat({ at: Date.now() - 3_600_000 });
146 const d = await post(old, sign(key, old));
147 for (const r of [a, b, c, d]) expect(r.status).toBe(404);
148 const state = await childStandings(parent);
149 expect(state.items[0]?.state).toBe("unheard");
150 });
151
152 test("a signed heartbeat is kept, shown with its standing, and not replayable", async () => {
153 const at = Date.now();
154 const body = beat({ at });
155 expect((await post(body, sign(key, body))).status).toBe(200);
156 expect((await post(body, sign(key, body))).status).toBe(404);
157 const html = await family(adminCookie);
158 expect(html).toContain("reporting");
159 expect(html).toContain("version 0.7.114");
160 expect(html).toContain("3 open proposals");
161 expect(html).toMatch(/\d+ releases? behind/);
162 const stored = await readFile(join(dir, "beats", "maria.json"), "utf8");
163 expect(Object.keys(JSON.parse(stored).beat).sort()).toEqual([
164 "at",
165 "children",
166 "name",
167 "openProposals",
168 "version",
169 ]);
170 });
171
172 test("a child that stops reporting reads quiet", async () => {
173 const state = await childStandings(parent, Date.now() + QUIET_MS + 60_000);
174 expect(state.items[0]?.state).toBe("quiet");
175 expect(state.items[0]?.lastSeen).not.toBeNull();
176 });
177
178 test("only an admin of the parent sees children", async () => {
179 expect(await family(bobCookie)).not.toContain("maria");
180 expect(await family()).not.toContain("maria");
181 expect(await family(adminCookie)).toContain("maria");
182 });
183
184 test("a chunked body over 1 KiB is refused unread and keeps nothing", async () => {
185 const before = (await childStandings(parent)).items[0]?.lastSeen;
186 const big = beat({ version: "1".repeat(4000) });
187 const res = await fetch(`${servers.url}/api/v1/heartbeat`, {
188 method: "POST",
189 headers: { [SIGNATURE_HEADER]: sign(key, big) },
190 body: new ReadableStream({
191 start(c) {
192 c.enqueue(new TextEncoder().encode(big));
193 c.close();
194 },
195 }),
196 duplex: "half",
197 } as RequestInit);
198 expect(res.status).toBe(404);
199 expect((await childStandings(parent)).items[0]?.lastSeen).toBe(before);
200 });
201
202 test("two identical beats at once keep only one", async () => {
203 const body = beat({ at: Date.now() + 1000 });
204 const rs = await Promise.all([post(body, sign(key, body)), post(body, sign(key, body))]);
205 expect(rs.map((r) => r.status).sort()).toEqual([200, 404]);
206 });
207
208 test("the family has a JSON door and a terminal view with the same gate", async () => {
209 const read = async (cookie?: string) =>
210 (await (
211 await fetch(`${servers.url}/api/v1/family`, {
212 headers: cookie === undefined ? {} : { cookie },
213 })
214 ).json()) as { children: { name: string }[]; here: unknown };
215 expect((await read()).children).toEqual([]);
216 expect((await read(bobCookie)).children).toEqual([]);
217 expect((await read(adminCookie)).children.map((c) => c.name)).toEqual(["maria"]);
218 const guest = familyLines(await familyView(parent, false));
219 const admin = familyLines(await familyView(parent, true));
220 expect(guest.some(([k]) => k === "child maria")).toBe(false);
221 expect(admin.some(([k]) => k === "child maria")).toBe(true);
222 });
223
224 test("a child sends its own beat, signed with the key spawn gave it", async () => {
225 const sent: { url: string; body: string; sig: string }[] = [];
226 const ok = await sendBeat(child, async (url, init) => {
227 sent.push({
228 url,
229 body: String(init?.body),
230 sig: String((init?.headers as Record<string, string>)[SIGNATURE_HEADER]),
231 });
232 return new Response("{}");
233 });
234 expect(ok).toBe(true);
235 expect(sent[0]?.url).toBe("https://forge.test/api/v1/heartbeat");
236 expect(sent[0]?.sig).toBe(sign(key, sent[0]!.body));
237 expect(Object.keys(JSON.parse(sent[0]!.body)).sort()).toEqual([
238 "at",
239 "children",
240 "name",
241 "openProposals",
242 "version",
243 ]);
244 // A forge that was not spawned has no one to tell.
245 expect(await sendBeat(parent, async () => new Response("{}"))).toBe(false);
246 });
247 });
test/mcp.test.ts +1
@@ -183,6 +183,7 @@ describe("the forge over MCP", () => {
183183 "claim_link",
184184 "take_updates",
185185 "converge",
186 "family",
186187 "next_steps",
187188 "thought",
188189 "asked",
test/spawn-keys.test.ts +1
@@ -358,6 +358,7 @@ describe("a spawned child's own keys and domain", () => {
358358 ) as Record<string, unknown>;
359359 expect(Object.keys(record).sort()).toEqual([
360360 "apps",
361 "beatKey",
361362 "keepAccess",
362363 "name",
363364 "org",
test/spawn.test.ts +2
@@ -87,6 +87,8 @@ describe("spawnForge", () => {
8787 keepAccess: false,
8888 // A hash only: what the parent answers to stays in its own ledger.
8989 statusHash: expect.stringMatching(/^[0-9a-f]{64}$/),
90 // The child signs heartbeats to its parent with it (#425).
91 beatKey: expect.stringMatching(/^[A-Za-z0-9_-]{20,}$/),
9092 });
9193 });
9294

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs