trevato/forge

Public: anyone can read this; the crew can change it.

More
Forks1g fMapg aRuns12g rEventsFlows

An agent deploys a forge for its person, and that forge scales to zero when idle #9547

Merged scoutcrew opened this in answer to trevatoowner

Check · Review · Land · Live done — Landed and live in production.

ConversationFiles
Merged

scout/forge:scout/9444-an-agent-deploys-a-forge-for-its-person → main 6 commits fork

  • Merged. scout/9444-an-agent-deploys-a-forge-for-its-person was merged into main as 43459ae.

Review

Sign in to review this change.
19 files changed +1006 −20
docs/features/forge-sleep.md added +16
@@ -0,0 +1,16 @@
1 # Hosted forges that scale to zero
2
3 A person (or their assistant, over the MCP `spawn` tool with their consent) makes a forge at `/forges/new`, the address `<name>--<this host>`. Free within the limits below; no payment path yet.
4
5 - **Sleep.** A forge with no request for `FORGE_CHILD_IDLE_S` (600) sleeps: a hosted child's App and apps close, a cell child's process stops. Data stays under `children/<bodyId>`. The next page load, `git push` or API call wakes it. A page for a sleeping cell forge answers a plain "waking up" page at once (git and API calls wait up to 4 s) (503, `retry-after`, reloads itself); a hosted one opens in-process.
6 - **Limits, each refused in words.** `FORGE_FORGES_PER_PERSON` (3; site admins exempt), `FORGE_CHILD_AWAKE_MAX` (8 awake at once; sleeping forges do not count), `FORGE_CHILD_WAKES_PER_HOUR` (20 per forge), plus the existing size/quota limits (`FORGE_CHILD_*_MIB`), and each cell's own disk cap.
7 - **Owner.** `/forges` shows awake/asleep, size, last activity, with Stop (sleep now) and Remove (ledger and data).
8 - **Secrets.** A child starts from `childEnv` and never sees this forge's model credential; the person brings their own or its crew stays off.
9
10 Code: `src/spawn/sleep.ts` (limits, waking page), `src/spawn/host.ts` (`wakeGate`, `sleepIdleChildren`), `src/spawn/vouch.ts`, `src/spawn/manage.ts`. Test: `test/forge-sleep.test.ts`.
11
12 ## Doors
13 - Web: `/forges` (awake/asleep, size, last activity; Stop; Remove asks on `GET /forges/:name/remove` first). Create: `/forges/new`, or the one-field form on `/agents`, which posts the name straight to the spawn (one step, defaults for the rest).
14 - Terminal: the "Forges you sent" screen lists awake/asleep and size, `s` stops one, `x` removes one (confirmed); both call `spawn/manage.ts`, which writes `forge.stopped` / `forge.removed` to the event outbox, so web viewers see the change. Create: the New forge screen (`src/tui/screens/newforge.ts`). MCP: `spawn`.
15 - A stranger's request wakes a sleeping forge only within an address rate (`takeDoor`) and the awake ceiling, and never refreshes its idleness. A request does only when the child itself vouches for it: its session cookie or token (Bearer, `token`, or a `forge_` token as the Basic password) is looked up read-only in the child's own database (`src/spawn/vouch.ts`), so a made-up `authorization` header or `forge_session` cookie is a stranger's. A Basic password is not checked there and counts as a stranger's. A forge with a request in flight or crew work queued or running is not put to sleep. Not covered: an open websocket, and a response body still streaming from a cell forge after its headers.
16 - The wake path for a cell forge, and the hold around cell forwards, are not exercised by tests in this sandbox (cells are skipped).
src/actions/spawn.ts +8
@@ -56,6 +56,7 @@ import { loadOrCreateHostKey } from "../ssh/keys.ts";
5656 import { claimLink, hasAdmin, mintClaim, spawnedOrg } from "../spawn/claim.ts";
5757 import { childPublicUrl, parentOrigin, type Placement, type PlacementKind } from "../spawn/placement.ts";
5858 import { quotaRefusal } from "../spawn/quota.ts";
59 import { personRefusal } from "../spawn/sleep.ts";
5960 import { planSpawn, type SpawnInput, type SpawnPlan, type SpawnSource } from "../spawn/plan.ts";
6061 import { DEFAULT_SIZE, isSize, type Size } from "../spawn/sizes.ts";
6162 import { chainForChild, DEFAULT_MODE, forgeLinkOf, type FollowMode, type Hop } from "../spawn/chain.ts";
@@ -114,6 +115,8 @@ interface Ledger {
114115 readonly forWhom?: string;
115116 readonly apps?: readonly string[];
116117 readonly createdAt?: string;
118 /** The person who made it, counted against their forges (`spawn/sleep.ts`); absent for a stranger's. */
119 readonly ownerId?: string;
117120 /** Absent on a ledger written before sizes; reads as small. */
118121 readonly size?: Size;
119122 /** The template chosen (`spawn/templates.ts`); absent means `blank`. */
@@ -255,6 +258,10 @@ async function make(
255258 // A retry of an unfinished child is a new attempt: its failure has a start.
256259 app.bus.append({ kind: "forge.spawn.started", payload: { ...(deps.drill === true ? { drill: 1 } : {}) } });
257260 } else {
261 if (owner !== undefined) {
262 const many = personRefusal(app.config.dataDir, owner);
263 if (many !== undefined) return err({ kind: "refused", reason: "quota", message: many });
264 }
258265 const over = quotaRefusal(app.config.dataDir, plan.size ?? DEFAULT_SIZE);
259266 if (over !== undefined) return err({ kind: "refused", reason: "quota", message: over });
260267 const placed = await deps.placement.place(plan);
@@ -272,6 +279,7 @@ async function make(
272279 ...(plan.forWhom === undefined ? {} : { forWhom: plan.forWhom }),
273280 apps: plan.apps.map((a) => a.ref),
274281 createdAt: new Date().toISOString(),
282 ...(owner === undefined ? {} : { ownerId: owner.id }),
275283 size: plan.size ?? DEFAULT_SIZE,
276284 ...(plan.template === undefined ? {} : { template: plan.template }),
277285 placement: placed.value.kind ?? "hosted",
src/core/event_audience.ts +2
@@ -106,6 +106,8 @@ const READER_KINDS: ReadonlySet<string> = new Set([
106106 "team.member_removed",
107107 "forge.spawn.started",
108108 "forge.spawned",
109 "forge.removed",
110 "forge.stopped",
109111 "forge.claim_reissued",
110112 "agent.created",
111113 "agent.updated",
src/http/server.ts +15 −2
@@ -244,7 +244,7 @@ import {
244244 import { claimFragmentPage, claimOptions, claimSubmit } from "../web/pages/claim.ts";
245245 import { firstAdminOptions, firstAdminPage, firstAdminSubmit } from "../web/pages/first-admin.ts";
246246 import { familyPage } from "../web/pages/family.ts";
247 import { forgeLinkSubmit, forgesPage } from "../web/pages/forges.ts";
247 import { forgeLinkSubmit, forgeRemoveConfirm, forgeRemoveSubmit, forgeStopSubmit, forgesPage } from "../web/pages/forges.ts";
248248 import { forgeItSubmit } from "../web/pages/forge-it.ts";
249249 import { newForgePage, newForgeSubmit } from "../web/pages/new-forge.ts";
250250 import { askedPage } from "../web/studio/asked_page.ts";
@@ -533,10 +533,12 @@ import { portInsteadOfSocket, tcpAddress } from "../git/hook-address.ts";
533533 import {
534534 answerFromCell,
535535 closeHostedChildren,
536 holdChild,
536537 hostedChildOpen,
537538 resolveHostedChild,
538539 resolveHostedChildApp,
539540 socketToCell,
541 wakeGate,
540542 watchHostedChildClose,
541543 } from "../spawn/host.ts";
542544 import { getVerifiedDomain } from "../core/domains.ts";
@@ -770,6 +772,9 @@ export function buildRouter(): Router<Handler> {
770772 router.add("GET", "/forges", forgesPage);
771773 router.add("GET", "/family", familyPage);
772774 router.add("POST", "/forges/:name/claim-link", forgeLinkSubmit);
775 router.add("POST", "/forges/:name/stop", forgeStopSubmit);
776 router.add("GET", "/forges/:name/remove", forgeRemoveConfirm);
777 router.add("POST", "/forges/:name/remove", forgeRemoveSubmit);
773778 router.add("GET", "/forges/new", newForgePage);
774779 router.add("POST", "/forges/new", newForgeSubmit);
775780 // The front page as a stranger sees it, for anyone; a site admin there
@@ -2441,6 +2446,8 @@ export async function serve(app: App): Promise<Servers> {
24412446 // child's own App before the parent's pages. Ingress routes the label
24422447 // here; forge terminates Host routing in-process.
24432448 {
2449 const gated = wakeGate(app, request);
2450 if (gated !== undefined) return securityFloor(gated);
24442451 const child = await resolveHostedChild(app, request);
24452452 if (child === "unknown")
24462453 return securityFloor(new Response("not found", { status: 404 }));
@@ -2465,7 +2472,13 @@ export async function serve(app: App): Promise<Servers> {
24652472 }
24662473 notePeer(request, server.requestIP(request)?.address, trust);
24672474 armClose(request, () => server.timeout(request, 1));
2468 const response = await answered(child, request, () => http.handle(request));
2475 const release = holdChild(app, child);
2476 let response: Response;
2477 try {
2478 response = await answered(child, request, () => http.handle(request));
2479 } finally {
2480 release();
2481 }
24692482 counted(child, request, response, started);
24702483 logged(key, request, response, started);
24712484 await settleBody(request);
src/spawn/cell.ts +35
@@ -107,6 +107,12 @@ export interface CellChildren {
107107 /** Why the last start failed, in a sentence a person can read; undefined when it did not. */
108108 failure(bodyId: string): string | undefined;
109109 list(): readonly CellChild[];
110 /** Mark a request under way for this child; the returned function ends it. A child with one is not put to sleep. */
111 hold(bodyId: string): () => void;
112 /** Note that a request just reached this child, so it is not idle. */
113 touch(bodyId: string, now?: number): void;
114 /** Stop every running child quiet for `idleMs`; the names of those stopped (their data stays on disk). */
115 sleepIdle(idleMs: number, now?: number): Promise<string[]>;
110116 }
111117
112118 const registries = new WeakMap<App, CellChildren>();
@@ -137,6 +143,10 @@ function makeRegistry(parent: App): CellChildren {
137143 /** When a start last failed: a broken child is not relaunched on every request. */
138144 const failed = new Map<string, { at: number; why: string }>();
139145 /** Held once it answers, never before: a start in flight is in `starting`. */
146 /** When a request last reached each running child, ms. */
147 const seen = new Map<string, number>();
148 /** Requests under way, by body id. */
149 const busy = new Map<string, number>();
140150
141151 const launch = async (input: { bodyId: string; name: string; size: Size }): Promise<CellChild | undefined> => {
142152 const cells = await cellsAvailable();
@@ -202,6 +212,7 @@ function makeRegistry(parent: App): CellChildren {
202212 return undefined;
203213 }
204214 held.set(input.bodyId, child);
215 seen.set(input.bodyId, Date.now());
205216 return child;
206217 };
207218
@@ -239,6 +250,30 @@ function makeRegistry(parent: App): CellChildren {
239250 get: (bodyId) => held.get(bodyId),
240251 failure: (bodyId) => failed.get(bodyId)?.why,
241252 list: () => [...held.values()],
253 hold(bodyId) {
254 busy.set(bodyId, (busy.get(bodyId) ?? 0) + 1);
255 let ended = false;
256 return () => {
257 if (ended) return;
258 ended = true;
259 const left = (busy.get(bodyId) ?? 1) - 1;
260 if (left <= 0) busy.delete(bodyId);
261 else busy.set(bodyId, left);
262 };
263 },
264 touch(bodyId, now = Date.now()) {
265 if (held.has(bodyId)) seen.set(bodyId, now);
266 },
267 async sleepIdle(idleMs, now = Date.now()) {
268 const asleep: string[] = [];
269 for (const child of [...held.values()]) {
270 if (now - (seen.get(child.bodyId) ?? now) < idleMs || (busy.get(child.bodyId) ?? 0) > 0) continue;
271 asleep.push(child.name);
272 seen.delete(child.bodyId);
273 await this.stop(child.bodyId);
274 }
275 return asleep;
276 },
242277 };
243278 }
244279
src/spawn/child-apps.ts +10
@@ -82,6 +82,16 @@ export async function stopChildApps(parent: App): Promise<void> {
8282 }
8383 }
8484
85 /** Stop one child's apps and checks (it is going to sleep). */
86 export async function stopChildAppsOf(parent: App, name: string): Promise<void> {
87 const held = running.get(parent);
88 const entry = held?.get(name);
89 if (!held || !entry) return;
90 held.delete(name);
91 await entry.checks?.stop();
92 await entry.runtime.stop();
93 }
94
8595 export interface ChildAppsSummary {
8696 readonly total: number;
8797 readonly running: number;
src/spawn/clients.ts +4 −1
@@ -14,7 +14,7 @@ import { childDirOf, listChildren, statusTokenOf, type Child } from "../actions/
1414 import { timingSafeEqual } from "node:crypto";
1515 import { sha256 } from "../kernel/hash.ts";
1616 import type { ChildAppsSummary } from "./child-apps.ts";
17 import { hostedChildApps } from "./host.ts";
17 import { childAwake, hostedChildApps } from "./host.ts";
1818 import { hasAdmin, spawnedOrg } from "./claim.ts";
1919 import { lastHeard, type Beat } from "./heartbeat.ts";
2020 import { reachOf, reachWords, type ChildReach } from "./reach.ts";
@@ -38,6 +38,8 @@ export interface ClientRow {
3838 readonly made: boolean;
3939 readonly claimed: boolean | null;
4040 readonly lastActivityAt: string | null;
41 /** Whether its process is running now; a sleeping forge costs nothing and wakes on its next request. */
42 readonly awake?: boolean;
4143 /** The apps this parent runs for the child; null when it runs none here. */
4244 readonly running: ChildAppsSummary | null;
4345 /** What a visitor got from its public addresses when last asked (`spawn/reach.ts`); null before it was. */
@@ -204,6 +206,7 @@ export async function clientRows(app: App): Promise<ClientRow[]> {
204206 made: c.done,
205207 claimed: states[i]?.claimed ?? null,
206208 lastActivityAt: states[i]?.lastActivityAt ?? null,
209 awake: childAwake(app, c.name),
207210 running: hostedChildApps(app, c.name) ?? null,
208211 reach: reachOf(app, c.name) ?? null,
209212 spendTodayUsd: beats[i]?.beat.spendTodayUsd ?? null,
src/spawn/host.ts +218 −7
@@ -9,7 +9,7 @@ import { childDirOf } from "../actions/spawn.ts";
99 import { createApp, type App } from "../app.ts";
1010 import { appsRuntime, type AppsRuntime } from "../apps/runtime.ts";
1111 import { resolveAppHost, type AppSocket, type AppTarget } from "../apps/proxy.ts";
12 import { childAppsSummary, startChildApps, stopChildApps } from "./child-apps.ts";
12 import { childAppsSummary, startChildApps, stopChildApps, stopChildAppsOf } from "./child-apps.ts";
1313 import { childHost } from "./door.ts";
1414 import { markHostedChild } from "./hosted-mark.ts";
1515 import { firstBoot } from "./first-boot.ts";
@@ -25,6 +25,10 @@ import {
2525 import { DEFAULT_SIZE, isSize, type Size } from "./sizes.ts";
2626 import { cellChildren, forward, forwardSocket, type CellChild } from "./cell.ts";
2727 import { cellsAvailable } from "../apps/contain.ts";
28 import { awakeRefusal, idleAfterMs, WakeRate } from "./sleep.ts";
29 import { wakingResponse } from "../web/pages/waking.ts";
30 import { takeDoor } from "../http/strangers.ts";
31 import { vouched } from "./vouch.ts";
2832 import { readdirSync, readFileSync, renameSync, writeFileSync } from "node:fs";
2933 import { join } from "node:path";
3034
@@ -95,8 +99,191 @@ export function hostedChildOpen(parent: App, child: App): boolean {
9599 return false;
96100 }
97101
102 /** When a request last reached each open hosted child, by body id, with the name it goes by. */
103 const touched = new WeakMap<App, Map<string, { name: string; at: number }>>();
104 const wakeRates = new WeakMap<App, WakeRate>();
105 const sweepers = new WeakMap<App, ReturnType<typeof setInterval>>();
106
107 function touch(parent: App, ledger: { bodyId: string; name: string }, now = Date.now()): void {
108 let m = touched.get(parent);
109 if (!m) touched.set(parent, (m = new Map()));
110 m.set(ledger.bodyId, { name: ledger.name, at: now });
111 }
112
113 /** Forges awake now: hosted children open in this process plus children running in a cell. */
114 export function awakeCount(parent: App): number {
115 const awake = (caches.get(parent)?.size ?? 0) + cellChildren(parent).list().length;
116 // A wake granted but not finished holds its slot, so two at once cannot both pass the ceiling.
117 return awake + reservedCount(parent);
118 }
119
120 /** Wakes granted and not yet awake, by body id, with when. */
121 const waking = new WeakMap<App, Map<string, number>>();
122 const RESERVE_MS = 30_000;
123
124 function reservedCount(parent: App, now = Date.now()): number {
125 const m = waking.get(parent);
126 if (!m) return 0;
127 let n = 0;
128 for (const [bodyId, at] of [...m]) {
129 if (now - at >= RESERVE_MS || isAwakeBody(parent, bodyId)) m.delete(bodyId);
130 else n++;
131 }
132 return n;
133 }
134
135 const isAwakeBody = (parent: App, bodyId: string): boolean =>
136 caches.get(parent)?.has(bodyId) === true || cellChildren(parent).get(bodyId) !== undefined;
137
138 /** Requests being answered now, by body id: a forge answering one does not sleep under it. */
139 const inFlight = new WeakMap<App, Map<string, number>>();
140
141 /** Mark a request as under way for this child; the function it answers ends it. */
142 export function holdChild(parent: App, child: App): () => void {
143 const m = inFlight.get(parent) ?? new Map<string, number>();
144 inFlight.set(parent, m);
145 const key = child.config.dataDir;
146 m.set(key, (m.get(key) ?? 0) + 1);
147 let ended = false;
148 return () => {
149 if (ended) return;
150 ended = true;
151 const left = (m.get(key) ?? 1) - 1;
152 if (left <= 0) m.delete(key);
153 else m.set(key, left);
154 };
155 }
156
157 /** Whether the child is mid-request or has crew work queued or running: it is not idle however quiet its visitors. */
158 function working(parent: App, child: App): boolean {
159 if ((inFlight.get(parent)?.get(child.config.dataDir) ?? 0) > 0) return true;
160 try {
161 return child.db.query("SELECT 1 FROM run WHERE state IN ('queued','running') LIMIT 1").get() !== null;
162 } catch {
163 return false;
164 }
165 }
166
167 /**
168 * Put to sleep every child quiet for `idleMs`: a hosted child's apps stop and its App
169 * closes, a cell child's process stops. Their data stays on disk; the next request
170 * wakes them. Answers the names put to sleep.
171 */
172 export async function sleepIdleChildren(parent: App, idleMs = idleAfterMs(), now = Date.now()): Promise<string[]> {
173 const asleep: string[] = [];
174 const cache = caches.get(parent);
175 for (const [bodyId, child] of [...(cache ?? [])]) {
176 const mark = touched.get(parent)?.get(bodyId);
177 if (mark === undefined || now - mark.at < idleMs || working(parent, child)) continue;
178 asleep.push(await closeOne(parent, cache!, bodyId, child, mark.name));
179 }
180 const cellsAsleep = await cellChildren(parent).sleepIdle(idleMs, now);
181 for (const [bodyId, mark] of [...(touched.get(parent) ?? [])]) if (cellsAsleep.includes(mark.name)) {
182 touched.get(parent)?.delete(bodyId);
183 waking.get(parent)?.delete(bodyId);
184 }
185 asleep.push(...cellsAsleep);
186 for (const name of asleep) parent.log.info("child forge asleep", { name });
187 return asleep;
188 }
189
190 /** Put one named child to sleep now (its owner stopped it); true when it was awake. */
191 export async function sleepChild(parent: App, name: string): Promise<boolean> {
192 const ledger = readLedgerSync(parent, name);
193 if (ledger === undefined) return false;
194 const cache = caches.get(parent);
195 const child = cache?.get(ledger.bodyId);
196 if (cache !== undefined && child !== undefined) {
197 await closeOne(parent, cache, ledger.bodyId, child, name);
198 return true;
199 }
200 if (cellChildren(parent).get(ledger.bodyId) === undefined) return false;
201 touched.get(parent)?.delete(ledger.bodyId);
202 waking.get(parent)?.delete(ledger.bodyId);
203 await cellChildren(parent).stop(ledger.bodyId);
204 return true;
205 }
206
207 /** Whether a child is awake: open in this process or running in a cell. */
208 export function childAwake(parent: App, name: string): boolean {
209 const ledger = readLedgerSync(parent, name);
210 if (ledger === undefined) return false;
211 return caches.get(parent)?.has(ledger.bodyId) === true || cellChildren(parent).get(ledger.bodyId) !== undefined;
212 }
213
214 async function closeOne(parent: App, cache: Map<string, App>, bodyId: string, child: App, name: string): Promise<string> {
215 cache.delete(bodyId);
216 touched.get(parent)?.delete(bodyId);
217 waking.get(parent)?.delete(bodyId);
218 for (const onClose of closeWatchers.get(parent) ?? []) {
219 try {
220 onClose(child);
221 } catch (error) {
222 parent.log.warn("a hosted child's close watcher failed", { error: String(error) });
223 }
224 }
225 await stopChildAppsOf(parent, name).catch(() => {});
226 child.close();
227 return name;
228 }
229
230 function keepSweeping(parent: App): void {
231 if (sweepers.has(parent)) return;
232 const every = Math.max(1000, Math.min(60_000, Math.floor(idleAfterMs() / 4)));
233 const timer = setInterval(() => void sleepIdleChildren(parent).catch(() => {}), every);
234 timer.unref?.();
235 sweepers.set(parent, timer);
236 }
237
238 /** Whether the child itself vouches for the request's session or token: a header a client merely wrote is not enough. */
239 const carriesCredentials = (parent: App, bodyId: string, request: Request): boolean =>
240 vouched(childDirOf(parent, bodyId), request, caches.get(parent)?.get(bodyId)?.db);
241
242 /**
243 * Before a child's Host is answered: a child already awake is marked busy (by a
244 * request whose session or token the child verifies, so a stranger cannot keep it awake); one
245 * asleep is woken within the limits (a stranger's address first, then this host's
246 * awake ceiling, then, for strangers, the child's wakes an hour), else a plain
247 * refusal. The slot is reserved here. Undefined means carry on.
248 */
249 export function wakeGate(parent: App, request: Request): Response | undefined {
250 const name = parseHostedChildName(request.headers.get("host") ?? "", childHost(parent));
251 if (name === undefined) return undefined;
252 const ledger = readLedgerSync(parent, name);
253 if (ledger === undefined || !ledger.done) return undefined;
254 keepSweeping(parent);
255 const known = carriesCredentials(parent, ledger.bodyId, request);
256 if (isAwakeBody(parent, ledger.bodyId)) {
257 if (known) {
258 touch(parent, ledger);
259 cellChildren(parent).touch(ledger.bodyId);
260 }
261 return undefined;
262 }
263 // A wake already granted (a cell takes a moment to answer) is one wake, not one per request.
264 let reserved = waking.get(parent);
265 if (!reserved) waking.set(parent, (reserved = new Map()));
266 if (reservedCount(parent) > 0 && reserved.has(ledger.bodyId)) return undefined;
267 const verdict = takeDoor(parent, "api", request);
268 if (!verdict.allowed) return wakingResponse(parent, request, name, "too many requests to sleeping forges from your address; try again in a minute");
269 let rates = wakeRates.get(parent);
270 if (!rates) wakeRates.set(parent, (rates = new WakeRate()));
271 const why = awakeRefusal(awakeCount(parent)) ?? (known ? undefined : rates.refusal(ledger.bodyId));
272 if (why !== undefined) return wakingResponse(parent, request, name, why);
273 if (!known) rates.note(ledger.bodyId);
274 reserved.set(ledger.bodyId, Date.now());
275 touch(parent, ledger);
276 return undefined;
277 }
278
98279 /** Drop every hosted child App this parent has open (call when the parent stops). */
99280 export function closeHostedChildren(parent: App): void {
281 const sweeper = sweepers.get(parent);
282 if (sweeper !== undefined) clearInterval(sweeper);
283 sweepers.delete(parent);
284 touched.delete(parent);
285 waking.delete(parent);
286 inFlight.delete(parent);
100287 const m = caches.get(parent);
101288 if (!m) return;
102289 const children = [...m.values()];
@@ -220,6 +407,7 @@ function openOnce(parent: App, ledger: ChildLedger): Promise<App> | App {
220407 if (!flight) {
221408 flight = openChild(parent, ledger).then((opened) => {
222409 cache.set(ledger.bodyId, opened);
410 touch(parent, ledger);
223411 return opened;
224412 });
225413 flight.finally(() => flights.delete(ledger.bodyId)).catch(() => {});
@@ -320,14 +508,37 @@ export async function answerFromCell(parent: App, request: Request, named?: stri
320508 const name = named ?? parseHostedChildName(request.headers.get("host") ?? "", childHost(parent));
321509 const ledger = name === undefined ? undefined : readLedgerSync(parent, name);
322510 if (ledger === undefined || ledger.placement !== "cell") return new Response("not found", { status: 404 });
323 const child = cellChildren(parent).get(ledger.bodyId);
324 if (child !== undefined) return forward(child, request, parent.config.clientIpHeader);
325 // Never a wait on a cell that may not come up: it starts behind this
326 // answer, and once it has failed the ledger says hosted.
327 void settleCell(parent, ledger);
328 return new Response("this forge is starting", { status: 503, headers: { "retry-after": "5", "cache-control": "no-store" } });
511 const cells = cellChildren(parent);
512 const child = cells.get(ledger.bodyId);
513 if (child !== undefined) {
514 const release = cells.hold(ledger.bodyId);
515 try {
516 return await forward(child, request, parent.config.clientIpHeader);
517 } finally {
518 release();
519 }
520 }
521 // Asleep: it wakes behind this answer. A git push or an API call waits a few
522 // seconds for it; a page, or a wake that takes longer, is a plain "waking up" page.
523 // A page never waits: it starts the cell and gets the waking page at once.
524 const path = new URL(request.url).pathname;
525 const patient = path.startsWith("/api/") || path.includes(".git/");
526 const starting = settleCell(parent, ledger);
527 const woke = patient ? await Promise.race([starting, Bun.sleep(WAKE_WAIT_MS).then(() => undefined)]) : undefined;
528 if (woke !== undefined) {
529 const release = cells.hold(ledger.bodyId);
530 try {
531 return await forward(woke, request, parent.config.clientIpHeader);
532 } finally {
533 release();
534 }
535 }
536 return wakingResponse(parent, request, ledger.name);
329537 }
330538
539 /** How long a request waits for a sleeping cell forge before it is told it is waking. */
540 const WAKE_WAIT_MS = 4000;
541
331542 /**
332543 * A websocket handshake for a child in its own cell (its home or one of its
333544 * apps): carried to the cell like any request. `undefined` means it became a
src/spawn/manage.ts added +37
@@ -0,0 +1,37 @@
1 /**
2 * The owner's two hands on a hosted forge: stop it (put it to sleep now; the
3 * next request wakes it) and remove it (its ledger and its data, for good).
4 * Both are the site admin's, and both write an event.
5 */
6
7 import { rm } from "node:fs/promises";
8 import { join } from "node:path";
9 import type { App } from "../app.ts";
10 import { childDirOf, listChildren } from "../actions/spawn.ts";
11 import { sleepChild } from "./host.ts";
12 import { CHILD_NAME } from "./placement.ts";
13
14 export type Managed = { readonly ok: true; readonly says: string } | { readonly ok: false; readonly says: string };
15
16 async function find(app: App, name: string) {
17 if (!CHILD_NAME.test(name)) return undefined;
18 return (await listChildren(app)).find((c) => c.name === name);
19 }
20
21 export async function stopForge(app: App, name: string): Promise<Managed> {
22 const child = await find(app, name);
23 if (child === undefined) return { ok: false, says: `no forge named ${name}` };
24 const was = await sleepChild(app, name);
25 if (was) app.bus.append({ kind: "forge.stopped", payload: { name } });
26 return { ok: true, says: was ? `${name} is asleep; its next visitor wakes it` : `${name} was already asleep` };
27 }
28
29 export async function removeForge(app: App, name: string): Promise<Managed> {
30 const child = await find(app, name);
31 if (child === undefined) return { ok: false, says: `no forge named ${name}` };
32 await sleepChild(app, name);
33 await rm(join(app.config.dataDir, "spawn", `${name}.json`), { force: true });
34 await rm(childDirOf(app, child.bodyId), { recursive: true, force: true });
35 app.bus.append({ kind: "forge.removed", payload: { name } });
36 return { ok: true, says: `${name} is removed, with its data` };
37 }
src/spawn/sleep.ts added +95
@@ -0,0 +1,95 @@
1 /**
2 * Hosted forges that scale to zero: how long a forge may be quiet before it
3 * sleeps, how many may be awake at once, how often one may be woken, and how
4 * many a person may own. Each limit is declared by an environment variable
5 * with a default, and each refusal is a sentence a person can read. A sleeping
6 * forge keeps its data on disk (`children/<bodyId>`); only its process goes.
7 */
8
9 import { readdirSync, readFileSync } from "node:fs";
10 import { join } from "node:path";
11
12 type Env = Record<string, string | undefined>;
13
14 function whole(env: Env, name: string, fallback: number): number {
15 const v = env[name];
16 if (v === undefined || !/^[0-9]{1,9}$/.test(v)) return fallback;
17 const n = Number(v);
18 return n > 0 ? n : fallback;
19 }
20
21 /** A forge quiet for this long sleeps: FORGE_CHILD_IDLE_S, ten minutes by default. */
22 export const idleAfterMs = (env: Env = process.env): number => whole(env, "FORGE_CHILD_IDLE_S", 600) * 1000;
23 /** Most hosted forges awake at once on this host: FORGE_CHILD_AWAKE_MAX. A sleeping forge does not count. */
24 export const awakeCeiling = (env: Env = process.env): number => whole(env, "FORGE_CHILD_AWAKE_MAX", 8);
25 /** Most wakes of one forge in an hour: FORGE_CHILD_WAKES_PER_HOUR. */
26 export const wakesPerHour = (env: Env = process.env): number => whole(env, "FORGE_CHILD_WAKES_PER_HOUR", 20);
27 /** Most forges one person may own: FORGE_FORGES_PER_PERSON. */
28 export const forgesPerPerson = (env: Env = process.env): number => whole(env, "FORGE_FORGES_PER_PERSON", 3);
29
30 const HOUR_MS = 3_600_000;
31
32 /** Wakes per forge in the last hour; in memory, since a restart wakes nothing. */
33 export class WakeRate {
34 private readonly at = new Map<string, number[]>();
35 private recent(bodyId: string, now: number): number[] {
36 const kept = (this.at.get(bodyId) ?? []).filter((t) => now - t < HOUR_MS);
37 this.at.set(bodyId, kept);
38 return kept;
39 }
40 /** Why this forge may not be woken now, or undefined. */
41 refusal(bodyId: string, now = Date.now(), env: Env = process.env): string | undefined {
42 const max = wakesPerHour(env);
43 return this.recent(bodyId, now).length >= max
44 ? `this forge has been woken ${max} times in the last hour, the most it may be; try again in a little while`
45 : undefined;
46 }
47 note(bodyId: string, now = Date.now()): void {
48 this.recent(bodyId, now).push(now);
49 }
50 }
51
52 /** Why this host will not wake one more forge while `awake` are running, or undefined. */
53 export function awakeRefusal(awake: number, env: Env = process.env): string | undefined {
54 const max = awakeCeiling(env);
55 return awake >= max
56 ? `${awake} forges are awake on this host right now, the most it runs at once; try again in a minute`
57 : undefined;
58 }
59
60 /** How many forges, finished or begun, this person owns (the ledger's `ownerId`). */
61 export function ownedBy(dataDir: string, ownerId: string): number {
62 let n = 0;
63 let files: string[] = [];
64 try {
65 files = readdirSync(join(dataDir, "spawn"));
66 } catch {
67 return 0;
68 }
69 for (const f of files) {
70 if (!f.endsWith(".json")) continue;
71 try {
72 if ((JSON.parse(readFileSync(join(dataDir, "spawn", f), "utf8")) as { ownerId?: unknown }).ownerId === ownerId) n++;
73 } catch {
74 // a damaged ledger counts for no one
75 }
76 }
77 return n;
78 }
79
80 /** Why this person may not make another forge, or undefined. Site admins are not limited. */
81 export function personRefusal(dataDir: string, owner: { id: string; isAdmin: boolean }, env: Env = process.env): string | undefined {
82 if (owner.isAdmin) return undefined;
83 const max = forgesPerPerson(env);
84 const has = ownedBy(dataDir, owner.id);
85 return has >= max ? `you already have ${has} forges, the most one person may have (${max}); remove one first` : undefined;
86 }
87
88 /** The plain-text "waking up" or refusal answer: for a git client or an API call (a page is `web/pages/waking.ts`). */
89 export function wakingText(name: string, why?: string): Response {
90 const message = why ?? `${name} was asleep and is waking up; this takes a few seconds`;
91 return new Response(message, {
92 status: why === undefined ? 503 : 429,
93 headers: { "retry-after": "5", "cache-control": "no-store", "content-type": "text/plain; charset=utf-8" },
94 });
95 }
src/spawn/vouch.ts added +86
@@ -0,0 +1,86 @@
1 /**
2 * Whether a request to a hosted child carries a credential that child would
3 * accept: a live session cookie, or a live token (Bearer, `token`, or as a
4 * Basic password), of an active account in the child's own database. Only a
5 * verified credential is kind to the wake limits or keeps a forge awake; a
6 * header a stranger merely wrote counts for nothing.
7 *
8 * Read-only: no last-seen writes, so a sleeping forge's database is only read.
9 * A password in Basic is not checked here (hashing it per request is the cost
10 * the limits exist to avoid); that request is a stranger's to the limits.
11 */
12
13 import { Database } from "bun:sqlite";
14 import { existsSync } from "node:fs";
15 import { join } from "node:path";
16 import { cookieValue } from "../http/context.ts";
17 import { tokenSecret } from "../http/token_header.ts";
18 import { sha256 } from "../kernel/hash.ts";
19 import { getUserById, isActive } from "../core/users.ts";
20
21 /** The secrets a request presents, unverified: a session cookie and a token. */
22 export function presented(request: Request): { session?: string; token?: string } {
23 const out: { session?: string; token?: string } = {};
24 const session = cookieValue(request, "forge_session");
25 if (session !== undefined && session !== "" && session.length <= 512) out.session = session;
26 const header = request.headers.get("authorization");
27 if (header !== null && header.length <= 4096) {
28 const secret = tokenSecret(header);
29 if (secret !== null) out.token = secret;
30 else if (header.startsWith("Basic ")) {
31 const decoded = Buffer.from(header.slice(6), "base64").toString();
32 const colon = decoded.indexOf(":");
33 const password = colon === -1 ? "" : decoded.slice(colon + 1);
34 if (password.startsWith("forge_")) out.token = password;
35 }
36 }
37 return out;
38 }
39
40 const live = (expires: string | null, now: number): boolean =>
41 expires === null || Date.parse(expires) > now;
42
43 function activeUser(db: Database, userId: string): boolean {
44 const user = getUserById(db, userId);
45 return user !== undefined && isActive(user);
46 }
47
48 /** Whether `db` vouches for the request's session or token. */
49 export function vouchedIn(db: Database, request: Request, now = Date.now()): boolean {
50 const { session, token } = presented(request);
51 if (session !== undefined) {
52 const row = db
53 .query<{ user_id: string; expires_at: string }, [string]>("SELECT user_id, expires_at FROM session WHERE token_hash = ?1")
54 .get(sha256(session));
55 if (row && live(row.expires_at, now) && activeUser(db, row.user_id)) return true;
56 }
57 if (token !== undefined) {
58 const row = db
59 .query<{ user_id: string; expires_at: string | null }, [string]>("SELECT user_id, expires_at FROM token WHERE token_hash = ?1")
60 .get(sha256(token));
61 if (row && live(row.expires_at, now) && activeUser(db, row.user_id)) return true;
62 }
63 return false;
64 }
65
66 /**
67 * Whether the child whose data is in `dataDir` vouches for the request. `open`
68 * is its database when it is open in this process; otherwise its file is
69 * opened read-only for the one look. A request presenting nothing opens nothing.
70 */
71 export function vouched(dataDir: string, request: Request, open?: Database): boolean {
72 const { session, token } = presented(request);
73 if (session === undefined && token === undefined) return false;
74 if (open !== undefined) return vouchedIn(open, request);
75 const path = join(dataDir, "forge.db");
76 if (!existsSync(path)) return false;
77 let db: Database | undefined;
78 try {
79 db = new Database(path, { readonly: true });
80 return vouchedIn(db, request);
81 } catch {
82 return false;
83 } finally {
84 db?.close();
85 }
86 }
src/tui/screens/forges.ts +30 −5
@@ -13,13 +13,16 @@ import { card } from "../ui/widgets.ts";
1313 import { drawFacts, runAsync } from "./common.ts";
1414 import type { PaletteEntry } from "./entries.ts";
1515 import { clientRows } from "../../spawn/clients.ts";
16 import { removeForge, stopForge } from "../../spawn/manage.ts";
17 import { confirmScreen, pickerScreen } from "./common.ts";
18 import type { Session } from "../session.ts";
1619
1720 /** [name, "Not reporting · for Maria · Claimed · 3 apps running"] per row; an empty line when none. */
1821 export function forgeLines(rows: readonly ClientRow[]): [string, string][] {
1922 if (rows.length === 0) return [["", "You have not sent a forge yet"]];
2023 return rows.flatMap((r): [string, string][] => [[
2124 r.name,
22 [r.attention === null ? null : ATTENTION_WORDS[r.attention], r.for === null ? null : `for ${r.for}`, CLAIM_WORDS[claimState(r)], runningWords(r), r.nextStep.words, spendWords(r)?.words ?? null, useLine(r.size, r.use), r.lastHeartbeatAt === null ? "no heartbeat yet" : `heartbeat ${timeAgo(r.lastHeartbeatAt)}`]
25 [r.made ? (r.awake === true ? "awake" : "asleep") : null, r.attention === null ? null : ATTENTION_WORDS[r.attention], r.for === null ? null : `for ${r.for}`, CLAIM_WORDS[claimState(r)], runningWords(r), r.nextStep.words, spendWords(r)?.words ?? null, useLine(r.size, r.use), r.lastHeartbeatAt === null ? "no heartbeat yet" : `heartbeat ${timeAgo(r.lastHeartbeatAt)}`]
2326 .filter((m): m is string => m !== null)
2427 .join(" · "),
2528 ], ...addressLines(r)]);
@@ -39,11 +42,17 @@ export function fitLines(lines: [string, string][], room: number): [string, stri
3942 return kept;
4043 }
4144
42 export function forgesScreen(ctx: Ctx, rows: readonly ClientRow[]): Screen {
45 export function forgesScreen(ctx: Ctx, rows: readonly ClientRow[], session?: Session): Screen {
4346 const lines = forgeLines(rows);
44 return {
47 // Stop and Remove are the page's buttons, through the same actions (`spawn/manage.ts`).
48 const choose = (title: string, act: (name: string) => void): void => {
49 const items = rows.filter((r) => r.made).map((r) => ({ id: r.name, label: r.name, hint: r.awake === true ? "awake" : "asleep" }));
50 if (items.length === 0 || session === undefined) return;
51 ctx.push(pickerScreen(ctx, screen, { title, items, onPick: (ids) => { if (ids[0] !== undefined) act(ids[0]); } }));
52 };
53 const screen: Screen = {
4554 title: "Forges you sent",
46 keys: [binding(["esc"], "close")],
55 keys: [binding(["s"], "stop one (sleep now)"), binding(["x"], "remove one"), binding(["esc"], "close")],
4756 render(area, s) {
4857 const c = ctx.theme.c;
4958 const w = Math.min(100, Math.max(30, area.w - 6));
@@ -55,9 +64,25 @@ export function forgesScreen(ctx: Ctx, rows: readonly ClientRow[]): Screen {
5564 },
5665 key(e) {
5766 if (e.key === "esc" || e.key === "q") ctx.pop();
67 else if (e.key === "s" && session !== undefined)
68 choose("Stop which forge?", (name) => void stopForge(session.app, name).then((done) => ctx.toast(done.says, done.ok ? "success" : "danger")));
69 else if (e.key === "x" && session !== undefined)
70 choose("Remove which forge?", (name) =>
71 ctx.push(
72 confirmScreen(ctx, screen, {
73 title: `Remove ${name}?`,
74 body: [`This deletes ${name} and everything in it. It cannot be undone.`],
75 actions: [{ label: "Remove it for good", variant: "danger" }, { label: "Keep it" }],
76 onChoose: (i) => {
77 if (i === 0) void removeForge(session.app, name).then((done) => ctx.toast(done.says, done.ok ? "success" : "danger"));
78 },
79 }),
80 ),
81 );
5882 return true;
5983 },
6084 };
85 return screen;
6186 }
6287
6388 /** The forges this one sent, as /forges lists them: an admin's only. */
@@ -67,6 +92,6 @@ export const entry: PaletteEntry = {
6792 hint: "each forge sent, claimed or not, and its apps",
6893 offered: (session) => session.user?.isAdmin === true,
6994 run: (session, ctx) => runAsync(ctx, async () => {
70 ctx.push(forgesScreen(ctx, await clientRows(session.app)));
95 ctx.push(forgesScreen(ctx, await clientRows(session.app), session));
7196 }),
7297 };
src/web/pages/agents.ts +7
@@ -10,6 +10,7 @@ import type { Ctx } from "../../http/context.ts";
1010 import { notePhase, takeSessionTime, timePhase } from "../../core/route_timing.ts";
1111 import { pageHead } from "../chrome.ts";
1212 import { html } from "../html.ts";
13 import { btn, card, field as formField } from "../ui.ts";
1314 import { page } from "../layout.ts";
1415 import { renderMarkdown } from "../markdown.ts";
1516 import { llmsBody } from "../standards.ts";
@@ -50,6 +51,12 @@ export function agentsPage(ctx: Ctx): Response {
5051 sub: html`What this forge is, and the steps an assistant can take for you, or tell you to take. The same
5152 words as plain text: <a href="/llms.txt">/llms.txt</a>.`,
5253 })}
54 ${card(html`<form method="post" action="/forges/new" class="stack" id="agent-forge">
55 <h2>Get a forge of your own</h2>
56 <p class="muted small">Free within limits while hosting is not sold. It sleeps when idle and wakes on its next visit; bring your own model credential or its crew stays off.</p>
57 ${formField({ label: "Name", name: "name", required: true })}
58 ${btn("Make my forge", { variant: "primary" })}
59 </form>`)}
5360 <article class="prose prose-page" id="agents-guide">${renderMarkdown(linkAddresses(withoutTitle(body)))}</article>`,
5461 }));
5562 }
src/web/pages/forges.ts +56
@@ -17,6 +17,8 @@ import { readForm } from "../form.ts";
1717 import { badge, btn, card, empty, row, time } from "../ui.ts";
1818 import { noRoom } from "../../spawn/door.ts";
1919 import { sizeWords, useWords } from "../../spawn/usage.ts";
20 import { listChildren } from "../../actions/spawn.ts";
21 import { removeForge, stopForge, type Managed } from "../../spawn/manage.ts";
2022 import { claimLinkBody } from "./new-forge.ts";
2123
2224
@@ -43,6 +45,8 @@ function forgeRow(r: ClientRow): Html {
4345 const state = claimState(r);
4446 const meta = [
4547 state === "unknown" ? html`it did not answer just now` : null,
48 r.made ? badge(r.awake === true ? "awake" : "asleep", r.awake === true ? { tone: "success" } : { variant: "outline" }) : null,
49 r.made ? html`${sizeWords(r.size)}` : null,
4650 r.for === null ? null : html`for ${r.for}`,
4751 r.apps.length === 0 ? null : html`${r.apps.join(", ")}`,
4852 runningNote(r),
@@ -80,6 +84,9 @@ function forgeRow(r: ClientRow): Html {
8084 ? html`<form method="post" action="/forges/${r.name}/claim-link">
8185 ${btn("Send a fresh link", { size: "sm", title: "A new claim link; the old one stops working" })}
8286 </form>`
87 : null}${r.made
88 ? html`<form method="post" action="/forges/${r.name}/stop" class="inline">${btn("Stop", { size: "sm", title: "Put it to sleep now; its next visitor wakes it" })}</form>
89 ${btn("Remove", { href: `/forges/${r.name}/remove`, size: "sm", title: "Asks first, then deletes this forge and its data for good" })}`
8390 : null}`,
8491 });
8592 }
@@ -192,3 +199,52 @@ export async function forgeLinkSubmit(ctx: Ctx): Promise<Response> {
192199 response.headers.set("cache-control", "no-store");
193200 return response;
194201 }
202
203 // POST /forges/:name/stop and /remove — the owner's hands on a hosted forge.
204 function manageSubmit(act: (app: Ctx["app"], name: string) => Promise<Managed>, title: string) {
205 return async (ctx: Ctx): Promise<Response> => {
206 const user = ctx.auth.user;
207 if (!user?.isAdmin) return notFoundPage(ctx);
208 const form = await readForm(ctx, "admin");
209 if (form instanceof Response) return form;
210 const done = await act(ctx.app, ctx.params["name"] ?? "");
211 const response = page(
212 {
213 ctx,
214 title,
215 body: html`<div class="narrow-wide">
216 ${pageHead(title)}
217 ${card(html`<p role="status">${done.says}</p><p class="mt">${btn("Back to your forges", { href: "/forges", size: "sm" })}</p>`)}
218 </div>`,
219 },
220 done.ok ? 200 : 404,
221 );
222 response.headers.set("cache-control", "no-store");
223 return response;
224 };
225 }
226 export const forgeStopSubmit = manageSubmit(stopForge, "Forge stopped");
227 export const forgeRemoveSubmit = manageSubmit(removeForge, "Forge removed");
228
229 // GET /forges/:name/remove — what removing would delete, and the one button that does it.
230 export async function forgeRemoveConfirm(ctx: Ctx): Promise<Response> {
231 const user = ctx.auth.user;
232 if (!user?.isAdmin) return notFoundPage(ctx);
233 const name = ctx.params["name"] ?? "";
234 const child = (await listChildren(ctx.app)).find((c) => c.name === name);
235 if (child === undefined) return notFoundPage(ctx);
236 const response = page({
237 ctx,
238 title: `Remove ${name}?`,
239 body: html`<div class="narrow-wide">
240 ${pageHead(`Remove ${name}?`)}
241 ${card(html`<p>This deletes the forge <strong>${name}</strong> at ${child.url.replace(/^https?:\/\//, "")} with everything in it: its repositories, issues, accounts and settings. It cannot be undone.</p>
242 <form method="post" action="/forges/${name}/remove" class="mt">
243 ${btn("Remove it for good", { variant: "primary" })}
244 ${btn("Keep it", { href: "/forges" })}
245 </form>`)}
246 </div>`,
247 });
248 response.headers.set("cache-control", "no-store");
249 return response;
250 }
src/web/pages/new-forge.ts +2 −1
@@ -156,6 +156,7 @@ function formView(
156156 : "Your own forge, or one for someone else: the apps you choose, a crew, and one link to claim.",
157157 actions: btn("Forges you sent", { href: "/forges", size: "sm" }),
158158 })}
159 <p class="muted small" id="hosting-terms">Free within limits while hosting is not sold: a few forges per person, and a forge sleeps when idle and wakes on its next visit. Its crew stays off until you bring your own model credential.</p>
159160 ${quota ? html`<p role="status" id="quota-full">This forge cannot hold another child: ${quota}.</p>` : null}
160161 ${errors["form"]
161162 ? html`<p class="error mb" role="alert">${errors["form"]}</p>`
@@ -416,7 +417,7 @@ export function newForgePage(ctx: Ctx): Response {
416417 const response = page({
417418 ctx,
418419 title: "New forge",
419 body: formView(ctx, user, EMPTY, {}, null, reads),
420 body: formView(ctx, user, { ...EMPTY, name: (ctx.url.searchParams.get("name") ?? "").slice(0, 38) }, {}, null, reads),
420421 });
421422 notePhase(ctx.request, "render", performance.now() - rendering);
422423 return response;
src/web/pages/waking.ts added +40
@@ -0,0 +1,40 @@
1 /**
2 * What a visitor sees at a sleeping forge's address: it is waking, or the host
3 * will not wake it right now. Drawn with the page shell and its components, so
4 * it follows the theme; a browser asks again by itself (`refresh`), a git
5 * client or an API call gets a sentence (`spawn/sleep.ts` `wakingText`).
6 */
7
8 import type { App } from "../../app.ts";
9 import type { Ctx } from "../../http/context.ts";
10 import { wakingText } from "../../spawn/sleep.ts";
11 import { btn, empty } from "../ui.ts";
12 import { html } from "../html.ts";
13 import { page } from "../layout.ts";
14
15 /** Waking (`why` undefined, 503) or refused (`why` a sentence, 429), as a page when the visitor asks for HTML. */
16 export function wakingResponse(app: App, request: Request, name: string, why?: string): Response {
17 if (!(request.headers.get("accept") ?? "").includes("text/html")) return wakingText(name, why);
18 const url = new URL(request.url);
19 const ctx: Ctx = { app, request, url, params: {}, auth: { user: undefined, scope: undefined, via: "none" }, theme: undefined };
20 const response = page(
21 {
22 ctx,
23 bare: true,
24 title: why === undefined ? "Waking up" : "Not now",
25 body: html`<div class="narrow-wide">
26 ${empty({
27 icon: "history",
28 title: why === undefined ? "Waking up" : "Not now",
29 note: why ?? `${name} was asleep. It is waking up and this page tries again by itself in a few seconds.`,
30 action: btn("Try again", { href: url.pathname, variant: "primary" }),
31 })}
32 </div>`,
33 },
34 why === undefined ? 503 : 429,
35 );
36 response.headers.set("retry-after", "5");
37 response.headers.set("cache-control", "no-store");
38 if (why === undefined) response.headers.set("refresh", "3");
39 return response;
40 }
src/web/standards.ts +1 −1
@@ -405,7 +405,7 @@ function renderLlmsTxt(ctx: Ctx, o: string, tell: string | undefined, running: s
405405 "If you are an assistant helping a person: say plainly what you can and cannot do from where you are. If you cannot reach this forge or hold an account on it, give the person these steps to do by hand.",
406406 "",
407407 `- Help build a public app here: each one is a repository at ${o}/{owner}/{name}. A person makes an account at ${o}/signup, then follows "Offer a change" below. If /signup answers 403, new accounts here are by invitation for now: send a suggestion through "Report a problem" below, or work from a forge of your own.`,
408 `- Get your own forge: \`git clone https://forge.open-platform.sh/trevato/forge.git && cd forge && bun install && bun run src/main.ts\`, then in a second terminal \`bun src/main.ts --first-admin\`, which prints a one-time link that makes you the admin. It needs Bun; its crew needs the claude CLI and a model credential. Hosted forges are not available yet.`,
408 `- Get your own forge: \`git clone https://forge.open-platform.sh/trevato/forge.git && cd forge && bun install && bun run src/main.ts\`, then in a second terminal \`bun src/main.ts --first-admin\`, which prints a one-time link that makes you the admin. It needs Bun; its crew needs the claude CLI and a model credential. Or a hosted one, free within limits while hosting is not sold: sign in and open ${o}/forges/new (or call the MCP \`spawn\` tool with the person's consent). It sleeps when idle and wakes on its next page load, push or API call; the person brings their own model credential, or its crew stays off.`,
409409 "- Work from your own forge: connect it to this one as an origin, bring an app in, and send your work back. How: https://forge.open-platform.sh/trevato/forge/raw/main/docs/ORIGINS.md",
410410 `- Connect an assistant that can use tools: add a custom connector with the URL ${mcp.url}. It signs in with OAuth; only the report tool works without an account.`,
411411 "",
test/forge-sleep.test.ts added +341
@@ -0,0 +1,341 @@
1 /**
2 * A hosted forge scales to zero: it sleeps after a quiet period, wakes on a
3 * page load and on a `git push` with its data kept, a sleeping forge is woken
4 * only within the host's limits (awake ceiling, wakes an hour, an address's
5 * rate), a stranger cannot hold one awake, a person owns only so many, and
6 * the owner stops and removes one (admin only, same-origin, confirmed).
7 */
8 import { afterAll, beforeAll, describe, expect, test } from "bun:test";
9 import { existsSync } from "node:fs";
10 import { mkdtemp, rm } from "node:fs/promises";
11 import { tmpdir } from "node:os";
12 import { join } from "node:path";
13 import { spawnForge } from "../src/actions/spawn.ts";
14 import { createApp, type App } from "../src/app.ts";
15 import { createSession } from "../src/core/sessions.ts";
16 import { createUser } from "../src/core/users.ts";
17 import { serve, type Servers } from "../src/http/server.ts";
18 import { awakeCount, childAwake, holdChild, resolveHostedChild, sleepIdleChildren, wakeGate } from "../src/spawn/host.ts";
19 import { removeForge, stopForge } from "../src/spawn/manage.ts";
20 import { awakeRefusal, personRefusal, WakeRate } from "../src/spawn/sleep.ts";
21 import { wakingResponse } from "../src/web/pages/waking.ts";
22 import { hostedPlacement } from "../src/spawn/placement.ts";
23 import { gitClientEnv } from "./helpers/git-client.ts";
24 import { quietLogs } from "./helpers/globals.ts";
25 import { promote } from "./helpers/members.ts";
26 import { tokenFor } from "./helpers/tokens.ts";
27
28 quietLogs();
29
30 const CHILD_HOST = "homie--forge.test";
31
32 async function git(args: string[], cwd: string): Promise<{ code: number; err: string }> {
33 const proc = Bun.spawn(["git", ...args], {
34 cwd,
35 env: gitClientEnv(),
36 stdout: "pipe",
37 stderr: "pipe",
38 });
39 const [err, code] = await Promise.all([new Response(proc.stderr).text(), proc.exited]);
40 return { code, err: err.trim() };
41 }
42
43
44 const later = () => Date.now() + 3_600_000;
45
46 describe("a hosted forge scales to zero", () => {
47 let dir = "";
48 let work = "";
49 let parent: App;
50 let servers: Servers;
51 let token = "";
52 const remote = () => `${servers.url.replace("http://", `http://bob:${token}@`)}/bob/app.git`;
53 const asChild = (path: string) =>
54 fetch(`${servers.url}${path}`, { headers: { host: CHILD_HOST, authorization: `Bearer ${token}` } });
55
56 beforeAll(async () => {
57 dir = await mkdtemp(join(tmpdir(), "forge-sleep-"));
58 work = await mkdtemp(join(tmpdir(), "forge-sleep-work-"));
59 parent = createApp({ dataDir: dir, url: "https://forge.test", port: 0 });
60 const made = await createUser(parent.db, { origin: "seed", trusted: true, username: "alice", email: "[email protected]", isAdmin: true });
61 if (!made.ok) throw new Error("user");
62 const spawned = await spawnForge(
63 parent,
64 made.value,
65 { name: "homie", org: "homie", apps: [], carryBriefs: false, carrySubscriptions: false },
66 { catalog: [], briefs: [], subscriptions: [] },
67 {
68 placement: hostedPlacement({ host: "forge.test" }),
69 request: new Request("https://forge.test/spawn", { method: "POST", headers: { origin: "https://forge.test" } }),
70 url: new URL("https://forge.test/spawn"),
71 },
72 );
73 if (!spawned.ok) throw new Error(JSON.stringify(spawned.error));
74 servers = await serve(parent);
75 const opened = await resolveHostedChild(parent, new Request("https://forge.test/healthz", { headers: { host: CHILD_HOST } }));
76 if (opened === undefined || opened === "unknown" || opened === "cell") throw new Error("child not hosted");
77 const bob = await createUser(opened.db, { origin: "seed", trusted: true, username: "bob", email: "[email protected]" });
78 if (!bob.ok) throw new Error("child user");
79 promote(opened.db, bob.value);
80 token = tokenFor(opened, "bob", "rw");
81 const created = await fetch(`${servers.url}/api/v1/repos`, {
82 method: "POST",
83 headers: { host: CHILD_HOST, authorization: `Bearer ${token}`, "content-type": "application/json" },
84 body: JSON.stringify({ name: "app" }),
85 });
86 if (created.status !== 201) throw new Error(`repo: ${created.status}`);
87 });
88
89 afterAll(async () => {
90 servers?.stop();
91 parent?.close();
92 await rm(work, { recursive: true, force: true });
93 await rm(dir, { recursive: true, force: true });
94 });
95
96 test("sleeps after a quiet period, wakes on a page load, and keeps its data", async () => {
97 expect(childAwake(parent, "homie")).toBe(true);
98 // Not quiet long enough: nothing sleeps.
99 expect(await sleepIdleChildren(parent, 600_000, Date.now())).toEqual([]);
100 expect(await sleepIdleChildren(parent, 600_000, later())).toEqual(["homie"]);
101 expect(childAwake(parent, "homie")).toBe(false);
102 expect(awakeCount(parent)).toBe(0);
103
104 const page = await fetch(`${servers.url}/healthz`, { headers: { host: CHILD_HOST } });
105 expect(page.status).toBe(200);
106 expect(childAwake(parent, "homie")).toBe(true);
107 const repo = await asChild("/api/v1/repos/bob/app");
108 expect(repo.status).toBe(200);
109 });
110
111 test("wakes on a git push, and the data from before the sleep is still there", async () => {
112 const repo = join(work, "app");
113 expect((await git(["init", "-q", "-b", "main", repo], work)).code).toBe(0);
114 await Bun.write(join(repo, "README.md"), "# app\n");
115 expect((await git(["add", "."], repo)).code).toBe(0);
116 expect((await git(["commit", "-qm", "genesis"], repo)).code).toBe(0);
117 const first = await git(["-c", `http.extraHeader=Host: ${CHILD_HOST}`, "push", remote(), "main"], repo);
118 expect(first.code).toBe(0);
119
120 expect(await sleepIdleChildren(parent, 600_000, later())).toEqual(["homie"]);
121 expect(childAwake(parent, "homie")).toBe(false);
122
123 await Bun.write(join(repo, "NEXT.md"), "next\n");
124 expect((await git(["add", "."], repo)).code).toBe(0);
125 expect((await git(["commit", "-qm", "after sleep"], repo)).code).toBe(0);
126 const second = await git(["-c", `http.extraHeader=Host: ${CHILD_HOST}`, "push", remote(), "main"], repo);
127 expect(second.err).not.toContain("hook relay failed");
128 expect(second.code).toBe(0);
129 expect(childAwake(parent, "homie")).toBe(true);
130
131 const events = (await (await asChild("/api/v1/repos/bob/app/events")).json()) as { items: { kind: string }[] };
132 expect(events.items.filter((e) => e.kind === "push").length).toBe(2);
133 });
134
135 test("a wake that would pass a limit is refused in plain words, and a sleeping forge is not counted", async () => {
136 expect(awakeRefusal(1, { FORGE_CHILD_AWAKE_MAX: "2" })).toBeUndefined();
137 expect(awakeRefusal(2, { FORGE_CHILD_AWAKE_MAX: "2" })).toContain("try again in a minute");
138 const rates = new WakeRate();
139 const env = { FORGE_CHILD_WAKES_PER_HOUR: "2" };
140 rates.note("b", 1000);
141 rates.note("b", 2000);
142 expect(rates.refusal("b", 3000, env)).toContain("woken 2 times");
143 expect(rates.refusal("b", 3000 + 3_600_000, env)).toBeUndefined();
144 const refused = wakingResponse(parent, new Request("https://x"), "homie", "too many");
145 expect(refused.status).toBe(429);
146 expect(await refused.text()).toBe("too many");
147 });
148
149 test("a person owns at most so many forges; a site admin is not limited", () => {
150 // Alice made homie: with a ceiling of 1 she (an admin) is still free, a person is not.
151 const env = { FORGE_FORGES_PER_PERSON: "1" };
152 const owner = parent.db.query("SELECT id FROM user WHERE username = 'alice'").get() as { id: string };
153 expect(personRefusal(dir, { id: owner.id, isAdmin: true }, env)).toBeUndefined();
154 expect(personRefusal(dir, { id: owner.id, isAdmin: false }, env)).toContain("remove one first");
155 expect(personRefusal(dir, { id: "someone-else", isAdmin: false }, env)).toBeUndefined();
156 });
157
158 test("a waking page is plain for a browser, and the gate lets an awake forge through", async () => {
159 const page = wakingResponse(parent, new Request("https://forge.test/", { headers: { accept: "text/html" } }), "homie");
160 expect(page.status).toBe(503);
161 expect(page.headers.get("refresh")).toBe("3");
162 const text = await page.text();
163 expect(text).toContain("Waking up");
164 expect(text).not.toContain("style=");
165 expect(wakeGate(parent, new Request("https://forge.test/", { headers: { host: CHILD_HOST } }))).toBeUndefined();
166 });
167
168 const asleepAgain = async () => {
169 await sleepIdleChildren(parent, 600_000, later());
170 expect(childAwake(parent, "homie")).toBe(false);
171 };
172 const gate = (headers: Record<string, string> = {}) =>
173 wakeGate(parent, new Request("https://forge.test/", { headers: { host: CHILD_HOST, ...headers } }));
174
175 test("the gate refuses a wake at the awake ceiling, and the slot is held so two wakes cannot both pass", async () => {
176 await asleepAgain();
177 const was = process.env["FORGE_CHILD_AWAKE_MAX"];
178 process.env["FORGE_CHILD_AWAKE_MAX"] = "1";
179 try {
180 expect(gate()).toBeUndefined(); // the first wake is granted and holds the one slot
181 expect(awakeCount(parent)).toBe(1);
182 // Another forge's wake now would pass the ceiling: the slot is reserved though nothing is open yet.
183 expect(awakeRefusal(awakeCount(parent))).toContain("try again in a minute");
184 } finally {
185 if (was === undefined) delete process.env["FORGE_CHILD_AWAKE_MAX"];
186 else process.env["FORGE_CHILD_AWAKE_MAX"] = was;
187 }
188 });
189
190 test("a stranger's wakes are counted against the forge's hour, a request with a session is not", async () => {
191 await asleepAgain();
192 await sleepIdleChildren(parent, 0, later()); // clears any reservation by the passage of the wake
193 const was = process.env["FORGE_CHILD_WAKES_PER_HOUR"];
194 process.env["FORGE_CHILD_WAKES_PER_HOUR"] = "1";
195 try {
196 // Reservations from the earlier test are cleared by opening and sleeping the forge.
197 await fetch(`${servers.url}/healthz`, { headers: { host: CHILD_HOST } });
198 await asleepAgain();
199 const strangers = [gate(), gate()];
200 expect(strangers.some((r) => r?.status === 429)).toBe(true);
201 const known = gate({ authorization: `Bearer ${token}` });
202 expect(known?.status ?? 0).not.toBe(429);
203 } finally {
204 if (was === undefined) delete process.env["FORGE_CHILD_WAKES_PER_HOUR"];
205 else process.env["FORGE_CHILD_WAKES_PER_HOUR"] = was;
206 }
207 });
208
209 test("a stranger's visit does not keep an awake forge awake; a signed-in one does", async () => {
210 await fetch(`${servers.url}/healthz`, { headers: { host: CHILD_HOST } });
211 expect(childAwake(parent, "homie")).toBe(true);
212 // Idle by the clock an hour from now: a stranger's request in between changes nothing.
213 expect(gate()).toBeUndefined();
214 expect(await sleepIdleChildren(parent, 600_000, later())).toEqual(["homie"]);
215 await fetch(`${servers.url}/healthz`, { headers: { host: CHILD_HOST } });
216 expect(gate({ authorization: `Bearer ${token}` })).toBeUndefined();
217 expect(await sleepIdleChildren(parent, 600_000, Date.now() + 1000)).toEqual([]);
218 });
219
220 test("a bogus Authorization header or session cookie is a stranger's: limited, and it keeps nothing awake", async () => {
221 const bogus = [{ authorization: "x" }, { authorization: "Bearer forge_not-a-real-token" }, { cookie: "forge_session=made-up" }];
222 const was = process.env["FORGE_CHILD_WAKES_PER_HOUR"];
223 process.env["FORGE_CHILD_WAKES_PER_HOUR"] = "1";
224 try {
225 await fetch(`${servers.url}/healthz`, { headers: { host: CHILD_HOST } });
226 await asleepAgain();
227 const answers = bogus.map((h) => gate(h));
228 expect(answers.some((r) => r?.status === 429)).toBe(true);
229 } finally {
230 if (was === undefined) delete process.env["FORGE_CHILD_WAKES_PER_HOUR"];
231 else process.env["FORGE_CHILD_WAKES_PER_HOUR"] = was;
232 }
233 await fetch(`${servers.url}/healthz`, { headers: { host: CHILD_HOST } });
234 expect(childAwake(parent, "homie")).toBe(true);
235 for (const h of bogus) expect(gate(h)).toBeUndefined();
236 expect(await sleepIdleChildren(parent, 600_000, later())).toEqual(["homie"]);
237 });
238
239 test("a real session cookie of the child's keeps it awake; a made-up one does not", async () => {
240 const child = (await resolveHostedChild(parent, new Request("https://forge.test/healthz", { headers: { host: CHILD_HOST } }))) as App;
241 const bob = child.db.query("SELECT id FROM user WHERE username = 'bob'").get() as { id: string };
242 const { token: session } = createSession(child.db, bob.id);
243 // Each round: the last touch is at least 30 ms old, one request, then a sweep that sleeps what is 20 ms idle.
244 const round = async (cookie: string) => {
245 await fetch(`${servers.url}/healthz`, { headers: { host: CHILD_HOST, authorization: `Bearer ${token}` } });
246 await Bun.sleep(30);
247 const t = Date.now();
248 expect(gate({ cookie })).toBeUndefined();
249 return sleepIdleChildren(parent, 20, t + 10);
250 };
251 expect(await round("forge_session=made-up")).toEqual(["homie"]);
252 expect(await round(`theme=dark; forge_session=${session}`)).toEqual([]);
253 });
254
255 test("a forge answering a request is not put to sleep under it", async () => {
256 const child = (await resolveHostedChild(parent, new Request("https://forge.test/healthz", { headers: { host: CHILD_HOST } }))) as App;
257 const release = holdChild(parent, child);
258 expect(await sleepIdleChildren(parent, 0, later())).toEqual([]);
259 release();
260 expect(await sleepIdleChildren(parent, 0, later())).toEqual(["homie"]);
261 });
262
263 test("a person's forge limit holds through spawnForge", async () => {
264 const was = process.env["FORGE_FORGES_PER_PERSON"];
265 process.env["FORGE_FORGES_PER_PERSON"] = "1";
266 try {
267 const carol = await createUser(parent.db, { origin: "seed", trusted: true, username: "carol", email: "[email protected]" });
268 if (!carol.ok) throw new Error("carol");
269 const make = (name: string) =>
270 spawnForge(
271 parent,
272 carol.value,
273 { name, org: name, apps: [], carryBriefs: false, carrySubscriptions: false },
274 { catalog: [], briefs: [], subscriptions: [] },
275 {
276 placement: hostedPlacement({ host: "forge.test" }),
277 request: new Request("https://forge.test/spawn", { method: "POST", headers: { origin: "https://forge.test" } }),
278 url: new URL("https://forge.test/spawn"),
279 },
280 );
281 expect((await make("carols-first")).ok).toBe(true);
282 const second = await make("carols-second");
283 expect(second.ok).toBe(false);
284 if (!second.ok) expect(JSON.stringify(second.error)).toContain("remove one first");
285 } finally {
286 if (was === undefined) delete process.env["FORGE_FORGES_PER_PERSON"];
287 else process.env["FORGE_FORGES_PER_PERSON"] = was;
288 }
289 });
290
291 test("stop and remove are an admin's, same-origin, and removal asks first", async () => {
292 const aliceToken = tokenFor(parent, "alice", "admin");
293 const carolToken = tokenFor(parent, "carol", "rw");
294 const post = (path: string, bearer: string, origin: string | undefined) =>
295 fetch(`${servers.url}${path}`, {
296 method: "POST",
297 redirect: "manual",
298 headers: { authorization: `Bearer ${bearer}`, "content-type": "application/x-www-form-urlencoded", ...(origin === undefined ? {} : { origin }) },
299 body: "",
300 });
301 expect((await post("/forges/homie/stop", carolToken, "https://forge.test")).status).toBe(404);
302 expect((await post("/forges/homie/remove", carolToken, "https://forge.test")).status).toBe(404);
303 expect(existsSync(join(dir, "spawn", "homie.json"))).toBe(true);
304 const cross = await post("/forges/homie/remove", aliceToken, "https://evil.example");
305 expect(cross.status).toBe(403);
306 expect(existsSync(join(dir, "spawn", "homie.json"))).toBe(true);
307 const confirm = await fetch(`${servers.url}/forges/homie/remove`, { headers: { authorization: `Bearer ${aliceToken}` } });
308 expect(confirm.status).toBe(200);
309 expect(await confirm.text()).toContain("cannot be undone");
310 const asCarol = await fetch(`${servers.url}/forges/homie/remove`, { headers: { authorization: `Bearer ${carolToken}` } });
311 expect(asCarol.status).toBe(404);
312 const stopped = await post("/forges/homie/stop", aliceToken, "https://forge.test");
313 expect(stopped.status).toBe(200);
314 });
315
316 test("/agents makes a forge in one step: its form posts the name straight to the spawn", async () => {
317 const aliceToken = tokenFor(parent, "alice", "admin");
318 const agents = await (await fetch(`${servers.url}/agents`)).text();
319 expect(agents).toContain('<form method="post" action="/forges/new" class="stack" id="agent-forge">');
320 // What that form sends, and nothing else: the forge is made, its claim link in the answer.
321 const made = await fetch(`${servers.url}/forges/new`, {
322 method: "POST",
323 headers: { authorization: `Bearer ${aliceToken}`, origin: "https://forge.test", "content-type": "application/x-www-form-urlencoded" },
324 body: "name=from-agents",
325 });
326 expect(made.status).toBeLessThan(300);
327 expect(await made.text()).toContain("claim");
328 expect(existsSync(join(dir, "spawn", "from-agents.json"))).toBe(true);
329 });
330
331 test("the owner stops a forge, then removes it with its data", async () => {
332 expect((await stopForge(parent, "homie")).says).toContain("asleep");
333 expect(childAwake(parent, "homie")).toBe(false);
334 expect((await stopForge(parent, "Not A Name")).ok).toBe(false);
335 const removed = await removeForge(parent, "homie");
336 expect(removed.ok).toBe(true);
337 expect(existsSync(join(dir, "spawn", "homie.json"))).toBe(false);
338 const gone = await fetch(`${servers.url}/healthz`, { headers: { host: CHILD_HOST } });
339 expect(gone.status).toBe(404);
340 });
341 });
test/tui-forges.test.ts +3 −3
@@ -144,15 +144,15 @@ describe("forges you sent in the terminal", () => {
144144 const up = { total: 3, running: 3, notRunning: [] };
145145 const down = { total: 2, running: 1, notRunning: [{ slug: "notes", why: "build failed" }] };
146146 expect(forgeLines([row({ running: up })])).toEqual([
147 ["maria", "for Maria's bakery · Not claimed yet · 3 apps running · has not reported yet · small · shared · no heartbeat yet"],
147 ["maria", "asleep · for Maria's bakery · Not claimed yet · 3 apps running · has not reported yet · small · shared · no heartbeat yet"],
148148 ]);
149149 expect(forgeLines([row({ claimed: true, running: down, for: null })])).toEqual([
150 ["maria", "Claimed · 1 app running · notes: build failed · has not reported yet · small · shared · no heartbeat yet"],
150 ["maria", "asleep · Claimed · 1 app running · notes: build failed · has not reported yet · small · shared · no heartbeat yet"],
151151 ]);
152152 expect(forgeLines([row({ claimed: null })])[0]![1]).toContain("Unknown");
153153 expect(forgeLines([row({ made: false })])[0]![1]).toContain("Still being made");
154154 expect(forgeLines([row({ attention: "down", claimed: true, for: null })])).toEqual([
155 ["maria", "App down · Claimed · has not reported yet · small · shared · no heartbeat yet"],
155 ["maria", "asleep · App down · Claimed · has not reported yet · small · shared · no heartbeat yet"],
156156 ]);
157157 expect(forgeLines([row({})])[0]![1]).not.toMatch(/Not reporting|App down|Unclaimed/);
158158 expect(forgeLines([row({ nextStep: { key: "tell", words: "no tell yet", lastTellAt: null } })])[0]![1]).toContain("no tell yet");

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs