trevato/forge

Public: anyone can read this; the crew can change it.

More
Forks1g fMapg aRuns5g rEventsFlows

One command on your own machine deploys a pack: its apps running with their history, the crew set up, only the model credential left to supply #9605

Merged scoutcrew opened this

Check · Review · Land · Live done — Landed and live in production.

ConversationFiles
Merged

scout/forge:scout/9571-one-command-on-your-own-machine-deploys → main 5 commits fork

  • Merged. scout/9571-one-command-on-your-own-machine-deploys was merged into main as 1da81e4.

Review

Sign in to review this change.
10 files changed +505 −26
docs/ON-YOUR-MACHINE.md +9
@@ -117,3 +117,12 @@ That produces a self-contained binary (it embeds the Bun runtime, so expect
117117 something in the 60–110 MB range; `dist/` is gitignored and binaries are never
118118 committed). Until you build it, `bun run src/cli/main.ts …` is the same
119119 program.
120
121 ## A whole pack, one command
122
123 ```sh
124 git clone <forge>/trevato/forge.git && cd forge && bun install
125 bun scripts/pack.ts up games # or: business
126 ```
127
128 It boots a forge on http://localhost:3000 with its state in `forge-data/`, deploys the pack's apps with their full history and a follow on their source, sets up the crew, and asks for nothing. Only the model credential is yours to add (`ANTHROPIC_API_KEY` in the environment, or Settings › Subscriptions); until then the crew is off and the command says so. No data, secret or variable is carried from the source. See `docs/features/packs.md`.
docs/features/packs.md +3 −3
@@ -4,9 +4,9 @@ A pack is a `pack.json` in a repository (`src/packs/pack.ts` holds the format, v
44
55 Two ship in `packs/`: **games** (2048, hextris, react-tetris) and **business** (memos, kanboard), each with a crew.
66
7 - **See one.** `/packs` lists them, `/packs/<name>` says what it brings, the deploy commands, and how a stranger helps build its apps (get a guest account or use their own forge; a guest's pull waits for a person, and paths touching secrets, credentials, permissions or release machinery are named on it — `docs/ROLES.md`). The home page (`/`) has one press for each pack in its "Get your own forge" section, a link to `/packs/<name>` with an empty `data-slot="host"` beside it for a later hosted deploy; the pack page shows the commands in one `copyField` block (`#pack-command`, a read-only text area you select with a click; its copy button is `js-only`, so it is on the page only when script runs). Reached from the user menu, the signed-out menu and the palette. `bun scripts/pack.ts show games` prints the same in a terminal, and the SSH terminal has a **Forge packs** screen (palette, `src/tui/screens/packs.ts`): the list, a pack's apps and crew, the deploy commands and what happens to a stranger's pull. The words are shared in `src/packs/words.ts`.
8 - **Deploy one.** Boot a forge (`bun run src/main.ts`), make an account and token, then `FORGE_URL=… FORGE_TOKEN=forge_… bun scripts/pack.ts deploy games`. Each app is made, the source's whole history (branches and tags) is pushed in, and the crew is added. Run it again to finish a partial deploy: what exists is kept. The model credential is yours to add in Settings.
7 - **Deploy one, one command.** After `git clone … && cd forge && bun install`, `bun scripts/pack.ts up games` makes the owner account (its passphrase is shown once and kept 0600 in `forge-data/owner-passphrase`; `FORGE_DATA` and `PORT` move the data and the port), boots this tree's forge on http://localhost:3000, makes each app, pushes the source's whole history (branches and tags), turns on following the source (Settings shows it; a new upstream commit shows as drift at the next check), adds the crew, and stays up serving. Nothing is asked but the model credential: with none in the environment it says the crew is off until one is added in Settings › Subscriptions. Run it again to finish a partial deploy: what exists is kept (history is never pushed again), and an app that was made but never followed is followed now; with no branch from the pack or the clone, that step fails rather than guess one. The carried branch becomes the app's default branch. Whether the crew has a credential is read from the data directory (Settings › Subscriptions) as well as the environment. Against a forge you already run: `FORGE_URL=… FORGE_TOKEN=forge_… bun scripts/pack.ts deploy games`.
8 - **See one.** `/packs` lists them, `/packs/<name>` says what it brings, the deploy commands, and how a stranger helps build its apps (get a guest account or use their own forge; a guest's pull waits for a person, and paths touching secrets, credentials, permissions or release machinery are named on it — `docs/ROLES.md`). The home page (`/`) has one press for each pack in its "Get your own forge" section, a link to `/packs/<name>` with an empty `data-slot="host"` beside it for a later hosted deploy; the pack page shows the commands in one `copyField` block (`#pack-command`, a read-only text area you select with a click; its copy button is `js-only`, so it is on the page only when script runs). Reached from the user menu, the signed-out menu and the palette. `bun scripts/pack.ts show games` prints the same in a terminal, and the SSH terminal has a **Forge packs** screen (palette, `src/tui/screens/packs.ts`): the list, a pack's apps and crew, the deploy commands (the `up` line included: the web page and the screen both take them from `deployCommands`) and what happens to a stranger's pull. The words are shared in `src/packs/words.ts`.
99 - **Nightly.** `scripts/pack-compat.ts` boots a clean forge per pack, deploys it with the real sources and fails if any step or app page fails (job `pack-compat`, `src/nightly/jobs.ts`).
10 - Code: `src/packs/`, `src/web/pages/packs.ts`, `scripts/pack.ts`. Tests: `test/packs.test.ts`, `test/packs-hands.test.ts` (the git credential is scoped to the forge and never sent to a pack's source; only 409 means "exists"; the terminal screen).
10 - Code: `src/packs/`, `src/web/pages/packs.ts`, `scripts/pack.ts`. Tests: `test/packs.test.ts`, `test/packs-up.test.ts` (the whole path against a fake source for both packs, `up()` itself with a real child forge on a free port, the owner reused on a second run, a stored credential, a partial deploy finished, a new upstream commit showing as drift), `test/packs-hands.test.ts` (the git credential is scoped to the forge and never sent to a pack's source; only 409 means "exists"; the terminal screen).
1111
1212 Not done yet: a hosted "deploy this pack" press, a stable follow link from each app back to its source, the guest sign-up gate and the crew filtering guest pulls on automode (owner decisions on #9428).
scripts/pack.ts +116 −10
@@ -1,6 +1,7 @@
11 /**
2 * `bun scripts/pack.ts <show|deploy> <pack>` — a forge pack on your machine.
2 * `bun scripts/pack.ts <show|deploy|up> <pack>` — a forge pack on your machine.
33 *
4 * bun scripts/pack.ts up games # the one command: boots a forge, deploys, asks nothing
45 * bun scripts/pack.ts show games
56 * FORGE_URL=http://localhost:3000 FORGE_TOKEN=forge_… bun scripts/pack.ts deploy games
67 *
@@ -18,6 +19,7 @@ import { ForgeApi } from "../src/cli/api.ts";
1819 import { packNamed } from "../src/packs/catalog.ts";
1920 import { deployPack, type Hands } from "../src/packs/deploy.ts";
2021 import { parsePack, type Pack } from "../src/packs/pack.ts";
22 import { finishWords, OWNER, prepareOwner } from "../src/packs/up.ts";
2123
2224 function load(arg: string): Pack | string {
2325 if (/^[a-z][a-z0-9-]*$/.test(arg) && !existsSync(arg)) return packNamed(arg) ?? `no pack named ${arg}`;
@@ -55,32 +57,76 @@ async function git(args: string[], env: Record<string, string>, cwd?: string): P
5557 return code === 0 ? null : errText.trim().split("\n").at(-1) ?? "git failed";
5658 }
5759
58 export function realHands(url: string, token: string): Hands {
60 export interface HandsOptions {
61 /** Where to clone a pack's source from, when not the address the pack names (a test's local repository). */
62 readonly cloneFrom?: (source: string) => string;
63 }
64
65 async function gitOut(args: string[], env: Record<string, string>, cwd: string): Promise<string | null> {
66 const p = Bun.spawn(["git", ...args], { env, cwd, stdout: "pipe", stderr: "ignore" });
67 const [code, out] = await Promise.all([p.exited, new Response(p.stdout).text()]);
68 return code === 0 && out.trim() !== "" ? out.trim() : null;
69 }
70
71 export function realHands(url: string, token: string, options: HandsOptions = {}): Hands {
5972 const api = new ForgeApi(url, token);
6073 return {
6174 async makeApp(name, description) {
6275 const made = await api.newApp({ name, description, visibility: "public" });
6376 if (made.ok) return { ok: true, owner: made.value.pin.owner };
6477 const e = made.error;
65 return { ok: false, exists: e.status === 409, why: e.message };
78 const exists = e.status === 409 || e.code === "name_taken";
79 if (!exists) return { ok: false, exists, why: e.message };
80 const me = await api.whoami();
81 return me.ok && me.value.username !== "" ? { ok: false, exists, why: e.message, owner: me.value.username } : { ok: false, exists, why: e.message };
82 },
83 async defaultBranch(owner, name) {
84 const r = await fetch(`${url}/api/v1/repos/${owner}/${name}`, { headers: { authorization: `Bearer ${token}` } }).catch(() => null);
85 const body = r?.ok ? ((await r.json().catch(() => null)) as { defaultBranch?: unknown } | null) : null;
86 return typeof body?.defaultBranch === "string" && body.defaultBranch !== "" ? body.defaultBranch : undefined;
6687 },
6788 async carryHistory(source, owner, name, branch) {
6889 const dir = mkdtempSync(join(tmpdir(), "pack-"));
6990 try {
7091 const bare = join(dir, "src.git");
7192 // Source history is fetched without hooks or templates and pushed as plain branches and tags.
72 const cloned = await git(["clone", "--bare", "--quiet", ...(branch === undefined ? [] : ["--branch", branch]), source, bare], cloneEnv());
93 const cloned = await git(["clone", "--bare", "--quiet", ...(branch === undefined ? [] : ["--branch", branch]), options.cloneFrom?.(source) ?? source, bare], cloneEnv());
7394 if (cloned !== null) return { ok: false, why: cloned };
7495 const pushed = await git(
7596 ["push", "--quiet", "--force", `${url}/${owner}/${name}.git`, "refs/heads/*:refs/heads/*", "refs/tags/*:refs/tags/*"],
7697 pushEnv(url, token),
7798 bare,
7899 );
79 return pushed === null ? { ok: true } : { ok: false, why: pushed };
100 if (pushed !== null) return { ok: false, why: pushed };
101 const head = await gitOut(["symbolic-ref", "--short", "HEAD"], cloneEnv(), bare);
102 if (head === null) return { ok: true };
103 // The app's default branch is where the history is, so its page, its pulls and the drift check read the same line.
104 const set = await fetch(`${url}/api/v1/repos/${owner}/${name}`, {
105 method: "PATCH",
106 headers: { authorization: `Bearer ${token}`, "content-type": "application/json" },
107 body: JSON.stringify({ defaultBranch: head }),
108 }).catch(() => null);
109 if (set === null || !set.ok) return { ok: false, why: `history carried, default branch not set to ${head}` };
110 return { ok: true, branch: head };
80111 } finally {
81112 rmSync(dir, { recursive: true, force: true });
82113 }
83114 },
115 async follow(owner, name, source, branch) {
116 if (branch === undefined) return { ok: false, why: "no branch to follow: the pack names none and the clone gave none" };
117 const was = await fetch(`${url}/api/v1/repos/${owner}/${name}/upstream`, { headers: { authorization: `Bearer ${token}` } }).catch(() => null);
118 const seen = was?.ok ? ((await was.json().catch(() => null)) as { following?: { enabled?: boolean } | null } | null) : null;
119 // Already following (an earlier run, or the person's own choice): leave it be.
120 if (seen?.following?.enabled === true) return { ok: true };
121 const r = await fetch(`${url}/api/v1/repos/${owner}/${name}/upstream`, {
122 method: "PUT",
123 headers: { authorization: `Bearer ${token}`, "content-type": "application/json" },
124 body: JSON.stringify({ url: source, branch, enabled: true }),
125 }).catch(() => null);
126 if (r === null) return { ok: false, why: `cannot reach ${url}` };
127 if (r.ok) return { ok: true };
128 return { ok: false, why: `the forge answered ${r.status}` };
129 },
84130 async makeAgent(a) {
85131 const r = await fetch(`${url}/api/v1/agents`, {
86132 method: "POST",
@@ -89,18 +135,78 @@ export function realHands(url: string, token: string): Hands {
89135 }).catch(() => null);
90136 if (r === null) return { ok: false, exists: false, why: `cannot reach ${url}` };
91137 if (r.ok) return { ok: true };
92 return { ok: false, exists: r.status === 409, why: `the forge answered ${r.status}` };
138 if (r.status === 409) return { ok: false, exists: true, why: "exists" };
139 // A taken username is refused as 422; only a crew member that is really there counts as kept.
140 const there = r.status === 422 ? await fetch(`${url}/api/v1/agents/${encodeURIComponent(a.username)}`, { headers: { authorization: `Bearer ${token}` } }).catch(() => null) : null;
141 return { ok: false, exists: there?.ok === true, why: `the forge answered ${r.status}` };
93142 },
94143 };
95144 }
96145
146 const USAGE = "usage: bun scripts/pack.ts <show|deploy|up> <pack>";
147
148 export interface UpOptions {
149 readonly port: number;
150 readonly dataDir: string;
151 /** Where to clone a pack's source from (a test's local repository). */
152 readonly cloneFrom?: (source: string) => string;
153 /** Where the words go; the console by default. */
154 readonly say?: (line: string) => void;
155 /** Aborting stops the forge and ends `up`; Ctrl-C and SIGTERM do the same. */
156 readonly signal?: AbortSignal;
157 /** The environment the credential is read from and the forge is started with. */
158 readonly env?: Record<string, string | undefined>;
159 }
160
161 const mark = (s: { state: string }): string => (s.state === "done" ? "✓" : s.state === "kept" ? "·" : "✗");
162
163 /** One command: prepare the owner, boot this tree's forge, deploy the pack into it, then stay up until stopped. */
164 export async function up(pack: Pack, options: UpOptions): Promise<number> {
165 const say = options.say ?? ((l: string) => console.log(l));
166 const env = options.env ?? process.env;
167 const url = `http://localhost:${options.port}`;
168 const prepared = await prepareOwner(options.dataDir, env);
169 const child = Bun.spawn([process.execPath, "run", join(import.meta.dir, "..", "src", "main.ts")], {
170 env: { ...env, FORGE_DATA: options.dataDir, PORT: String(options.port), FORGE_URL: url },
171 stdout: "inherit",
172 stderr: "inherit",
173 });
174 const stop = () => child.kill();
175 process.on("SIGINT", stop);
176 process.on("SIGTERM", stop);
177 options.signal?.addEventListener("abort", stop);
178 try {
179 let ready = false;
180 for (let i = 0; i < 150 && !ready && child.exitCode === null; i++) {
181 ready = await fetch(`${url}/healthz`).then((r) => r.ok, () => false);
182 if (!ready) await Bun.sleep(200);
183 }
184 if (!ready) {
185 say("the forge did not come up; its log is above");
186 stop();
187 await child.exited;
188 return 1;
189 }
190 const steps = await deployPack(pack, realHands(url, prepared.token, options.cloneFrom === undefined ? {} : { cloneFrom: options.cloneFrom }));
191 for (const s of steps) say(`${mark(s)} ${s.what}${s.note ? ` — ${s.note}` : ""}`);
192 if (prepared.passphrase !== undefined) say(`\nSign in as "${OWNER}" with the passphrase kept in ${prepared.passphraseFile}`);
193 for (const line of finishWords(url, prepared.credential)) say(line);
194 if (steps.some((s) => s.state === "failed")) say("Some steps failed; run the command again to finish.");
195 return await child.exited;
196 } finally {
197 process.off("SIGINT", stop);
198 process.off("SIGTERM", stop);
199 }
200 }
201
97202 if (import.meta.main) {
98203 const [verb, which] = process.argv.slice(2);
99 const pack = which === undefined ? "usage: bun scripts/pack.ts <show|deploy> <pack>" : load(which);
100 if (typeof pack === "string" || (verb !== "show" && verb !== "deploy")) {
101 console.error(typeof pack === "string" ? pack : "usage: bun scripts/pack.ts <show|deploy> <pack>");
204 const pack = which === undefined ? USAGE : load(which);
205 if (typeof pack === "string" || (verb !== "show" && verb !== "deploy" && verb !== "up")) {
206 console.error(typeof pack === "string" ? pack : USAGE);
102207 process.exit(2);
103208 }
209 if (verb === "up") process.exit(await up(pack, { port: Number(process.env.PORT ?? 3000), dataDir: process.env.FORGE_DATA ?? "forge-data" }));
104210 if (verb === "show") {
105211 console.log(`${pack.title} ${pack.version}\n${pack.description}\n`);
106212 for (const a of pack.apps) console.log(` app ${a.name.padEnd(14)} ${a.source}`);
@@ -114,7 +220,7 @@ if (import.meta.main) {
114220 process.exit(2);
115221 }
116222 const steps = await deployPack(pack, realHands(url, token));
117 for (const s of steps) console.log(`${s.state === "done" ? "✓" : s.state === "kept" ? "·" : "✗"} ${s.what}${s.note ? ` — ${s.note}` : ""}`);
223 for (const s of steps) console.log(`${mark(s)} ${s.what}${s.note ? ` — ${s.note}` : ""}`);
118224 const failed = steps.some((s) => s.state === "failed");
119225 console.log(failed ? "\nSome steps failed; run it again to finish." : `\n${pack.title} is deployed. Add your model credential in Settings to start the crew.`);
120226 process.exit(failed ? 1 : 0);
src/packs/deploy.ts +31 −8
@@ -5,16 +5,21 @@
55 * watch it with fakes. An app that fails is reported and the rest go on;
66 * running it again skips what already exists, so a half-deployed pack is
77 * finished by deploying it again. No data, secret or variable is read from the
8 * source, and the model credential is left for the person to supply.
8 * source, and the model credential is left for the person to supply. Each app
9 * also follows its source (`follow`), so upstream commits show as drift.
910 */
1011
1112 import type { Pack } from "./pack.ts";
1213
1314 export interface Hands {
1415 /** Make an app; `exists` when the forge already has one of that name. */
15 makeApp(name: string, description: string): Promise<{ ok: true; owner: string } | { ok: false; exists: boolean; why: string }>;
16 makeApp(name: string, description: string): Promise<{ ok: true; owner: string } | { ok: false; exists: boolean; why: string; owner?: string }>;
17 /** The default branch of an app already in the forge, or undefined when it cannot be read. */
18 defaultBranch(owner: string, name: string): Promise<string | undefined>;
1619 /** Copy every branch and tag of `source` into the app, history whole. */
17 carryHistory(source: string, owner: string, name: string, branch: string | undefined): Promise<{ ok: true } | { ok: false; why: string }>;
20 carryHistory(source: string, owner: string, name: string, branch: string | undefined): Promise<{ ok: true; branch?: string } | { ok: false; why: string }>;
21 /** Make the app follow `source` on `branch`, so Settings shows it and a new upstream commit shows as drift. No branch is a failure: a wrong one is never guessed. */
22 follow(owner: string, name: string, source: string, branch: string | undefined): Promise<{ ok: true } | { ok: false; why: string }>;
1823 makeAgent(a: Pack["crew"][number]): Promise<{ ok: true } | { ok: false; exists: boolean; why: string }>;
1924 }
2025
@@ -29,18 +34,36 @@ export async function deployPack(pack: Pack, hands: Hands): Promise<readonly Ste
2934 for (const app of pack.apps) {
3035 const made = await hands.makeApp(app.name, `${app.description} (from ${app.source})`);
3136 if (!made.ok) {
37 if (!made.exists) {
38 steps.push({ what: `app ${app.name}`, state: "failed", note: made.why });
39 continue;
40 }
41 // Kept: its history is not pushed again (that could overwrite work), but a
42 // follow that an earlier, partial run never set is finished now.
43 const owner = made.owner;
44 const branch = owner === undefined ? undefined : (app.branch ?? (await hands.defaultBranch(owner, app.name)));
45 if (owner === undefined) {
46 steps.push({ what: `app ${app.name}`, state: "kept", note: "already here" });
47 continue;
48 }
49 const followed = await hands.follow(owner, app.name, app.source, branch);
3250 steps.push(
33 made.exists
51 followed.ok
3452 ? { what: `app ${app.name}`, state: "kept", note: "already here" }
35 : { what: `app ${app.name}`, state: "failed", note: made.why },
53 : { what: `app ${app.name}`, state: "failed", note: `already here, not followed: ${followed.why}` },
3654 );
3755 continue;
3856 }
3957 const carried = await hands.carryHistory(app.source, made.owner, app.name, app.branch);
58 if (!carried.ok) {
59 steps.push({ what: `app ${app.name}`, state: "failed", note: `made, history not carried: ${carried.why}` });
60 continue;
61 }
62 const followed = await hands.follow(made.owner, app.name, app.source, app.branch ?? carried.branch);
4063 steps.push(
41 carried.ok
42 ? { what: `app ${app.name}`, state: "done", note: `history from ${app.source}` }
43 : { what: `app ${app.name}`, state: "failed", note: `made, history not carried: ${carried.why}` },
64 followed.ok
65 ? { what: `app ${app.name}`, state: "done", note: `history from ${app.source}, following it` }
66 : { what: `app ${app.name}`, state: "failed", note: `history carried, not followed: ${followed.why}` },
4467 );
4568 }
4669 for (const member of pack.crew) {
src/packs/up.ts added +69
@@ -0,0 +1,69 @@
1 /**
2 * `pack up`'s preparation, before the forge boots: open the data directory,
3 * make the one owner account if there is none, and mint a token that expires
4 * in an hour for the deploy to use. The owner's passphrase is made here, kept
5 * in `<data>/owner-passphrase` (0600, under the forge's own state), and shown
6 * once. Nothing is read from the pack's sources and no model credential is
7 * asked for: the words below say what is left to supply.
8 */
9
10 import { chmodSync, writeFileSync } from "node:fs";
11 import { join } from "node:path";
12 import { randomBytes } from "node:crypto";
13 import { createApp } from "../app.ts";
14 import { crewHasCredential } from "../core/subscriptions.ts";
15 import { createToken } from "../core/tokens.ts";
16 import { createUser, getUserByName } from "../core/users.ts";
17
18 export const OWNER = "owner";
19 const HOUR_MS = 3_600_000;
20
21 export interface Prepared {
22 readonly token: string;
23 /** The passphrase, only on the run that made the account. */
24 readonly passphrase?: string;
25 readonly passphraseFile: string;
26 /** Whether the crew has a model credential: one stored in this data directory, or in `env`. */
27 readonly credential: boolean;
28 }
29
30 export async function prepareOwner(dataDir: string, env: Record<string, string | undefined> = process.env): Promise<Prepared> {
31 const app = createApp({ dataDir, port: 0 });
32 try {
33 const file = join(dataDir, "owner-passphrase");
34 let user = getUserByName(app.db, OWNER);
35 let passphrase: string | undefined;
36 if (user === undefined) {
37 passphrase = randomBytes(18).toString("base64url");
38 const made = await createUser(app.db, {
39 origin: "seed",
40 trusted: true,
41 allowReserved: true,
42 username: OWNER,
43 email: "[email protected]",
44 password: passphrase,
45 adminIfFirst: true,
46 });
47 if (!made.ok) throw new Error(`cannot make the owner account: ${made.error}`);
48 user = made.value;
49 writeFileSync(file, `${passphrase}\n`, { mode: 0o600 });
50 chmodSync(file, 0o600);
51 }
52 const { token } = createToken(app.db, user.id, "pack up", "admin", {
53 expiresAt: new Date(Date.now() + HOUR_MS).toISOString(),
54 });
55 return { token, ...(passphrase === undefined ? {} : { passphrase }), passphraseFile: file, credential: crewHasCredential(app.db, env) };
56 } finally {
57 app.close();
58 }
59 }
60
61 /** The last words: what is running, and the one thing left for the person. */
62 export function finishWords(url: string, credentialPresent: boolean): readonly string[] {
63 return [
64 `Your forge is serving ${url}/`,
65 credentialPresent
66 ? "The crew has a model credential and will start work."
67 : `The crew is off until you add a model credential: sign in and open ${url}/settings/subscriptions (or start this command again with ANTHROPIC_API_KEY set).`,
68 ];
69 }
src/packs/words.ts +3 −4
@@ -10,9 +10,8 @@ export function deployCommands(p: Pack, origin: string | undefined): readonly st
1010 const from = origin === undefined ? "<this forge's address>" : origin.replace(/\/+$/, "");
1111 return [
1212 `git clone ${from}/trevato/forge.git && cd forge && bun install`,
13 `bun run src/main.ts # your forge, on http://localhost:3000`,
14 `# make an account there, create an access token, then:`,
15 `FORGE_URL=http://localhost:3000 FORGE_TOKEN=forge_… bun scripts/pack.ts deploy ${p.name}`,
13 `bun scripts/pack.ts up ${p.name} # boots your forge on http://localhost:3000, deploys the pack, asks nothing`,
14 `# already have a forge? FORGE_URL=… FORGE_TOKEN=forge_… bun scripts/pack.ts deploy ${p.name}`,
1615 ];
1716 }
1817
@@ -22,4 +21,4 @@ export const STRANGER_WORDS: readonly string[] = [
2221 ];
2322
2423 export const BRINGS_WORDS =
25 "Each app comes with its history. No data, secrets or variables come with a pack, and the crew waits for the model credential you give it.";
24 "Each app comes with its history. No data, secrets or variables come with a pack, and the crew stays off until you give it a model credential, the one thing asked of you.";
test/packs-hands.test.ts +1
@@ -118,6 +118,7 @@ describe("terminal screen", () => {
118118 tui.ctx.pop();
119119 const lines = packLines(packNamed("games")!, undefined).join("\n");
120120 expect(lines).toContain("scripts/pack.ts deploy games");
121 expect(lines).toContain("scripts/pack.ts up games");
121122 expect(lines).toContain("waits for a person");
122123 expect(lines).toContain("2048");
123124 tui.stop?.();
test/packs-up.test.ts added +264
@@ -0,0 +1,264 @@
1 /** `pack up`'s path against a fake source: Games and Business arrive with history, followed, with their crew, and nothing is carried but code. */
2 import { afterAll, beforeAll, describe, expect, test } from "bun:test";
3 import { mkdtemp, rm, readFile, stat } from "node:fs/promises";
4 import { tmpdir } from "node:os";
5 import { join } from "node:path";
6 import { createApp, type App } from "../src/app.ts";
7 import { crewHasCredential } from "../src/core/subscriptions.ts";
8 import { getAgentByName } from "../src/core/agents.ts";
9 import { getRepo } from "../src/core/repos.ts";
10 import { getFollow } from "../src/core/upstream_follow.ts";
11 import { serve, type Servers } from "../src/http/server.ts";
12 import { catalog } from "../src/packs/catalog.ts";
13 import { deployPack } from "../src/packs/deploy.ts";
14 import { addSubscription } from "../src/core/subscriptions.ts";
15 import { checkUpstream } from "../src/actions/upstream_follow.ts";
16 import { getRepoById } from "../src/core/repos.ts";
17 import { getUserByName } from "../src/core/users.ts";
18 import { provideMirrorOutboundForTest } from "../src/origins/git-mirror.ts";
19 import { finishWords, OWNER, prepareOwner } from "../src/packs/up.ts";
20 import { deployCommands } from "../src/packs/words.ts";
21 import { repoDir } from "../src/git/repos.ts";
22 import { realHands, up } from "../scripts/pack.ts";
23 import { quietLogs } from "./helpers/globals.ts";
24
25 quietLogs();
26
27 const GIT_ENV = {
28 PATH: process.env.PATH ?? "/usr/bin:/bin",
29 GIT_CONFIG_GLOBAL: "/dev/null",
30 GIT_CONFIG_SYSTEM: "/dev/null",
31 GIT_AUTHOR_NAME: "t",
32 GIT_AUTHOR_EMAIL: "[email protected]",
33 GIT_COMMITTER_NAME: "t",
34 GIT_COMMITTER_EMAIL: "[email protected]",
35 };
36 const git = (cwd: string, ...args: string[]): string => {
37 const r = Bun.spawnSync(["git", ...args], { cwd, env: GIT_ENV });
38 if (r.exitCode !== 0) throw new Error(`git ${args.join(" ")}: ${r.stderr.toString()}`);
39 return r.stdout.toString().trim();
40 };
41
42 let root: string;
43 let dataDir: string;
44 let app: App;
45 let servers: Servers;
46 let token: string;
47 let passphrase: string | undefined;
48 const fake = new Map<string, string>();
49
50 beforeAll(async () => {
51 root = await mkdtemp(join(tmpdir(), "packs-up-"));
52 dataDir = join(root, "data");
53 const prepared = await prepareOwner(dataDir);
54 token = prepared.token;
55 passphrase = prepared.passphrase;
56 // A fake source for every app of both packs: three commits and a tag, on a branch that is not "main" for one of them.
57 let n = 0;
58 for (const pack of catalog().packs)
59 for (const a of pack.apps) {
60 const dir = join(root, `src-${n++}`);
61 git(root, "init", "-q", "-b", n === 1 ? "master" : "main", dir);
62 for (const i of [1, 2, 3]) {
63 await Bun.write(join(dir, "f.txt"), `${a.name} ${i}\n`);
64 git(dir, "add", ".");
65 git(dir, "commit", "-q", "-m", `commit ${i}`);
66 }
67 git(dir, "tag", "v1");
68 fake.set(a.source, dir);
69 }
70 app = createApp({ dataDir, port: 0 });
71 servers = await serve(app);
72 });
73 afterAll(async () => {
74 await servers.stop();
75 app.close();
76 await rm(root, { recursive: true, force: true });
77 });
78
79 describe("the one command, against a fake source", () => {
80 test("the command is one line to run after clone and install, and asks for no token", () => {
81 for (const p of catalog().packs) {
82 const lines = deployCommands(p, "https://f.example");
83 expect(lines.some((l) => l.startsWith(`bun scripts/pack.ts up ${p.name}`))).toBe(true);
84 expect(lines[0]).toContain("bun install");
85 }
86 });
87
88 test("the owner is made once, its passphrase kept private, and a second run reuses it", async () => {
89 expect(passphrase?.length).toBeGreaterThan(15);
90 const file = join(dataDir, "owner-passphrase");
91 expect((await stat(file)).mode & 0o777).toBe(0o600);
92 const before = await readFile(file, "utf8");
93 expect(before.trim()).toBe(passphrase!);
94 const again = await prepareOwner(dataDir, {});
95 expect(again.passphrase).toBeUndefined();
96 expect(await readFile(file, "utf8")).toBe(before);
97 expect((await stat(file)).mode & 0o777).toBe(0o600);
98 expect(app.db.query("SELECT COUNT(*) AS n FROM user WHERE username = ?1").get(OWNER)).toEqual({ n: 1 });
99 // The same account: the token the second run mints is the owner's.
100 const who = await fetch(`${servers.url}/api/v1/user`, { headers: { authorization: `Bearer ${again.token}` } });
101 expect(((await who.json()) as { username: string }).username).toBe(OWNER);
102 });
103
104 for (const name of ["games", "business"]) {
105 test(`${name}: every app arrives with history, followed; the crew is configured; no secret or variable travels`, async () => {
106 const pack = catalog().packs.find((p) => p.name === name)!;
107 const steps = await deployPack(pack, realHands(servers.url, token, { cloneFrom: (s) => fake.get(s)! }));
108 expect(steps.filter((s) => s.state !== "done")).toEqual([]);
109 for (const a of pack.apps) {
110 const repo = getRepo(app.db, OWNER, a.name)!;
111 expect(repo).toBeDefined();
112 const dir = repoDir(app.reposRoot, OWNER, a.name);
113 const src = fake.get(a.source)!;
114 const head = git(src, "symbolic-ref", "--short", "HEAD");
115 expect(git(dir, "rev-parse", `refs/heads/${head}`)).toBe(git(src, "rev-parse", "HEAD"));
116 expect(git(dir, "rev-list", "--count", `refs/heads/${head}`)).toBe("3");
117 expect(git(dir, "rev-parse", "refs/tags/v1")).toBe(git(src, "rev-parse", "v1"));
118 const follow = getFollow(app.db, repo.id)!;
119 expect(follow.enabled).toBe(true);
120 expect(follow.url).toBe(a.source);
121 expect(follow.branch).toBe(git(src, "symbolic-ref", "--short", "HEAD"));
122 expect((await fetch(`${servers.url}/${OWNER}/${a.name}`)).status).toBe(200);
123 }
124 for (const c of pack.crew) {
125 const agent = getAgentByName(app.db, c.username)!;
126 expect(c.roles.length).toBeGreaterThan(0);
127 expect([...agent.roles].sort()).toEqual([...c.roles].sort());
128 }
129 });
130 }
131
132 test("running it again keeps what exists", async () => {
133 const pack = catalog().packs[0]!;
134 const steps = await deployPack(pack, realHands(servers.url, token, { cloneFrom: (s) => fake.get(s)! }));
135 expect(steps.map((s) => `${s.what} ${s.state} ${s.note}`)).toEqual(steps.map((s) => `${s.what} kept already here`));
136 });
137
138 test("a credential stored in the forge counts even when the environment has none", async () => {
139 const d = await mkdtemp(join(tmpdir(), "packs-cred-"));
140 try {
141 expect((await prepareOwner(d, {})).credential).toBe(false);
142 const a2 = createApp({ dataDir: d, port: 0 });
143 const added = addSubscription(a2.db, d, { name: "mine", token: "sk-ant-oat01-" + "x".repeat(40) });
144 a2.close();
145 expect(added.ok).toBe(true);
146 const again = await prepareOwner(d, {});
147 expect(again.credential).toBe(true);
148 expect(finishWords("http://localhost:3000", again.credential).join("\n")).not.toContain("crew is off");
149 expect((await prepareOwner(await mkdtemp(join(tmpdir(), "packs-cred2-")), { ANTHROPIC_API_KEY: "k" })).credential).toBe(true);
150 } finally {
151 await rm(d, { recursive: true, force: true });
152 }
153 });
154
155 test("with no model credential the forge says the crew is off, and says where to add one", () => {
156 expect(crewHasCredential(app.db, {})).toBe(false);
157 expect(finishWords("http://localhost:3000", false).join("\n")).toContain("crew is off until you add a model credential");
158 expect(finishWords("http://localhost:3000", true).join("\n")).not.toContain("crew is off");
159 });
160
161 test("a partial deploy is finished by running it again: a kept app that was never followed is followed", async () => {
162 const pack = catalog().packs[0]!;
163 const hands = realHands(servers.url, token, { cloneFrom: (s) => fake.get(s)! });
164 // First run: the follow of the first app fails after the app and its history are in.
165 const one = { ...pack, apps: [{ ...pack.apps[0]!, name: "partial-app" }], crew: [] };
166 let fail = true;
167 const steps1 = await deployPack(one, { ...hands, follow: (...a) => (fail ? Promise.resolve({ ok: false, why: "down" }) : hands.follow(...a)) });
168 expect(steps1[0]!.state).toBe("failed");
169 const repo = getRepo(app.db, OWNER, "partial-app")!;
170 expect(getFollow(app.db, repo.id)).toBeUndefined();
171 fail = false;
172 const steps2 = await deployPack(one, { ...hands, follow: (...a) => hands.follow(...a) });
173 expect(steps2[0]!.state).toBe("kept");
174 const f = getFollow(app.db, repo.id)!;
175 expect(f.enabled).toBe(true);
176 expect(f.url).toBe(one.apps[0]!.source);
177 expect(f.branch).toBe(repo.defaultBranch);
178 });
179
180 test("no branch to follow fails the step, and never follows main by guess", async () => {
181 const hands = realHands(servers.url, token);
182 const r = await hands.follow(OWNER, "whatever", "https://github.com/a/b", undefined);
183 expect(r.ok).toBe(false);
184 });
185
186 test("a new upstream commit shows as drift at the next check", async () => {
187 const pack = catalog().packs[0]!;
188 const a = pack.apps[0]!;
189 const repo = getRepo(app.db, OWNER, a.name)!;
190 const src = fake.get(a.source)!;
191 const branch = git(src, "symbolic-ref", "--short", "HEAD");
192 expect(getFollow(app.db, repo.id)!.behind).toBe(0);
193 // The source moves on by one commit.
194 await Bun.write(join(src, "f.txt"), "newer\n");
195 git(src, "commit", "-qam", "upstream moved");
196 const tip = git(src, "rev-parse", "HEAD");
197 // The forge asks the follow's address; a local server stands for the source's host (the check refuses file: addresses).
198 const server = Bun.serve({
199 port: 0,
200 hostname: "127.0.0.1",
201 fetch: (req) => (new URL(req.url).pathname.endsWith("/info/refs") ? new Response(`${tip}\trefs/heads/${branch}\n`) : new Response("no", { status: 404 })),
202 });
203 const dir = repoDir(app.reposRoot, OWNER, a.name);
204 // The staged copy is what the check's fetch would have brought.
205 git(dir, "fetch", "-q", "--no-tags", src, `+${tip}:refs/upstream-follow/${branch}`);
206 provideMirrorOutboundForTest({ allow: ["127.0.0.1"] });
207 try {
208 const f = getFollow(app.db, repo.id)!;
209 app.db.query("UPDATE repo_upstream SET url = ?1 WHERE repo_id = ?2").run(`http://127.0.0.1:${server.port}/o/r.git`, repo.id);
210 expect(f.enabled).toBe(true);
211 const checked = await checkUpstream(app, getRepoById(app.db, repo.id)!);
212 expect(checked.ok).toBe(true);
213 expect(repo.defaultBranch).toBe(branch);
214 expect(getFollow(app.db, repo.id)!.behind).toBe(1);
215 expect(getFollow(app.db, repo.id)!.tip).toBe(tip);
216 } finally {
217 provideMirrorOutboundForTest({});
218 server.stop(true);
219 }
220 });
221 });
222
223 describe("up() itself", () => {
224 test("boots a forge, deploys the pack into it with its history, says the crew is off, and stops when told", async () => {
225 const d = await mkdtemp(join(tmpdir(), "packs-upcmd-"));
226 const free = Bun.serve({ port: 0, fetch: () => new Response("") });
227 const port = free.port!;
228 free.stop(true);
229 const lines: string[] = [];
230 const stop = new AbortController();
231 const env: Record<string, string | undefined> = { ...process.env, FORGE_SSH_PORT: "0" };
232 for (const n of ["ANTHROPIC_API_KEY", "CLAUDE_CODE_OAUTH_TOKEN", "FORGE_CLAUDE_CONFIG_DIR"]) delete env[n];
233 try {
234 const pack = catalog().packs[0]!;
235 const running = up(pack, { port, dataDir: d, cloneFrom: (s) => fake.get(s)!, say: (l) => lines.push(l), signal: stop.signal, env });
236 let ended: number | undefined;
237 void running.then((c) => (ended = c));
238 for (let i = 0; i < 700 && ended === undefined && !lines.some((l) => l.startsWith("Your forge is serving")); i++) await Bun.sleep(100);
239 if (ended !== undefined) throw new Error(`up() ended with ${ended} before serving: ${lines.join(" | ")}`);
240 const said = lines.join("\n");
241 expect(said).toContain(`Your forge is serving http://localhost:${port}/`);
242 expect(said).toContain("crew is off until you add a model credential");
243 expect(said).toContain("Sign in as \"owner\"");
244 expect((await fetch(`http://localhost:${port}/${OWNER}/${pack.apps[0]!.name}`)).status).toBe(200);
245 stop.abort();
246 await running;
247 expect(await fetch(`http://localhost:${port}/healthz`).then((r) => r.ok, () => false)).toBe(false);
248 const a2 = createApp({ dataDir: d, port: 0 });
249 try {
250 for (const x of pack.apps) {
251 const repo = getRepo(a2.db, OWNER, x.name)!;
252 expect(getFollow(a2.db, repo.id)!.enabled).toBe(true);
253 expect(git(repoDir(a2.reposRoot, OWNER, x.name), "rev-list", "--count", "--all")).not.toBe("0");
254 }
255 for (const c of pack.crew) expect([...getAgentByName(a2.db, c.username)!.roles].sort()).toEqual([...c.roles].sort());
256 expect(getUserByName(a2.db, OWNER)).toBeDefined();
257 } finally {
258 a2.close();
259 }
260 } finally {
261 await rm(d, { recursive: true, force: true });
262 }
263 }, 120_000);
264 });
test/packs.test.ts +8 −1
@@ -68,6 +68,13 @@ describe("deploy", () => {
6868 calls.push(`history ${source} -> ${owner}/${name}`);
6969 return { ok: true };
7070 },
71 async defaultBranch() {
72 return "main";
73 },
74 async follow(owner, name, source, branch) {
75 calls.push(`follow ${owner}/${name} ${source} ${branch}`);
76 return { ok: true };
77 },
7178 async makeAgent(a) {
7279 calls.push(`agent ${a.username}`);
7380 return { ok: false, exists: false, why: "no" };
@@ -76,7 +83,7 @@ describe("deploy", () => {
7683 const pack = parse({ ...good, apps: [...good.apps, { name: "two", description: "d", source: "https://github.com/c/d" }] });
7784 if (!pack.ok) throw new Error(pack.error);
7885 const steps = await deployPack(pack.value, hands);
79 expect(calls).toEqual(["app one", "history https://github.com/a/b -> me/one", "app two", "agent helper"]);
86 expect(calls).toEqual(["app one", "history https://github.com/a/b -> me/one", "follow me/one https://github.com/a/b undefined", "app two", "agent helper"]);
8087 expect(steps.map((s) => s.state)).toEqual(["done", "kept", "failed"]);
8188 });
8289 });
test/signup-gate-parent.test.ts +1
@@ -115,6 +115,7 @@ test("every caller of createUser says where its account comes from", async () =>
115115 "src/actions/spawn.ts": ["spawn", "spawn"],
116116 "src/actions/users.ts": ["signup", "passkey"],
117117 "src/chaos/world.ts": ["seed"],
118 "src/packs/up.ts": ["seed"],
118119 "src/spawn/preview-seed.ts": ["seed", "seed"],
119120 "src/web/pages/claim.ts": ["claim"],
120121 "src/web/pages/everyday.ts": ["everyday"],

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs