Public: anyone can read this; the crew can change it.
A pack app's source can be a forge repo, and deploying it forks from the forge with lineage #10607
Check · Review · Land · Live done — Landed and live in production.
mason/forge:mason/10529-a-pack-app-s-source-can-be-a-forge-repo → main 3 commits fork 4 threads · 4 unresolved
docs/features/packs.md +3 −2
| @@ -1,13 +1,14 @@ | ||
| 1 | 1 | # Forge packs: a premade forge to deploy (`/packs`) |
| 2 | 2 | |
| 3 | A pack is a `pack.json` in a repository (`src/packs/pack.ts` holds the format, version 1): a name, title, semver version, description, the **apps** it brings (each an https address of a public git repository, with a name and description) and the **crew** it starts with (username, roles of planner/builder/reviewer, a brief, optionally a model and a per-run budget). Anyone can publish one. A pack may not carry data, secrets or variables: any key that names one (`secret`, `env`, `token`, `data`, `key`…) at any depth is refused, not ignored. | |
| 3 | A pack is a `pack.json` in a repository (`src/packs/pack.ts` holds the format, version 1): a name, title, semver version, description, the **apps** it brings (each a `source` that is either an https address of a public git repository or `owner/name` of a repository on a forge, with a name and description) and the **crew** it starts with (username, roles of planner/builder/reviewer, a brief, optionally a model and a per-run budget). Anyone can publish one. A pack may not carry data, secrets or variables: any key that names one (`secret`, `env`, `token`, `data`, `key`…) at any depth is refused, not ignored. | |
| 4 | 4 | |
| 5 | 5 | Two ship in `packs/`: **games** (2048, hextris, react-tetris) and **business** (memos, kanboard), each with a crew. |
| 6 | 6 | |
| 7 | 7 | - **Deploy one, one command.** After `git clone … && cd forge && bun install`, `bun scripts/pack.ts up games` makes the owner account (its passphrase is shown once and kept 0600 in `forge-data/owner-passphrase`; `FORGE_DATA` and `PORT` move the data and the port), boots this tree's forge on http://localhost:3000, makes each app, pushes the source's whole history (branches and tags), turns on following the source (Settings shows it; a new upstream commit shows as drift at the next check), adds the crew, and stays up serving. Nothing is asked but the model credential: with none in the environment it says the crew is off until one is added in Settings › Subscriptions. Run it again to finish a partial deploy: what exists is kept (history is never pushed again), and an app that was made but never followed is followed now; with no branch from the pack or the clone, that step fails rather than guess one. The carried branch becomes the app's default branch. Whether the crew has a credential is read from the data directory (Settings › Subscriptions) as well as the environment. Against a forge you already run: `FORGE_URL=… FORGE_TOKEN=forge_… bun scripts/pack.ts deploy games`. |
| 8 | 8 | - **See one.** `/packs` lists them, `/packs/<name>` says what it brings, the deploy commands, and how a stranger helps build its apps (get a guest account or use their own forge; a guest's pull waits for a person, and paths touching secrets, credentials, permissions or release machinery are named on it — `docs/ROLES.md`). The home page (`/`) has one press for each pack in its "Get your own forge" section, a link to `/packs/<name>` with an empty `data-slot="host"` beside it for a later hosted deploy; the pack page shows the commands in one `copyField` block (`#pack-command`, a read-only text area you select with a click; its copy button is `js-only`, so it is on the page only when script runs). Reached from the user menu, the signed-out menu and the palette. `bun scripts/pack.ts show games` prints the same in a terminal, and the SSH terminal has a **Forge packs** screen (palette, `src/tui/screens/packs.ts`): the list, a pack's apps and crew, the deploy commands (the `up` line included: the web page and the screen both take them from `deployCommands`) and what happens to a stranger's pull. The words are shared in `src/packs/words.ts`. |
| 9 | 9 | - **Nightly.** `scripts/pack-compat.ts` boots a clean forge per pack, deploys it with the real sources and fails if any step or app page fails (job `pack-compat`, `src/nightly/jobs.ts`). |
| 10 | - Code: `src/packs/`, `src/web/pages/packs.ts`, `scripts/pack.ts`. Tests: `test/packs.test.ts`, `test/packs-up.test.ts` (the whole path against a fake source for both packs, `up()` itself with a real child forge on a free port, the owner reused on a second run, a stored credential, a partial deploy finished, a new upstream commit showing as drift), `test/packs-hands.test.ts` (the git credential is scoped to the forge and never sent to a pack's source; only 409 means "exists"; the terminal screen). | |
| 10 | - Code: `src/packs/`, `src/web/pages/packs.ts`, `scripts/pack.ts`. Tests: `test/packs.test.ts`, `test/packs-up.test.ts` (the whole path against a fake source for both packs, `up()` itself with a real child forge on a free port, the owner reused on a second run, a stored credential, a partial deploy finished, a new upstream commit showing as drift), `test/packs-hands.test.ts` (the git credential is scoped to the forge and never sent to a pack's source; only 409 means "exists"; the terminal screen; an `owner/name` source is forked with lineage, kept on 409, a 422 or 429 refusal failing that step with the forge's message, an unreachable forge, and `absorb` absorbing the fork as its owner). | |
| 11 | - **A source on a forge (`owner/name`).** Deploying forks it with lineage (`POST /forks`) instead of pushing history: nothing is carried over the network and nothing is followed (no follow is set; the fork's lineage is the link). The fork keeps the app's name. `owner/name` is resolved on whichever forge the deploy targets, not on the pack author's forge. An existing fork (409 `already_forked`) counts as kept; a name already taken by something else fails that step. Non-admins are held to the fork limit (`FORKS_PER_DAY`, 5 a day), so a pack with more than five forge apps fails the rest with "at most 5 forks a day"; run it again the next day. Mirroring (below) is for https sources only: a forge-form source is reported as failed ("not an address a mirror may follow"). | |
| 11 | 12 | |
| 12 | 13 | Not done yet: a hosted "deploy this pack" press, a stable follow link from each app back to its source, the guest sign-up gate and the crew filtering guest pulls on automode (owner decisions on #9428). |
| 13 | 14 |
scripts/pack.ts +14
| @@ -80,6 +80,20 @@ export function realHands(url: string, token: string, options: HandsOptions = {} | ||
| 80 | 80 | const me = await api.whoami(); |
| 81 | 81 | return me.ok && me.value.username !== "" ? { ok: false, exists, why: e.message, owner: me.value.username } : { ok: false, exists, why: e.message }; |
| 82 | 82 | }, |
| 83 | async forkFrom(source, name) { | |
| 84 | const r = await fetch(`${url}/api/v1/repos/${source}/forks`, { | |
| 85 | method: "POST", | |
| 86 | headers: { authorization: `Bearer ${token}`, "content-type": "application/json" }, | |
| 87 | body: JSON.stringify({ name }), | |
| 88 | }).catch(() => null); | |
| 89 | if (r === null) return { ok: false, exists: false, why: `cannot reach ${url}` }; | |
| 90 | const body = (await r.json().catch(() => null)) as { ownerName?: unknown; name?: unknown; message?: unknown } | null; | |
| 91 | if (r.ok && typeof body?.ownerName === "string") return { ok: true, owner: body.ownerName }; | |
| 92 | const why = typeof body?.message === "string" ? body.message : `the forge answered ${r.status}`; | |
| 93 | if (r.status !== 409) return { ok: false, exists: false, why }; | |
| 94 | const me = await api.whoami(); | |
| 95 | return me.ok && me.value.username !== "" ? { ok: false, exists: true, why, owner: me.value.username } : { ok: false, exists: true, why }; | |
| 96 | }, | |
| 83 | 97 | async defaultBranch(owner, name) { |
| 84 | 98 | const r = await fetch(`${url}/api/v1/repos/${owner}/${name}`, { headers: { authorization: `Bearer ${token}` } }).catch(() => null); |
| 85 | 99 | const body = r?.ok ? ((await r.json().catch(() => null)) as { defaultBranch?: unknown } | null) : null; |
src/packs/deploy.ts +19 −1
| @@ -9,11 +9,13 @@ | ||
| 9 | 9 | * also follows its source (`follow`), so upstream commits show as drift. |
| 10 | 10 | */ |
| 11 | 11 | |
| 12 | import type { Pack } from "./pack.ts"; | |
| 12 | import { isForgeSource, type Pack } from "./pack.ts"; | |
| 13 | 13 | |
| 14 | 14 | export interface Hands { |
| 15 | 15 | /** Make an app; `exists` when the forge already has one of that name. */ |
| 16 | 16 | makeApp(name: string, description: string): Promise<{ ok: true; owner: string } | { ok: false; exists: boolean; why: string; owner?: string }>; |
| 17 | /** Fork the forge repo `source` (`owner/name`) into the person's account as `name`, with lineage: the fork records its parent. No network but the forge's own. */ | |
| 18 | forkFrom(source: string, name: string): Promise<{ ok: true; owner: string } | { ok: false; exists: boolean; why: string; owner?: string }>; | |
| 17 | 19 | /** The default branch of an app already in the forge, or undefined when it cannot be read. */ |
| 18 | 20 | defaultBranch(owner: string, name: string): Promise<string | undefined>; |
| 19 | 21 | /** Copy every branch and tag of `source` into the app, history whole. */ |
| @@ -37,9 +39,25 @@ async function absorbStep(hands: Hands, owner: string, name: string): Promise<St | ||
| 37 | 39 | return r.ok ? { what: `absorb ${name}`, state: "done", note: "a feature in the menu" } : { what: `absorb ${name}`, state: "failed", note: r.why }; |
| 38 | 40 | } |
| 39 | 41 | |
| 42 | /** A forge-repo source is forked, not copied: the fork's lineage is the follow, so nothing is carried or followed over the network. */ | |
| 43 | async function forkStep(hands: Hands, app: Pack["apps"][number]): Promise<Step[]> { | |
| 44 | const forked = await hands.forkFrom(app.source, app.name); | |
| 45 | if (!forked.ok) { | |
| 46 | if (!forked.exists) return [{ what: `app ${app.name}`, state: "failed", note: forked.why }]; | |
| 47 | return [{ what: `app ${app.name}`, state: "kept", note: "already here" }]; | |
| 48 | } | |
| 49 | const steps: Step[] = [{ what: `app ${app.name}`, state: "done", note: `forked from ${app.source}` }]; | |
| 50 | if (app.absorb === true) steps.push(await absorbStep(hands, forked.owner, app.name)); | |
Untested: the absorb step after a fork. Add a deployPack test with absorb:true on a forge source asserting the step and the owner passed. |
||
| 51 | return steps; | |
| 52 | } | |
| 53 | ||
| 40 | 54 | export async function deployPack(pack: Pack, hands: Hands): Promise<readonly Step[]> { |
| 41 | 55 | const steps: Step[] = []; |
| 42 | 56 | for (const app of pack.apps) { |
| 57 | if (isForgeSource(app.source)) { | |
| 58 | steps.push(...(await forkStep(hands, app))); | |
| 59 | continue; | |
| 60 | } | |
| 43 | 61 | const made = await hands.makeApp(app.name, `${app.description} (from ${app.source})`); |
| 44 | 62 | if (!made.ok) { |
| 45 | 63 | if (!made.exists) { |
src/packs/pack.ts +8 −2
| @@ -25,7 +25,7 @@ const FORBIDDEN = /secret|password|token|credential|env|variable|data|key/i; | ||
| 25 | 25 | export interface PackApp { |
| 26 | 26 | readonly name: string; |
| 27 | 27 | readonly description: string; |
| 28 | /** An https address of a public git repository, no credentials in it. */ | |
| 28 | /** An https address of a public git repository, no credentials in it, or `owner/name` of a repository on this forge. */ | |
| 29 | 29 | readonly source: string; |
| 30 | 30 | readonly branch?: string; |
| 31 | 31 | /** Deploy absorbs the app as a feature, as the deploying admin's act (an event). Only a yes is carried; it sets no setting. */ |
| @@ -55,9 +55,15 @@ const text = (v: unknown, max = MAX_TEXT): string | undefined => | ||
| 55 | 55 | ? v.trim() |
| 56 | 56 | : undefined; |
| 57 | 57 | |
| 58 | /** The source's address if it names a public https repository, else undefined. */ | |
| 58 | const FORGE_SOURCE = /^([A-Za-z0-9][A-Za-z0-9._-]{0,99})\/([A-Za-z0-9][A-Za-z0-9._-]{0,99})$/; | |
Fine, but note that |
||
| 59 | ||
| 60 | /** True when a source names a repository on this forge (`owner/name`) rather than an https address. */ | |
| 61 | export const isForgeSource = (source: string): boolean => FORGE_SOURCE.test(source); | |
| 62 | ||
| 63 | /** The source's address if it names a public https repository or a forge repo (`owner/name`), else undefined. */ | |
| 59 | 64 | export function sourceOf(v: unknown): string | undefined { |
| 60 | 65 | const raw = text(v, 300); |
| 66 | if (raw !== undefined && FORGE_SOURCE.test(raw)) return raw; | |
| 61 | 67 | const url = raw === undefined ? null : URL.parse(raw); |
| 62 | 68 | if (url === null || url.protocol !== "https:" || url.username !== "" || url.password !== "") return undefined; |
| 63 | 69 | if (url.search !== "" || url.hash !== "") return undefined; |
test/packs-hands.test.ts +116
| @@ -6,6 +6,8 @@ import { join } from "node:path"; | ||
| 6 | 6 | import { createApp, type App } from "../src/app.ts"; |
| 7 | 7 | import { createUser } from "../src/core/users.ts"; |
| 8 | 8 | import { serve } from "../src/http/server.ts"; |
| 9 | import { deployPack } from "../src/packs/deploy.ts"; | |
| 10 | import { parsePack } from "../src/packs/pack.ts"; | |
| 9 | 11 | import { packNamed } from "../src/packs/catalog.ts"; |
| 10 | 12 | import { deployCommands } from "../src/packs/words.ts"; |
| 11 | 13 | import { createForgeApp, forgeCommands } from "../src/tui/forge.ts"; |
| @@ -178,3 +180,117 @@ describe("absorb hand", () => { | ||
| 178 | 180 | if (!r.ok) expect(r.why).toContain("cannot reach"); |
| 179 | 181 | }); |
| 180 | 182 | }); |
| 183 | ||
| 184 | describe("forge-repo sources", () => { | |
| 185 | const parsed = parsePack( | |
| 186 | JSON.stringify({ | |
| 187 | pack: 1, | |
| 188 | name: "mine", | |
| 189 | title: "Mine", | |
| 190 | version: "1.0.0", | |
| 191 | description: "Apps from this forge.", | |
| 192 | apps: [ | |
| 193 | { name: "radar", description: "A radar.", source: "trevato/radar" }, | |
| 194 | { name: "notes", description: "Notes.", source: "trevato/notes" }, | |
| 195 | ], | |
| 196 | }), | |
| 197 | ); | |
| 198 | if (!parsed.ok) throw new Error(parsed.error); | |
| 199 | ||
| 200 | test("deploying forks from the forge: one POST each, no network anywhere else, no history carried", async () => { | |
| 201 | const posts: string[] = []; | |
| 202 | const forge = Bun.serve({ | |
| 203 | port: 0, | |
| 204 | fetch: async (req) => { | |
| 205 | const u = new URL(req.url); | |
| 206 | posts.push(`${req.method} ${u.pathname} ${await req.text()}`); | |
| 207 | return u.pathname.endsWith("/notes/forks") | |
| 208 | ? Response.json({ error: "already_forked" }, { status: 409 }) | |
| 209 | : Response.json({ ownerName: "me", name: "radar" }, { status: 201 }); | |
| 210 | }, | |
| 211 | }); | |
| 212 | const real = globalThis.fetch; | |
| 213 | const hosts: string[] = []; | |
| 214 | globalThis.fetch = ((input: string | URL | Request, init?: RequestInit) => { | |
| 215 | const host = new URL(typeof input === "string" ? input : input instanceof URL ? input.href : input.url).host; | |
| 216 | hosts.push(host); | |
| 217 | if (host.includes("github.com")) return Promise.reject(new Error("network to github.com refused")); | |
| 218 | return real(input, init); | |
| 219 | }) as typeof fetch; | |
| 220 | try { | |
| 221 | const hands = realHands(`http://127.0.0.1:${forge.port}`, "t", { | |
| 222 | cloneFrom: () => { | |
| 223 | throw new Error("history must not be carried over the network"); | |
| 224 | }, | |
| 225 | }); | |
| 226 | const steps = await deployPack(parsed.value, hands); | |
| 227 | expect(steps.map((s) => [s.what, s.state])).toEqual([ | |
| 228 | ["app radar", "done"], | |
| 229 | ["app notes", "kept"], | |
| 230 | ]); | |
| 231 | expect(posts).toEqual([ | |
| 232 | 'POST /api/v1/repos/trevato/radar/forks {"name":"radar"}', | |
| 233 | 'POST /api/v1/repos/trevato/notes/forks {"name":"notes"}', | |
| 234 | "GET /api/v1/user ", | |
| 235 | ]); | |
| 236 | expect(hosts.every((h) => h === `127.0.0.1:${forge.port}`)).toBe(true); | |
| 237 | } finally { | |
| 238 | globalThis.fetch = real; | |
Only 201 and 409 are exercised. Add a case where the fake forge answers 422/429 and assert state 'failed' with the message (the docs promise this), and one with the forge unreachable. |
||
| 239 | await forge.stop(true); | |
| 240 | } | |
| 241 | }); | |
| 242 | ||
| 243 | test("a refusal other than 409 (name taken, fork limit) fails that app's step with the forge's message, and the next app is still forked", async () => { | |
| 244 | const forge = Bun.serve({ | |
| 245 | port: 0, | |
| 246 | fetch: (req) => { | |
| 247 | const u = new URL(req.url); | |
| 248 | if (u.pathname.endsWith("/radar/forks")) return Response.json({ error: "name_taken", message: "you already have a repository named radar" }, { status: 422 }); | |
| 249 | return Response.json({ error: "rate_limited", message: "at most 5 forks a day" }, { status: 429 }); | |
| 250 | }, | |
| 251 | }); | |
| 252 | try { | |
| 253 | const steps = await deployPack(parsed.value, realHands(`http://127.0.0.1:${forge.port}`, "t")); | |
| 254 | expect(steps).toEqual([ | |
| 255 | { what: "app radar", state: "failed", note: "you already have a repository named radar" }, | |
| 256 | { what: "app notes", state: "failed", note: "at most 5 forks a day" }, | |
| 257 | ]); | |
| 258 | } finally { | |
| 259 | await forge.stop(true); | |
| 260 | } | |
| 261 | }); | |
| 262 | ||
| 263 | test("an unreachable forge fails the fork step and says so", async () => { | |
| 264 | const r = await realHands("http://127.0.0.1:1", "t").forkFrom("trevato/radar", "radar"); | |
| 265 | expect(r).toEqual({ ok: false, exists: false, why: "cannot reach http://127.0.0.1:1" }); | |
| 266 | }); | |
| 267 | ||
| 268 | test("absorb: true on a forge source absorbs the fork as its new owner", async () => { | |
| 269 | const absorbed: string[] = []; | |
| 270 | const refuse = () => Promise.reject(new Error("not used for a forge source")); | |
| 271 | const hands = { | |
| 272 | makeApp: refuse, | |
| 273 | defaultBranch: refuse, | |
| 274 | carryHistory: refuse, | |
| 275 | follow: refuse, | |
| 276 | makeAgent: refuse, | |
| 277 | forkFrom: async () => ({ ok: true as const, owner: "me" }), | |
| 278 | absorb: async (owner: string, name: string) => { | |
| 279 | absorbed.push(`${owner}/${name}`); | |
| 280 | return { ok: true as const }; | |
| 281 | }, | |
| 282 | }; | |
| 283 | const pack = { ...parsed.value, apps: [{ ...parsed.value.apps[0]!, absorb: true }] }; | |
| 284 | const steps = await deployPack(pack, hands); | |
| 285 | expect(steps.map((s) => [s.what, s.state])).toEqual([ | |
| 286 | ["app radar", "done"], | |
| 287 | ["absorb radar", "done"], | |
| 288 | ]); | |
| 289 | expect(absorbed).toEqual(["me/radar"]); | |
| 290 | }); | |
| 291 | ||
| 292 | test("the https form still parses", () => { | |
| 293 | const r = parsePack(JSON.stringify({ pack: 1, name: "mine", title: "Mine", version: "1.0.0", description: "d", apps: [{ name: "x", description: "d", source: "https://github.com/a/b.git" }] })); | |
| 294 | expect(r.ok && r.value.apps[0]!.source).toBe("https://github.com/a/b"); | |
| 295 | }); | |
| 296 | }); |
test/packs.test.ts +9 −1
| @@ -34,7 +34,9 @@ describe("pack definition", () => { | ||
| 34 | 34 | }); |
| 35 | 35 | test("sources are public https repositories without credentials", () => { |
| 36 | 36 | expect(sourceOf("https://github.com/a/b.git")).toBe("https://github.com/a/b"); |
| 37 | for (const bad of ["http://github.com/a/b", "https://u:[email protected]/a/b", "[email protected]:a/b", "https://github.com/a", "https://github.com/a/b?x=1", "file:///etc"]) | |
| 37 | expect(sourceOf("trevato/forge")).toBe("trevato/forge"); | |
| 38 | expect(parse({ ...good, apps: [{ name: "one", description: "d", source: "trevato/forge" }] }).ok).toBe(true); | |
| 39 | for (const bad of ["a/b/c", "/a", "a/", "../x", "http://github.com/a/b", "https://u:[email protected]/a/b", "[email protected]:a/b", "https://github.com/a", "https://github.com/a/b?x=1", "file:///etc"]) | |
| 38 | 40 | expect(sourceOf(bad)).toBeUndefined(); |
| 39 | 41 | }); |
| 40 | 42 | test("each field is checked", () => { |
| @@ -57,6 +59,9 @@ describe("pack absorb", () => { | ||
| 57 | 59 | for (const v of ["yes", 1, {}, null]) expect(withAbsorb(v).ok).toBe(false); |
| 58 | 60 | }); |
| 59 | 61 | const hands = (calls: string[], absorbed: { ok: boolean }): Hands => ({ |
| 62 | async forkFrom() { | |
| 63 | return { ok: false, exists: false, why: "unused" }; | |
| 64 | }, | |
| 60 | 65 | async makeApp(name) { |
| 61 | 66 | return name === "kept" ? { ok: false, exists: true, why: "x", owner: "me" } : { ok: true, owner: "me" }; |
| 62 | 67 | }, |
| @@ -120,6 +125,9 @@ describe("deploy", () => { | ||
| 120 | 125 | calls.push(`history ${source} -> ${owner}/${name}`); |
| 121 | 126 | return { ok: true }; |
| 122 | 127 | }, |
| 128 | async forkFrom() { | |
| 129 | return { ok: false, exists: false, why: "unused" }; | |
| 130 | }, | |
| 123 | 131 | async defaultBranch() { |
| 124 | 132 | return "main"; |
| 125 | 133 | }, |
Docs say the hands test covers a name collision; it does not. Add the test or drop the claim. Also remove the blank line above this bullet.