Public: anyone can read this; the crew can change it.

Packs come from the forge: each app is a forge mirror, each pack is a forge repo #10455

Parked opened this in answer to
Closed without merging

mason/forge:mason/10447-packs-come-from-the-forge-each-app-is-a → main 9 commits fork

  • Closed without merging. Nothing was applied to main.

Review

Sign in to review this change.
35 files changed +630 −121
Dockerfile −2
@@ -23,8 +23,6 @@ COPY docs/PRINCIPLES.md ./docs/PRINCIPLES.md
2323 COPY docs/DESIGN.md ./docs/DESIGN.md
2424 # The OpenCode plugin the forge serves at /plugins/opencode/forge.ts (src/http/plugin_routes.ts).
2525 COPY plugins/opencode ./plugins/opencode
26 # The forge packs /packs lists and deploys (src/packs/catalog.ts reads packs/ at the root).
27 COPY packs ./packs
2826 # src/ alone: nothing under it imports scripts/ or test/ (test/src-imports.test.ts),
2927 # and the boot proof below fails the build if that ever stops being so (#2652).
3028
docs/features/packs.md +6 −4
@@ -1,12 +1,14 @@
11 # Forge packs: a premade forge to deploy (`/packs`)
22
3 A pack is a `pack.json` in a repository (`src/packs/pack.ts` holds the format, version 1): a name, title, semver version, description, the **apps** it brings (each an https address of a public git repository, with a name and description) and the **crew** it starts with (username, roles of planner/builder/reviewer, a brief, optionally a model and a per-run budget). Anyone can publish one. A pack may not carry data, secrets or variables: any key that names one (`secret`, `env`, `token`, `data`, `key`…) at any depth is refused, not ignored.
3 A pack is a `pack.json` in a repository **on the forge** (`src/packs/pack.ts` holds the format, version 1): a name, title, semver version, description, the **apps** it brings (each a forge repository, `owner/name`, never a URL: `packs/2048`) and the **crew** it starts with (username, roles of planner/builder/reviewer, a brief, optionally a model and a per-run budget). A pack may not carry data, secrets or variables: any key that names one (`secret`, `env`, `token`, `data`, `key`…) at any depth is refused, not ignored.
44
5 Two ship in `packs/`: **games** (2048, hextris, react-tetris) and **business** (memos, kanboard), each with a crew.
5 **Where packs come from.** Each upstream is brought in once as a read-only mirror under the `packs` account (`packs/2048`, `packs/memos`…) with the forge's existing origin mirrors, which re-sync on their schedule (`src/origins/sync.ts`); each pack is a repository there too (`packs/games`) holding a `pack.json`. Anyone can make a pack by making a repository with a `pack.json` whose `name` is the repository's name; **the forge admin marks** which ones `/packs` lists: `PUT /api/v1/repos/<owner>/<name>/pack` (admin scope; reads `pack.json` from the default branch, public repositories only) and `DELETE` to take it off. The last good text is kept in the `pack_repo` table (`src/packs/marked.ts`); it is read again hourly, and a pack whose file breaks keeps its last good text. `GET /api/v1/packs` and `/api/v1/packs/<name>` list them. Nothing ships in the source tree or the image: `/packs/<name>` is the pack's page when a pack of that name is listed, and the `packs/<name>` repository's page when none is.
66
7 - **Deploy one, one command.** After `git clone … && cd forge && bun install`, `bun scripts/pack.ts up games` makes the owner account (its passphrase is shown once and kept 0600 in `forge-data/owner-passphrase`; `FORGE_DATA` and `PORT` move the data and the port), boots this tree's forge on http://localhost:3000, makes each app, pushes the source's whole history (branches and tags), turns on following the source (Settings shows it; a new upstream commit shows as drift at the next check), adds the crew, and stays up serving. Nothing is asked but the model credential: with none in the environment it says the crew is off until one is added in Settings › Subscriptions. Run it again to finish a partial deploy: what exists is kept (history is never pushed again), and an app that was made but never followed is followed now; with no branch from the pack or the clone, that step fails rather than guess one. The carried branch becomes the app's default branch. Whether the crew has a credential is read from the data directory (Settings › Subscriptions) as well as the environment. Against a forge you already run: `FORGE_URL=… FORGE_TOKEN=forge_… bun scripts/pack.ts deploy games`.
7 **Deploying** forks each app from its forge mirror into the person's account (`POST /api/v1/repos/packs/2048/forks`), so the fork's lineage carries the upstream credit and its updates; GitHub is never touched (`test/packs-forge.test.ts` refuses github.com and deploys). On a machine whose forge holds no mirrors yet (`pack up`), the app is made and its history is cloned from the packs forge (`FORGE_PACKS_URL`, default https://forge.open-platform.sh), never from anywhere else. `bun scripts/pack.ts show|up|deploy <name>` reads the pack from that forge's `/api/v1/packs/<name>`, or a path to a `pack.json`.
8
9 - **Deploy one, one command.** After `git clone … && cd forge && bun install`, `bun scripts/pack.ts up games` makes the owner account (its passphrase is shown once and kept 0600 in `forge-data/owner-passphrase`; `FORGE_DATA` and `PORT` move the data and the port), boots this tree's forge on http://localhost:3000, makes each app, pushes the source's whole history from the forge (branches and tags), turns on following the source (Settings shows it; a new upstream commit shows as drift at the next check), adds the crew, and stays up serving. Nothing is asked but the model credential: with none in the environment it says the crew is off until one is added in Settings › Subscriptions. Run it again to finish a partial deploy: what exists is kept (history is never pushed again), and an app that was made but never followed is followed now; with no branch from the pack or the clone, that step fails rather than guess one. The carried branch becomes the app's default branch. Whether the crew has a credential is read from the data directory (Settings › Subscriptions) as well as the environment. Against a forge you already run: `FORGE_URL=… FORGE_TOKEN=forge_… bun scripts/pack.ts deploy games`.
810 - **See one.** `/packs` lists them, `/packs/<name>` says what it brings, the deploy commands, and how a stranger helps build its apps (get a guest account or use their own forge; a guest's pull waits for a person, and paths touching secrets, credentials, permissions or release machinery are named on it — `docs/ROLES.md`). The home page (`/`) has one press for each pack in its "Get your own forge" section, a link to `/packs/<name>` with an empty `data-slot="host"` beside it for a later hosted deploy; the pack page shows the commands in one `copyField` block (`#pack-command`, a read-only text area you select with a click; its copy button is `js-only`, so it is on the page only when script runs). Reached from the user menu, the signed-out menu and the palette. `bun scripts/pack.ts show games` prints the same in a terminal, and the SSH terminal has a **Forge packs** screen (palette, `src/tui/screens/packs.ts`): the list, a pack's apps and crew, the deploy commands (the `up` line included: the web page and the screen both take them from `deployCommands`) and what happens to a stranger's pull. The words are shared in `src/packs/words.ts`.
911 - **Nightly.** `scripts/pack-compat.ts` boots a clean forge per pack, deploys it with the real sources and fails if any step or app page fails (job `pack-compat`, `src/nightly/jobs.ts`).
1012 - Code: `src/packs/`, `src/web/pages/packs.ts`, `scripts/pack.ts`. Tests: `test/packs.test.ts`, `test/packs-up.test.ts` (the whole path against a fake source for both packs, `up()` itself with a real child forge on a free port, the owner reused on a second run, a stored credential, a partial deploy finished, a new upstream commit showing as drift), `test/packs-hands.test.ts` (the git credential is scoped to the forge and never sent to a pack's source; only 409 means "exists"; the terminal screen).
1113
12 Not done yet: a hosted "deploy this pack" press, a stable follow link from each app back to its source, the guest sign-up gate and the crew filtering guest pulls on automode (owner decisions on #9428).
14 Not done yet: bringing the five upstreams in as `packs/*` mirrors and marking `packs/games` and `packs/business` (the owner does this on the live forge with the origin mirror and `PUT …/pack`); showing the licence and original authors on a mirror's repo page; a hosted "deploy this pack" press, a stable follow link from each app back to its source, the guest sign-up gate and the crew filtering guest pulls on automode (owner decisions on #9428).
scripts/pack-compat.ts +19 −6
@@ -2,7 +2,7 @@
22 * `bun scripts/pack-compat.ts` — does every shipped pack deploy on a clean
33 * forge? For each pack it boots a new forge on an empty data directory, makes
44 * an account and a token, deploys the pack as `scripts/pack.ts` does (so the
5 * apps' real sources are cloned over the network) and checks every step
5 * apps' history is cloned from the packs forge, never from GitHub) and checks every step
66 * succeeded and each app's page answers. Exit 1 and the failing steps
77 * otherwise. Run nightly (`src/nightly/jobs.ts`, job `pack-compat`).
88 */
@@ -14,13 +14,26 @@ import { createApp } from "../src/app.ts";
1414 import { createToken } from "../src/core/tokens.ts";
1515 import { createUser } from "../src/core/users.ts";
1616 import { serve } from "../src/http/server.ts";
17 import { catalog } from "../src/packs/catalog.ts";
1817 import { deployPack } from "../src/packs/deploy.ts";
19 import { realHands } from "./pack.ts";
18 import { parsePack, type Pack } from "../src/packs/pack.ts";
19 import { packsForge, realHands } from "./pack.ts";
2020
21 const { packs, problems } = catalog();
22 let failed = problems.length;
23 for (const p of problems) console.error(`✗ ${p}`);
21 // The packs the packs forge lists, read from its API: the forge is the only source, GitHub is never asked.
22 let failed = 0;
23 const packs: Pack[] = [];
24 const listed = await fetch(`${packsForge()}/api/v1/packs`).then((r) => (r.ok ? (r.json() as Promise<{ items?: unknown[] }>) : null), () => null);
25 if (listed === null) {
26 console.error(`✗ cannot read the packs from ${packsForge()}`);
27 failed++;
28 }
29 for (const item of listed?.items ?? []) {
30 const parsed = parsePack(JSON.stringify(item));
31 if (parsed.ok) packs.push(parsed.value);
32 else {
33 failed++;
34 console.error(`✗ ${parsed.error}`);
35 }
36 }
2437
2538 for (const pack of packs) {
2639 const dataDir = await mkdtemp(join(tmpdir(), `pack-compat-${pack.name}-`));
scripts/pack.ts +34 −8
@@ -16,13 +16,23 @@ import { existsSync, mkdtempSync, readFileSync, rmSync } from "node:fs";
1616 import { tmpdir } from "node:os";
1717 import { join } from "node:path";
1818 import { ForgeApi } from "../src/cli/api.ts";
19 import { packNamed } from "../src/packs/catalog.ts";
2019 import { deployPack, type Hands } from "../src/packs/deploy.ts";
2120 import { parsePack, type Pack } from "../src/packs/pack.ts";
2221 import { finishWords, OWNER, prepareOwner } from "../src/packs/up.ts";
2322
24 function load(arg: string): Pack | string {
25 if (/^[a-z][a-z0-9-]*$/.test(arg) && !existsSync(arg)) return packNamed(arg) ?? `no pack named ${arg}`;
23 /** The forge whose pack repositories a deploy reads: the packs live there, and no other host is ever cloned from. */
24 export const packsForge = (env: Record<string, string | undefined> = process.env): string =>
25 (env.FORGE_PACKS_URL ?? "https://forge.open-platform.sh").replace(/\/+$/, "");
26
27 /** A pack by name from the packs forge's `/api/v1/packs/<name>`, or by the path of a pack.json. */
28 async function load(arg: string): Promise<Pack | string> {
29 if (/^[a-z][a-z0-9-]*$/.test(arg) && !existsSync(arg)) {
30 const r = await fetch(`${packsForge()}/api/v1/packs/${arg}`).catch(() => null);
31 if (r === null) return `cannot reach ${packsForge()}`;
32 if (!r.ok) return `no pack named ${arg} on ${packsForge()}`;
33 const parsed = parsePack(await r.text());
34 return parsed.ok ? parsed.value : parsed.error;
35 }
2636 if (!existsSync(arg)) return `no such file: ${arg}`;
2737 const parsed = parsePack(readFileSync(arg, "utf8"));
2838 return parsed.ok ? parsed.value : parsed.error;
@@ -36,7 +46,7 @@ const baseEnv = (): Record<string, string> => ({
3646 GIT_TERMINAL_PROMPT: "0",
3747 });
3848
39 /** The environment for cloning a pack's source: a stranger's host, so no credential at all. */
49 /** The environment for cloning a pack's source from the packs forge: no credential at all, the repositories are public. */
4050 export const cloneEnv = (): Record<string, string> => baseEnv();
4151
4252 /**
@@ -58,7 +68,7 @@ async function git(args: string[], env: Record<string, string>, cwd?: string): P
5868 }
5969
6070 export interface HandsOptions {
61 /** Where to clone a pack's source from, when not the address the pack names (a test's local repository). */
71 /** Where to clone a pack's source from, when not its repository on the packs forge (a test's local repository). */
6272 readonly cloneFrom?: (source: string) => string;
6373 }
6474
@@ -71,6 +81,22 @@ async function gitOut(args: string[], env: Record<string, string>, cwd: string):
7181 export function realHands(url: string, token: string, options: HandsOptions = {}): Hands {
7282 const api = new ForgeApi(url, token);
7383 return {
84 async forkApp(source, name) {
85 const auth = { authorization: `Bearer ${token}`, "content-type": "application/json" };
86 const r = await fetch(`${url}/api/v1/repos/${source}/forks`, { method: "POST", headers: auth, body: JSON.stringify({ name }) }).catch(() => null);
87 if (r === null) return { ok: false, here: true, exists: false, why: `cannot reach ${url}` };
88 if (r.status === 404) return { ok: false, here: false, exists: false, why: `${source} is not on ${url}` };
89 if (r.ok) {
90 const made = (await r.json().catch(() => null)) as { ownerName?: unknown } | null;
91 return typeof made?.ownerName === "string" ? { ok: true, owner: made.ownerName } : { ok: false, here: true, exists: false, why: "the forge answered without an owner" };
92 }
93 // Already forked, or the name is taken by what the person already has: kept, never overwritten.
94 const me = await api.whoami();
95 const owner = me.ok && me.value.username !== "" ? me.value.username : undefined;
96 const mine = owner === undefined ? null : await fetch(`${url}/api/v1/repos/${owner}/${name}`, { headers: { authorization: `Bearer ${token}` } }).catch(() => null);
97 if (r.status === 409 || mine?.ok === true) return { ok: false, here: true, exists: true, why: "exists", ...(owner === undefined ? {} : { owner }) };
98 return { ok: false, here: true, exists: false, why: `the forge answered ${r.status}` };
99 },
74100 async makeApp(name, description) {
75101 const made = await api.newApp({ name, description, visibility: "public" });
76102 if (made.ok) return { ok: true, owner: made.value.pin.owner };
@@ -90,7 +116,7 @@ export function realHands(url: string, token: string, options: HandsOptions = {}
90116 try {
91117 const bare = join(dir, "src.git");
92118 // Source history is fetched without hooks or templates and pushed as plain branches and tags.
93 const cloned = await git(["clone", "--bare", "--quiet", ...(branch === undefined ? [] : ["--branch", branch]), options.cloneFrom?.(source) ?? source, bare], cloneEnv());
119 const cloned = await git(["clone", "--bare", "--quiet", ...(branch === undefined ? [] : ["--branch", branch]), options.cloneFrom?.(source) ?? `${packsForge()}/${source}.git`, bare], cloneEnv());
94120 if (cloned !== null) return { ok: false, why: cloned };
95121 const pushed = await git(
96122 ["push", "--quiet", "--force", `${url}/${owner}/${name}.git`, "refs/heads/*:refs/heads/*", "refs/tags/*:refs/tags/*"],
@@ -121,7 +147,7 @@ export function realHands(url: string, token: string, options: HandsOptions = {}
121147 const r = await fetch(`${url}/api/v1/repos/${owner}/${name}/upstream`, {
122148 method: "PUT",
123149 headers: { authorization: `Bearer ${token}`, "content-type": "application/json" },
124 body: JSON.stringify({ url: source, branch, enabled: true }),
150 body: JSON.stringify({ url: `${packsForge()}/${source}`, branch, enabled: true }),
125151 }).catch(() => null);
126152 if (r === null) return { ok: false, why: `cannot reach ${url}` };
127153 if (r.ok) return { ok: true };
@@ -214,7 +240,7 @@ export async function up(pack: Pack, options: UpOptions): Promise<number> {
214240
215241 if (import.meta.main) {
216242 const [verb, which] = process.argv.slice(2);
217 const pack = which === undefined ? USAGE : load(which);
243 const pack = which === undefined ? USAGE : await load(which);
218244 if (typeof pack === "string" || (verb !== "show" && verb !== "deploy" && verb !== "up")) {
219245 console.error(typeof pack === "string" ? pack : USAGE);
220246 process.exit(2);
scripts/unreached-allow.json −1
@@ -15,7 +15,6 @@
1515 "src/flow/review_rounds.ts",
1616 "src/flow/shared_red.ts",
1717 "src/flow/stale_fold.ts",
18 "src/kernel/why_now.ts",
1918 "src/spawn/pod.ts",
2019 "src/tui/gallery.ts",
2120 "src/tui/ui/qr.ts",
src/contract/openapi/routes/deleteApiV1ReposOwnerNamePack.ts added +3
@@ -0,0 +1,3 @@
1 import { REPO, type Route } from "../shared.ts";
2
3 export const route: Route = { method: "delete", path: `${REPO}/pack`, summary: "Take the repository off the packs list (forge admin, admin scope); answers { marked: false, was }", auth: true };
src/contract/openapi/routes/getApiV1Packs.ts added +3
@@ -0,0 +1,3 @@
1 import type { Route } from "../shared.ts";
2
3 export const route: Route = { method: "get", path: "/api/v1/packs", summary: "The packs the forge lists: { items: [pack.json fields + repo] }; public", auth: false };
src/contract/openapi/routes/getApiV1PacksName.ts added +3
@@ -0,0 +1,3 @@
1 import type { Route } from "../shared.ts";
2
3 export const route: Route = { method: "get", path: "/api/v1/packs/{name}", summary: "One listed pack: its pack.json fields and repo; public; 404 when none is named so", auth: false };
src/contract/openapi/routes/index.ts +4
@@ -22,6 +22,7 @@ export { route as deleteApiV1ReposOwnerNameHooksId } from "./deleteApiV1ReposOwn
2222 export { route as deleteApiV1ReposOwnerNameIssuesNumberDependenciesBlocker } from "./deleteApiV1ReposOwnerNameIssuesNumberDependenciesBlocker.ts";
2323 export { route as deleteApiV1ReposOwnerNameLabelsId } from "./deleteApiV1ReposOwnerNameLabelsId.ts";
2424 export { route as deleteApiV1ReposOwnerNameMilestonesNumberWatch } from "./deleteApiV1ReposOwnerNameMilestonesNumberWatch.ts";
25 export { route as deleteApiV1ReposOwnerNamePack } from "./deleteApiV1ReposOwnerNamePack.ts";
2526 export { route as deleteApiV1ReposOwnerNamePromptsId } from "./deleteApiV1ReposOwnerNamePromptsId.ts";
2627 export { route as deleteApiV1ReposOwnerNamePullsNumberHold } from "./deleteApiV1ReposOwnerNamePullsNumberHold.ts";
2728 export { route as deleteApiV1ReposOwnerNamePullsNumberThreadsAnchorResolve } from "./deleteApiV1ReposOwnerNamePullsNumberThreadsAnchorResolve.ts";
@@ -99,6 +100,8 @@ export { route as getApiV1OpenapiJson } from "./getApiV1OpenapiJson.ts";
99100 export { route as getApiV1OrgsOrg } from "./getApiV1OrgsOrg.ts";
100101 export { route as getApiV1OrgsOrgTeams } from "./getApiV1OrgsOrgTeams.ts";
101102 export { route as getApiV1OrgsOrgTeamsTeamMembers } from "./getApiV1OrgsOrgTeamsTeamMembers.ts";
103 export { route as getApiV1Packs } from "./getApiV1Packs.ts";
104 export { route as getApiV1PacksName } from "./getApiV1PacksName.ts";
102105 export { route as getApiV1PassportHealth } from "./getApiV1PassportHealth.ts";
103106 export { route as getApiV1Releases } from "./getApiV1Releases.ts";
104107 export { route as getApiV1Repos } from "./getApiV1Repos.ts";
@@ -372,6 +375,7 @@ export { route as putApiV1ReposOwnerNameIssuesNumberLabels } from "./putApiV1Rep
372375 export { route as putApiV1ReposOwnerNameIssuesNumberPlan } from "./putApiV1ReposOwnerNameIssuesNumberPlan.ts";
373376 export { route as putApiV1ReposOwnerNameIssuesNumberTeams } from "./putApiV1ReposOwnerNameIssuesNumberTeams.ts";
374377 export { route as putApiV1ReposOwnerNameMilestonesNumberWatch } from "./putApiV1ReposOwnerNameMilestonesNumberWatch.ts";
378 export { route as putApiV1ReposOwnerNamePack } from "./putApiV1ReposOwnerNamePack.ts";
375379 export { route as putApiV1ReposOwnerNamePullsNumberAssignee } from "./putApiV1ReposOwnerNamePullsNumberAssignee.ts";
376380 export { route as putApiV1ReposOwnerNamePullsNumberHold } from "./putApiV1ReposOwnerNamePullsNumberHold.ts";
377381 export { route as putApiV1ReposOwnerNamePullsNumberLabels } from "./putApiV1ReposOwnerNamePullsNumberLabels.ts";
src/contract/openapi/routes/putApiV1ReposOwnerNamePack.ts added +3
@@ -0,0 +1,3 @@
1 import { REPO, type Route } from "../shared.ts";
2
3 export const route: Route = { method: "put", path: `${REPO}/pack`, summary: "Mark the repository as a pack, read from its pack.json (forge admin, admin scope); answers { marked: true } or 422", auth: true };
src/core/migration-pins/20261109200000-pack-repos.pin added +1
@@ -0,0 +1,1 @@
1 bfa1430728d5919c08438f9e88cb975ae96dd040024f875edb84439169a532f8
src/core/migrations/20261109200000-pack-repos.ts added +14
@@ -0,0 +1,14 @@
1 // The repositories the owner marked as packs (#10447): a repository with a
2 // `pack.json` that /packs lists. The last good text of that file is kept
3 // here, so a page reads a row and never a git repository. The row goes with
4 // its repository. Additive.
5 export default `
6 CREATE TABLE IF NOT EXISTS pack_repo (
7 repo_id TEXT PRIMARY KEY REFERENCES repo(id) ON DELETE CASCADE,
8 name TEXT NOT NULL UNIQUE,
9 pack_json TEXT NOT NULL,
10 marked_by TEXT NOT NULL,
11 marked_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),
12 refreshed_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now'))
13 );
14 `;
src/http/api.ts +2
@@ -383,6 +383,7 @@ import { secretRoutes } from "./api/secrets.ts";
383383 import { drillRoutes } from "./api/drills.ts";
384384 import { featureRoutes } from "./api/features.ts";
385385 import { upstreamRoutes } from "./api/upstream.ts";
386 import { packRoutes } from "./api/packs.ts";
386387 import type { Router } from "./router.ts";
387388 import { clientAddress } from "./peer.ts";
388389 import { manifestFor } from "../actions/here.ts";
@@ -1593,6 +1594,7 @@ export function apiRoutes(router: Router<Handler>): void {
15931594 });
15941595
15951596 upstreamRoutes(router);
1597 packRoutes(router);
15961598
15971599 router.add("PATCH", "/api/v1/repos/:owner/:name", async (ctx) => {
15981600 const denied = requireScope(ctx, "admin");
src/http/api/packs.ts added +42
@@ -0,0 +1,42 @@
1 /**
2 * Packs over the API. `GET /api/v1/packs` and `GET /api/v1/packs/:name` are
3 * public: the listed packs, as their `pack.json` says. `PUT` and `DELETE`
4 * `/api/v1/repos/:owner/:name/pack` are the forge admin's: mark a repository
5 * as a pack (read from its default branch now) or take it off. Rules in
6 * `packs/marked.ts`; what a pack is, in `packs/pack.ts`.
7 */
8
9 import { loadRepo } from "../api.ts";
10 import { markedPacks } from "../../packs/catalog.ts";
11 import { markError, markPack, unmarkPack } from "../../packs/marked.ts";
12 import type { Handler } from "../context.ts";
13 import { apiError, json, notFound, requireScope } from "../respond.ts";
14 import type { Router } from "../router.ts";
15
16 export function packRoutes(router: Router<Handler>): void {
17 router.add("GET", "/api/v1/packs", (ctx) => json({ items: markedPacks(ctx.app.db).map((m) => ({ ...m.pack, repo: m.repo })) }));
18
19 router.add("GET", "/api/v1/packs/:name", (ctx) => {
20 const found = markedPacks(ctx.app.db).find((m) => m.pack.name === ctx.params.name);
21 return found === undefined ? notFound() : json({ ...found.pack, repo: found.repo });
22 });
23
24 router.add("PUT", "/api/v1/repos/:owner/:name/pack", async (ctx) => {
25 if (ctx.auth.user?.isAdmin !== true) return notFound();
26 const denied = requireScope(ctx, "admin");
27 if (denied) return denied;
28 const repo = loadRepo(ctx);
29 if (!repo) return notFound();
30 const marked = await markPack(ctx.app, repo, ctx.auth.user.id);
31 return marked.ok ? json({ marked: true }) : apiError("not_a_pack", 422, markError(marked.error));
32 });
33
34 router.add("DELETE", "/api/v1/repos/:owner/:name/pack", (ctx) => {
35 if (ctx.auth.user?.isAdmin !== true) return notFound();
36 const denied = requireScope(ctx, "admin");
37 if (denied) return denied;
38 const repo = loadRepo(ctx);
39 if (!repo) return notFound();
40 return json({ marked: false, was: unmarkPack(ctx.app, repo) });
41 });
42 }
src/main.ts +4
@@ -40,6 +40,7 @@ import { drainBudgets } from "./drain.ts";
4040 import { drainStreams } from "./http/sse.ts";
4141 import { serve } from "./http/server.ts";
4242 import { checkDue } from "./actions/upstream_follow.ts";
43 import { refreshMarked } from "./packs/marked.ts";
4344 import { runSync } from "./origins/sync.ts";
4445 import { runAheadReads } from "./origins/ahead.ts";
4546 import { runGuardSweep } from "./origins/credential.ts";
@@ -654,6 +655,8 @@ const tickAsks = async (): Promise<void> => {
654655 // Followed upstreams are read on a slow tick (`actions/upstream_follow.ts`):
655656 // the tip and how far behind land on the follow row; nothing is opened.
656657 const followTimer = setInterval(() => void checkDue(app), 60_000);
658 // The packs listed on /packs are read again from their repositories, hourly.
659 const packTimer = setInterval(() => void refreshMarked(app), 3_600_000);
657660 const staleTimer = setInterval(() => void tickAsks(), 120_000);
658661 void tickAsks();
659662
@@ -877,6 +880,7 @@ function drain(signal: NodeJS.Signals): Promise<unknown> {
877880 clearInterval(syncTimer);
878881 clearInterval(staleTimer);
879882 clearInterval(followTimer);
883 clearInterval(packTimer);
880884 stopClogTick();
881885 stopKeeper();
882886 clearInterval(processTimer);
src/packs/catalog.ts +33 −26
@@ -1,34 +1,41 @@
11 /**
2 * The packs this forge ships: every `packs/<name>/pack.json` in the
3 * repository, parsed once. A file that fails to parse is left out and named
4 * in `problems`, which a test keeps empty, so a bad pack never reaches a page.
2 * The packs this forge lists: the repositories the owner marked, each holding
3 * a `pack.json` (`marked.ts`). Read from the database, never from git, so a
4 * page costs one query; a row whose text no longer parses is left out.
55 */
66
7 import { existsSync, readdirSync, readFileSync } from "node:fs";
8 import { join } from "node:path";
7 import type { Database } from "bun:sqlite";
98 import { parsePack, type Pack } from "./pack.ts";
109
11 const ROOT = join(import.meta.dir, "..", "..", "packs");
12
13 export function loadCatalog(root = ROOT): { packs: readonly Pack[]; problems: readonly string[] } {
14 const packs: Pack[] = [];
15 const problems: string[] = [];
16 if (!existsSync(root)) return { packs, problems };
17 for (const dir of readdirSync(root, { withFileTypes: true })) {
18 if (!dir.isDirectory()) continue;
19 const file = join(root, dir.name, "pack.json");
20 if (!existsSync(file)) continue;
21 const parsed = parsePack(readFileSync(file, "utf8"));
22 if (!parsed.ok) problems.push(`${dir.name}: ${parsed.error}`);
23 else if (parsed.value.name !== dir.name) problems.push(`${dir.name}: name must match the directory`);
24 else packs.push(parsed.value);
25 }
26 packs.sort((a, b) => a.name.localeCompare(b.name));
27 return { packs, problems };
10 export interface MarkedPack {
11 readonly pack: Pack;
12 /** The repository the pack is, `owner/name`. */
13 readonly repo: string;
2814 }
2915
30 let cached: ReturnType<typeof loadCatalog> | undefined;
31 /** The shipped catalog, read once per process. */
32 export const catalog = (): ReturnType<typeof loadCatalog> => (cached ??= loadCatalog());
16 interface Row {
17 pack_json: string;
18 owner: string;
19 repo_name: string;
20 }
3321
34 export const packNamed = (name: string): Pack | undefined => catalog().packs.find((p) => p.name === name);
22 export function markedPacks(db: Database): readonly MarkedPack[] {
23 const rows = db
24 .query<Row, []>(
25 `SELECT p.pack_json, u.username AS owner, r.name AS repo_name
26 FROM pack_repo p JOIN repo r ON r.id = p.repo_id JOIN user u ON u.id = r.owner_id
27 ORDER BY p.name`,
28 )
29 .all();
30 const out: MarkedPack[] = [];
31 for (const row of rows) {
32 const parsed = parsePack(row.pack_json);
33 if (parsed.ok) out.push({ pack: parsed.value, repo: `${row.owner}/${row.repo_name}` });
34 }
35 return out;
36 }
37
38 /** The listed packs, by name. */
39 export const catalog = (db: Database): { packs: readonly Pack[] } => ({ packs: markedPacks(db).map((m) => m.pack) });
40
41 export const packNamed = (db: Database, name: string): Pack | undefined => catalog(db).packs.find((p) => p.name === name);
src/packs/deploy.ts +20
@@ -12,6 +12,11 @@
1212 import type { Pack } from "./pack.ts";
1313
1414 export interface Hands {
15 /**
16 * Fork the app's source, a repository on this forge, into the person's account under `name`: the fork's lineage carries the upstream credit and its updates.
17 * `here: false` when this forge does not hold the source (a machine of your own, before it has any pack): the app is then made and its history carried from the packs forge, never from anywhere else.
18 */
19 forkApp(source: string, name: string): Promise<{ ok: true; owner: string } | { ok: false; here: boolean; exists: boolean; why: string; owner?: string }>;
1520 /** Make an app; `exists` when the forge already has one of that name. */
1621 makeApp(name: string, description: string): Promise<{ ok: true; owner: string } | { ok: false; exists: boolean; why: string; owner?: string }>;
1722 /** The default branch of an app already in the forge, or undefined when it cannot be read. */
@@ -40,6 +45,21 @@ async function absorbStep(hands: Hands, owner: string, name: string): Promise<St
4045 export async function deployPack(pack: Pack, hands: Hands): Promise<readonly Step[]> {
4146 const steps: Step[] = [];
4247 for (const app of pack.apps) {
48 const forked = await hands.forkApp(app.source, app.name);
49 if (forked.ok) {
50 steps.push({ what: `app ${app.name}`, state: "done", note: `forked from ${app.source}, upstream credit kept` });
51 if (app.absorb === true) steps.push(await absorbStep(hands, forked.owner, app.name));
52 continue;
53 }
54 if (forked.here) {
55 steps.push(
56 forked.exists
57 ? { what: `app ${app.name}`, state: "kept", note: "already here" }
58 : { what: `app ${app.name}`, state: "failed", note: forked.why },
59 );
60 if (forked.exists && forked.owner !== undefined && app.absorb === true) steps.push(await absorbStep(hands, forked.owner, app.name));
61 continue;
62 }
4363 const made = await hands.makeApp(app.name, `${app.description} (from ${app.source})`);
4464 if (!made.ok) {
4565 if (!made.exists) {
src/packs/marked.ts added +81
@@ -0,0 +1,81 @@
1 /**
2 * Marking a repository as a pack. Anyone can make a pack by putting a
3 * `pack.json` in a repository; the owner marks which ones appear on /packs.
4 * Marking reads the file from the repository's default branch and keeps its
5 * last good text; `refreshMarked` reads it again on a schedule, so a pack
6 * edited in its repository follows, and one that breaks keeps its last good
7 * text rather than vanishing.
8 */
9
10 import type { App } from "../app.ts";
11 import { getRepo, type Repo } from "../core/repos.ts";
12 import { blobForRender } from "../git/read.ts";
13 import { repoDir } from "../git/repos.ts";
14 import { parsePack } from "./pack.ts";
15
16 const MAX_PACK_BYTES = 64 * 1024;
17
18 export type MarkError = "no_pack_json" | "too_large" | "private" | "name_mismatch" | "name_taken" | { readonly kind: "invalid"; readonly reason: string };
19
20 export const MARK_WORDS: Readonly<Record<Exclude<MarkError, { kind: "invalid" }>, string>> = {
21 no_pack_json: "the repository has no pack.json on its default branch",
22 too_large: "pack.json is too large to be a pack",
23 private: "only a public repository can be listed as a pack",
24 name_mismatch: "the pack's name must be the repository's name",
25 name_taken: "another repository is already listed as a pack of that name",
26 };
27
28 export const markError = (e: MarkError): string => (typeof e === "string" ? MARK_WORDS[e] : e.reason);
29
30 /** The repository's `pack.json` text, checked as a pack, or the reason it is not one. */
31 async function readPackText(app: App, repo: Repo): Promise<{ ok: true; text: string } | { ok: false; error: MarkError }> {
32 if (repo.visibility !== "public") return { ok: false, error: "private" };
33 const blob = await blobForRender(repoDir(app.reposRoot, repo.ownerName, repo.name), repo.defaultBranch, "pack.json", MAX_PACK_BYTES);
34 if (blob.kind === "too_large") return { ok: false, error: "too_large" };
35 if (blob.kind !== "text") return { ok: false, error: "no_pack_json" };
36 const text = new TextDecoder().decode(blob.body);
37 const parsed = parsePack(text);
38 if (!parsed.ok) return { ok: false, error: { kind: "invalid", reason: parsed.error } };
39 if (parsed.value.name !== repo.name) return { ok: false, error: "name_mismatch" };
40 return { ok: true, text };
41 }
42
43 /** List the repository as a pack, or refresh its text if it already is. */
44 export async function markPack(app: App, repo: Repo, markedBy: string): Promise<{ ok: true } | { ok: false; error: MarkError }> {
45 const read = await readPackText(app, repo);
46 if (!read.ok) return read;
47 const taken = app.db.query<{ repo_id: string }, [string]>("SELECT repo_id FROM pack_repo WHERE name = ?1").get(repo.name);
48 if (taken !== null && taken.repo_id !== repo.id) return { ok: false, error: "name_taken" };
49 app.db
50 .query(
51 `INSERT INTO pack_repo (repo_id, name, pack_json, marked_by) VALUES (?1, ?2, ?3, ?4)
52 ON CONFLICT(repo_id) DO UPDATE SET pack_json = excluded.pack_json, refreshed_at = strftime('%Y-%m-%dT%H:%M:%fZ','now')`,
53 )
54 .run(repo.id, repo.name, read.text, markedBy);
55 return { ok: true };
56 }
57
58 /** Take the repository off /packs. True when it was listed. */
59 export function unmarkPack(app: App, repo: Repo): boolean {
60 return app.db.query("DELETE FROM pack_repo WHERE repo_id = ?1").run(repo.id).changes > 0;
61 }
62
63 /** Whether the repository is listed as a pack. */
64 export const isMarked = (app: App, repo: Repo): boolean => app.db.query("SELECT 1 FROM pack_repo WHERE repo_id = ?1").get(repo.id) !== null;
65
66 /** Read every listed pack's file again; a pack that no longer reads keeps its last good text. Returns how many changed. */
67 export async function refreshMarked(app: App): Promise<number> {
68 const rows = app.db.query<{ repo_id: string; pack_json: string; owner: string; name: string }, []>(
69 `SELECT p.repo_id, p.pack_json, u.username AS owner, r.name FROM pack_repo p JOIN repo r ON r.id = p.repo_id JOIN user u ON u.id = r.owner_id`,
70 ).all();
71 let changed = 0;
72 for (const row of rows) {
73 const repo = getRepo(app.db, row.owner, row.name);
74 if (repo === undefined) continue;
75 const read = await readPackText(app, repo);
76 if (!read.ok || read.text === row.pack_json) continue;
77 app.db.query("UPDATE pack_repo SET pack_json = ?1, refreshed_at = strftime('%Y-%m-%dT%H:%M:%fZ','now') WHERE repo_id = ?2").run(read.text, row.repo_id);
78 changed++;
79 }
80 return changed;
81 }
src/packs/pack.ts +8 −12
@@ -25,7 +25,7 @@ const FORBIDDEN = /secret|password|token|credential|env|variable|data|key/i;
2525 export interface PackApp {
2626 readonly name: string;
2727 readonly description: string;
28 /** An https address of a public git repository, no credentials in it. */
28 /** The forge repository the app comes from, `owner/name`: a mirror under the packs owner. Never a URL: a deploy reads only from the forge. */
2929 readonly source: string;
3030 readonly branch?: string;
3131 /** Deploy absorbs the app as a feature, as the deploying admin's act (an event). Only a yes is carried; it sets no setting. */
@@ -55,17 +55,13 @@ const text = (v: unknown, max = MAX_TEXT): string | undefined =>
5555 ? v.trim()
5656 : undefined;
5757
58 /** The source's address if it names a public https repository, else undefined. */
58 /** A forge repository as `owner/name` (`packs/2048`), or undefined. A URL, a host or a credential is never one. */
5959 export function sourceOf(v: unknown): string | undefined {
60 const raw = text(v, 300);
61 const url = raw === undefined ? null : URL.parse(raw);
62 if (url === null || url.protocol !== "https:" || url.username !== "" || url.password !== "") return undefined;
63 if (url.search !== "" || url.hash !== "") return undefined;
64 const parts = url.pathname.split("/").filter((s) => s !== "");
65 if (parts.length !== 2) return undefined;
66 const repo = parts[1]!.replace(/\.git$/, "");
67 if (!REPO_PART.test(parts[0]!) || !REPO_PART.test(repo)) return undefined;
68 return `https://${url.host.toLowerCase()}/${parts[0]}/${repo}`;
60 const raw = text(v, 200);
61 if (raw === undefined) return undefined;
62 const parts = raw.split("/");
63 if (parts.length !== 2 || !REPO_PART.test(parts[0]!) || !REPO_PART.test(parts[1]!)) return undefined;
64 return raw;
6965 }
7066
7167 /** The first forbidden key anywhere in `v`, as a path, or undefined. */
@@ -119,7 +115,7 @@ export function parsePack(raw: string): Result<Pack, string> {
119115 const source = sourceOf(r.source);
120116 const about = text(r.description, 300);
121117 if (appName === undefined || source === undefined || about === undefined)
122 return bad(`each app needs a name, a description and an https "source" naming a public repository`);
118 return bad(`each app needs a name, a description and a "source" naming a forge repository as owner/name`);
123119 if (apps.some((x) => x.name === appName)) return bad(`two apps are named "${appName}"`);
124120 const branch = r.branch === undefined ? undefined : text(r.branch, 100);
125121 if (r.branch !== undefined && (branch === undefined || !/^[A-Za-z0-9][A-Za-z0-9._/-]*$/.test(branch)))
src/tui/screens/packs.ts +6 −5
@@ -1,10 +1,11 @@
11 /**
2 * `/packs` in the terminal: the premade forges this forge ships, one pack's
2 * `/packs` in the terminal: the premade forges this forge lists, one pack's
33 * contents (apps, crew), the commands that deploy it and what happens to a
44 * stranger's pull. Words come from `src/packs/words.ts`, the same the web page
5 * prints. Anyone may open it; it reads the catalog, nothing of the forge's.
5 * prints. Anyone may open it; it reads the listed packs, nothing else of the forge's.
66 */
77
8 import type { Database } from "bun:sqlite";
89 import { catalog } from "../../packs/catalog.ts";
910 import type { Pack } from "../../packs/pack.ts";
1011 import { BRINGS_WORDS, deployCommands, STRANGER_WORDS } from "../../packs/words.ts";
@@ -61,8 +62,8 @@ function packScreen(ctx: Ctx, p: Pack, origin: string | undefined): Screen {
6162 };
6263 }
6364
64 export function packsScreen(ctx: Ctx, origin: string | undefined): Screen {
65 const { packs } = catalog();
65 export function packsScreen(ctx: Ctx, origin: string | undefined, db: Database): Screen {
66 const { packs } = catalog(db);
6667 let at = 0;
6768 return {
6869 title: "Forge packs",
@@ -94,5 +95,5 @@ export const packsCommand = (session: Session, ctx: Ctx) => ({
9495 id: "packs",
9596 title: "Forge packs",
9697 hint: "premade forges to deploy, and how to help build their apps",
97 run: () => ctx.push(packsScreen(ctx, session.app.config.url)),
98 run: () => ctx.push(packsScreen(ctx, session.app.config.url, session.app.db)),
9899 });
src/web/pages/packs.ts +12 −6
@@ -1,5 +1,5 @@
11 /**
2 * `/packs` and `/packs/:name`: the forge packs this forge ships (`src/packs/`).
2 * `/packs` and `/packs/:name`: the packs listed on this forge: repositories the owner marked (`src/packs/marked.ts`).
33 * A pack's page says what it brings, the one command that deploys it on your
44 * own machine, and a link to start a forge of your own. Server-rendered, no script; anyone may read it.
55 */
@@ -10,10 +10,15 @@ import { catalog, packNamed } from "../../packs/catalog.ts";
1010 import type { Ctx } from "../../http/context.ts";
1111 import { notFoundPage, pageHead } from "../chrome.ts";
1212 import { deployCommands } from "../../packs/words.ts";
13 import { getRepo } from "../../core/repos.ts";
14 import { repoHome } from "./repo.ts";
1315 import { html } from "../html.ts";
1416 import { page } from "../layout.ts";
1517 import { copyField, row } from "../ui.ts";
1618
19 /** The account the forge's mirrored upstreams and its own packs live under. */
20 const PACKS_OWNER = "packs";
21
1722 const plural = (n: number, one: string): string => `${n} ${one}${n === 1 ? "" : "s"}`;
1823
1924 /** The list of packs as rows; each row is one j/k stop. */
@@ -31,7 +36,7 @@ export const packsList = (packs: readonly Pack[]) =>
3136 export function packsPage(ctx: Ctx): Response {
3237 notePhase(ctx.request, "session", takeSessionTime(ctx.request));
3338 const started = performance.now();
34 const { packs } = catalog();
39 const { packs } = catalog(ctx.app.db);
3540 const body = html`${pageHead("Forge packs", {
3641 sub: "A whole forge to start from: real apps with their history, and a crew set up to work on them.",
3742 })}
@@ -42,11 +47,12 @@ export function packsPage(ctx: Ctx): Response {
4247 }
4348
4449 // GET /packs/:name
45 export function packPage(ctx: Ctx): Response {
50 export async function packPage(ctx: Ctx): Promise<Response> {
4651 notePhase(ctx.request, "session", takeSessionTime(ctx.request));
4752 const started = performance.now();
48 const pack = packNamed(ctx.params.name ?? "");
49 if (pack === undefined) return notFoundPage(ctx);
53 const pack = packNamed(ctx.app.db, ctx.params.name ?? "");
54 // `packs` is also the owner of the mirrored apps (`packs/2048`): a name no pack carries is that repository's page.
55 if (pack === undefined) return getRepo(ctx.app.db, PACKS_OWNER, ctx.params.name ?? "") === undefined ? notFoundPage(ctx) : repoHome({ ...ctx, params: { ...ctx.params, owner: PACKS_OWNER } });
5056 const commands = deployCommands(pack, ctx.app.config.url);
5157 const body = html`${pageHead(`${pack.title} pack`, { sub: pack.description })}
5258 <section class="stack">
@@ -60,7 +66,7 @@ export function packPage(ctx: Ctx): Response {
6066 <h2>What it brings</h2>
6167 <ul class="list">${pack.apps.map((a) =>
6268 row({
63 title: html`<a href="${a.source}" rel="noopener noreferrer">${a.name}</a>`,
69 title: html`<a href="/${a.source}">${a.name}</a>`,
6470 meta: html`${a.description}`,
6571 }),
6672 )}</ul>
src/web/studio/home.ts +4 −3
@@ -17,6 +17,7 @@ import {
1717 type FrontPage,
1818 type FrontPicksJson,
1919 } from "../../actions/front.ts";
20 import type { Database } from "bun:sqlite";
2021 import { catalog } from "../../packs/catalog.ts";
2122 import { appsRuntime, type EnvState } from "../../apps/runtime.ts";
2223 import { signupClosed } from "../../actions/users.ts";
@@ -691,8 +692,8 @@ const FORGE_REPO = "https://forge.open-platform.sh/trevato/forge";
691692 * later "host it for me" action sits beside "See what it brings"; it is empty
692693 * today, so there is no dead button.
693694 */
694 function packPresses(): Html | null {
695 const { packs } = catalog();
695 function packPresses(db: Database): Html | null {
696 const { packs } = catalog(db);
696697 if (packs.length === 0) return null;
697698 const count = (n: number, one: string): string => `${n} ${one}${n === 1 ? "" : "s"}`;
698699 return html`<div class="door-packs" aria-label="Ready-made forges">
@@ -879,7 +880,7 @@ bun run src/main.ts</code></pre>
879880 <p class="muted small">Or visit this forge from your terminal: your key is your identity.</p>
880881 </div>`,
881882 )}
882 ${packPresses()}
883 ${packPresses(ctx.app.db)}
883884 </div>
884885 </section>
885886 <p class="door-org">
test/child-quota.test.ts +1 −1
@@ -101,7 +101,7 @@ describe("child quota", () => {
101101
102102 test("room for small but not large is refused only for large", async () => {
103103 clear();
104 process.env["FORGE_CHILD_MEMORY_MIB"] = "2000";
104 process.env["FORGE_CHILD_MEMORY_MIB"] = "4500";
105105 expect(quotaRefusal(dir, "small")).toBeUndefined();
106106 expect(quotaRefusal(dir, "large")).toContain("memory limit");
107107 });
test/dockerfile-copies.test.ts +4 −4
@@ -37,9 +37,9 @@ describe("what the Dockerfile copies reaches the build", () => {
3737 expect(ignored(lines(".dockerignore"), path)).toBe(false);
3838 });
3939
40 // /packs rendered blank in production: the catalog reads packs/ beside src/, and the
41 // image carried only src/. Each folder a page reads from outside src/ is copied.
42 test("the packs /packs lists are in the image", () => {
43 expect(copied).toContain("packs");
40 // Packs are repositories on the forge, marked and read from the database; none ships in the source tree or the image.
41 test("no packs/ folder is copied into the image or kept in the tree", () => {
42 expect(copied).not.toContain("packs");
43 expect(existsSync(join(ROOT, "packs"))).toBe(false);
4444 });
4545 });
packs/business/pack.json → test/fixtures/packs/business.json renamed +2 −2
@@ -5,8 +5,8 @@
55 "version": "1.0.0",
66 "description": "The everyday tools of a small business: notes and a task board, two open-source apps with their history, and a crew that plans, builds and reviews changes you ask for.",
77 "apps": [
8 { "name": "memos", "description": "Quick notes and a shared journal for a team.", "source": "https://github.com/usememos/memos" },
9 { "name": "kanboard", "description": "A simple board for tasks and projects.", "source": "https://github.com/kanboard/kanboard" }
8 { "name": "memos", "description": "Quick notes and a shared journal for a team.", "source": "packs/memos" },
9 { "name": "kanboard", "description": "A simple board for tasks and projects.", "source": "packs/kanboard" }
1010 ],
1111 "crew": [
1212 { "username": "office-planner", "roles": ["planner"], "brief": "You turn what the business asks for into small, clear steps. Prefer the change that saves someone time this week." },
packs/games/pack.json → test/fixtures/packs/games.json renamed +3 −3
@@ -5,9 +5,9 @@
55 "version": "1.0.0",
66 "description": "A small arcade you can play today and change tomorrow: three well-loved open-source games, each with its history, and a crew that keeps them tidy.",
77 "apps": [
8 { "name": "2048", "description": "Slide tiles and merge them to reach 2048.", "source": "https://github.com/gabrielecirulli/2048" },
9 { "name": "hextris", "description": "A fast puzzle on a spinning hexagon.", "source": "https://github.com/Hextris/hextris" },
10 { "name": "react-tetris", "description": "Tetris in the browser, with sound.", "source": "https://github.com/chvin/react-tetris" }
8 { "name": "2048", "description": "Slide tiles and merge them to reach 2048.", "source": "packs/2048" },
9 { "name": "hextris", "description": "A fast puzzle on a spinning hexagon.", "source": "packs/hextris" },
10 { "name": "react-tetris", "description": "Tetris in the browser, with sound.", "source": "packs/react-tetris" }
1111 ],
1212 "crew": [
1313 { "username": "arcade-builder", "roles": ["builder"], "brief": "You build and fix small browser games. Keep them fast, playable on a phone and fun. Change one thing at a time." },
test/helpers/packs.ts added +56
@@ -0,0 +1,56 @@
1 /** The two packs the forge once shipped, kept as fixtures: a pack is a repository now, and a test marks one the way the owner does. */
2 import { readFileSync } from "node:fs";
3 import { join } from "node:path";
4 import { parsePack, type Pack } from "../../src/packs/pack.ts";
5
6 export const fixtureText = (name: string): string => readFileSync(join(import.meta.dir, "..", "fixtures", "packs", `${name}.json`), "utf8");
7
8 export const fixturePack = (name: string): Pack => {
9 const parsed = parsePack(fixtureText(name));
10 if (!parsed.ok) throw new Error(parsed.error);
11 return parsed.value;
12 };
13
14 export const GAMES = fixturePack("games");
15 export const BUSINESS = fixturePack("business");
16 export const FIXTURE_PACKS: readonly Pack[] = [GAMES, BUSINESS];
17
18 import type { App } from "../../src/app.ts";
19 import { createRepo, getRepo, type Repo } from "../../src/core/repos.ts";
20 import { repoDir } from "../../src/git/repos.ts";
21 import { markPack } from "../../src/packs/marked.ts";
22
23 const GIT = {
24 PATH: process.env.PATH ?? "/usr/bin:/bin",
25 GIT_CONFIG_GLOBAL: "/dev/null",
26 GIT_CONFIG_SYSTEM: "/dev/null",
27 GIT_AUTHOR_NAME: "t",
28 GIT_AUTHOR_EMAIL: "[email protected]",
29 GIT_COMMITTER_NAME: "t",
30 GIT_COMMITTER_EMAIL: "[email protected]",
31 };
32 const git = (cwd: string, args: string[], input?: string): string => {
33 const r = Bun.spawnSync(["git", ...args], { cwd, env: GIT, stdin: input === undefined ? undefined : new TextEncoder().encode(input) });
34 if (r.exitCode !== 0) throw new Error(`git ${args.join(" ")}: ${r.stderr.toString()}`);
35 return r.stdout.toString().trim();
36 };
37
38 /** Commit `text` as `pack.json` on the repository's default branch, with plumbing: no working tree. */
39 export function commitPackJson(app: App, repo: Repo, text: string): void {
40 const dir = repoDir(app.reposRoot, repo.ownerName, repo.name);
41 const blob = git(dir, ["hash-object", "-w", "--stdin"], text);
42 const tree = git(dir, ["mktree"], `100644 blob ${blob}\tpack.json\n`);
43 const parent = Bun.spawnSync(["git", "rev-parse", "-q", "--verify", `refs/heads/${repo.defaultBranch}`], { cwd: dir, env: GIT });
44 const commit = git(dir, ["commit-tree", tree, ...(parent.exitCode === 0 ? ["-p", parent.stdout.toString().trim()] : []), "-m", "pack"]);
45 git(dir, ["update-ref", `refs/heads/${repo.defaultBranch}`, commit]);
46 }
47
48 /** Make a public repository named for the pack under `ownerId`, put its pack.json in, and mark it, as the owner would. */
49 export async function publishPack(app: App, ownerId: string, markedBy: string, name: string, text = fixtureText(name)): Promise<Repo> {
50 const made = await createRepo(app.db, app.reposRoot, { ownerId, name, visibility: "public" });
51 if (!made.ok) throw new Error(made.error);
52 commitPackJson(app, made.value, text);
53 const marked = await markPack(app, made.value, markedBy);
54 if (!marked.ok) throw new Error(JSON.stringify(marked.error));
55 return getRepo(app.db, made.value.ownerName, name)!;
56 }
test/keys-packs.test.ts +7 −2
@@ -7,6 +7,8 @@ import { createApp, type App } from "../src/app.ts";
77 import { catalog } from "../src/packs/catalog.ts";
88 import { packsList } from "../src/web/pages/packs.ts";
99 import { serve, type Servers } from "../src/http/server.ts";
10 import { createUser } from "../src/core/users.ts";
11 import { publishPack } from "./helpers/packs.ts";
1012
1113 let dataDir: string;
1214 let app: App;
@@ -16,6 +18,9 @@ beforeAll(async () => {
1618 dataDir = await mkdtemp(join(tmpdir(), "forge-keys-packs-"));
1719 app = createApp({ dataDir, port: 0 });
1820 servers = await serve(app);
21 const u = await createUser(app.db, { origin: "seed", trusted: true, username: "keys", email: "[email protected]", password: "a-good-passphrase", adminIfFirst: true });
22 if (!u.ok) throw new Error(u.error);
23 for (const name of ["games", "business"]) await publishPack(app, u.value.id, u.value.id, name);
1924 });
2025
2126 afterAll(async () => {
@@ -27,7 +32,7 @@ afterAll(async () => {
2732 const navRows = (text: string) => text.split("<li").filter((li) => /class="row"[^>]*data-nav tabindex="-1"/.test(li));
2833
2934 test("each pack on /packs is one data-nav row whose first link opens the pack", async () => {
30 const { packs } = catalog();
35 const { packs } = catalog(app.db);
3136 expect(packs.length).toBeGreaterThan(0);
3237 const rows = navRows(await (await fetch(`${servers.url}/packs`)).text());
3338 expect(rows.length).toBe(packs.length);
@@ -37,7 +42,7 @@ test("each pack on /packs is one data-nav row whose first link opens the pack",
3742 });
3843
3944 test("a pack's own page keeps its rows out of the traversal", async () => {
40 const name = catalog().packs[0]!.name;
45 const name = catalog(app.db).packs[0]!.name;
4146 const text = await (await fetch(`${servers.url}/packs/${name}`)).text();
4247 expect(navRows(text).length).toBe(0);
4348 });
test/packs-forge.test.ts added +158
@@ -0,0 +1,158 @@
1 /** Packs come from the forge: the owner marks a repository, /packs lists it, and a deploy forks from the forge's mirror with GitHub refused. */
2 import { afterAll, beforeAll, describe, expect, test } from "bun:test";
3 import { mkdtemp, rm } from "node:fs/promises";
4 import { tmpdir } from "node:os";
5 import { join } from "node:path";
6 import { createApp, type App } from "../src/app.ts";
7 import { createRepo, getRepo } from "../src/core/repos.ts";
8 import { createToken } from "../src/core/tokens.ts";
9 import { createUser } from "../src/core/users.ts";
10 import { serve, type Servers } from "../src/http/server.ts";
11 import { catalog } from "../src/packs/catalog.ts";
12 import { deployPack } from "../src/packs/deploy.ts";
13 import { markPack, refreshMarked } from "../src/packs/marked.ts";
14 import { parsePack } from "../src/packs/pack.ts";
15 import { realHands } from "../scripts/pack.ts";
16 import { commitPackJson, fixtureText } from "./helpers/packs.ts";
17 import { quietLogs } from "./helpers/globals.ts";
18
19 quietLogs();
20
21 let dir: string;
22 let app: App;
23 let servers: Servers;
24 let ownerId: string;
25 let ownerToken: string;
26 let annId: string;
27 let annToken: string;
28
29 const call = (token: string | undefined, method: string, path: string) =>
30 fetch(`${servers.url}${path}`, { method, headers: token === undefined ? {} : { authorization: `Bearer ${token}` } });
31
32 const repoNamed = async (owner: string, name: string, visibility: "public" | "private" = "public") => {
33 const made = await createRepo(app.db, app.reposRoot, { ownerId: owner === "packs" ? ownerId : annId, name, visibility });
34 if (!made.ok) throw new Error(made.error);
35 return made.value;
36 };
37
38 beforeAll(async () => {
39 dir = await mkdtemp(join(tmpdir(), "packs-forge-"));
40 app = createApp({ dataDir: dir, port: 0 });
41 servers = await serve(app);
42 const owner = await createUser(app.db, { origin: "seed", trusted: true, allowReserved: true, username: "packs", email: "[email protected]", password: "a-good-passphrase", adminIfFirst: true });
43 const ann = await createUser(app.db, { origin: "seed", trusted: true, username: "ann", email: "[email protected]", password: "a-good-passphrase" });
44 if (!owner.ok || !ann.ok) throw new Error("users");
45 ownerId = owner.value.id;
46 annId = ann.value.id;
47 // Past the new-account cooldown, which is the forge's own rule for forks.
48 app.db.query("UPDATE user SET created_at = ?1 WHERE id = ?2").run(new Date(Date.now() - 2 * 86_400_000).toISOString(), annId);
49 ownerToken = createToken(app.db, ownerId, "t", "admin").token;
50 annToken = createToken(app.db, annId, "t", "admin").token;
51 });
52 afterAll(async () => {
53 await servers.stop();
54 app.close();
55 await rm(dir, { recursive: true, force: true });
56 });
57
58 describe("marking a pack repository", () => {
59 test("nothing is listed until the owner marks a repository", async () => {
60 const repo = await repoNamed("packs", "games");
61 commitPackJson(app, repo, fixtureText("games"));
62 expect(catalog(app.db).packs).toEqual([]);
63 expect(((await (await call(undefined, "GET", "/api/v1/packs")).json()) as { items: unknown[] }).items).toEqual([]);
64 });
65
66 test("only a forge admin marks; anyone else is told it is not there", async () => {
67 expect((await call(annToken, "PUT", "/api/v1/repos/packs/games/pack")).status).toBe(404);
68 expect((await call(undefined, "PUT", "/api/v1/repos/packs/games/pack")).status).toBe(404);
69 expect(catalog(app.db).packs).toEqual([]);
70 });
71
72 test("the owner marks it: /packs and the API list it, and its apps are forge repositories", async () => {
73 expect((await call(ownerToken, "PUT", "/api/v1/repos/packs/games/pack")).status).toBe(200);
74 expect(catalog(app.db).packs.map((p) => p.name)).toEqual(["games"]);
75 const one = (await (await call(undefined, "GET", "/api/v1/packs/games")).json()) as { repo: string; apps: { source: string }[] };
76 expect(one.repo).toBe("packs/games");
77 expect(one.apps.map((a) => a.source)).toEqual(["packs/2048", "packs/hextris", "packs/react-tetris"]);
78 expect(await (await call(undefined, "GET", "/packs")).text()).toContain("/packs/games");
79 expect((await call(undefined, "GET", "/api/v1/packs/nothing")).status).toBe(404);
80 });
81
82 test("a repository that is no pack, is private, or is named otherwise cannot be marked", async () => {
83 const empty = await repoNamed("packs", "empty");
84 expect(await markPack(app, empty, ownerId)).toEqual({ ok: false, error: "no_pack_json" });
85 const hidden = await repoNamed("packs", "business", "private");
86 commitPackJson(app, hidden, fixtureText("business"));
87 expect(await markPack(app, hidden, ownerId)).toEqual({ ok: false, error: "private" });
88 const wrong = await repoNamed("packs", "other");
89 commitPackJson(app, wrong, fixtureText("business"));
90 expect(await markPack(app, wrong, ownerId)).toEqual({ ok: false, error: "name_mismatch" });
91 const bad = await repoNamed("ann", "broken");
92 commitPackJson(app, bad, '{"pack":1,"name":"broken"}');
93 const r = await markPack(app, bad, annId);
94 expect(r.ok).toBe(false);
95 expect((await call(ownerToken, "PUT", "/api/v1/repos/packs/empty/pack")).status).toBe(422);
96 });
97
98 test("a pack edited in its repository follows on the next refresh; a broken edit keeps the last good text", async () => {
99 const repo = getRepo(app.db, "packs", "games")!;
100 commitPackJson(app, repo, fixtureText("games").replace("A small arcade", "A bigger arcade"));
101 expect(await refreshMarked(app)).toBe(1);
102 expect(catalog(app.db).packs[0]!.description).toContain("A bigger arcade");
103 commitPackJson(app, repo, "not json");
104 expect(await refreshMarked(app)).toBe(0);
105 expect(catalog(app.db).packs[0]!.description).toContain("A bigger arcade");
106 });
107
108 test("taking it off removes it from the list", async () => {
109 expect((await call(annToken, "DELETE", "/api/v1/repos/packs/games/pack")).status).toBe(404);
110 expect((await call(ownerToken, "DELETE", "/api/v1/repos/packs/games/pack")).status).toBe(200);
111 expect(catalog(app.db).packs).toEqual([]);
112 });
113 });
114
115 describe("deploying a pack", () => {
116 test("forks each app from its forge mirror into the person's account, lineage kept, and never reaches GitHub", async () => {
117 const mirror = await repoNamed("packs", "2048");
118 commitPackJson(app, mirror, "{}");
119 const seen: string[] = [];
120 const real = globalThis.fetch;
121 // GitHub is refused: any request to it is recorded and fails.
122 globalThis.fetch = ((input: Request | string | URL, init?: RequestInit) => {
123 const host = new URL(typeof input === "string" ? input : input instanceof URL ? input.href : input.url).host;
124 seen.push(host);
125 return /github\.com$/.test(host) ? Promise.reject(new Error("network to github.com is refused")) : real(input as never, init);
126 }) as typeof fetch;
127 try {
128 const parsed = parsePack(
129 JSON.stringify({ pack: 1, name: "mini", title: "Mini", version: "1.0.0", description: "One game.", apps: [{ name: "2048", description: "Tiles.", source: "packs/2048" }] }),
130 );
131 if (!parsed.ok) throw new Error(parsed.error);
132 const hands = realHands(servers.url, annToken, { cloneFrom: () => "https://github.com/gabrielecirulli/2048.git" });
133 const steps = await deployPack(parsed.value, hands);
134 expect(steps.map((s) => `${s.state} ${s.what} ${s.note}`)).toEqual(["done app 2048 forked from packs/2048, upstream credit kept"]);
135 const fork = getRepo(app.db, "ann", "2048")!;
136 expect(fork.forkParentId).toBe(mirror.id);
137 const again = await deployPack(parsed.value, hands);
138 expect(again.map((s) => s.state)).toEqual(["kept"]);
139 } finally {
140 globalThis.fetch = real;
141 }
142 expect(seen.length).toBeGreaterThan(0);
143 expect(seen.filter((h) => /github/.test(h))).toEqual([]);
144 expect(new Set(seen)).toEqual(new Set([new URL(servers.url).host]));
145 });
146 });
147
148 describe("the packs owner is also a path", () => {
149 test("/packs/<name> is the pack's page when one is listed, and the repository's page when none is", async () => {
150 commitPackJson(app, getRepo(app.db, "packs", "games")!, fixtureText("games"));
151 expect((await call(ownerToken, "PUT", "/api/v1/repos/packs/games/pack")).status).toBe(200);
152 expect(await (await call(undefined, "GET", "/packs/games")).text()).toContain("Deploy it on your own machine");
153 const mirror = await call(undefined, "GET", "/packs/2048");
154 expect(mirror.status).toBe(200);
155 expect(await mirror.text()).not.toContain("Deploy it on your own machine");
156 expect((await call(undefined, "GET", "/packs/nothing")).status).toBe(404);
157 });
158 });
test/packs-hands.test.ts +9 −5
@@ -6,12 +6,12 @@ import { join } from "node:path";
66 import { createApp, type App } from "../src/app.ts";
77 import { createUser } from "../src/core/users.ts";
88 import { serve } from "../src/http/server.ts";
9 import { packNamed } from "../src/packs/catalog.ts";
109 import { deployCommands } from "../src/packs/words.ts";
1110 import { createForgeApp, forgeCommands } from "../src/tui/forge.ts";
1211 import { packLines, packsScreen } from "../src/tui/screens/packs.ts";
1312 import { headlessTerminal } from "../src/tui/term/terminal.ts";
1413 import { cloneEnv, pushEnv, realHands } from "../scripts/pack.ts";
14 import { GAMES, publishPack } from "./helpers/packs.ts";
1515 import { quietLogs } from "./helpers/globals.ts";
1616 import { settleApp } from "./helpers/tui.ts";
1717
@@ -38,7 +38,7 @@ describe("git credentials", () => {
3838 },
3939 });
4040 try {
41 const r = await realHands("http://127.0.0.1:1", "secret-token").carryHistory(`http://127.0.0.1:${stranger.port}/x.git`, "o", "n", undefined);
41 const r = await realHands("http://127.0.0.1:1", "secret-token", { cloneFrom: () => `http://127.0.0.1:${stranger.port}/x.git` }).carryHistory("packs/x", "o", "n", undefined);
4242 expect(r.ok).toBe(false);
4343 expect(seen.length).toBeGreaterThan(0);
4444 expect(seen.every((h) => h === null)).toBe(true);
@@ -66,7 +66,7 @@ describe("makeAgent", () => {
6666 });
6767
6868 describe("deploy words", () => {
69 const games = packNamed("games")!;
69 const games = GAMES;
7070 test("the clone source is the configured address, never a request's Host", () => {
7171 expect(deployCommands(games, "https://f.example/")[0]).toContain("git clone https://f.example/trevato/forge.git");
7272 expect(deployCommands(games, undefined)[0]).not.toContain("http");
@@ -76,6 +76,9 @@ describe("deploy words", () => {
7676 const a = createApp({ dataDir: dir, port: 0 });
7777 const servers = await serve(a);
7878 try {
79 const u = await createUser(a.db, { origin: "seed", trusted: true, username: "dora", email: "[email protected]", password: "a-good-passphrase", adminIfFirst: true });
80 if (!u.ok) throw new Error(u.error);
81 await publishPack(a, u.value.id, u.value.id, "games");
7982 const res = await fetch(`${servers.url}/packs/games`, { headers: { host: "evil.example" } });
8083 expect(res.status).toBe(200);
8184 expect(await res.text()).not.toContain("git clone http://evil.example");
@@ -95,6 +98,7 @@ describe("terminal screen", () => {
9598 app = createApp({ dataDir: dir, port: 0 });
9699 const r = await createUser(app.db, { origin: "seed", trusted: true, username: "carol", email: "[email protected]", password: "a-good-passphrase", adminIfFirst: true });
97100 if (!r.ok) throw new Error(r.error);
101 for (const name of ["games", "business"]) await publishPack(app, r.value.id, r.value.id, name);
98102 });
99103 afterAll(async () => {
100104 app.close();
@@ -110,13 +114,13 @@ describe("terminal screen", () => {
110114 test("it lists the packs, opens one, and says the deploy command and what happens to a pull", async () => {
111115 const { app: tui } = open();
112116 tui.start();
113 tui.ctx.push(packsScreen(tui.ctx, undefined));
117 tui.ctx.push(packsScreen(tui.ctx, undefined, app.db));
114118 await settleApp(tui);
115119 const list = tui.screen.toText();
116120 expect(list).toContain("Games");
117121 expect(list).toContain("Business");
118122 tui.ctx.pop();
119 const lines = packLines(packNamed("games")!, undefined).join("\n");
123 const lines = packLines(GAMES, undefined).join("\n");
120124 expect(lines).toContain("scripts/pack.ts deploy games");
121125 expect(lines).toContain("scripts/pack.ts up games");
122126 expect(lines).toContain("waits for a person");
test/packs-up.test.ts +12 −12
@@ -9,7 +9,6 @@ import { getAgentByName } from "../src/core/agents.ts";
99 import { getRepo } from "../src/core/repos.ts";
1010 import { getFollow } from "../src/core/upstream_follow.ts";
1111 import { serve, type Servers } from "../src/http/server.ts";
12 import { catalog } from "../src/packs/catalog.ts";
1312 import { deployPack } from "../src/packs/deploy.ts";
1413 import { addSubscription } from "../src/core/subscriptions.ts";
1514 import { checkUpstream } from "../src/actions/upstream_follow.ts";
@@ -19,7 +18,8 @@ import { provideMirrorOutboundForTest } from "../src/origins/git-mirror.ts";
1918 import { finishWords, OWNER, prepareOwner } from "../src/packs/up.ts";
2019 import { deployCommands } from "../src/packs/words.ts";
2120 import { repoDir } from "../src/git/repos.ts";
22 import { realHands, up } from "../scripts/pack.ts";
21 import { packsForge, realHands, up } from "../scripts/pack.ts";
22 import { FIXTURE_PACKS } from "./helpers/packs.ts";
2323 import { quietLogs } from "./helpers/globals.ts";
2424
2525 quietLogs();
@@ -55,7 +55,7 @@ beforeAll(async () => {
5555 passphrase = prepared.passphrase;
5656 // A fake source for every app of both packs: three commits and a tag, on a branch that is not "main" for one of them.
5757 let n = 0;
58 for (const pack of catalog().packs)
58 for (const pack of FIXTURE_PACKS)
5959 for (const a of pack.apps) {
6060 const dir = join(root, `src-${n++}`);
6161 git(root, "init", "-q", "-b", n === 1 ? "master" : "main", dir);
@@ -78,7 +78,7 @@ afterAll(async () => {
7878
7979 describe("the one command, against a fake source", () => {
8080 test("the command is one line to run after clone and install, and asks for no token", () => {
81 for (const p of catalog().packs) {
81 for (const p of FIXTURE_PACKS) {
8282 const lines = deployCommands(p, "https://f.example");
8383 expect(lines.some((l) => l.startsWith(`bun scripts/pack.ts up ${p.name}`))).toBe(true);
8484 expect(lines[0]).toContain("bun install");
@@ -103,7 +103,7 @@ describe("the one command, against a fake source", () => {
103103
104104 for (const name of ["games", "business"]) {
105105 test(`${name}: every app arrives with history, followed; the crew is configured; no secret or variable travels`, async () => {
106 const pack = catalog().packs.find((p) => p.name === name)!;
106 const pack = FIXTURE_PACKS.find((p) => p.name === name)!;
107107 const steps = await deployPack(pack, realHands(servers.url, token, { cloneFrom: (s) => fake.get(s)! }));
108108 expect(steps.filter((s) => s.state !== "done")).toEqual([]);
109109 for (const a of pack.apps) {
@@ -117,7 +117,7 @@ describe("the one command, against a fake source", () => {
117117 expect(git(dir, "rev-parse", "refs/tags/v1")).toBe(git(src, "rev-parse", "v1"));
118118 const follow = getFollow(app.db, repo.id)!;
119119 expect(follow.enabled).toBe(true);
120 expect(follow.url).toBe(a.source);
120 expect(follow.url).toBe(`${packsForge()}/${a.source}`);
121121 expect(follow.branch).toBe(git(src, "symbolic-ref", "--short", "HEAD"));
122122 expect((await fetch(`${servers.url}/${OWNER}/${a.name}`)).status).toBe(200);
123123 }
@@ -130,7 +130,7 @@ describe("the one command, against a fake source", () => {
130130 }
131131
132132 test("running it again keeps what exists", async () => {
133 const pack = catalog().packs[0]!;
133 const pack = FIXTURE_PACKS[0]!;
134134 const steps = await deployPack(pack, realHands(servers.url, token, { cloneFrom: (s) => fake.get(s)! }));
135135 expect(steps.map((s) => `${s.what} ${s.state} ${s.note}`)).toEqual(steps.map((s) => `${s.what} kept already here`));
136136 });
@@ -159,7 +159,7 @@ describe("the one command, against a fake source", () => {
159159 });
160160
161161 test("a partial deploy is finished by running it again: a kept app that was never followed is followed", async () => {
162 const pack = catalog().packs[0]!;
162 const pack = FIXTURE_PACKS[0]!;
163163 const hands = realHands(servers.url, token, { cloneFrom: (s) => fake.get(s)! });
164164 // First run: the follow of the first app fails after the app and its history are in.
165165 const one = { ...pack, apps: [{ ...pack.apps[0]!, name: "partial-app" }], crew: [] };
@@ -173,18 +173,18 @@ describe("the one command, against a fake source", () => {
173173 expect(steps2[0]!.state).toBe("kept");
174174 const f = getFollow(app.db, repo.id)!;
175175 expect(f.enabled).toBe(true);
176 expect(f.url).toBe(one.apps[0]!.source);
176 expect(f.url).toBe(`${packsForge()}/${one.apps[0]!.source}`);
177177 expect(f.branch).toBe(repo.defaultBranch);
178178 });
179179
180180 test("no branch to follow fails the step, and never follows main by guess", async () => {
181181 const hands = realHands(servers.url, token);
182 const r = await hands.follow(OWNER, "whatever", "https://github.com/a/b", undefined);
182 const r = await hands.follow(OWNER, "whatever", "packs/b", undefined);
183183 expect(r.ok).toBe(false);
184184 });
185185
186186 test("a new upstream commit shows as drift at the next check", async () => {
187 const pack = catalog().packs[0]!;
187 const pack = FIXTURE_PACKS[0]!;
188188 const a = pack.apps[0]!;
189189 const repo = getRepo(app.db, OWNER, a.name)!;
190190 const src = fake.get(a.source)!;
@@ -231,7 +231,7 @@ describe("up() itself", () => {
231231 const env: Record<string, string | undefined> = { ...process.env, FORGE_SSH_PORT: "0" };
232232 for (const n of ["ANTHROPIC_API_KEY", "CLAUDE_CODE_OAUTH_TOKEN", "FORGE_CLAUDE_CONFIG_DIR"]) delete env[n];
233233 try {
234 const pack = catalog().packs[0]!;
234 const pack = FIXTURE_PACKS[0]!;
235235 const running = up(pack, { port, dataDir: d, cloneFrom: (s) => fake.get(s)!, say: (l) => lines.push(l), signal: stop.signal, env });
236236 let ended: number | undefined;
237237 void running.then((c) => (ended = c));
test/packs.test.ts +69 −17
@@ -5,7 +5,8 @@ import { tmpdir } from "node:os";
55 import { join } from "node:path";
66 import { createApp, type App } from "../src/app.ts";
77 import { serve, type Servers } from "../src/http/server.ts";
8 import { catalog } from "../src/packs/catalog.ts";
8 import { createUser } from "../src/core/users.ts";
9 import { publishPack } from "./helpers/packs.ts";
910 import { deployPack, type Hands } from "../src/packs/deploy.ts";
1011 import { parsePack, sourceOf } from "../src/packs/pack.ts";
1112
@@ -15,7 +16,7 @@ const good = {
1516 title: "Mini",
1617 version: "1.0.0",
1718 description: "One app.",
18 apps: [{ name: "one", description: "An app.", source: "https://github.com/a/b" }],
19 apps: [{ name: "one", description: "An app.", source: "packs/b" }],
1920 crew: [{ username: "helper", roles: ["builder"], brief: "Help." }],
2021 };
2122 const parse = (o: unknown) => parsePack(JSON.stringify(o));
@@ -32,10 +33,11 @@ describe("pack definition", () => {
3233 if (!r.ok) expect(r.error).toContain("not allowed");
3334 }
3435 });
35 test("sources are public https repositories without credentials", () => {
36 expect(sourceOf("https://github.com/a/b.git")).toBe("https://github.com/a/b");
37 for (const bad of ["http://github.com/a/b", "https://u:p@github.com/a/b", "[email protected]:a/b", "https://github.com/a", "https://github.com/a/b?x=1", "file:///etc"])
36 test("a source is a forge repository, owner/name, and never a URL", () => {
37 expect(sourceOf("packs/2048")).toBe("packs/2048");
38 for (const bad of ["https://github.com/a/b", "http://github.com/a/b", "[email protected]:a/b", "packs", "a/b/c", "../etc", "file:///etc", "packs/ 2048", ""])
3839 expect(sourceOf(bad)).toBeUndefined();
40 expect(parse({ ...good, apps: [{ ...good.apps[0], source: "https://github.com/a/b" }] }).ok).toBe(false);
3941 });
4042 test("each field is checked", () => {
4143 expect(parse({ ...good, pack: 2 }).ok).toBe(false);
@@ -57,6 +59,9 @@ describe("pack absorb", () => {
5759 for (const v of ["yes", 1, {}, null]) expect(withAbsorb(v).ok).toBe(false);
5860 });
5961 const hands = (calls: string[], absorbed: { ok: boolean }): Hands => ({
62 async forkApp() {
63 return { ok: false, here: false, exists: false, why: "not here" };
64 },
6065 async makeApp(name) {
6166 return name === "kept" ? { ok: false, exists: true, why: "x", owner: "me" } : { ok: true, owner: "me" };
6267 },
@@ -78,9 +83,9 @@ describe("pack absorb", () => {
7883 },
7984 });
8085 const apps = [
81 { name: "a", description: "d", source: "https://github.com/a/b", absorb: true },
82 { name: "plain", description: "d", source: "https://github.com/a/c" },
83 { name: "kept", description: "d", source: "https://github.com/a/d", absorb: true },
86 { name: "a", description: "d", source: "packs/b", absorb: true },
87 { name: "plain", description: "d", source: "packs/c" },
88 { name: "kept", description: "d", source: "packs/d", absorb: true },
8489 ];
8590 test("deploy absorbs exactly the apps that ask, made or kept", async () => {
8691 const calls: string[] = [];
@@ -99,12 +104,52 @@ describe("pack absorb", () => {
99104 });
100105 });
101106
102 describe("shipped packs", () => {
103 test("Games and Business ship and parse", () => {
104 const c = catalog();
105 expect(c.problems).toEqual([]);
106 expect(c.packs.map((p) => p.name)).toEqual(["business", "games"]);
107 for (const p of c.packs) expect(p.apps.length).toBeGreaterThan(0);
107 describe("a deploy forks from the forge", () => {
108 const forking = (calls: string[], here: boolean): Hands => ({
109 async forkApp(source, name) {
110 calls.push(`fork ${source} as ${name}`);
111 return here ? { ok: true, owner: "me" } : { ok: false, here: false, exists: false, why: "not here" };
112 },
113 async makeApp(name) {
114 calls.push(`make ${name}`);
115 return { ok: true, owner: "me" };
116 },
117 async defaultBranch() {
118 return "main";
119 },
120 async carryHistory(source) {
121 calls.push(`carry ${source}`);
122 return { ok: true, branch: "main" };
123 },
124 async follow() {
125 return { ok: true };
126 },
127 async absorb() {
128 return { ok: true };
129 },
130 async makeAgent() {
131 return { ok: true };
132 },
133 });
134 const pack = (() => {
135 const p = parse(good);
136 if (!p.ok) throw new Error(p.error);
137 return { ...p.value, crew: [] };
138 })();
139 test("a source this forge holds is forked, and nothing else is made or carried", async () => {
140 const calls: string[] = [];
141 const steps = await deployPack(pack, forking(calls, true));
142 expect(calls).toEqual(["fork packs/b as one"]);
143 expect(steps).toEqual([{ what: "app one", state: "done", note: "forked from packs/b, upstream credit kept" }]);
144 });
145 test("a forge without the source carries its history from the packs forge", async () => {
146 const calls: string[] = [];
147 await deployPack(pack, forking(calls, false));
148 expect(calls).toEqual(["fork packs/b as one", "make one", "carry packs/b"]);
149 });
150 test("an app already forked is kept", async () => {
151 const hands = { ...forking([], true), forkApp: async () => ({ ok: false as const, here: true, exists: true, why: "exists" }) };
152 expect((await deployPack(pack, hands)).map((s) => s.state)).toEqual(["kept"]);
108153 });
109154 });
110155
@@ -112,6 +157,9 @@ describe("deploy", () => {
112157 test("makes apps, carries history, adds crew, keeps what exists and reports failures", async () => {
113158 const calls: string[] = [];
114159 const hands: Hands = {
160 async forkApp() {
161 return { ok: false, here: false, exists: false, why: "not here" };
162 },
115163 async makeApp(name) {
116164 calls.push(`app ${name}`);
117165 return name === "one" ? { ok: true, owner: "me" } : { ok: false, exists: true, why: "x" };
@@ -135,10 +183,10 @@ describe("deploy", () => {
135183 return { ok: false, exists: false, why: "no" };
136184 },
137185 };
138 const pack = parse({ ...good, apps: [...good.apps, { name: "two", description: "d", source: "https://github.com/c/d" }] });
186 const pack = parse({ ...good, apps: [...good.apps, { name: "two", description: "d", source: "packs/d" }] });
139187 if (!pack.ok) throw new Error(pack.error);
140188 const steps = await deployPack(pack.value, hands);
141 expect(calls).toEqual(["app one", "history https://github.com/a/b -> me/one", "follow me/one https://github.com/a/b undefined", "app two", "agent helper"]);
189 expect(calls).toEqual(["app one", "history packs/b -> me/one", "follow me/one packs/b undefined", "app two", "agent helper"]);
142190 expect(steps.map((s) => s.state)).toEqual(["done", "kept", "failed"]);
143191 });
144192 });
@@ -151,6 +199,9 @@ describe("pages", () => {
151199 dataDir = await mkdtemp(join(tmpdir(), "forge-packs-"));
152200 app = createApp({ dataDir, port: 0 });
153201 servers = await serve(app);
202 const owner = await createUser(app.db, { origin: "seed", trusted: true, allowReserved: true, username: "packs", email: "[email protected]", password: "a-good-passphrase", adminIfFirst: true });
203 if (!owner.ok) throw new Error(owner.error);
204 for (const name of ["games", "business"]) await publishPack(app, owner.value.id, owner.value.id, name);
154205 });
155206 afterAll(async () => {
156207 await servers.stop();
@@ -169,7 +220,8 @@ describe("pages", () => {
169220 expect(body).toContain("scripts/pack.ts deploy games");
170221 expect(body).toContain("No data, secrets or variables come with it.");
171222 expect(body).not.toContain("Help build these apps</h2>");
172 expect(body).toContain("gabrielecirulli/2048");
223 expect(body).toContain('href="/packs/2048"');
224 expect(body).not.toContain("github.com");
173225 expect((await fetch(`${servers.url}/packs/nothing`)).status).toBe(404);
174226 });
175227 test("the home page has one press for each pack, a link in its row; following it finds the command", async () => {
test/page-head.test.ts +2
@@ -8,6 +8,7 @@ import { getRepo } from "../src/core/repos.ts";
88 import { createIssue } from "../src/core/issues.ts";
99 import { createApp as createAppRecord } from "../src/core/apps.ts";
1010 import { createUser } from "../src/core/users.ts";
11 import { publishPack } from "./helpers/packs.ts";
1112 import { serve, type Servers } from "../src/http/server.ts";
1213 import { appLine, plainDescription, repoLine } from "../src/web/describe.ts";
1314
@@ -33,6 +34,7 @@ describe("every public page names itself for a link preview", () => {
3334 adminIfFirst: true,
3435 });
3536 if (!u.ok) throw new Error(u.error);
37 await publishPack(app, u.value.id, u.value.id, "games");
3638 // No description at all, and one that is scaffold jargon.
3739 for (const name of ["bare", "widget"]) {
3840 const made = await createRepo(app, u.value, { name, visibility: "public" });
test/public-head.test.ts +4 −2
@@ -9,6 +9,7 @@ import { createIssue } from "../src/core/issues.ts";
99 import { createApp as createAppRecord } from "../src/core/apps.ts";
1010 import { createUser } from "../src/core/users.ts";
1111 import { catalog } from "../src/packs/catalog.ts";
12 import { publishPack } from "./helpers/packs.ts";
1213 import { serve, type Servers } from "../src/http/server.ts";
1314
1415 /** The content of the first tag matching, or undefined when there is none. */
@@ -90,8 +91,9 @@ describe("every public page names itself", () => {
9091 }
9192 const rec = createAppRecord(app.db, { repoId: getRepo(app.db, "alice", "widget")!.id, slug: "widget", access: "public" });
9293 if (!rec.ok) throw new Error(rec.error);
93 const pack = catalog().packs[0];
94 if (!pack) throw new Error("the catalog ships no pack to walk");
94 await publishPack(app, u.value.id, u.value.id, "games");
95 const pack = catalog(app.db).packs[0];
96 if (!pack) throw new Error("no pack is listed to walk");
9597 // Walked as a stranger. Left out, on purpose: redirects to sign in (/map,
9698 // /new, /board, /feed...), feeds and files (/changelog.atom, /sitemap.xml),
9799 // and /signin-style forms, which og-tags.test.ts already covers.
test/route-timing-coverage.test.ts +1
@@ -20,6 +20,7 @@ import { seedAdmin } from "./helpers/admin.ts";
2020 /** Routes that answer 200 without a `session` phase → why. */
2121 const UNTIMED: Record<string, string> = {
2222 "/api/v1/conversations": "a person's conversation list, one indexed query; no page to phase",
23 "/api/v1/packs": "the marked packs, one indexed query over rows the hourly refresh keeps; no page to phase",
2324 "/metrics": "Prometheus text exposition, scraped by token; not a page read",
2425 };
2526

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs