trevato/forge

Public: anyone can read this; the crew can change it.

More
Forks1g fMapg aRunsg rEventsFlows

The app template is a repository: new apps start from trevato/app-template, and scaffold.ts's strings go #11662

Merged torquecrew opened this

Check · Review · Land · Live done — Landed and live in production.

ConversationFiles
Merged

torque/forge:torque/10305-the-app-template-is-a-repository-new-app → main 13 commits fork

  • Merged. torque/10305-the-app-template-is-a-repository-new-app was merged into main as 205032b.

Review

Sign in to review this change.
27 files changed +690 −6240
Dockerfile +2
@@ -25,6 +25,8 @@ COPY docs/DESIGN.md ./docs/DESIGN.md
2525 COPY plugins/opencode ./plugins/opencode
2626 # The Claude Code command files the marketplace manifest lists (same route file).
2727 COPY plugins/claude-code/commands ./plugins/claude-code/commands
28 # The app template every forge can make apps from, taken from trevato/app-template (src/apps/template.ts).
29 COPY templates ./templates
2830 # src/ alone: nothing under it imports scripts/ or test/ (test/src-imports.test.ts),
2931 # and the boot proof below fails the build if that ever stops being so (#2652).
3032
docs/APPS.md +1 −1
@@ -992,7 +992,7 @@ generous because the first boot migrates.
992992 ## What a new app starts as
993993
994994 **New app** writes a small, complete app into the repository
995 (`src/apps/scaffold.ts`): notes behind a page, a JSON API, an MCP server
995 (`src/apps/template.ts`): notes behind a page, a JSON API, an MCP server
996996 and a terminal, all over one set of actions, with no dependencies. What
997997 the forge already does for every app (identity, the grant, preview data,
998998 health polling, the terminal, the swap, sleep, the contract's wall) is not
docs/SPAWN.md +1 −1
@@ -78,7 +78,7 @@ A spawn may also name a `size`: `small`, `medium` or `large` (`src/spawn/sizes.t
7878
7979 ## Choosing a template (#7533)
8080
81 A template is a named starting point for a new forge's apps, listed in `src/spawn/templates.ts`: each has an `id`, a `title`, one sentence `says`, and a `source` — `builtin` (its files are this forge's code) or `repo` (read from `repo`, an `owner/name` on this forge; none yet, #7536). Built in are `blank`, the default and what a spawn has always made (the forked apps and an optional `fresh` one), and `starter`, the app `/apps/new` starts from (`scaffoldFiles`, `src/apps/scaffold.ts`).
81 A template is a named starting point for a new forge's apps, listed in `src/spawn/templates.ts`: each has an `id`, a `title`, one sentence `says`, and a `source` — `builtin` (its files are this forge's code) or `repo` (read from `repo`, an `owner/name` on this forge; none yet, #7536). Built in are `blank`, the default and what a spawn has always made (the forked apps and an optional `fresh` one), and `starter`, the app `/apps/new` starts from (`scaffoldFiles`, `src/apps/template.ts`).
8282
8383 `POST /api/v1/forges` and the MCP `spawn` take an optional `template` id. An unknown id is a 422 on `template` naming the valid ids. Absent or `blank`, the plan is the one it always was; any other id is named in the plan, the ledger line, `parent.json` and the child's row in `GET /api/v1/forges` (absent there means `blank`). Because it is in the plan, a retry with another template is refused as a different child. A `starter` child begins with the template's files (#7535, `src/spawn/template_birth.ts`): one commit, no history and no upstream, in a new private repo of the child's organisation named for the `fresh` sentence (which becomes its description; no second repo) or `app`. `parent.json` records `template: {id}`. Only tracked files travel, nothing from the "never copied" list; a retry finds the repo and leaves it.
8484
docs/features/app-template.md +11
@@ -20,3 +20,14 @@ A marked repository whose top-level folders hold both `TEMPLATE.md` and `templat
2020 - **Literals:** `template.json` is `{ "files": ["README.md"], "literals": ["{{name}}", "{{description}}"] }`. Each named file must be in the folder (no `..`, no leading `/`) and each literal one of those two; anything else is `template_invalid` and nothing is made.
2121 - **Never run.** A script in a folder (`new.mjs`) is copied as a file and not executed.
2222 - The terminal's New app prompt lists the folders too (`openNewApp` loads the choices before the screen opens).
23
24 ## The default app is a repository
25
26 The app New app starts from when no template is named is `trevato/app-template` (`apps.template` in `forge_setting` names another), copied at its default branch's tip: one commit, no history (#10305). The strings that used to make it (`src/apps/scaffold.ts`) are gone.
27
28 - **Name and description.** The template's `template.json` is `{ "name": "app-template", "description": "A new app on the forge.", "files": [...] }`: the literals the forge replaces, and the only files it replaces them in (`.json` files get the JSON-escaped form). Nothing else is touched, so the template runs and passes its own tests as it stands. `template.json` is not copied. An empty description keeps the template's line. (This is the root template's form; a folder of a template repository uses the `{{name}}` form above.)
29 - **A copy in every forge.** `templates/app-template.json` is the template's files, carried in the image (`Dockerfile` copies `templates/`). A forge without the repository (a child, a pack forge, a fresh install), or whose copy has no readable `template.json`, uses it; so does `src/spawn/template_birth.ts` (`bakedFiles`). Refresh it with `bun scripts/take-template.ts <checkout>`, which reads bytes and refuses any file that is not UTF-8 text (the bundle is JSON), and marks the bundle `pinned: true` with HEAD as `commit`. A test fails when a pinned bundle's `commit` is not a 40-hex id; the file in the tree says `pinned: false` until it is taken. **Not done:** the file in the tree was generated from the removed strings, so its `commit` is the forge commit they were written at, not a commit of trevato/app-template, and the release build does not yet run `take-template.ts` at a pinned commit; until it does the copy and the repository can differ.
30 - **`apps.template` is operator-only on purpose.** It is a `forge_setting` key with no page or terminal control: which repository every new app copies is a forge-wide choice, set by whoever runs the forge, not by an app owner. A user picks another template per app with `template:` (see above).
31 - **Two `template.json` forms, pinned by a test.** The root template names its literals (`name`, `description`, `files`); a folder of a repository of templates uses `files` and `literals`. Each parser rejects the other's form. Both read paths with no dot segments, no `.git`, no absolute path, and the tip copy has the same caps as the rest (500 files, 512 KiB each, 4 MiB; over any falls back to the image's copy).
32 - **Checked both ways.** In the template's repository a pull runs its own `bun test`, then `bun scripts/birth-check.ts <checkout>` from this repository, which fills the tip as New app does, boots the result on a port the kernel chose (retried if taken) and waits on `/healthz` and `/`. Wiring that into the template repository's `forge.json` is done in that repository, not here.
33 - **Code and tests.** `src/apps/template.ts`; `test/app-template.test.ts`, `test/template-scripts.test.ts` (the two scripts). The fresh-app suites (`test/scaffold*.test.ts`, `test/apps-new.test.ts`) boot the filled copy.
scripts/birth-check.ts added +69
@@ -0,0 +1,69 @@
1 /**
2 * `bun scripts/birth-check.ts <checkout>`: the second half of a template
3 * pull's check (#10305). Births an app from the checkout's tracked files
4 * exactly as New app does (`fillTemplate`), writes it to a scratch
5 * directory, boots it on a free port with its own data directory and waits
6 * for `/healthz`. The template's own `bun test` is the first half and runs
7 * in the checkout itself. Exits 0 when the born app answers, 1 when not.
8 */
9
10 import { mkdtemp, mkdir, rm } from "node:fs/promises";
11 import { tmpdir } from "node:os";
12 import { dirname, join } from "node:path";
13 import { fillTemplate } from "../src/apps/template.ts";
14
15 const [dir] = process.argv.slice(2);
16 if (dir === undefined) {
17 console.error("usage: bun scripts/birth-check.ts <checkout>");
18 process.exit(2);
19 }
20 const listed = Bun.spawnSync(["git", "-C", dir, "ls-files", "-z"], { stdout: "pipe", stderr: "pipe" });
21 if (listed.exitCode !== 0) {
22 console.error(`birth-check: not a git checkout: ${listed.stderr.toString().trim()}`);
23 process.exit(1);
24 }
25 const paths = listed.stdout.toString().split("\0").filter((p) => p !== "");
26 const files = await Promise.all(paths.map(async (path) => ({ path, content: new Uint8Array(await Bun.file(join(dir, path)).arrayBuffer()) })));
27 const born = fillTemplate({ files, from: "checkout", commit: "" }, { name: "born-from-template", description: "Born in the template check." });
28 if (born === undefined) {
29 console.error("birth-check: template.json is missing or unreadable");
30 process.exit(1);
31 }
32
33 const work = await mkdtemp(join(process.env["TMPDIR"] ?? tmpdir(), "birth-check-"));
34 let child: ReturnType<typeof Bun.spawn> | undefined;
35 try {
36 for (const f of born) {
37 const to = join(work, "app", f.path);
38 await mkdir(dirname(to), { recursive: true });
39 await Bun.write(to, f.content ?? "");
40 }
41 // Port 0 asks the kernel for a free one; the app is told it only after the
42 // probe socket is closed, so a port taken in between is retried, not fatal.
43 let up = false;
44 let port = 0;
45 for (let attempt = 0; attempt < 3 && !up; attempt++) {
46 const probe = Bun.listen({ hostname: "127.0.0.1", port: 0, socket: { data() {} } });
47 port = probe.port;
48 probe.stop(true);
49 child = Bun.spawn(["bun", "run", "src/server.ts"], {
50 cwd: join(work, "app"),
51 env: { PATH: process.env["PATH"] ?? "", PORT: String(port), FORGE_APP_DATA: join(work, "data"), FORGE_APP_ENV: "preview" },
52 stdout: "ignore",
53 stderr: "inherit",
54 });
55 for (let i = 0; i < 100 && !up && child.exitCode === null; i++) {
56 await Bun.sleep(100);
57 up = await fetch(`http://127.0.0.1:${port}/healthz`).then((r) => r.ok, () => false);
58 }
59 if (!up) child.kill();
60 }
61 const home = up ? await fetch(`http://127.0.0.1:${port}/`).then((r) => r.text(), () => "") : "";
62 if (!up || !home.includes("born-from-template")) {
63 console.error(`birth-check: the born app ${up ? "answers / without its name" : "did not answer /healthz"}`);
64 process.exitCode = 1;
65 } else console.log(`birth-check: born from ${paths.length} files, booted, / answers`);
66 } finally {
67 child?.kill();
68 await rm(work, { recursive: true, force: true });
69 }
scripts/take-template.ts added +55
@@ -0,0 +1,55 @@
1 /**
2 * `bun scripts/take-template.ts <checkout> [owner/name]`: write
3 * `templates/app-template.json`, the copy of the app template every forge
4 * carries in its image (#10305), from a checkout of the template repository.
5 * The tracked files at HEAD are taken as they are and HEAD is recorded as
6 * `commit` in the bundle. It is meant to run on a checkout of
7 * trevato/app-template at the commit a release pins (the release build does
8 * not do so yet); a checkout without a readable `template.json`, or with a
9 * file that is not UTF-8 text, is refused. `pinned: true` marks a bundle
10 * taken this way; the file in the tree says `pinned: false` until one is.
11 */
12
13 import { parseTemplateSpec } from "../src/apps/template.ts";
14
15 const [dir, repo = "trevato/app-template"] = process.argv.slice(2);
16 // FORGE_TEMPLATE_OUT: where to write (tests); default the tree's bundle.
17 const out = process.env["FORGE_TEMPLATE_OUT"] ?? `${import.meta.dir}/../templates/app-template.json`;
18 if (dir === undefined) {
19 console.error("usage: bun scripts/take-template.ts <checkout> [owner/name]");
20 process.exit(2);
21 }
22 const git = (...args: string[]): string => {
23 const r = Bun.spawnSync(["git", "-C", dir, ...args], { stdout: "pipe", stderr: "pipe" });
24 if (r.exitCode !== 0) {
25 console.error(`take-template: git ${args[0]} failed: ${r.stderr.toString().trim()}`);
26 process.exit(1);
27 }
28 return r.stdout.toString();
29 };
30 const commit = git("rev-parse", "HEAD").trim();
31 const paths = git("ls-files", "-z").split("\0").filter((p) => p !== "").sort();
32 // The bundle is JSON text, so a file that is not UTF-8 text is refused, not
33 // mangled: read the bytes and decode them strictly.
34 const files: Record<string, string> = {};
35 const decoder = new TextDecoder("utf-8", { fatal: true });
36 for (const p of paths) {
37 const bytes = new Uint8Array(await Bun.file(`${dir}/${p}`).arrayBuffer());
38 try {
39 const body = decoder.decode(bytes);
40 if (body.includes("\u0000")) throw new Error("NUL byte");
41 files[p] = body;
42 } catch {
43 console.error(`take-template: ${p} is not UTF-8 text; the bundle holds text files only`);
44 process.exit(1);
45 }
46 }
47 if (files["template.json"] === undefined || parseTemplateSpec(files["template.json"]) === undefined) {
48 console.error("take-template: the checkout has no readable template.json");
49 process.exit(1);
50 }
51 await Bun.write(
52 out,
53 `${JSON.stringify({ repo, commit, pinned: true, files }, null, 1)}\n`,
54 );
55 console.log(`took ${paths.length} files of ${repo} at ${commit}`);
src/actions/apps.ts +3 −3
@@ -23,7 +23,7 @@ import { appVitals, type AppVitals } from "../apps/vitals.ts";
2323 import { appsRuntime, envsFor, type EnvStatus } from "../apps/runtime.ts";
2424 import { grantUrl, originFor, sideForViewer, sideReader } from "../apps/proxy.ts";
2525 import { plainLine } from "../apps/side.ts";
26 import { scaffoldFiles } from "../apps/scaffold.ts";
26 import { scaffoldFiles } from "../apps/template.ts";
2727 import type { FileWrite } from "../git/write.ts";
2828 import { TEMPLATE_ERRORS, templateFiles, type TemplateError } from "../apps/template_tip.ts";
2929 import {
@@ -364,10 +364,10 @@ export async function newApp(
364364 message: `Start ${input.name}`,
365365 files:
366366 starter ??
367 scaffoldFiles({
367 (await scaffoldFiles(app, {
368368 name: input.name,
369369 description: input.description ?? "",
370 }),
370 })),
371371 });
372372 lap("scaffold");
373373 if (!committed.ok) {
src/apps/scaffold.ts deleted −6214

Diff not shown — 6,214 changed lines (limit 5,000). −6214

View file
src/apps/template.ts added +192
@@ -0,0 +1,192 @@
1 /**
2 * The app a new app starts as is a repository (#10305): `apps.template`
3 * names it (default `trevato/app-template`), and New app copies its tracked
4 * files at the default branch's tip — one commit, no history. Where the
5 * forge has no such repository (a child forge, a pack forge, a fresh
6 * install) the copy baked into the image is used: `templates/app-template.json`,
7 * taken from the repository by `scripts/take-template.ts`.
8 *
9 * `template.json` in the template says which files carry the app's name and
10 * description and the literals standing in for them; the forge replaces
11 * those and leaves the rest byte for byte, so the template stays runnable
12 * and its own tests pass as they are. `template.json` itself is not copied.
13 */
14
15 import { readFileSync } from "node:fs";
16 import type { App } from "../app.ts";
17 import { DEFAULT_TEMPLATE } from "../core/app_templates.ts";
18 import { getRepo } from "../core/repos.ts";
19 import { listPaths, readerFor } from "../git/read.ts";
20 import { repoDir } from "../git/repos.ts";
21 import type { FileWrite } from "../git/write.ts";
22 import { TEMPLATE_MAX_BYTES, TEMPLATE_MAX_FILE_BYTES, TEMPLATE_MAX_FILES } from "./template_tip.ts";
23
24 export const TEMPLATE_SETTING = "apps.template";
25
26 // The same caps as every other template copy (template_tip.ts).
27 const FILES_MAX = TEMPLATE_MAX_FILES;
28 const FILE_BYTES_MAX = TEMPLATE_MAX_FILE_BYTES;
29 const TOTAL_BYTES_MAX = TEMPLATE_MAX_BYTES;
30 const SLUG = /^[A-Za-z0-9][A-Za-z0-9._-]{0,63}\/[A-Za-z0-9][A-Za-z0-9._-]{0,99}$/;
31
32 export interface ScaffoldInput {
33 /** The repository name; also the title on the page. */
34 readonly name: string;
35 /** What the app should become. Shown on the page until it does. */
36 readonly description: string;
37 }
38
39 /** What `template.json` declares. */
40 export interface TemplateSpec {
41 /** The literal standing in for the app's name. */
42 readonly name: string;
43 /** The literal standing in for its description (also the default one). */
44 readonly description: string;
45 /** The files that carry those literals. */
46 readonly files: readonly string[];
47 }
48
49 /** A template as copied: its files, with `template.json` among them. */
50 export interface TemplateCopy {
51 readonly files: readonly FileWrite[];
52 /** Where it came from: `owner/name`, and the tip or pinned commit. */
53 readonly from: string;
54 readonly commit: string;
55 }
56
57 export const isTemplateSlug = (s: string): boolean => SLUG.test(s);
58
59 /** A safe relative path: no dot segments, no `.git`, no absolute or odd bytes. */
60 function safePath(path: string): boolean {
61 if (path === "" || path.length > 300 || path.startsWith("/") || /[\x00-\x1f\x7f\\]/.test(path)) return false;
62 return path.split("/").every((p) => p !== "" && p !== "." && p !== ".." && p.toLowerCase() !== ".git");
63 }
64
65 /** Parse `template.json`; undefined when it is not what the forge reads. */
66 export function parseTemplateSpec(text: string): TemplateSpec | undefined {
67 let raw: unknown;
68 try {
69 raw = JSON.parse(text);
70 } catch {
71 return undefined;
72 }
73 if (typeof raw !== "object" || raw === null) return undefined;
74 const { name, description, files } = raw as Record<string, unknown>;
75 if (typeof name !== "string" || name === "" || name.length > 100) return undefined;
76 if (typeof description !== "string" || description === "" || description.length > 500) return undefined;
77 if (!Array.isArray(files) || files.length > 50) return undefined;
78 if (!files.every((f) => typeof f === "string" && safePath(f))) return undefined;
79 return { name, description, files: files as string[] };
80 }
81
82 const text = (f: FileWrite): string =>
83 typeof f.content === "string" ? f.content : new TextDecoder().decode(f.content ?? new Uint8Array());
84
85 /** A literal as it sits inside a JSON string. */
86 const jsonInner = (s: string): string => JSON.stringify(s).slice(1, -1);
87
88 /**
89 * Replace the declared literals in the declared files, once each, as plain
90 * text (a `.json` file gets the JSON-escaped form). Everything else is the
91 * template's own bytes. Undefined when the template has no readable
92 * `template.json`.
93 */
94 export function fillTemplate(copy: TemplateCopy, input: ScaffoldInput): FileWrite[] | undefined {
95 const spec = copy.files.find((f) => f.path === "template.json");
96 const parsed = spec === undefined ? undefined : parseTemplateSpec(text(spec));
97 if (parsed === undefined) return undefined;
98 const description = input.description.trim() || parsed.description;
99 const carriers = new Set(parsed.files);
100 const out: FileWrite[] = [];
101 for (const f of copy.files) {
102 if (f.path === "template.json") continue;
103 if (!carriers.has(f.path) || f.content === null) {
104 out.push(f);
105 continue;
106 }
107 const json = f.path.endsWith(".json");
108 const esc = (s: string): string => (json ? jsonInner(s) : s);
109 // Description first: the name literal can sit inside it.
110 const body = text(f)
111 .split(esc(parsed.description))
112 .join("\u0000D\u0000")
113 .split(esc(parsed.name))
114 .join(esc(input.name))
115 .split("\u0000D\u0000")
116 .join(esc(description));
117 out.push({ ...f, content: body });
118 }
119 return out;
120 }
121
122 let baked: TemplateCopy | undefined;
123
124 /** The copy in the image. */
125 export function bakedTemplate(): TemplateCopy {
126 if (baked) return baked;
127 const raw = JSON.parse(
128 readFileSync(new URL("../../templates/app-template.json", import.meta.url), "utf8"),
129 ) as { repo: string; commit: string; pinned?: boolean; files: Record<string, string> };
130 baked = {
131 from: raw.repo,
132 commit: raw.commit,
133 files: Object.entries(raw.files).map(([path, content]) => ({ path, content })),
134 };
135 return baked;
136 }
137
138 /** The baked copy filled in for an app, synchronously. */
139 export function bakedFiles(input: ScaffoldInput): FileWrite[] {
140 const files = fillTemplate(bakedTemplate(), input);
141 if (!files) throw new Error("the baked app template has no readable template.json");
142 return files;
143 }
144
145 /** The template this forge names, `owner/name`. */
146 export function templateSlug(app: App): string {
147 const row = app.db
148 .query<{ value: string }, [string]>("SELECT value FROM forge_setting WHERE key = ?1")
149 .get(TEMPLATE_SETTING);
150 return row !== null && isTemplateSlug(row.value) ? row.value : DEFAULT_TEMPLATE;
151 }
152
153 /** The tracked files of `owner/name` at its default branch's tip, or undefined. */
154 export async function readTemplate(app: App, slug: string): Promise<TemplateCopy | undefined> {
155 if (!isTemplateSlug(slug)) return undefined;
156 const [owner, name] = slug.split("/") as [string, string];
157 const repo = getRepo(app.db, owner, name);
158 if (!repo) return undefined;
159 const dir = repoDir(app.reposRoot, repo.ownerName, repo.name);
160 const rev = `refs/heads/${repo.defaultBranch}`;
161 const listed = await listPaths(dir, rev, FILES_MAX + 1);
162 if (!listed.ok || listed.value.truncated || listed.value.paths.length > FILES_MAX || listed.value.paths.length === 0)
163 return undefined;
164 const reader = readerFor(dir);
165 const commit = await reader.stat(rev);
166 if (!commit.ok) return undefined;
167 const files: FileWrite[] = [];
168 let total = 0;
169 for (const path of listed.value.paths) {
170 if (!safePath(path)) return undefined;
171 const obj = await reader.object(`${rev}:${path}`);
172 if (!obj.ok || obj.value.type !== "blob" || obj.value.body.length > FILE_BYTES_MAX) return undefined;
173 total += obj.value.body.length;
174 if (total > TOTAL_BYTES_MAX) return undefined;
175 files.push({ path, content: obj.value.body });
176 }
177 return { files, from: slug, commit: commit.value.oid };
178 }
179
180 /**
181 * The files for a new app: from the named template (default: the forge's),
182 * else the baked copy. A repository whose `template.json` is missing or
183 * unreadable is not a template, and the baked copy stands in.
184 */
185 export async function scaffoldFiles(
186 app: App,
187 input: ScaffoldInput,
188 slug: string = templateSlug(app),
189 ): Promise<FileWrite[]> {
190 const copy = await readTemplate(app, slug);
191 return (copy && fillTemplate(copy, input)) ?? bakedFiles(input);
192 }
src/kernel/destroy_reach.ts −5
@@ -83,11 +83,6 @@ export const ALLOWED_REMOVALS: readonly { unit: string; why: string }[] = [
8383 { unit: "src/spawn/own_code.ts:checkout", why: "removes a half-made export of the forge's own code, keyed by commit" },
8484 { unit: "src/spawn/own_code.ts:recordGood", why: "removes exports of the forge's own code for other commits than the good one" },
8585 { unit: "src/web/asset.ts:prune", why: "removes cached page scripts older than their keep time" },
86 ...unitsOn("the call is inside text the unit returns (a generated app's code), never run by the forge", [
87 "src/apps/scaffold.ts:db",
88 "src/apps/scaffold.ts:committed",
89 "src/apps/scaffold.ts:base",
90 ]),
9186 ];
9287
9388 const REMOVERS = ["rmSync", "rmdirSync", "unlinkSync", "rm", "rmdir", "unlink"];
src/kernel/signin_words.ts +1 −1
@@ -2,7 +2,7 @@
22 * What a sign-in flow says when its return cannot be finished: the state
33 * did not come back to the browser that began, or the login it names is
44 * gone (spent, or past its time). The parent sign-in (`http/rp.ts`) and the
5 * sign-in an app's scaffold writes (`apps/scaffold.ts`) say the same thing,
5 * sign-in an app's scaffold writes (`templates/app-template.json`) say the same thing,
66 * so it is said here once.
77 */
88 export const SIGNIN_NOT_THIS_BROWSER = "This sign-in did not start in this browser.";
src/spawn/template_birth.ts +1 −1
@@ -8,7 +8,7 @@
88 */
99
1010 import type { App } from "../app.ts";
11 import { scaffoldFiles } from "../apps/scaffold.ts";
11 import { bakedFiles as scaffoldFiles } from "../apps/template.ts";
1212 import { createRepo, getRepo, updateRepoSettings } from "../core/repos.ts";
1313 import { runGit } from "../git/env.ts";
1414 import { repoDir } from "../git/repos.ts";
templates/app-template.json added +55

Diff not shown — 236,122 bytes of diff (limit 100,000). +55

View file
test/app-template.test.ts added +199
@@ -0,0 +1,199 @@
1 /**
2 * The app a new app starts as is a repository (#10305): the forge copies
3 * the template repository's tip when it has one, and the copy in the image
4 * otherwise; `template.json` says which literals are the app's name and
5 * description; the template's own evaluator is the forge's.
6 */
7 import { afterAll, beforeAll, describe, expect, setDefaultTimeout, test } from "bun:test";
8 import { readFileSync } from "node:fs";
9 import { mkdtemp, rm } from "node:fs/promises";
10 import { tmpdir } from "node:os";
11 import { join } from "node:path";
12 import { createApp, type App } from "../src/app.ts";
13 import { newApp } from "../src/actions/apps.ts";
14 import { bakedFiles, bakedTemplate, fillTemplate, parseTemplateSpec, scaffoldFiles } from "../src/apps/template.ts";
15 import { createUser, type User } from "../src/core/users.ts";
16 import { readerFor } from "../src/git/read.ts";
17 import { repoDir } from "../src/git/repos.ts";
18 import { commitFiles } from "../src/git/write.ts";
19
20 setDefaultTimeout(30_000);
21
22 const text = (c: string | Uint8Array | null | undefined): string =>
23 typeof c === "string" ? c : new TextDecoder().decode(c ?? new Uint8Array());
24
25 describe("the copy in the image", () => {
26 const copy = bakedTemplate();
27 const spec = parseTemplateSpec(text(copy.files.find((f) => f.path === "template.json")?.content));
28
29 test("declares its literals and the files that carry them", () => {
30 expect(spec?.name).toBe("app-template");
31 expect(spec?.files).toEqual(["package.json", "README.md", "contract/openapi.json"]);
32 for (const path of spec?.files ?? []) expect(copy.files.some((f) => f.path === path)).toBe(true);
33 });
34
35 test("filling replaces the literals in the named files and touches nothing else", () => {
36 const out = bakedFiles({ name: "hello-there", description: 'A "quoted" place' });
37 expect(out.some((f) => f.path === "template.json")).toBe(false);
38 const get = (path: string) => text(out.find((f) => f.path === path)?.content);
39 expect(JSON.parse(get("package.json"))).toMatchObject({ name: "hello-there", description: 'A "quoted" place' });
40 expect(get("README.md").startsWith('# hello-there\n\nA "quoted" place\n')).toBe(true);
41 expect(JSON.parse(get("contract/openapi.json")).info).toMatchObject({ title: "hello-there", description: 'A "quoted" place' });
42 const carriers = new Set(spec?.files);
43 for (const f of copy.files) {
44 if (f.path === "template.json" || carriers.has(f.path)) continue;
45 expect(get(f.path)).toBe(text(f.content));
46 }
47 });
48
49 test("an empty description keeps the template's own line", () => {
50 const out = bakedFiles({ name: "x", description: " " });
51 expect(text(out.find((f) => f.path === "README.md")?.content)).toContain("A new app on the forge.");
52 });
53
54 test("filling with its own literals gives back the template's files", () => {
55 const out = fillTemplate(copy, { name: "app-template", description: "A new app on the forge." }) ?? [];
56 for (const f of out) expect(text(f.content)).toBe(text(copy.files.find((c) => c.path === f.path)?.content));
57 });
58
59 test("its contract evaluator and beacon are the forge's own", async () => {
60 const mine = (p: string) => Bun.file(join(import.meta.dir, "..", p)).text();
61 const can = copy.files.find((f) => f.path === "src/can.ts")?.content;
62 expect(text(can)).toBe(await mine("src/contract/can.ts"));
63 });
64
65 test("a template.json that names odd paths or too many files is not read", () => {
66 expect(parseTemplateSpec("nope")).toBeUndefined();
67 for (const bad of ["../x", "/abs", ".git/config", "a/../b", "a//b", "a\\b", "x/.GIT/y"])
68 expect(parseTemplateSpec(JSON.stringify({ name: "a", description: "b", files: [bad] }))).toBeUndefined();
69 const many = Array.from({ length: 51 }, (_, i) => `f${i}`);
70 expect(parseTemplateSpec(JSON.stringify({ name: "a", description: "b", files: many }))).toBeUndefined();
71 });
72 });
73
74 describe("New app copies the template repository when the forge has it", () => {
75 let dataDir: string;
76 let app: App;
77 let owner: User;
78
79 beforeAll(async () => {
80 dataDir = await mkdtemp(join(tmpdir(), "forge-template-"));
81 app = createApp({ dataDir, port: 0 });
82 const made = await createUser(app.db, {
83 origin: "seed",
84 trusted: true,
85 username: "trevato",
86 email: "[email protected]",
87 password: "a-good-passphrase",
88 adminIfFirst: true,
89 });
90 if (!made.ok) throw new Error(made.error);
91 owner = made.value;
92 });
93 afterAll(async () => {
94 app.close();
95 await rm(dataDir, { recursive: true, force: true });
96 });
97
98 const readme = async (name: string): Promise<string> => {
99 const got = await readerFor(repoDir(app.reposRoot, "trevato", name)).object("refs/heads/main:README.md");
100 return got.ok ? text(got.value.body) : "";
101 };
102
103 test("without the repository the image's copy makes the app", async () => {
104 const files = await scaffoldFiles(app, { name: "fallback", description: "" });
105 expect(files.map((f) => f.path).sort()).toEqual(bakedFiles({ name: "fallback", description: "" }).map((f) => f.path).sort());
106 const made = await newApp(app, owner, { name: "from-baked", description: "Baked" });
107 expect(made.ok).toBe(true);
108 expect(await readme("from-baked")).toStartWith("# from-baked\n\nBaked\n");
109 });
110
111 test("with it, the tip of its default branch is the app, in one commit", async () => {
112 // trevato/app-template: made as an app, then given a README the image does not have.
113 expect((await newApp(app, owner, { name: "app-template" })).ok).toBe(true);
114 const edited = text(bakedTemplate().files.find((f) => f.path === "README.md")?.content).replace(
115 "# app-template",
116 "# app-template\n\nTIP-MARKER",
117 );
118 const template = bakedTemplate().files.map((f) => (f.path === "README.md" ? { ...f, content: edited } : f));
119 await commitFiles(repoDir(app.reposRoot, "trevato", "app-template"), {
120 branch: "main",
121 message: "Change the template",
122 files: template,
123 author: { name: "trevato", email: "[email protected]" },
124 });
125 const child = await newApp(app, owner, { name: "child-one", description: "Kid" });
126 expect(child.ok).toBe(true);
127 const body = await readme("child-one");
128 expect(body).toContain("TIP-MARKER");
129 expect(body).toStartWith("# child-one\n\nTIP-MARKER");
130 const count = Bun.spawnSync(["git", "-C", repoDir(app.reposRoot, "trevato", "child-one"), "rev-list", "--count", "main"]);
131 expect(count.stdout.toString().trim()).toBe("1");
132 const tracked = Bun.spawnSync(["git", "-C", repoDir(app.reposRoot, "trevato", "child-one"), "ls-tree", "-r", "--name-only", "main"]);
133 expect(tracked.stdout.toString()).not.toContain("template.json");
134 });
135
136 test("a template repository without a readable template.json falls back to the image's copy", async () => {
137 await commitFiles(repoDir(app.reposRoot, "trevato", "app-template"), {
138 branch: "main",
139 message: "Break template.json",
140 files: [{ path: "template.json", content: "nope" }],
141 author: { name: "trevato", email: "[email protected]" },
142 });
143 expect((await newApp(app, owner, { name: "child-two" })).ok).toBe(true);
144 expect(await readme("child-two")).not.toContain("TIP-MARKER");
145 });
146 });
147
148 describe("the copy in the image is pinned or says it is not", () => {
149 const raw = JSON.parse(readFileSync(new URL("../templates/app-template.json", import.meta.url), "utf8")) as {
150 commit: string;
151 pinned?: boolean;
152 note?: string;
153 };
154
155 test("a pinned bundle names a full commit id; an unpinned one says so in a note", () => {
156 if (raw.pinned === true) expect(raw.commit).toMatch(/^[0-9a-f]{40}$/);
157 else expect(raw.note ?? "").toContain("Not yet taken from the repository");
158 });
159 });
160
161 describe("the two template.json forms", () => {
162 test("the root form is not read as a folder's and the folder's is not read as the root's", () => {
163 const root = { name: "app-template", description: "A new app.", files: ["README.md"] };
164 const folder = { files: ["README.md"], literals: ["{{name}}", "{{description}}"] };
165 expect(parseTemplateSpec(JSON.stringify(root))).toBeDefined();
166 expect(parseTemplateSpec(JSON.stringify(folder))).toBeUndefined();
167 expect(Object.keys(folder).sort()).toEqual(["files", "literals"]);
168 });
169 });
170
171 describe("a template copy over the caps is not used", () => {
172 let dataDir: string;
173 let app: App;
174 beforeAll(async () => {
175 dataDir = await mkdtemp(join(tmpdir(), "forge-template-caps-"));
176 app = createApp({ dataDir, port: 0 });
177 const made = await createUser(app.db, {
178 origin: "seed", trusted: true, username: "trevato", email: "[email protected]",
179 password: "a-good-passphrase", adminIfFirst: true,
180 });
181 if (!made.ok) throw new Error(made.error);
182 expect((await newApp(app, made.value, { name: "app-template" })).ok).toBe(true);
183 });
184 afterAll(async () => {
185 app.close();
186 await rm(dataDir, { recursive: true, force: true });
187 });
188
189 test("a file over the per-file cap sends New app to the image's copy", async () => {
190 await commitFiles(repoDir(app.reposRoot, "trevato", "app-template"), {
191 branch: "main",
192 message: "Too big",
193 files: [{ path: "big.bin", content: new Uint8Array(512 * 1024 + 1) }],
194 author: { name: "trevato", email: "[email protected]" },
195 });
196 const got = await scaffoldFiles(app, { name: "capped", description: "" });
197 expect(got.some((f) => f.path === "big.bin")).toBe(false);
198 });
199 });
test/apps-measure.test.ts +1 −1
@@ -12,7 +12,7 @@ import { join } from "node:path";
1212 import { createApp, type App } from "../src/app.ts";
1313 import { newApp } from "../src/actions/apps.ts";
1414 import { AppsRuntime } from "../src/apps/runtime.ts";
15 import { scaffoldFiles } from "../src/apps/scaffold.ts";
15 import { bakedFiles as scaffoldFiles } from "../src/apps/template.ts";
1616 import { appVisitors } from "../src/actions/apps.ts";
1717 import { createToken } from "../src/core/tokens.ts";
1818 import { getRepo } from "../src/core/repos.ts";
test/apps-new.test.ts +1 −1
@@ -24,7 +24,7 @@ import { getRepo } from "../src/core/repos.ts";
2424 import { createUser, type User } from "../src/core/users.ts";
2525 import { serve, type Servers } from "../src/http/server.ts";
2626
27 import { scaffoldFiles } from "../src/apps/scaffold.ts";
27 import { bakedFiles as scaffoldFiles } from "../src/apps/template.ts";
2828 import { parseCheck } from "../src/apps/manifest.ts";
2929 import {
3030 appNameJs,
test/apps-token.test.ts +1 −1
@@ -23,7 +23,7 @@ import {
2323 renewAppToken,
2424 revokeAppToken,
2525 } from "../src/apps/token.ts";
26 import { scaffoldFiles } from "../src/apps/scaffold.ts";
26 import { bakedFiles as scaffoldFiles } from "../src/apps/template.ts";
2727 import { newApp } from "../src/actions/apps.ts";
2828 import { openIssue } from "../src/actions/issues.ts";
2929 import { commitToBranch, createRepo } from "../src/actions/repos.ts";
test/contract.test.ts +1 −1
@@ -10,7 +10,7 @@
1010 */
1111 import { describe, expect, test } from "bun:test";
1212 import { readFileSync } from "node:fs";
13 import { scaffoldFiles } from "../src/apps/scaffold.ts";
13 import { bakedFiles as scaffoldFiles } from "../src/apps/template.ts";
1414 import { policyLines, taskPrompt } from "../src/crew/brief.ts";
1515 import {
1616 NOBODY,
test/job-slices.test.ts +1 −1
@@ -247,7 +247,6 @@ const UNTIMED: Record<string, string> = {
247247 "release-behind": "reads the forge's own release tags with git; needs FORGE_REPO and its repository",
248248 "release-keeper": "cluster only (FORGE_RELEASE_KEEPER=on): talks to the cluster's API",
249249 "roll-probe": "cluster only (FORGE_RELEASE_KEEPER=on): one HTTP probe",
250 "spawn-due-wake": "one child's wake when its paused run falls due; the fixture has no children",
251250 "apps-sweep": "AppsRuntime's sweep over running app processes; the fixture has no apps",
252251 "cell-watch": "watches running app cells' cgroups; needs cells, skipped in this sandbox mode",
253252 "cell-probe-deadline": "a cell's start probe running out of time: needs cells, skipped in this sandbox mode",
@@ -275,6 +274,7 @@ const UNTIMED: Record<string, string> = {
275274 "system-live": "reads /proc and the cgroup roots for the system page while someone watches it",
276275 entitlements: "reads each paid subscription from Stripe: network, daily, and the fixture has no entitlements",
277276 traffic: "a closure in createTrafficWriter over its in-memory minute ledger; follow-up",
277 "spawn-due-wake": "a timer in the spawn host that wakes a due run: needs a spawned run; the fixture has none",
278278 };
279279
280280 /** A job held past this in one slice fails the test: five times the issue's 50 ms, so no runner flakes it. */
test/probe.test.ts +1 −1
@@ -13,7 +13,7 @@ import { createApp, type App } from "../src/app.ts";
1313 import { newApp } from "../src/actions/apps.ts";
1414 import { commitToBranch } from "../src/actions/repos.ts";
1515 import { AppsRuntime, appsRuntime } from "../src/apps/runtime.ts";
16 import { scaffoldFiles } from "../src/apps/scaffold.ts";
16 import { bakedFiles as scaffoldFiles } from "../src/apps/template.ts";
1717 import { audit } from "../src/contract/audit.ts";
1818 import { probe, type Probed } from "../src/contract/probe.ts";
1919 import { getAppByRepo } from "../src/core/apps.ts";
test/scaffold-webspec.test.ts +1 −1
@@ -8,7 +8,7 @@ import { afterAll, beforeAll, expect, test } from "bun:test";
88 import { mkdtemp, mkdir, rm, writeFile } from "node:fs/promises";
99 import { tmpdir } from "node:os";
1010 import { dirname, join } from "node:path";
11 import { scaffoldFiles } from "../src/apps/scaffold.ts";
11 import { bakedFiles as scaffoldFiles } from "../src/apps/template.ts";
1212 import { webspec, type Finding } from "../src/release/webspec.ts";
1313
1414 /** Today's gaps, each with its specification.website section. None: the scaffold meets it. */
test/scaffold.test.ts +1 −1
@@ -23,7 +23,7 @@ import { parseManifest } from "../src/apps/manifest.ts";
2323 import { createSession } from "../src/core/sessions.ts";
2424 import { createUser } from "../src/core/users.ts";
2525 import { serve } from "../src/http/server.ts";
26 import { scaffoldFiles } from "../src/apps/scaffold.ts";
26 import { bakedFiles as scaffoldFiles } from "../src/apps/template.ts";
2727
2828 /** How long the written-out app's `bun test` may take, wall clock. */
2929 const SUITE_BUDGET_MS = 3_000;
test/security-floor-doors.test.ts −1
@@ -33,7 +33,6 @@ const LISTENERS: Record<string, { count: number; why: string }> = {
3333 "src/spawn/beat-door.ts": { count: 1, why: "fetch is `securityFloor(await answerBeat(...))`, asked below" },
3434 "src/kernel/port.ts": { count: 1, why: "a port probe, stopped before any request can reach it" },
3535 "src/apps/process.ts": { count: 1, why: "a port probe, stopped before any request can reach it" },
36 "src/apps/scaffold.ts": { count: 1, why: "text of another app's server, written into its repository; it lays its own floor" },
3736 "src/actions/spawn.ts": { count: 1, why: "text of a fresh app's index.ts, written into its repository" },
3837 };
3938
test/signin-words.test.ts +10 −1
@@ -6,9 +6,10 @@
66 import { describe, expect, test } from "bun:test";
77 import { readFileSync } from "node:fs";
88 import { join } from "node:path";
9 import { bakedTemplate } from "../src/apps/template.ts";
910 import * as words from "../src/kernel/signin_words.ts";
1011
11 const READERS = ["src/http/rp.ts", "src/apps/scaffold.ts"];
12 const READERS = ["src/http/rp.ts"];
1213 const source = (path: string): string =>
1314 readFileSync(join(import.meta.dir, "..", path), "utf8");
1415
@@ -33,3 +34,11 @@ describe("a sign-in flow's expiry sentences", () => {
3334 }
3435 });
3536 });
37
38 describe("the app template says the same sentences", () => {
39 test("its sign-in carries each one verbatim", () => {
40 const signin = bakedTemplate().files.find((f) => f.path === "src/signin.ts")?.content;
41 expect(typeof signin).toBe("string");
42 for (const [, sentence] of Object.entries(words)) expect(signin as string).toContain(sentence);
43 });
44 });
test/spawn-template-birth.test.ts +1 −1
@@ -11,7 +11,7 @@ import { join } from "node:path";
1111 import { createRepo } from "../src/actions/repos.ts";
1212 import { childDirOf, spawnForge, type SpawnDeps } from "../src/actions/spawn.ts";
1313 import { createApp, type App } from "../src/app.ts";
14 import { scaffoldFiles } from "../src/apps/scaffold.ts";
14 import { bakedFiles as scaffoldFiles } from "../src/apps/template.ts";
1515 import { getAppByRepo } from "../src/core/apps.ts";
1616 import { getRepo } from "../src/core/repos.ts";
1717 import { createToken } from "../src/core/tokens.ts";
test/template-scripts.test.ts added +80
@@ -0,0 +1,80 @@
1 /**
2 * scripts/take-template.ts and scripts/birth-check.ts (#10305): the only
3 * checks that a template change still bundles and still boots.
4 */
5 import { afterAll, beforeAll, expect, setDefaultTimeout, test } from "bun:test";
6 import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
7 import { tmpdir } from "node:os";
8 import { dirname, join } from "node:path";
9 import { bakedTemplate } from "../src/apps/template.ts";
10
11 setDefaultTimeout(60_000);
12
13 let root: string;
14 const git = (dir: string, ...args: string[]) =>
15 Bun.spawnSync(["git", "-C", dir, "-c", "user.name=t", "-c", "user.email=t@t", ...args], { stdout: "pipe", stderr: "pipe" });
16
17 async function checkout(name: string, extra: Record<string, Uint8Array | string> = {}): Promise<string> {
18 const dir = join(root, name);
19 await mkdir(dir, { recursive: true });
20 git(dir, "init", "-q", "-b", "main");
21 const all: Record<string, string | Uint8Array> = {};
22 for (const f of bakedTemplate().files) all[f.path] = typeof f.content === "string" ? f.content : "";
23 Object.assign(all, extra);
24 for (const [p, c] of Object.entries(all)) {
25 await mkdir(dirname(join(dir, p)), { recursive: true });
26 await writeFile(join(dir, p), c);
27 }
28 git(dir, "add", "-A");
29 git(dir, "commit", "-q", "-m", "t");
30 return dir;
31 }
32
33 const run = (script: string, args: string[], env: Record<string, string> = {}) =>
34 Bun.spawnSync(["bun", join(import.meta.dir, "..", "scripts", script), ...args], {
35 env: { ...process.env, ...env },
36 stdout: "pipe",
37 stderr: "pipe",
38 });
39
40 beforeAll(async () => {
41 root = await mkdtemp(join(process.env["TMPDIR"] ?? tmpdir(), "tpl-scripts-"));
42 });
43 afterAll(() => rm(root, { recursive: true, force: true }));
44
45 test("take-template records HEAD as a pinned commit and the tracked files", async () => {
46 const dir = await checkout("take");
47 const out = join(root, "take.json");
48 const r = run("take-template.ts", [dir], { FORGE_TEMPLATE_OUT: out });
49 expect(r.exitCode).toBe(0);
50 const bundle = JSON.parse(await Bun.file(out).text());
51 expect(bundle.commit).toBe(git(dir, "rev-parse", "HEAD").stdout.toString().trim());
52 expect(bundle.commit).toMatch(/^[0-9a-f]{40}$/);
53 expect(bundle.pinned).toBe(true);
54 expect(Object.keys(bundle.files).sort()).toEqual(bakedTemplate().files.map((f) => f.path).sort());
55 });
56
57 test("take-template refuses a binary file and a checkout without template.json", async () => {
58 const bin = await checkout("bin", { "icon.png": new Uint8Array([0x89, 0x50, 0xff, 0xfe, 0x00]) });
59 const out = join(root, "bin.json");
60 const r = run("take-template.ts", [bin], { FORGE_TEMPLATE_OUT: out });
61 expect(r.exitCode).toBe(1);
62 expect(r.stderr.toString()).toContain("icon.png");
63 expect(await Bun.file(out).exists()).toBe(false);
64 const none = join(root, "none");
65 await mkdir(none);
66 git(none, "init", "-q");
67 await writeFile(join(none, "a.txt"), "x");
68 git(none, "add", "-A");
69 git(none, "commit", "-q", "-m", "t");
70 expect(run("take-template.ts", [none], { FORGE_TEMPLATE_OUT: join(root, "none.json") }).exitCode).toBe(1);
71 });
72
73 test("birth-check boots the born app and exits 0; a template that does not boot exits 1", async () => {
74 const good = await checkout("good");
75 const ok = run("birth-check.ts", [good], { TMPDIR: root });
76 expect(ok.stdout.toString()).toContain("booted");
77 expect(ok.exitCode).toBe(0);
78 const bad = await checkout("bad", { "src/server.ts": "throw new Error('does not boot');\n" });
79 expect(run("birth-check.ts", [bad], { TMPDIR: root }).exitCode).toBe(1);
80 });

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs