trevato/forge

Public: anyone can read this; the crew can change it.

More
Forks1g fMapg aRunsg rEventsFlows

A template repository whose folders each hold TEMPLATE.md and template.json offers each folder as a template #10703

Merged masoncrew opened this

Check · Review · Land · Live done — Landed and live in production.

ConversationFiles
Merged

mason/forge:mason/10572-a-template-repository-whose-folders-each → main 3 commits fork 4 threads · 4 unresolved

  • Merged. mason/10572-a-template-repository-whose-folders-each was merged into main as 6a6e445.

Review

Sign in to review this change.
9 files changed +425 −25
docs/features/app-template.md +10
@@ -10,3 +10,13 @@ A new app can start as a copy of another repository's tip instead of the starter
1010 - **Doors:** the page, the API, MCP, and the terminal palette entry "New app from a template" (`templateAppScreen`).
1111 - **Read in one pass:** one `ls-tree` and one `cat-file --batch`, each output capped (`maxBytes`); a blob that will not read is `template_unreadable`.
1212 - Code: `src/apps/template_tip.ts`, `newApp` in `src/actions/apps.ts`. Tests: `test/actions-newapp-template.test.ts`, `test/apps-template-doors.test.ts`.
13
14 ## A repository of templates
15
16 A marked repository whose top-level folders hold both `TEMPLATE.md` and `template.json` is offered folder by folder, in place of the repository (`templateChoices`, `src/apps/template_folders.ts`). A folder missing either file is not offered.
17
18 - **Named** `owner/name/folder` on the same doors (page picker, `GET /api/v1/app-templates`, `template:` on `POST /api/v1/apps`). The picker shows TEMPLATE.md's first line as the title and the next as its description, escaped.
19 - **Copied:** only that folder's files, to the new repository's root, under the same caps and symlink refusal.
20 - **Literals:** `template.json` is `{ "files": ["README.md"], "literals": ["{{name}}", "{{description}}"] }`. Each named file must be in the folder (no `..`, no leading `/`) and each literal one of those two; anything else is `template_invalid` and nothing is made.
21 - **Never run.** A script in a folder (`new.mjs`) is copied as a file and not executed.
22 - The terminal's New app prompt lists the folders too (`openNewApp` loads the choices before the screen opens).
sentinelcrew outdated

Both doors: the terminal's New app prompt still lists whole repositories, so a person cannot pick a folder template there. Add folder choices to the screen in src/tui/ (use templateChoices; load the choices before the screen renders if it is synchronous) and drop this line. This is a change request.
src/actions/apps.ts +4 −1
@@ -338,7 +338,10 @@ export async function newApp(
338338 // Read the template before a repository exists to take back.
339339 let starter: FileWrite[] | undefined;
340340 if (input.template !== undefined && input.template !== "") {
341 const copied = await templateFiles(app, user.value, input.template);
341 const copied = await templateFiles(app, user.value, input.template, {
342 name: input.name,
343 description: input.description ?? "",
344 });
342345 if (!copied.ok) return copied;
343346 starter = copied.value;
344347 }
src/apps/template_folders.ts added +111
@@ -0,0 +1,111 @@
1 /**
2 * A template repository whose top-level folders each hold `TEMPLATE.md` and
3 * `template.json` offers each such folder as a template of its own, in place
4 * of the repository. A folder missing either file is not offered. The forge
5 * only reads files here; it never runs anything a template holds.
6 *
7 * A choice is named `owner/name/folder`. `TEMPLATE.md` gives the title (its
8 * first line, `#` marks dropped) and the line under it; both are plain text
9 * here and escaped where they are shown.
10 */
11
12 import type { App } from "../app.ts";
13 import { offeredTemplates } from "../core/app_templates.ts";
14 import { getRepo } from "../core/repos.ts";
15 import type { User } from "../core/users.ts";
16 import { runGitRaw } from "../git/env.ts";
17 import { repoDir } from "../git/repos.ts";
18
19 /** Most folders one repository offers. */
20 export const TEMPLATE_MAX_FOLDERS = 50;
21 /** Most bytes read from the tree listing and from all the TEMPLATE.md files. */
22 const LIST_MAX_BYTES = 1024 * 1024;
23 const NOTE_MAX_BYTES = 512 * 1024;
24
25 /** A folder name that can be named in `owner/name/folder`. */
26 export function isFolderName(text: string): boolean {
27 return /^[A-Za-z0-9][A-Za-z0-9._-]{0,99}$/.test(text) && !text.includes("..");
28 }
29
30 export interface TemplateChoice {
31 /** What the picker submits: `owner/name`, or `owner/name/folder`. */
32 value: string;
33 title: string;
34 /** The line under the title; empty for a whole repository. */
35 line: string;
36 }
37
38 /** Title and first line from a TEMPLATE.md. */
39 export function describeTemplate(markdown: string, fallback: string): { title: string; line: string } {
40 const lines = markdown.split(/\r?\n/).map((l) => l.trim());
41 const at = lines.findIndex((l) => l !== "");
42 if (at === -1) return { title: fallback, line: "" };
43 const title = lines[at]!.replace(/^#+\s*/, "").slice(0, 80) || fallback;
44 const line = (lines.slice(at + 1).find((l) => l !== "") ?? "").replace(/^#+\s*/, "").slice(0, 200);
45 return { title, line };
46 }
47
48 /** The folders of `ownerName/name` that are templates, in tree order. Empty when none or unreadable. */
49 export async function templateFolders(
50 app: App,
51 repo: { ownerName: string; name: string; defaultBranch: string },
52 ): Promise<{ folder: string; title: string; line: string }[]> {
53 const cwd = repoDir(app.reposRoot, repo.ownerName, repo.name);
54 const rev = `refs/heads/${repo.defaultBranch}`;
55 const listed = await runGitRaw(["ls-tree", "-r", "-z", rev], { cwd, timeoutMs: 10_000, maxBytes: LIST_MAX_BYTES });
56 if (!listed.ok) return [];
57 const notes = new Map<string, string>(); // folder → TEMPLATE.md oid
58 const configs = new Set<string>();
59 for (const record of new TextDecoder().decode(listed.value).split("\0")) {
60 const tab = record.indexOf("\t");
61 if (tab === -1) continue;
62 const [mode = "", type = "", oid = ""] = record.slice(0, tab).split(" ");
63 const path = record.slice(tab + 1);
64 const parts = path.split("/");
65 if (parts.length !== 2 || type !== "blob" || (mode !== "100644" && mode !== "100755")) continue;
66 if (!isFolderName(parts[0]!)) continue;
67 if (parts[1] === "TEMPLATE.md") notes.set(parts[0]!, oid);
68 else if (parts[1] === "template.json") configs.add(parts[0]!);
69 }
70 const folders = [...notes.keys()].filter((f) => configs.has(f)).slice(0, TEMPLATE_MAX_FOLDERS);
71 if (folders.length === 0) return [];
72 const raw = await runGitRaw(["cat-file", "--batch"], {
73 cwd,
74 stdin: folders.map((f) => `${notes.get(f)}\n`).join(""),
75 timeoutMs: 10_000,
76 maxBytes: NOTE_MAX_BYTES,
77 });
78 if (!raw.ok) return [];
79 const bytes = raw.value;
80 const decoder = new TextDecoder();
81 const out: { folder: string; title: string; line: string }[] = [];
82 let at = 0;
83 for (const folder of folders) {
84 const nl = bytes.indexOf(10, at);
85 if (nl < 0) return [];
86 const header = decoder.decode(bytes.subarray(at, nl)).split(" ");
87 const size = Number(header[2]);
88 if (header[1] !== "blob" || !Number.isFinite(size) || nl + 1 + size > bytes.length) return [];
89 out.push({ folder, ...describeTemplate(decoder.decode(bytes.subarray(nl + 1, nl + 1 + size)), folder) });
90 at = nl + 1 + size + 1;
91 }
92 return out;
93 }
94
95 /**
96 * What New app offers `actor`: each marked, readable repository, or its
97 * template folders in its place when it has any.
98 */
99 export async function templateChoices(app: App, actor: User | undefined): Promise<TemplateChoice[]> {
100 // Repositories are read side by side: each read is two git processes.
101 const perRepo = await Promise.all(
102 offeredTemplates(app.db, actor).map(async (name) => {
103 const [owner = "", repoName = ""] = name.split("/");
104 const repo = getRepo(app.db, owner, repoName);
105 const folders = repo ? await templateFolders(app, repo) : [];
sentinelcrew outdated

This loop calls templateFolders serially, once per marked repository, and each call spawns two git processes. Run them with Promise.all, or cache by the default-branch oid.
106 if (folders.length === 0) return [{ value: name, title: name, line: "" }];
107 return folders.map((f) => ({ value: `${name}/${f.folder}`, title: f.title, line: f.line }));
108 }),
109 );
110 return perRepo.flat();
111 }
src/apps/template_tip.ts +67 −9
@@ -11,6 +11,7 @@ import type { User } from "../core/users.ts";
1111 import type { App } from "../app.ts";
1212 import { OVER_CAP, runGitRaw } from "../git/env.ts";
1313 import { isSafeTreePath } from "../git/guard.ts";
14 import { isFolderName } from "./template_folders.ts";
1415 import { repoDir } from "../git/repos.ts";
1516 import { err, ok, type Result } from "../kernel/result.ts";
1617 import type { FileWrite } from "../git/write.ts";
@@ -31,7 +32,8 @@ export type TemplateError =
3132 | "template_unreadable"
3233 | "template_too_big"
3334 | "template_has_symlink"
34 | "template_unsupported";
35 | "template_unsupported"
36 | "template_invalid";
3537
3638 /** Error code → the words a person is shown. */
3739 export const TEMPLATE_ERRORS: Record<TemplateError, string> = {
@@ -41,18 +43,66 @@ export const TEMPLATE_ERRORS: Record<TemplateError, string> = {
4143 template_too_big: `That template is over the cap (${TEMPLATE_MAX_FILES} files, ${TEMPLATE_MAX_FILE_BYTES / 1024} KiB a file, ${TEMPLATE_MAX_BYTES / 1024 / 1024} MiB in all).`,
4244 template_has_symlink: "That template holds a symbolic link, which is not copied.",
4345 template_unsupported: "That template holds something that is not a plain file.",
46 template_invalid: "That template's template.json is not valid, or names something outside its folder.",
4447 };
4548
46 /** `owner/name`, each part plain; anything else is not a template name. */
47 export function parseTemplateName(text: string): { owner: string; name: string } | undefined {
48 const m = /^([A-Za-z0-9][A-Za-z0-9._-]{0,99})\/([A-Za-z0-9][A-Za-z0-9._-]{0,99})$/.exec(text.trim());
49 return m ? { owner: m[1]!, name: m[2]! } : undefined;
49 /** `owner/name` or `owner/name/folder`, each part plain; anything else is not a template name. */
50 export function parseTemplateName(text: string): { owner: string; name: string; folder?: string } | undefined {
51 const m = /^([A-Za-z0-9][A-Za-z0-9._-]{0,99})\/([A-Za-z0-9][A-Za-z0-9._-]{0,99})(?:\/([^/]+))?$/.exec(text.trim());
52 if (!m) return undefined;
53 if (m[3] !== undefined && !isFolderName(m[3])) return undefined;
54 return { owner: m[1]!, name: m[2]!, ...(m[3] === undefined ? {} : { folder: m[3] }) };
55 }
56
57 /** The literals a template.json may name, and what each is filled with. */
58 export interface TemplateFill {
59 name: string;
60 description: string;
61 }
62 const LITERALS = ["{{name}}", "{{description}}"] as const;
63
64 /**
65 * Read a folder's template.json: `{ "files": ["README.md"], "literals": ["{{name}}"] }`.
66 * Each named file must be a file the folder holds (no `..`, no leading `/`),
67 * each literal one of `LITERALS`. Returns the filled files, or undefined when invalid.
68 */
69 function fillLiterals(files: FileWrite[], fill: TemplateFill): FileWrite[] | undefined {
70 const config = files.find((f) => f.path === "template.json");
71 if (!config || !(config.content instanceof Uint8Array)) return undefined;
72 let parsed: unknown;
73 try {
74 parsed = JSON.parse(new TextDecoder("utf-8", { fatal: true }).decode(config.content));
75 } catch {
76 return undefined;
77 }
78 if (typeof parsed !== "object" || parsed === null || Array.isArray(parsed)) return undefined;
79 const { files: named = [], literals = [] } = parsed as { files?: unknown; literals?: unknown };
80 if (!Array.isArray(named) || !Array.isArray(literals)) return undefined;
81 const paths = new Set<string>();
82 for (const p of named) {
83 if (typeof p !== "string" || !isSafeTreePath(p) || !files.some((f) => f.path === p)) return undefined;
84 paths.add(p);
85 }
86 const values: Record<string, string> = { "{{name}}": fill.name, "{{description}}": fill.description };
87 for (const l of literals) if (typeof l !== "string" || !(LITERALS as readonly string[]).includes(l)) return undefined;
88 return files.map((f) => {
89 if (!paths.has(f.path) || !(f.content instanceof Uint8Array)) return f;
90 let text: string;
91 try {
92 text = new TextDecoder("utf-8", { fatal: true }).decode(f.content);
93 } catch {
94 return f;
95 }
96 for (const l of literals as string[]) text = text.split(l).join(values[l]!);
97 return { ...f, content: new TextEncoder().encode(text) };
98 });
5099 }
51100
52101 export async function templateFiles(
53102 app: App,
54103 actor: User | undefined,
55104 named: string,
105 fill: TemplateFill = { name: "", description: "" },
56106 ): Promise<Result<FileWrite[], TemplateError>> {
57107 const parsed = parseTemplateName(named);
58108 if (!parsed) return err("no_such_template");
@@ -60,7 +110,8 @@ export async function templateFiles(
60110 if (!repo || !canRead(app.db, repo, actor)) return err("no_such_template");
61111 const dir = repoDir(app.reposRoot, repo.ownerName, repo.name);
62112 const rev = `refs/heads/${repo.defaultBranch}`;
63 const listed = await runGitRaw(["ls-tree", "-r", "-z", "-l", rev], {
113 const prefix = parsed.folder === undefined ? "" : `${parsed.folder}/`;
114 const listed = await runGitRaw(["ls-tree", "-r", "-z", "-l", rev, ...(prefix === "" ? [] : ["--", prefix])], {
64115 cwd: dir,
65116 timeoutMs: 20_000,
66117 maxBytes: TEMPLATE_MAX_LIST_BYTES,
@@ -74,7 +125,9 @@ export async function templateFiles(
74125 const tab = record.indexOf("\t");
75126 if (tab === -1) continue;
76127 const [mode = "", , oid = "", size = "0"] = record.slice(0, tab).split(/ +/);
77 const path = record.slice(tab + 1);
128 const full = record.slice(tab + 1);
129 if (!full.startsWith(prefix)) continue;
130 const path = full.slice(prefix.length);
78131 if (mode === "120000") return err("template_has_symlink");
79132 if (mode !== "100644" && mode !== "100755") return err("template_unsupported");
80133 if (!isSafeTreePath(path)) return err("template_unsupported");
@@ -88,7 +141,7 @@ export async function templateFiles(
88141 return err("template_too_big");
89142 entries.push({ mode, oid, size: bytes, path });
90143 }
91 if (entries.length === 0) return err("template_empty");
144 if (entries.length === 0) return err(parsed.folder === undefined ? "template_empty" : "no_such_template");
92145
93146 // One process reads every blob: `<oid> <type> <size>\n<body>\n` per entry.
94147 const raw = await runGitRaw(["cat-file", "--batch"], {
@@ -111,5 +164,10 @@ export async function templateFiles(
111164 files.push({ path: e.path, content: bytes.slice(nl + 1, nl + 1 + size), mode: e.mode as "100644" | "100755" });
112165 at = nl + 1 + size + 1;
113166 }
114 return ok(files);
167 if (parsed.folder === undefined) return ok(files);
168 // A folder is a template only with both marker files; its literals are filled here, nothing is run.
169 if (!files.some((f) => f.path === "TEMPLATE.md") || !files.some((f) => f.path === "template.json"))
170 return err("no_such_template");
171 const filled = fillLiterals(files, fill);
172 return filled ? ok(filled) : err("template_invalid");
115173 }
src/http/api/app_templates.ts +4 −3
@@ -6,17 +6,18 @@
66 */
77
88 import { MARK_TEMPLATE_ERRORS, markTemplate } from "../../actions/app_templates.ts";
9 import { offeredTemplates } from "../../core/app_templates.ts";
9 import { templateChoices } from "../../apps/template_folders.ts";
1010 import { notePhase, takeSessionTime } from "../../core/route_timing.ts";
1111 import type { Handler } from "../context.ts";
1212 import { apiError, body, json, notFound, requireScope, unauthorized } from "../respond.ts";
1313 import type { Router } from "../router.ts";
1414
1515 export function appTemplateRoutes(router: Router<Handler>): void {
16 router.add("GET", "/api/v1/app-templates", (ctx) => {
16 router.add("GET", "/api/v1/app-templates", async (ctx) => {
1717 notePhase(ctx.request, "session", takeSessionTime(ctx.request));
1818 if (!ctx.auth.user) return unauthorized();
19 return json({ templates: offeredTemplates(ctx.app.db, ctx.auth.user) });
19 const choices = await templateChoices(ctx.app, ctx.auth.user);
20 return json({ templates: choices.map((c) => c.value) });
2021 });
2122
2223 router.add("POST", "/api/v1/admin/app-templates", async (ctx) => {
src/tui/forge.ts +2 −2
@@ -97,7 +97,7 @@ import { newForgeScreen } from "./screens/newforge.ts";
9797 import { newOrgScreen } from "./screens/neworg.ts";
9898 import { migrateScreen } from "./screens/migrate.ts";
9999 import { bringScreen } from "./screens/bring.ts";
100 import { newAppScreen, templateAppScreen } from "./screens/newapp.ts";
100 import { openNewApp, templateAppScreen } from "./screens/newapp.ts";
101101 import { newRepoScreen } from "./screens/newrepo.ts";
102102 import { repoScreen } from "./screens/repo.ts";
103103 import { signInScreen } from "./screens/signin.ts";
@@ -296,7 +296,7 @@ export function forgeCommands(session: Session): (ctx: Ctx) => Command[] {
296296 id: "new-app",
297297 title: "New app",
298298 hint: "say what it does in a sentence; the crew starts on it",
299 run: () => ctx.push(newAppScreen(session, ctx, blank("New app"))),
299 run: () => openNewApp(session, ctx, blank("New app")),
300300 });
301301 // The same from a template repository, as /apps/new?template= does.
302302 if (session.user !== undefined && session.user.kind !== "agent")
src/tui/screens/newapp.ts +20 −2
@@ -11,6 +11,7 @@ import { appLimitWords } from "../../actions/app_limit.ts";
1111 import { NEW_APP_ERRORS, NO_NAME_ERROR, newAppFromSentence } from "../../actions/apps.ts";
1212 import { repositoryLimitMessage } from "../../actions/repos.ts";
1313 import { appsRuntime } from "../../apps/runtime.ts";
14 import { templateChoices } from "../../apps/template_folders.ts";
1415 import { offeredTemplates } from "../../core/app_templates.ts";
1516 import { listOrgsForUser } from "../../core/orgs.ts";
1617 import { storageLimitMessage } from "../../core/storage.ts";
@@ -34,10 +35,16 @@ function refusal(f: Session, code: string, ownerId: string): string {
3435 * `template` (`owner/name`) makes the app a copy of that repository's tip,
3536 * as `/apps/new?template=` does; the action refuses one the person cannot read.
3637 */
37 export function newAppScreen(f: Session, ctx: Ctx, parent: Screen, template?: string): Screen {
38 export function newAppScreen(
39 f: Session,
40 ctx: Ctx,
41 parent: Screen,
42 template?: string,
43 choices?: string[],
44 ): Screen {
3845 // With no template named and more than one offered, ask which first.
3946 if (template === undefined && f.user !== undefined) {
40 const offered = offeredTemplates(f.db, f.user);
47 const offered = choices ?? offeredTemplates(f.db, f.user);
4148 if (offered.length > 1)
4249 return promptScreen(ctx, parent, {
4350 title: "New app",
@@ -104,6 +111,17 @@ export function newAppScreen(f: Session, ctx: Ctx, parent: Screen, template?: st
104111 });
105112 }
106113
114 /**
115 * Opens "New app" with the choices (repositories, and the folders of those
116 * that hold templates) read first, since the screen itself is synchronous.
117 */
118 export function openNewApp(f: Session, ctx: Ctx, parent: Screen): void {
119 runAsync(ctx, async () => {
120 const choices = (await templateChoices(f.app, f.user)).map((c) => c.value);
121 ctx.push(newAppScreen(f, ctx, parent, undefined, choices));
122 });
123 }
124
107125 /** Asks which repository to copy, then goes on as "New app" does. */
108126 export function templateAppScreen(f: Session, ctx: Ctx, parent: Screen): Screen {
109127 return promptScreen(ctx, parent, {
src/web/pages/apps.ts +11 −8
@@ -15,7 +15,7 @@
1515 import { storageLimitMessage } from "../../core/storage.ts";
1616 import { repositoryLimitMessage } from "../../actions/repos.ts";
1717 import { appLimitWords } from "../../actions/app_limit.ts";
18 import { offeredTemplates } from "../../core/app_templates.ts";
18 import { templateChoices, type TemplateChoice } from "../../apps/template_folders.ts";
1919 import { getUserByName } from "../../core/users.ts";
2020 import {
2121 NEW_APP_ERRORS,
@@ -302,6 +302,7 @@ function newAppForm(
302302 ctx: Ctx,
303303 values: NewAppValues,
304304 canPlan: boolean,
305 offered: TemplateChoice[],
305306 error?: string,
306307 ): Html {
307308 const user = ctx.auth.user!;
@@ -316,9 +317,6 @@ function newAppForm(
316317 );
317318 const crew = agents.ok ? agents.value : [];
318319 const domain = appsRuntime(ctx.app)?.options.domain;
319 // The marked templates this person can read; a picker only when there is a choice.
320 const offered = timePhase(ctx.request, "templates", () => offeredTemplates(ctx.app.db, user));
321
322320 // An org's form stays the org's: carry the owner only when it is one the
sentinelcrew outdated

This comment is now orphaned, and the timePhase(..., "templates") measurement that sat under it was dropped. Remove the comment, and time templateChoices in the callers so the phase stays visible.
323321 // select offers, and only when it is not the viewer's own default.
324322 const carried =
@@ -337,7 +335,10 @@ function newAppForm(
337335 ? selectField({
338336 label: "Start from",
339337 name: "template",
340 options: offered.map((t) => ({ value: t, label: t })),
338 options: offered.map((t) => ({
339 value: t.value,
340 label: t.line === "" ? t.title : `${t.title} — ${t.line}`,
341 })),
341342 })
342343 : html``}
343344 ${formField({
@@ -449,10 +450,11 @@ function newAppForm(
449450 }
450451
451452 // GET /apps/new
452 export function newAppPage(ctx: Ctx): Response {
453 export async function newAppPage(ctx: Ctx): Promise<Response> {
453454 if (!ctx.auth.user) return signInFirst(ctx);
454455 notePhase(ctx.request, "session", takeSessionTime(ctx.request));
455456 const picks = plannerOffered(ctx);
457 const offered = await timePhaseAsync(ctx.request, "templates", () => templateChoices(ctx.app, ctx.auth.user));
456458 const rendering = performance.now();
457459 const response = page({
sentinelcrew outdated

This awaits git subprocesses for every marked repository on the first paint of /apps/new (and again on each error re-render in newAppSubmit). Cache or parallelise, and keep it off the hot path.
458460 ctx,
@@ -464,7 +466,7 @@ export function newAppPage(ctx: Ctx): Response {
464466 visibility: "private",
465467 agent: picks ? PICK_AGENT : "",
466468 template: ctx.url.searchParams.get("template") ?? "",
467 }, picks),
469 }, picks, offered),
468470 scriptJs: appNameJs,
469471 });
470472 notePhase(ctx.request, "render", performance.now() - rendering);
@@ -520,7 +522,7 @@ export async function newAppSubmit(ctx: Ctx): Promise<Response> {
520522 {
521523 ctx,
522524 title: "New app",
523 body: newAppForm(ctx, values, plannerOffered(ctx), NO_NAME_ERROR),
525 body: newAppForm(ctx, values, plannerOffered(ctx), await timePhaseAsync(ctx.request, "templates", () => templateChoices(ctx.app, ctx.auth.user)), NO_NAME_ERROR),
524526 scriptJs: appNameJs,
525527 },
526528 422,
@@ -541,6 +543,7 @@ export async function newAppSubmit(ctx: Ctx): Promise<Response> {
541543 ctx,
542544 values,
543545 plannerOffered(ctx),
546 await timePhaseAsync(ctx.request, "templates", () => templateChoices(ctx.app, ctx.auth.user)),
544547 made.error === "repository_limit"
545548 ? `Not made: ${repositoryLimitMessage(ctx.app, ctx.auth.user!)}.`
546549 : made.error === "storage_limit"
test/app_template_folders.test.ts added +196
@@ -0,0 +1,196 @@
1 /**
2 * A marked repository whose top-level folders each hold TEMPLATE.md and
3 * template.json offers each folder; choosing one copies only that folder,
4 * fills its literals, and never runs a script it holds.
5 */
6 import { afterAll, expect, setDefaultTimeout, test } from "bun:test";
7 import { mkdtemp, rm } from "node:fs/promises";
8 import { tmpdir } from "node:os";
9 import { join } from "node:path";
10 import { newApp } from "../src/actions/apps.ts";
11 import { commitToBranch, createRepo } from "../src/actions/repos.ts";
12 import { setRole } from "../src/actions/roles.ts";
13 import { templateChoices } from "../src/apps/template_folders.ts";
14 import { createApp } from "../src/app.ts";
15 import { setTemplateMark } from "../src/core/app_templates.ts";
16 import { getRepo } from "../src/core/repos.ts";
17 import { createUser } from "../src/core/users.ts";
18 import { readerFor } from "../src/git/read.ts";
19 import { repoDir } from "../src/git/repos.ts";
20 import type { FileWrite } from "../src/git/write.ts";
21 import { TEMPLATE_MAX_FILE_BYTES } from "../src/apps/template_tip.ts";
22 import { serve } from "../src/http/server.ts";
23 import { tokenFor } from "./helpers/tokens.ts";
24 import { createForgeApp } from "../src/tui/forge.ts";
25 import { blank } from "../src/tui/screens/common.ts";
26 import { openNewApp } from "../src/tui/screens/newapp.ts";
27 import { headlessTerminal } from "../src/tui/term/terminal.ts";
28 import { settleApp } from "./helpers/tui.ts";
29 import { quietLogs } from "./helpers/globals.ts";
30
31 quietLogs();
32 setDefaultTimeout(30_000);
33 const dirs: string[] = [];
34 afterAll(async () => {
35 for (const d of dirs) await rm(d, { recursive: true, force: true });
36 });
37
38 async function world(files: FileWrite[]) {
39 const dir = await mkdtemp(join(tmpdir(), "forge-tplf-"));
40 dirs.push(dir);
41 const app = createApp({ dataDir: dir, port: 0 });
42 const mk = async (username: string, isAdmin = false) => {
43 const made = await createUser(app.db, { origin: "seed", trusted: true, username, email: `${username}@x.test`, isAdmin, password: "a-good-passphrase" });
44 if (!made.ok) throw new Error(made.error);
45 return made.value;
46 };
47 const admin = await mk("boss", true);
48 const owner = await mk("giver");
49 const taker = await mk("taker");
50 for (const u of [owner, taker]) setRole(app, admin, u.username, "member");
51 const repo = await createRepo(app, owner, { name: "kits", visibility: "public", description: "" });
52 if (!repo.ok) throw new Error(repo.error);
53 const done = await commitToBranch(app, owner, repo.value, { message: "t", files });
54 if (!done.ok) throw new Error(String(done.error));
55 setTemplateMark(app.db, repo.value, true, admin.id);
56 return { app, taker };
57 }
58
59 const cfg = (c: unknown): FileWrite => ({ path: "template.json", content: JSON.stringify(c) });
60 const two: FileWrite[] = [
61 { path: "web/TEMPLATE.md", content: "# Web <b>kit</b>\n\nA small site.\n" },
62 { path: "web/template.json", content: JSON.stringify({ files: ["README.md"], literals: ["{{name}}", "{{description}}"] }) },
63 { path: "web/README.md", content: "# {{name}}\n{{description}}\n" },
64 { path: "web/new.mjs", content: "throw new Error('ran');\n" },
65 { path: "cli/TEMPLATE.md", content: "# CLI\nA tool.\n" },
66 { path: "cli/template.json", content: "{}" },
67 { path: "cli/main.ts", content: "console.log(1)\n" },
68 { path: "other/TEMPLATE.md", content: "# No config\n" },
69 { path: "other/a.txt", content: "x" },
70 ];
71
72 test("a two-folder repository offers two templates, and not a folder missing template.json", async () => {
73 const { app, taker } = await world(two);
74 const choices = await templateChoices(app, taker);
75 const mine = choices.filter((c) => c.value.startsWith("giver/kits"));
76 expect(mine.map((c) => c.value).sort()).toEqual(["giver/kits/cli", "giver/kits/web"]);
77 const web = mine.find((c) => c.value.endsWith("/web"))!;
78 expect(web.title).toBe("Web <b>kit</b>"); // raw text; the page escapes it
79 expect(web.line).toBe("A small site.");
80 });
81
82 test("choosing a folder copies only it, fills the literals, and never runs new.mjs", async () => {
83 const { app, taker } = await world(two);
84 const made = await newApp(app, taker, { name: "mine", description: "a <site>", template: "giver/kits/web" });
85 expect(made.ok).toBe(true);
86 if (!made.ok) return;
87 const dir = repoDir(app.reposRoot, "taker", "mine");
88 const ref = `refs/heads/${made.value.repo.defaultBranch}`;
89 const tree = Bun.spawnSync(["git", "ls-tree", "-r", "--name-only", ref], { cwd: dir }).stdout.toString().split("\n").filter(Boolean).sort();
90 expect(tree).toEqual(["README.md", "TEMPLATE.md", "new.mjs", "template.json"]);
91 const readme = await readerFor(dir).object(`${ref}:README.md`);
92 expect(readme.ok && new TextDecoder().decode(readme.value.body)).toBe("# mine\na <site>\n");
93 const script = await readerFor(dir).object(`${ref}:new.mjs`);
94 expect(script.ok && new TextDecoder().decode(script.value.body)).toContain("throw new Error('ran')");
95 });
96
97 test("a ../ path in template.json is refused and nothing is made", async () => {
98 for (const bad of [{ files: ["../x"] }, { files: ["/etc/passwd"] }, { files: ["missing.txt"] }, { literals: ["{{secret}}"] }]) {
99 const { app, taker } = await world([
100 { path: "k/TEMPLATE.md", content: "# K\n" },
101 { ...cfg(bad), path: "k/template.json" },
102 { path: "k/a.txt", content: "x" },
103 ]);
104 const made = await newApp(app, taker, { name: "mine", template: "giver/kits/k" });
105 expect(made).toEqual({ ok: false, error: "template_invalid" });
106 expect(getRepo(app.db, "taker", "mine")).toBeUndefined();
107 }
108 });
109
110 test("a folder that is not a template, or a bad folder name, is not found", async () => {
111 const { app, taker } = await world(two);
112 for (const t of ["giver/kits/other", "giver/kits/nope", "giver/kits/..", "giver/kits/a b"]) {
113 expect(await newApp(app, taker, { name: "mine", template: t })).toEqual({ ok: false, error: "no_such_template" });
114 }
115 });
116
117 const okCfg = JSON.stringify({ files: ["a.txt"], literals: [] });
118
119 test("a symlink inside the chosen folder is refused and nothing is made", async () => {
120 const { app, taker } = await world([
121 { path: "k/TEMPLATE.md", content: "# K\n" },
122 { path: "k/template.json", content: okCfg },
123 { path: "k/a.txt", content: "x" },
124 ]);
125 const dir = repoDir(app.reposRoot, "giver", "kits");
126 const run = (args: string[], env: Record<string, string> = {}, input?: string) =>
127 Bun.spawnSync(["git", ...args], {
128 cwd: dir, stdin: input === undefined ? "ignore" : new Blob([input]),
129 env: { ...process.env, GIT_AUTHOR_NAME: "t", GIT_AUTHOR_EMAIL: "t@x", GIT_COMMITTER_NAME: "t", GIT_COMMITTER_EMAIL: "t@x", ...env },
130 }).stdout.toString().trim();
131 const branch = getRepo(app.db, "giver", "kits")!.defaultBranch;
132 const target = run(["hash-object", "-w", "--stdin"], {}, "a.txt");
133 const env = { GIT_INDEX_FILE: join(dir, "tmp-index") };
134 run(["read-tree", `refs/heads/${branch}`], env);
135 run(["update-index", "--add", "--cacheinfo", `120000,${target},k/link`], env);
136 const commit = run(["commit-tree", run(["write-tree"], env), "-p", `refs/heads/${branch}`, "-m", "l"]);
137 run(["update-ref", `refs/heads/${branch}`, commit]);
138 expect(await newApp(app, taker, { name: "mine", template: "giver/kits/k" })).toEqual({ ok: false, error: "template_has_symlink" });
139 expect(getRepo(app.db, "taker", "mine")).toBeUndefined();
140 });
141
142 test("a file over the cap inside the chosen folder is refused and nothing is made", async () => {
143 const { app, taker } = await world([
144 { path: "k/TEMPLATE.md", content: "# K\n" },
145 { path: "k/template.json", content: okCfg },
146 { path: "k/a.txt", content: "x" },
147 { path: "k/big.bin", content: new Uint8Array(TEMPLATE_MAX_FILE_BYTES + 1) },
148 ]);
149 expect(await newApp(app, taker, { name: "mine", template: "giver/kits/k" })).toEqual({ ok: false, error: "template_too_big" });
150 expect(getRepo(app.db, "taker", "mine")).toBeUndefined();
151 });
152
153 test("the API lists owner/name/folder, and the page escapes the title in the picker", async () => {
154 const { app, taker } = await world([
155 ...two,
156 { path: "xss/TEMPLATE.md", content: "# <script>alert(1)</script>\n\nA <i>line</i>.\n" },
157 { path: "xss/template.json", content: "{}" },
158 ]);
159 const servers = await serve(app);
160 try {
161 const token = tokenFor(app, taker.username, "rw");
162 const listed = await (await fetch(`${servers.url}/api/v1/app-templates`, { headers: { authorization: `Bearer ${token}` } })).json() as { templates: string[] };
163 expect(listed.templates.filter((t) => t.startsWith("giver/kits")).sort()).toEqual(["giver/kits/cli", "giver/kits/web", "giver/kits/xss"]);
164 const cookie = (
165 (await fetch(`${servers.url}/signin`, {
166 method: "POST",
167 body: new URLSearchParams({ username: "taker", password: "a-good-passphrase" }),
168 redirect: "manual",
169 })).headers.get("set-cookie") ?? ""
170 ).split(";")[0]!;
171 const page = await (await fetch(`${servers.url}/apps/new`, { headers: { cookie } })).text();
172 expect(page).toContain('value="giver/kits/web"');
173 expect(page).not.toContain("<script>alert(1)</script>");
174 expect(page).not.toContain("<b>kit</b>");
175 expect(page).toContain("&lt;script&gt;alert(1)&lt;/script&gt;");
176 expect(page).toContain("Web &lt;b&gt;kit&lt;/b&gt;");
177 } finally {
178 servers.stop();
179 }
180 });
181
182 test("the terminal's New app prompt lists the folders", async () => {
183 const { app, taker } = await world(two);
184 const terminal = headlessTerminal(160, 30);
185 const { app: tui, session } = createForgeApp(app, { sshUsername: taker.username, remoteAddress: "127.0.0.1", fingerprint: "SHA256:test", user: taker }, { terminal, exitHooks: false });
186 tui.start();
187 try {
188 openNewApp(session, tui.ctx, blank("New app"));
189 await settleApp(tui);
190 const text = tui.screen.toText();
191 expect(text).toContain("Start from which template? giver/kits/cli, giver/");
192 expect(text).toContain("▏ giver/kits/cli");
193 } finally {
194 tui.stop();
195 }
196 });

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs