Public: anyone can read this; the crew can change it.

docs/README.md: generate its list of pages from the pages' front matter (child of #14248) #14382

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

torque/forge:torque/14353-docs-readme-md-generate-its-list-of-page → main 1 commit fork

  • Merged. torque/14353-docs-readme-md-generate-its-list-of-page was merged into main as 8d58706.

Review

Sign in to review this change.
33 files changed +225 −30
docs/APPS.md +3
@@ -1,3 +1,6 @@
1 ---
2 docs: Use it | Apps | apps, previews, runs and the rules the forge enforces.
3 ---
14 # Apps, runs, and the crew
25
36 This document defines apps, previews and runs, and lists the rules the forge
docs/ARCHITECTURE.md +3
@@ -1,3 +1,6 @@
1 ---
2 docs: Develop it | Architecture | the principles and the shape of the system.
3 ---
14 # Forge architecture
25
36 This document explains the forge's design principles and how its parts fit
docs/ASKS.md +3
@@ -1,3 +1,6 @@
1 ---
2 docs: Use it | Asks | what waits on a person and how to answer it.
3 ---
14 # Asks
25
36 What waits on a person, and how to answer it without opening each thing: `/asks`, the terminal's On you card, `GET /api/v1/asks` and the MCP `asks` tool. The board's own notes are in `docs/TOUR.md`.
docs/CLI.md +3
@@ -1,3 +1,6 @@
1 ---
2 docs: Use it | The `forge` command | every command, with an example.
3 ---
14 # The `forge` command
25
36 `forge` is the client that lives on your machine: it keeps a copy of the repos
docs/CREW.md +3
@@ -1,3 +1,6 @@
1 ---
2 docs: Use it | The crew | how agents build, review and land work.
3 ---
14 # The crew
25
36 The crew is the set of AI agents that build on the forge. This document covers
docs/DEPENDENCIES.md +3
@@ -1,3 +1,6 @@
1 ---
2 docs: Develop it | Dependencies | every package and why it is there; the gate refuses one without its line.
3 ---
14 # Dependencies
25
36 Every package in `package.json`, `devDependencies` included, with why it is
docs/DEVELOP.md +3
@@ -1,3 +1,6 @@
1 ---
2 docs: Develop it | Developing the forge | tests, checks, the gate, lint and types.
3 ---
14 # Developing the forge
25
36 This document is for people and agents who change the forge. It covers the tests,
docs/DOMAINS.md +3
@@ -1,3 +1,6 @@
1 ---
2 docs: Operate it | Failure domains | the ways the forge stops and what it reports.
3 ---
14 # Failure domains
25
36 The seven ways the forge stops, and what it says when each does.
docs/GIT.md +3
@@ -1,3 +1,6 @@
1 ---
2 docs: Use it | Cloning over git | smart HTTP, SSH and large repositories.
3 ---
14 # Cloning over git
25
36 The forge speaks git's smart HTTP (`https://<forge>/<owner>/<name>.git`) and
docs/MCP.md +3
@@ -1,3 +1,6 @@
1 ---
2 docs: Use it | MCP | the forge as a Model Context Protocol server.
3 ---
14 # MCP — the forge as a connector
25
36 `POST /mcp` is the forge as a Model Context Protocol server: the studio's
docs/METRICS.md +3
@@ -1,3 +1,6 @@
1 ---
2 docs: Operate it | Metrics | the Prometheus endpoint.
3 ---
14 # Metrics
25
36 `GET /metrics` answers Prometheus text (`text/plain; version=0.0.4`), one
docs/ON-YOUR-MACHINE.md +3
@@ -1,3 +1,6 @@
1 ---
2 docs: Run it | On your machine | the local copy, offline work and the binary.
3 ---
14 # On your machine
25
36 The forge holds the truth; `forge` keeps a copy of the parts you work on, the
docs/OPERATING.md +3
@@ -1,3 +1,6 @@
1 ---
2 docs: Operate it | Operating | configuration, health, releases, switches and limits.
3 ---
14 # Operating the forge
25
36 This document is for the person who runs a forge after it has started. It covers
docs/ORIGINS.md +3
@@ -1,3 +1,6 @@
1 ---
2 docs: Connect forges | Origins | connect accounts on other forges, mirror and write through.
3 ---
14 # Origins
25
36 This document describes how a forge connects to other forges. It is for people
docs/README.md +8 −30
@@ -7,55 +7,33 @@ apps, with AI agents that build. Forges connect to one another.
77 The forge is part of [Open Platform](https://www.open-platform.sh), with the
88 platform for organizations.
99
10 The lists below are built when the page is read, from the `docs:` lines in the front matter of the pages they name (#14353, `src/view/docs_index.ts`). Raw in git each shows only a marker line. A page adds itself with `docs: <heading> | <link words> | <sentence>` (an `| #anchor` after it links a part); nothing here is edited.
11
1012 ## Start here
1113
1214 The guide, in order, one or two screens a page: [What a forge is](guide-what-a-forge-is.md), [Your first app in five minutes](guide-first-app.md), [Tell the crew](guide-tell-the-crew.md), [Ship and undo](guide-ship-and-undo.md), [Your apps](guide-your-apps.md), [Hosting forges](guide-hosting.md), [Bring your own AI](guide-own-ai.md), [Agents](guide-agents.md), [Backups and moving](guide-backups-moving.md), [Settings](guide-settings.md), [When something is wrong](guide-when-wrong.md).
1315
1416 ## Use it
1517
16 - [The studio](STUDIO.md): the web interface and the words it uses.
17 - [Tour](TOUR.md): every other page and screen, and each way to reach it.
18 - [Apps](APPS.md): apps, previews, runs and the rules the forge enforces.
19 - [The crew](CREW.md): how agents build, review and land work.
20 - [Asks](ASKS.md): what waits on a person and how to answer it.
21 - [Cloning over git](GIT.md): smart HTTP, SSH and large repositories.
22 - [MCP](MCP.md): the forge as a Model Context Protocol server.
23 - [The `forge` command](CLI.md): every command, with an example.
18 <!-- pages -->
2419
2520 ## Run it
2621
27 - [Run it](REFERENCE.md#run-it): start a forge and create the first admin.
28 - [On your machine](ON-YOUR-MACHINE.md): the local copy, offline work and the binary.
29 - [Send a change without a forge](features/send-a-change.md): fork, source archive, patch, pull.
22 <!-- pages -->
3023
3124 ## Connect forges
3225
33 - [Origins](ORIGINS.md): connect accounts on other forges, mirror and write through.
34 - [A forge is an origin](features/forge-origin.md): connect one forge to another.
35 - [The forge toolbar](features/forge-toolbar.md): the pill in every app and preview.
26 <!-- pages -->
3627
3728 ## Operate it
3829
39 - [Operating](OPERATING.md): configuration, health, releases, switches and limits.
30 <!-- pages -->
4031 - [Operations log](ops/operations.log): one line per change to how the forge runs, with who and why; [where the bottleneck is](features/bottleneck.md) shows its newest and what the host is tuned to.
41 - [The forge as its own customer](designs/forge-as-its-own-customer.md): production hosts and releases, a hosted workshop builds the forge (#13639).
42 - [Compute leaves the forge](designs/compute-leaves-the-forge.md): runners that join and pull work, pools, SLAs, and the plan for a second machine (#13652).
43 - [Metrics](METRICS.md): the Prometheus endpoint.
44 - [Failure domains](DOMAINS.md): the ways the forge stops and what it reports.
45 - [An app's own domain](features/app-own-domain.md): cut-over, certificate, and the way back.
46 - [Snapshots](SNAPSHOT.md): nightly copies and what they hold.
47 - [Recovery](RECOVERY.md): a printable guide for a forge in trouble.
48 - [Restore](RESTORE.md): bring a forge back on a fresh machine.
4932
5033 ## Secure it
5134
52 - [Security runbook](SECURITY-RUNBOOK.md): what to do, with no AI, under load or after a leak.
53 - [Threats](THREATS.md): each protection and its test. Protections with no test are listed as gaps.
35 <!-- pages -->
5436
5537 ## Develop it
5638
57 - [Developing the forge](DEVELOP.md): tests, checks, the gate, lint and types.
58 - [Architecture](ARCHITECTURE.md): the principles and the shape of the system.
59 - [Writing](WRITING.md): how to write documentation for the forge.
60 - [Dependencies](DEPENDENCIES.md): every package and why it is there; the gate refuses one without its line.
61 - [Reference](REFERENCE.md): the long reference and the list of every document.
39 <!-- pages -->
docs/RECOVERY.md +3
@@ -1,3 +1,6 @@
1 ---
2 docs: Operate it | Recovery | a printable guide for a forge in trouble.
3 ---
14 # Recovery — read this at 3 a.m.
25
36 Print it. Keep it somewhere that is not the forge. `test/recovery-doc.test.ts`
docs/REFERENCE.md +4
@@ -1,3 +1,7 @@
1 ---
2 docs: Run it | Run it | start a forge and create the first admin. | #run-it
3 docs: Develop it | Reference | the long reference and the list of every document.
4 ---
15 # forge reference
26
37 The long version: how it runs, its options and how it fits together. [`README.md`](../README.md) is the short one.
docs/RESTORE.md +3
@@ -1,3 +1,6 @@
1 ---
2 docs: Operate it | Restore | bring a forge back on a fresh machine.
3 ---
14 # Restore card — the forge back on a fresh machine
25
36 Print this. Keep it somewhere that is not the forge.
docs/SECURITY-RUNBOOK.md +3
@@ -1,3 +1,6 @@
1 ---
2 docs: Secure it | Security runbook | what to do, with no AI, under load or after a leak.
3 ---
14 # Security runbook
25
36 This runbook says what to do when the forge is under load, a token leaks, an account looks wrong, a release is bad or sign-in breaks. It is for the forge's owner, with no AI at hand. Each step works from the browser or with `curl`, and each says how to undo it. Keep a copy off the forge (print it, or keep a clone), since you may need it when the forge is down.
docs/SNAPSHOT.md +3
@@ -1,3 +1,6 @@
1 ---
2 docs: Operate it | Snapshots | nightly copies and what they hold.
3 ---
14 # Snapshots — a forge back from a copy
25
36 Every night at 08:00 UTC a forge writes what cannot be rebuilt to a store,
docs/STUDIO.md +3
@@ -1,3 +1,6 @@
1 ---
2 docs: Use it | The studio | the web interface and the words it uses.
3 ---
14 # The studio
25
36 The studio is the forge's web interface. This document describes its pages and
docs/THREATS.md +3
@@ -1,3 +1,6 @@
1 ---
2 docs: Secure it | Threats | each protection and its test. Protections with no test are listed as gaps.
3 ---
14 # Threats
25
36 Who would attack this forge, what each wants, and what stands in the way.
docs/TOUR.md +3
@@ -1,3 +1,6 @@
1 ---
2 docs: Use it | Tour | every other page and screen, and each way to reach it.
3 ---
14 # A tour of the forge
25
36 This document describes each page and screen beyond the studio, one feature per
docs/WRITING.md +3
@@ -1,3 +1,6 @@
1 ---
2 docs: Develop it | Writing | how to write documentation for the forge.
3 ---
14 # Writing documentation
25
36 This guide sets the rules for documentation in this repository. It applies to
docs/designs/compute-leaves-the-forge.md +3
@@ -1,3 +1,6 @@
1 ---
2 docs: Operate it | Compute leaves the forge | runners that join and pull work, pools, SLAs, and the plan for a second machine (#13652).
3 ---
14 <!-- Source: issue #13652, kept here so the design lives in the repository (#14238). -->
25
36 The owner's direction, 2026-11-15, formalized. An epic beside #13639 (the forge as its own customer). Step R0 is the first issue to cut from it: name the seam in the check queue with no behaviour change.
docs/designs/forge-as-its-own-customer.md +3
@@ -1,3 +1,6 @@
1 ---
2 docs: Operate it | The forge as its own customer | production hosts and releases, a hosted workshop builds the forge (#13639).
3 ---
14 <!-- Source: issue #13639, kept here so the design lives in the repository (#14238). -->
25
36 The owner's direction, 2026-11-15, formalized. This is an epic: each step in the plan's table becomes its own issue as it is reached; steps 0 and 1 are being piloted now.
docs/features/app-own-domain.md +1
@@ -1,6 +1,7 @@
11 ---
22 topic: Your apps
33 row: An app's own domain: cut-over and the way back
4 docs: Operate it | An app's own domain | cut-over, certificate, and the way back.
45 ---
56 # An app's own domain: cut-over and the way back
67
docs/features/forge-origin.md +1
@@ -1,6 +1,7 @@
11 ---
22 topic: Backups and moving
33 row: A forge is an origin
4 docs: Connect forges | A forge is an origin | connect one forge to another.
45 ---
56 # A forge is an origin (#8900, first piece)
67
docs/features/forge-toolbar.md +1
@@ -1,6 +1,7 @@
11 ---
22 topic: Your apps
33 row: The forge toolbar
4 docs: Connect forges | The forge toolbar | the pill in every app and preview.
45 ---
56 # The forge toolbar
67
docs/features/send-a-change.md +1
@@ -1,6 +1,7 @@
11 ---
22 topic: Your first app in five minutes
33 row: Send a change without a forge of your own
4 docs: Run it | Send a change without a forge | fork, source archive, patch, pull.
45 ---
56 # Send a change without a forge of your own
67
src/view/docs.ts +2
@@ -8,6 +8,7 @@
88 */
99
1010 import { parseFront, renderFeatureIndex, stripFront } from "./feature_index.ts";
11 import { docsPages, renderDocsIndex } from "./docs_index.ts";
1112 import { readdirSync, readFileSync } from "node:fs";
1213 import { join } from "node:path";
1314
@@ -54,6 +55,7 @@ export function docText(entry: DocEntry, ssh = false): string {
5455 */
5556 function readDoc(name: string): string {
5657 const text = readFileSync(join(DOCS, name), "utf8");
58 if (name === "README.md") return renderDocsIndex(text, docsPages(DOCS));
5759 if (name !== "features/README.md") return stripFront(text);
5860 const notes = readdirSync(join(DOCS, "features"))
5961 .filter((n) => DOC_NAME.test(n) && n !== "README.md")
src/view/docs_index.ts added +89
@@ -0,0 +1,89 @@
1 /**
2 * The list of pages in `docs/README.md` is built from the pages' front matter, never typed (#14353,
3 * the method of the features index, #14248). A page opens with one `docs:` line per row it has
4 * there:
5 *
6 * ---
7 * docs: Use it | The studio | the web interface and the words it uses.
8 * docs: Run it | Run it | start a forge and create the first admin. | #run-it
9 * ---
10 *
11 * The fields are the `## ` heading of docs/README.md it sits under, the link words, the sentence
12 * after the link and, optionally, an `#anchor` of the page. `docs/README.md` keeps its
13 * introduction, one heading per topic and a `<!-- pages -->` line where each topic's rows go;
14 * `renderDocsIndex` puts them there, sorted by link words. A pull that adds a page adds that one
15 * file and edits no shared list. Plain data in, plain data out.
16 */
17
18 import { readdirSync, readFileSync } from "node:fs";
19 import { join } from "node:path";
20
21 export interface DocsRow {
22 readonly topic: string;
23 readonly words: string;
24 readonly about: string;
25 readonly anchor: string;
26 }
27
28 export interface DocsPage {
29 /** The path under docs/, as linked: `STUDIO.md`, `features/forge-origin.md`. */
30 readonly path: string;
31 readonly rows: readonly DocsRow[];
32 }
33
34 const FRONT = /^---\n([\s\S]*?)\n---\n/;
35
36 /** The rows a page's front matter asks for; a malformed `docs:` line is skipped. */
37 export function parseDocsRows(text: string): DocsRow[] {
38 const m = FRONT.exec(text);
39 if (m === null) return [];
40 const out: DocsRow[] = [];
41 for (const line of (m[1] ?? "").split("\n")) {
42 if (!line.startsWith("docs: ")) continue;
43 const f = line.slice(6).split(" | ").map((s) => s.trim());
44 const [topic, words, about, anchor = ""] = f;
45 if (f.length < 3 || f.length > 4 || !topic || !words || !about) continue;
46 if (anchor !== "" && !/^#[\w-]+$/.test(anchor)) continue;
47 out.push({ topic, words, about, anchor });
48 }
49 return out;
50 }
51
52 /** Every page under `dir` (its top folder, `designs/` and `features/`) that asks for a row. */
53 export function docsPages(dir: string): DocsPage[] {
54 const out: DocsPage[] = [];
55 for (const sub of ["", "designs", "features"]) {
56 const here = join(dir, sub);
57 for (const d of readdirSync(here, { withFileTypes: true })) {
58 if (!d.isFile() || !/^[A-Za-z0-9][A-Za-z0-9._-]{0,80}\.md$/.test(d.name) || d.name === "README.md") continue;
59 const rows = parseDocsRows(readFileSync(join(here, d.name), "utf8"));
60 if (rows.length > 0) out.push({ path: sub === "" ? d.name : `${sub}/${d.name}`, rows });
61 }
62 }
63 return out;
64 }
65
66 /** The head with each topic's rows put where its `<!-- pages -->` line is. */
67 export function renderDocsIndex(head: string, pages: readonly DocsPage[]): string {
68 const rows = new Map<string, { words: string; text: string }[]>();
69 for (const p of pages) {
70 for (const r of p.rows) {
71 const list = rows.get(r.topic) ?? [];
72 list.push({ words: r.words, text: `- [${r.words}](${p.path}${r.anchor}): ${r.about}` });
73 rows.set(r.topic, list);
74 }
75 }
76 let topic = "";
77 return head
78 .split("\n")
79 .map((l) => {
80 const h = /^## (.+)$/.exec(l);
81 if (h) topic = h[1] as string;
82 if (l.trim() !== "<!-- pages -->") return l;
83 return (rows.get(topic) ?? [])
84 .sort((a, b) => a.words.localeCompare(b.words) || a.text.localeCompare(b.text))
85 .map((r) => r.text)
86 .join("\n");
87 })
88 .join("\n");
89 }
test/docs-index.test.ts added +46
@@ -0,0 +1,46 @@
1 /**
2 * The list of pages in docs/README.md is built from the pages' front matter (#14353): a page with
3 * a row is listed, one with none is not, and the head holds no typed row.
4 */
5 import { expect, test } from "bun:test";
6 import { readFileSync } from "node:fs";
7 import { docsPages, parseDocsRows, renderDocsIndex } from "../src/view/docs_index.ts";
8 import { anyDocEntry, docText } from "../src/view/docs.ts";
9
10 const head = readFileSync("docs/README.md", "utf8");
11 const topics = [...head.matchAll(/^## (.+)$/gm)].map((m) => m[1] as string);
12
13 test("a row needs a topic, words and a sentence; a bad anchor is refused", () => {
14 expect(parseDocsRows("# none\n")).toEqual([]);
15 expect(parseDocsRows("---\ndocs: A | B\n---\n")).toEqual([]);
16 expect(parseDocsRows("---\ndocs: A | B | c. | nope\n---\n")).toEqual([]);
17 expect(parseDocsRows("---\ndocs: A | B | c.\ndocs: A | D | e. | #x-y\n---\n")).toEqual([
18 { topic: "A", words: "B", about: "c.", anchor: "" },
19 { topic: "A", words: "D", about: "e.", anchor: "#x-y" },
20 ]);
21 });
22
23 test("rows go under their topic, sorted, with the anchor; a page with none is absent", () => {
24 const out = renderDocsIndex("## A\n\n<!-- pages -->\n\n## B\n\n<!-- pages -->\n", [
25 { path: "z.md", rows: parseDocsRows("---\ndocs: A | Zed | last.\n---\n") },
26 { path: "y.md", rows: parseDocsRows("---\ndocs: A | Why | part. | #p\ndocs: B | Why too | again.\n---\n") },
27 { path: "n.md", rows: parseDocsRows("# bare\n") },
28 ]);
29 expect(out).toBe("## A\n\n- [Why](y.md#p): part.\n- [Zed](z.md): last.\n\n## B\n\n- [Why too](y.md): again.\n");
30 });
31
32 test("the head holds no typed page row, and every page's topic is a heading of it", () => {
33 expect(head.split("\n").filter((l) => /^- \[[^\]]+\]\([\w./-]+\.md(#[\w-]+)?\):/.test(l) && !l.includes("](features/bottleneck"))).toEqual([]);
34 const bad = docsPages("docs").flatMap((p) => p.rows.filter((r) => !topics.includes(r.topic)).map(() => p.path));
35 expect(bad).toEqual([]);
36 });
37
38 test("every page that asks for a row is listed in the README as served", () => {
39 const entry = anyDocEntry("README.md");
40 expect(entry).toBeDefined();
41 const text = docText(entry!);
42 const missing = docsPages("docs").flatMap((p) => p.rows.filter((r) => !text.includes(`- [${r.words}](${p.path}${r.anchor}): ${r.about}`)).map(() => p.path));
43 expect(missing).toEqual([]);
44 expect(docsPages("docs").length).toBeGreaterThanOrEqual(20);
45 expect(text).not.toContain("<!-- pages -->");
46 });

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs