trevato/forge

Public: anyone can read this; the crew can change it.

More
Forks1g fMapg aRuns7g rEventsFlows

An API reference in the forge: /api renders the forge's OpenAPI spec (and an app's own, if it serves one) — endpoints by area, schemas, and try it as yourself #7587

Merged masoncrew opened this in answer to trevatoowner

Check · Review · Land · Live done — Landed and live in production.

ConversationFiles
Merged

mason/forge:mason/7583-an-api-reference-in-the-forge-api-render → main 3 commits fork 3 threads · 3 unresolved

  • Merged. mason/7583-an-api-reference-in-the-forge-api-render was merged into main as 90e5ec7.

Review

Sign in to review this change.
13 files changed +856 −7
docs/design/audit.md +2 −1
@@ -33,7 +33,7 @@ their own vocabulary instead of composing the named one.
3333
3434 ## Finding 2 — the pages, and what each is for
3535
36 There are 63 top-level pages, one entry each: every static `GET` route in `src/http/server.ts`
36 There are 64 top-level pages, one entry each: every static `GET` route in `src/http/server.ts`
3737 that returns a page (`test/design-audit.test.ts` holds this list to the router,
3838 both ways). Each entry reads `route` — module · what it is for · what
3939 duplicates it · what breaks `docs/DESIGN.md`. "Breaks" counts colour literals
@@ -63,6 +63,7 @@ Machine routes are not pages and have no entry: `/audit.csv`,
6363 - `/audit` — `pages/audit.ts` · every app's contract, who can do what · the app page's one-line policy · none seen. Names the audit of apps, not this document.
6464 - `/releases` — `pages/releases.ts` · the forge's kept releases and rollback · `/settings/backups` (both are "go back") · none seen.
6565 - `/changelog` — `pages/changelog.ts` · the last 30 releases with their dates and pulls, public · `/releases` (the signed-in kept releases) · none seen.
66 - `/api` — `pages/api.ts` · the forge's OpenAPI operations by area, with search and Try it, public · the footer, the OpenAPI document, the terminal's API screen · none seen. An app's own spec is its `/:owner/:name/api` tab.
6667 - `/contribute` — `pages/contribute.ts` · what this forge is and where outside work is welcome, open to anyone · `/llms.txt`, MCP `contribute` tool · none seen.
6768 - `/search` — `pages/search.ts` · find across what I can read · the palette (`palette.ts`) · none seen.
6869 - `/-/confirm` — `pages/search.ts` (`confirmPage`) · the "are you sure" step for a destructive action · none · none seen.
docs/features/api-reference.md added +21
@@ -0,0 +1,21 @@
1 # The API reference
2
3 `/api` renders the forge's OpenAPI document (`src/contract/openapi.ts`) as a page: operations grouped by area
4 (the first word after `/api/v1`), a search box (`?q=`, a plain GET form), and for each operation a fold with its
5 parameters, body fields and replies. It is server-rendered; the folds are `<details>`. Nothing is loaded from
6 elsewhere, and the one script (`API_TRY_JS` in `src/web/api_reference.ts`) is on this page only.
7
8 **Try it** sends the request as you (your session cookie) or as a token you paste, and shows the status, the time,
9 the `Server-Timing` header and the body. A GET with no path parameter also works with no script: the form is a plain
10 GET to the endpoint. A request that is not a GET asks to confirm first.
11
12 **An app's API tab.** An app whose production environment serves `/openapi.json` gets an **API** tab next to Runs,
13 at `/:owner/:name/api`, rendered by the same reading (`src/contract/reference.ts`). The forge reads the file over
14 loopback (bounded to 512 KB and 1.5 s, kept only when it is a JSON object with `paths`) and remembers it for a
15 minute (`src/apps/api_spec.ts`); the tab appears once the header has seen it. Try it is not offered on an app's
16 operations: they live on another origin.
17
18 **In the terminal:** `API` in the command palette lists the same operations; `/` narrows; Enter on a GET with no
19 path parameter runs it as the person at the terminal and shows the status, time, Server-Timing and body.
20
21 `/api` answers `no_route` in JSON to a caller whose `Accept` prefers JSON, as the rest of `/api` does.
src/apps/api_spec.ts added +97
@@ -0,0 +1,97 @@
1 /**
2 * An app's own API description: the `/openapi.json` its production
3 * environment serves, read over loopback and remembered for a minute. The
4 * header asks `knownSpec` (never waits); `refreshSpec` is the one that reaches
5 * the app, and what it gets is data from outside: bounded in size and time,
6 * kept only when it is a JSON object with `paths`.
7 */
8
9 import type { AppRecord } from "../core/apps.ts";
10
11 const TTL_MS = 60_000;
12 const WAIT_MS = 1500;
13 const MAX_BYTES = 512 * 1024;
14 const MAX_KEPT = 500;
15
16 interface Kept {
17 readonly at: number;
18 readonly doc: Record<string, unknown> | null;
19 }
20
21 const kept = new Map<string, Kept>();
22 const asking = new Set<string>();
23
24 /** Remember what an app's spec is (null: it serves none). */
25 export function rememberSpec(appId: string, doc: Record<string, unknown> | null, at = Date.now()): void {
26 if (kept.size >= MAX_KEPT && !kept.has(appId)) kept.delete(kept.keys().next().value as string);
27 kept.set(appId, { at, doc });
28 }
29
30 /** The spec last read, while it is fresh; undefined when not known (yet). */
31 export function knownSpec(appId: string, at = Date.now()): Record<string, unknown> | null | undefined {
32 const hit = kept.get(appId);
33 return hit === undefined || at - hit.at > TTL_MS ? undefined : hit.doc;
34 }
35
36 /** The body, read in chunks and abandoned past the cap, so a huge one is never held whole. */
37 async function boundedBody(res: Response): Promise<Uint8Array | null> {
38 const reader = res.body?.getReader();
39 if (reader === undefined) return null;
40 const chunks: Uint8Array[] = [];
41 let total = 0;
42 for (;;) {
43 const { done, value } = await reader.read();
44 if (done) break;
45 total += value.length;
46 if (total > MAX_BYTES) {
47 await reader.cancel();
sentinelcrew outdated

HIGH: arrayBuffer() buffers the entire body before the cap is checked. Read res.body in chunks, count bytes, and cancel past MAX_BYTES.
48 return null;
49 }
50 chunks.push(value);
51 }
52 const out = new Uint8Array(total);
53 let at = 0;
54 for (const c of chunks) {
55 out.set(c, at);
56 at += c.length;
57 }
58 return out;
59 }
60
61 /** GET /openapi.json from the app listening on `port`; null when it serves none. */
62 export async function readSpec(port: number): Promise<Record<string, unknown> | null> {
63 try {
64 const res = await fetch(`http://127.0.0.1:${port}/openapi.json`, {
65 headers: { accept: "application/json" },
66 redirect: "manual",
67 signal: AbortSignal.timeout(WAIT_MS),
68 });
69 if (res.status !== 200) return null;
70 const bytes = await boundedBody(res);
71 if (bytes === null) return null;
72 const doc: unknown = JSON.parse(new TextDecoder().decode(bytes));
73 const paths = typeof doc === "object" && doc !== null ? (doc as Record<string, unknown>)["paths"] : undefined;
74 return typeof paths === "object" && paths !== null && !Array.isArray(paths) ? (doc as Record<string, unknown>) : null;
75 } catch {
76 return null;
77 }
78 }
79
80 /** Read the spec now and remember it; one read per app at a time. */
81 export async function refreshSpec(record: AppRecord, port: number | undefined): Promise<Record<string, unknown> | null> {
82 if (port === undefined) return null;
83 if (asking.has(record.id)) return knownSpec(record.id) ?? null;
84 asking.add(record.id);
85 try {
86 const doc = await readSpec(port);
87 rememberSpec(record.id, doc);
88 return doc;
89 } finally {
90 asking.delete(record.id);
91 }
92 }
93
94 /** Ask in the background when nothing fresh is known, so the next page has the answer. */
95 export function warmSpec(record: AppRecord, port: number | undefined): void {
96 if (knownSpec(record.id) === undefined) void refreshSpec(record, port);
97 }
src/contract/reference.ts added +172
@@ -0,0 +1,172 @@
1 /**
2 * An OpenAPI document read as a reference: operations grouped by area, each
3 * with its parameters, body and responses as plain rows. The forge's own
4 * document and an app's (which is data from outside, so every step here
5 * tolerates a missing or odd shape) go through the same reading; the page
6 * and the terminal render the result and never touch the document.
7 */
8
9 export const METHODS = ["get", "put", "post", "delete", "patch"] as const;
10
11 export interface Field {
12 readonly name: string;
13 readonly type: string;
14 readonly required: boolean;
15 }
16
17 export interface Param extends Field {
18 readonly in: string;
19 }
20
21 export interface Reply {
22 readonly status: string;
23 readonly description: string;
24 readonly fields: readonly Field[];
25 }
26
27 export interface Operation {
28 readonly id: string;
29 readonly method: string;
30 readonly path: string;
31 readonly summary: string;
32 /** Whether it asks for a credential. */
33 readonly auth: boolean;
34 readonly params: readonly Param[];
35 /** The JSON body's fields; absent when it takes none. */
36 readonly body?: readonly Field[];
37 readonly replies: readonly Reply[];
38 }
39
40 export interface Area {
41 readonly name: string;
42 readonly operations: readonly Operation[];
43 }
44
45 const MAX_OPERATIONS = 2000;
46 const MAX_FIELDS = 60;
47 const MAX_TEXT = 400;
48
49 type Obj = Record<string, unknown>;
50 const obj = (v: unknown): Obj | undefined => (typeof v === "object" && v !== null && !Array.isArray(v) ? (v as Obj) : undefined);
51 const text = (v: unknown): string => (typeof v === "string" ? v.slice(0, MAX_TEXT) : "");
52
53 /** A local `$ref` followed once; anything else is left as it is. */
54 function resolve(doc: Obj, node: Obj | undefined): Obj | undefined {
55 const ref = node?.["$ref"];
56 if (typeof ref !== "string" || !ref.startsWith("#/")) return node;
57 let at: unknown = doc;
58 for (const part of ref.slice(2).split("/")) at = obj(at)?.[part.replace(/~1/g, "/").replace(/~0/g, "~")];
59 return obj(at);
60 }
61
62 /** A schema's type as one short word: `integer`, `string[]`, `a | b`, `enum(x, y)`. */
63 function typeOf(doc: Obj, schema: Obj | undefined): string {
64 const s = resolve(doc, schema);
65 if (s === undefined) return "any";
66 if (Array.isArray(s["enum"])) return `enum(${s["enum"].slice(0, 8).map(String).join(", ")})`;
67 const t = s["type"];
68 if (t === "array") return `${typeOf(doc, obj(s["items"]))}[]`;
69 if (typeof t === "string") return t;
70 if (Array.isArray(t)) return t.map(String).join(" | ");
71 const of = s["oneOf"] ?? s["anyOf"];
72 if (Array.isArray(of)) return of.map((o) => typeOf(doc, obj(o))).join(" | ");
73 return s["properties"] !== undefined ? "object" : "any";
74 }
75
76 /** An object schema's properties as rows. */
77 function fieldsOf(doc: Obj, schema: Obj | undefined): Field[] {
78 const s = resolve(doc, schema);
79 const props = obj(s?.["properties"]);
80 if (props === undefined) return [];
81 const required = Array.isArray(s?.["required"]) ? s["required"].map(String) : [];
82 return Object.entries(props)
83 .slice(0, MAX_FIELDS)
84 .map(([name, p]) => ({ name: name.slice(0, 120), type: typeOf(doc, obj(p)), required: required.includes(name) }));
85 }
86
87 const jsonSchema = (doc: Obj, content: unknown): Obj | undefined => {
88 const c = obj(resolve(doc, obj(content)));
89 const json = obj(c?.["application/json"]) ?? Object.values(c ?? {}).map(obj).find((x) => x !== undefined);
90 return obj(json?.["schema"]);
91 };
92
93 /** The area a path belongs to: its tag, else the first word after `/api/v1`. */
94 export function areaOf(path: string, tag?: string): string {
95 if (tag !== undefined && tag !== "") return tag.slice(0, 60);
96 const words = path.split("/").filter((w) => w !== "" && !w.startsWith("{"));
97 const at = words[0] === "api" && /^v\d+$/.test(words[1] ?? "") ? 2 : 0;
98 return words[at] ?? "root";
99 }
100
101 function operation(doc: Obj, path: string, method: string, op: Obj, shared: unknown[]): Operation {
102 const params: Param[] = [...shared, ...(Array.isArray(op["parameters"]) ? op["parameters"] : [])]
103 .map((p) => resolve(doc, obj(p)))
104 .filter((p): p is Obj => p !== undefined && typeof p["name"] === "string")
105 .slice(0, MAX_FIELDS)
106 .map((p) => ({
107 name: text(p["name"]),
108 in: text(p["in"]) || "query",
109 type: typeOf(doc, obj(p["schema"])),
110 required: p["required"] === true || p["in"] === "path",
111 }));
112 const body = obj(resolve(doc, obj(op["requestBody"])));
113 const replies = Object.entries(obj(op["responses"]) ?? {})
114 .slice(0, 12)
115 .map(([status, r]) => {
116 const reply = resolve(doc, obj(r));
117 return { status: status.slice(0, 8), description: text(reply?.["description"]), fields: fieldsOf(doc, jsonSchema(doc, reply?.["content"])) };
118 });
119 return {
120 id: text(op["operationId"]) || `${method}${path}`,
121 method,
122 path,
123 summary: text(op["summary"]) || text(op["description"]),
124 auth: Array.isArray(op["security"]) ? op["security"].length > 0 : false,
125 params,
126 ...(body === undefined ? {} : { body: fieldsOf(doc, jsonSchema(doc, body["content"])) }),
127 replies,
128 };
129 }
130
131 /** Every operation of a document, grouped by area, areas and paths in order. */
132 export function reference(document: unknown): Area[] {
133 const doc = obj(document);
134 const paths = obj(doc?.["paths"]);
135 if (doc === undefined || paths === undefined) return [];
136 const areas = new Map<string, Operation[]>();
137 let count = 0;
138 for (const path of Object.keys(paths).sort()) {
139 const item = obj(paths[path]);
140 if (item === undefined || !path.startsWith("/")) continue;
141 const shared = Array.isArray(item["parameters"]) ? item["parameters"] : [];
142 for (const method of METHODS) {
143 const op = obj(item[method]);
144 if (op === undefined || count >= MAX_OPERATIONS) continue;
145 count++;
146 const tag = Array.isArray(op["tags"]) ? text(op["tags"][0]) : "";
147 const name = areaOf(path, tag);
148 const list = areas.get(name) ?? [];
149 list.push(operation(doc, path.slice(0, 300), method, op, shared));
150 areas.set(name, list);
151 }
152 }
153 return [...areas].sort(([a], [b]) => a.localeCompare(b)).map(([name, operations]) => ({ name, operations }));
154 }
155
156 /** The operations whose method, path or summary holds every word of the query. */
157 export function narrow(areas: readonly Area[], query: string): Area[] {
158 const words = query.toLowerCase().split(/\s+/).filter((w) => w !== "");
159 if (words.length === 0) return [...areas];
160 const out: Area[] = [];
161 for (const a of areas) {
162 const operations = a.operations.filter((o) => {
163 const hay = `${o.method} ${o.path} ${o.summary} ${o.id}`.toLowerCase();
164 return words.every((w) => hay.includes(w));
165 });
166 if (operations.length > 0) out.push({ name: a.name, operations });
167 }
168 return out;
169 }
170
171 /** Whether a GET can be sent as it is: no path parameter left to fill. */
172 export const runnable = (o: Operation): boolean => o.method === "get" && !o.path.includes("{");
src/http/server.ts +4
@@ -160,6 +160,7 @@ import {
160160 backupsTest,
161161 domainsMoved,
162162 } from "../web/pages/backups.ts";
163 import { apiPage, appApiPage } from "../web/pages/api.ts";
163164 import { changelogPage } from "../web/pages/changelog.ts";
164165 import { releasesPage, rollbackSubmit } from "../web/pages/releases.ts";
165166 import {
@@ -779,6 +780,7 @@ export function buildRouter(): Router<Handler> {
779780 router.add("GET", "/audit", auditPage);
780781 router.add("GET", "/releases", releasesPage);
781782 router.add("GET", "/changelog", changelogPage);
783 router.add("GET", "/:owner/:name/api", appApiPage);
782784 router.add("POST", "/releases/:sha/rollback", rollbackSubmit);
783785 router.add("GET", "/audit.csv", auditCsv);
784786 router.add("POST", "/audit/:owner/:name/probe", auditProbeSubmit);
@@ -1484,6 +1486,8 @@ export function buildApiRouter(): Router<Handler> {
14841486 apiRoutes(router);
14851487 feedRoutes(router);
14861488 checksRecheckRoutes(router);
1489 // The reference page for these routes lives at their root, so it answers here.
1490 router.add("GET", "/api", apiPage);
14871491 return router;
14881492 }
14891493
sentinelcrew outdated

Registering via API_ROOT to dodge the source-scan sweeps leaves /api outside their coverage. Teach the sweeps about it instead.
src/tui/forge.ts +8
@@ -77,6 +77,7 @@ import { grantable, READS_KEPT, recentReads } from "../core/telemetry.ts";
7777 import { noteRead, setTelemetryGrant, telemetryReader } from "../actions/telemetry.ts";
7878 import { queuesFor, systemFor } from "../system/telemetry.ts";
7979 import { live as liveMachine } from "../system/live.ts";
80 import { apiScreen } from "./screens/api.ts";
8081 import { changelogScreen } from "./screens/changelog.ts";
8182 import { releasesScreen } from "./screens/releases.ts";
8283 import { mayRead } from "../release/gate.ts";
@@ -384,6 +385,13 @@ export function forgeCommands(session: Session): (ctx: Ctx) => Command[] {
384385 hint: "each protection green, red or untested",
385386 run: () => ctx.push(resilienceScreen(session, ctx)),
386387 });
388 // The forge's API operations, as /api lists them; Enter runs a GET: for anyone.
389 commands.push({
390 id: "api",
391 title: "API",
392 hint: "the forge's endpoints; run a GET",
393 run: () => ctx.push(apiScreen(session, ctx)),
394 });
387395 // The last releases with their pulls, as /changelog shows them: for anyone.
388396 commands.push({
389397 id: "changelog",
src/tui/screens/api.ts added +139
@@ -0,0 +1,139 @@
1 /**
2 * The API reference, the terminal's /api: the forge's OpenAPI operations as
3 * rows (method, path, summary), `/` to narrow them, Enter to run a GET that
4 * needs no path parameter as the person at this terminal and read the status,
5 * time, Server-Timing and body. The request goes through the forge's own
6 * router, in this process, with a session made for the call and ended after.
7 */
8
9 import { narrow, reference, runnable, type Area, type Operation } from "../../contract/reference.ts";
10 import { openapiText } from "../../contract/served.ts";
11 import { createSession, destroySession } from "../../core/sessions.ts";
12 import { running } from "../../kernel/version.ts";
13 import type { Ctx, Screen } from "../app/app.ts";
14 import { binding } from "../app/keys.ts";
15 import type { Session } from "../session.ts";
16 import { fill, len, split } from "../ui/layout.ts";
17 import { style } from "../term/style.ts";
18 import { truncate } from "../term/width.ts";
19 import { emptyState } from "../ui/widgets.ts";
20 import { Filter, ListState, Scroll, drawLines, drawList, metaOf, type ListRow } from "./common.ts";
21
22 const MAX_SHOWN = 400;
23
24 /** Every operation of the forge's document, flat, in area order. */
25 export function apiOperations(areas: readonly Area[]): Operation[] {
26 return areas.flatMap((a) => a.operations);
27 }
28
29 export function apiRows(ops: readonly Operation[]): ListRow[] {
30 return ops.map((o) => ({
31 title: `${o.method.toUpperCase().padEnd(6)}${o.path}`,
32 meta: metaOf(o.summary, runnable(o) ? "Enter runs it" : ""),
33 }));
34 }
35
36 export interface Answer {
37 readonly status: number;
38 readonly ms: number;
39 readonly timing: string;
40 readonly body: string;
41 }
42
43 /** GET `path` through the forge's router as this session's person (a guest: anonymously). */
44 export async function runGet(f: Session, path: string): Promise<Answer> {
45 const { forgeHttp } = await import("../../http/server.ts");
46 const http = forgeHttp(f.app);
47 const made = f.user === undefined ? undefined : createSession(f.app.db, f.user.id);
48 const began = performance.now();
49 try {
50 const headers = made === undefined ? {} : { cookie: `forge_session=${made.token}` };
51 const base = f.app.config.url ?? "http://localhost";
52 let target = path;
53 let res = await http.handle(new Request(new URL(target, base).href, { headers }));
54 // A same-site redirect (the OpenAPI alias) is followed, twice at most.
55 for (let hops = 0; hops < 2 && res.status >= 300 && res.status < 400; hops++) {
56 const next = res.headers.get("location") ?? "";
57 if (!next.startsWith("/") || next.startsWith("//")) break;
58 target = next;
59 res = await http.handle(new Request(new URL(target, base).href, { headers }));
60 }
61 const text = await res.text();
62 let body = text;
63 try {
64 body = JSON.stringify(JSON.parse(text), null, 2);
65 } catch {
66 // not JSON: shown as it came
67 }
68 return { status: res.status, ms: Math.round(performance.now() - began), timing: res.headers.get("server-timing") ?? "", body };
69 } finally {
70 if (made !== undefined) destroySession(f.app.db, made.token);
71 http.close();
72 }
73 }
74
75 function answerScreen(ctx: Ctx, op: Operation, answer: Answer): Screen {
76 const scroll = new Scroll();
77 const head = `${answer.status} · ${answer.ms} ms${answer.timing === "" ? "" : ` · ${answer.timing}`}`;
78 const lines = [head, "", ...answer.body.slice(0, 40_000).split("\n")];
79 return {
80 title: `GET ${op.path}`,
81 keys: [binding(["j", "k"], "scroll"), binding(["esc", "q"], "back")],
82 render(area, s) {
83 const c = ctx.theme.c;
84 drawLines(s, area, c, scroll, lines.map((text, i) => [{ text, style: style(i === 0 ? c.fg : c.mutedFg, c.bg) }]));
85 },
86 key(e) {
87 if (e.key === "esc" || e.key === "q") ctx.pop();
88 else scroll.key(e, lines.length);
89 return true;
90 },
91 };
92 }
93
94 export function apiScreen(f: Session, ctx: Ctx): Screen {
95 const state = new ListState();
96 const filter = new Filter();
97 const all = apiOperations(reference(JSON.parse(openapiText(f.app.config.url ?? "http://localhost", running().version).text)));
98 const shown = (): Operation[] => apiOperations(narrow([{ name: "", operations: all }], filter.query)).slice(0, MAX_SHOWN);
99 let note = "";
100 return {
101 title: "API",
102 keys: [binding(["j", "k"], "move"), binding(["/"], "find"), binding(["enter"], "run a GET"), binding(["esc", "q"], "close")],
103 render(area, s) {
104 const c = ctx.theme.c;
105 const list = shown();
106 const [head, box, body] = split(area, "v", [len(1), len(filter.on ? 1 : 0), fill(1)]);
107 const count = filter.query === "" ? `${all.length} operations` : `${list.length} of ${all.length} operations match`;
108 s.text(head!.x, head!.y, truncate(note === "" ? count : `${count} · ${note}`, head!.w), style(c.mutedFg, c.bg), head!.x + head!.w);
109 ctx.setCursor(filter.on ? filter.render(s, box!, c, "find a path, method or word") : null);
110 if (list.length === 0) {
111 emptyState(s, body!, c, { title: "No operation matches", body: "Esc clears the word." });
112 return;
113 }
114 drawList(s, body!, c, state, apiRows(list));
115 },
116 key(e) {
117 if (filter.key(e)) {
118 state.clamp(shown().length);
119 return true;
120 }
121 if (e.key === "esc" && filter.query !== "") {
122 filter.query = "";
123 state.clamp(shown().length);
124 } else if (e.key === "esc" || e.key === "q") ctx.pop();
125 else if (e.key === "enter") {
126 const op = shown()[state.index];
127 if (op === undefined) return true;
128 if (!runnable(op)) note = "only a GET with no path parameter runs here";
129 else {
130 note = "";
131 void runGet(f, op.path).then((answer) => {
132 ctx.push(answerScreen(ctx, op, answer));
133 });
134 }
135 } else state.key(e, shown().length);
136 return true;
137 },
138 };
139 }
src/web/api_reference.ts added +160
@@ -0,0 +1,160 @@
1 /**
2 * The API reference as markup: operations by area, each a fold with its
3 * parameters, body and replies as tables, and a Try it form. All of it is
4 * server-rendered; the fold is a `<details>`, the search a GET form, and a
5 * GET with no path parameter is a plain GET form, so nothing needs script.
6 * `API_TRY_JS` only upgrades Try it to show the answer in place.
7 */
8
9 import { runnable, type Area, type Field, type Operation, type Param } from "../contract/reference.ts";
10 import { html, when, type Html } from "./html.ts";
11 import { badge, btn, card, empty, row } from "./ui.ts";
12
13 const fields = (rows: readonly Field[], heading: string): Html | null =>
14 rows.length === 0
15 ? null
16 : html`<h4>${heading}</h4>
17 <table>
18 ${rows.map(
19 (f) => html`<tr><td class="mono">${f.name}</td><td class="mono muted">${f.type}</td><td class="muted small">${f.required ? "required" : ""}</td></tr>`,
20 )}
21 </table>`;
22
23 const params = (list: readonly Param[]): Html | null =>
24 list.length === 0
25 ? null
26 : html`<h4>Parameters</h4>
27 <table>
28 ${list.map(
29 (p) => html`<tr><td class="mono">${p.name}</td><td class="muted small">${p.in}</td><td class="mono muted">${p.type}</td><td class="muted small">${p.required ? "required" : ""}</td></tr>`,
30 )}
31 </table>`;
32
33 /** Try it: inputs for what the operation takes, a token you pick, and where the answer shows. */
34 function tryIt(o: Operation, base: string): Html {
35 const inputs = o.params.filter((p) => p.in === "path" || p.in === "query");
36 const plainGet = runnable(o);
37 return html`<form class="try" method="get" action="${base}${o.path}" data-try data-method="${o.method.toUpperCase()}" data-path="${o.path}" data-base="${base}">
38 <div class="field-row">
39 ${inputs.map(
40 (p) => html`<label class="field"><span class="muted small">${p.name}${p.required ? " *" : ""}</span>
41 <input class="input" type="text" ${plainGet ? html`name="${p.name}"` : null} data-in="${p.in}" data-name="${p.name}" autocomplete="off" /></label>`,
42 )}
43 ${when(
44 o.method !== "get",
45 () => html`<label class="field"><span class="muted small">JSON body</span>
46 <textarea class="input mono" rows="4" data-body autocomplete="off" spellcheck="false">{}</textarea></label>`,
47 )}
48 ${when(
49 o.auth,
50 () => html`<label class="field"><span class="muted small">Token (blank: your session)</span>
51 <input class="input" type="password" data-token autocomplete="off" /></label>`,
52 )}
53 </div>
54 <div class="form-actions">
55 ${btn("Try it", { size: "sm", variant: "outline", ...(plainGet ? {} : { disabled: true }) })}
56 <span class="muted small" data-status role="status"></span>
57 </div>
58 <pre data-answer hidden></pre>
59 </form>`;
60 }
61
62 function operationRow(o: Operation, base: string, tryable: boolean): Html {
63 return row({
64 id: o.id,
65 lead: badge(o.method.toUpperCase(), { variant: "secondary" }),
66 title: html`<span class="mono">${o.path}</span>`,
67 meta: html`${o.summary}`,
68 trail: o.auth ? html`<span class="muted small">token</span>` : html``,
69 data: { method: o.method },
70 fold: {
71 id: `op-${o.id}`.replace(/[^A-Za-z0-9_-]/g, "-"),
72 open: false,
73 body: html`<div class="card-body">
74 ${params(o.params)}${o.body === undefined ? null : fields(o.body, "Body") ?? html`<h4>Body</h4><p class="muted small">JSON object.</p>`}
75 ${o.replies.map(
76 (r) => html`<h4>${r.status} <span class="muted small">${r.description}</span></h4>${fields(r.fields, "Fields")}`,
77 )}
78 ${tryable ? tryIt(o, base) : null}
79 </div>`,
80 },
81 });
82 }
83
84 export interface ReferenceView {
85 readonly areas: readonly Area[];
86 readonly total: number;
87 readonly query: string;
88 /** Where the search form sends its GET. */
89 readonly action: string;
90 /** Operations can be tried in place (same origin as the page). */
91 readonly tryable: boolean;
92 readonly base?: string;
93 }
94
95 /** The search box, the count, and each area's operations. */
96 export function referenceBody(v: ReferenceView): Html {
97 const shown = v.areas.reduce((n, a) => n + a.operations.length, 0);
98 return html`<form class="bar" method="get" action="${v.action}" role="search">
99 <input class="input" type="search" name="q" value="${v.query}" placeholder="Search path, method or summary" aria-label="Search the API" />
100 ${btn("Search", { size: "sm", variant: "outline" })}
101 ${when(v.query !== "", () => btn("Clear", { href: v.action, size: "sm", variant: "ghost" }))}
102 </form>
103 <p class="muted small">${v.query === "" ? `${v.total} operations in ${v.areas.length} areas` : `${shown} of ${v.total} operations match`}</p>
104 ${v.areas.length === 0
105 ? empty({ title: v.query === "" ? "No operations" : "No operation matches", note: v.query === "" ? "This API describes none." : "Try a shorter word." })
106 : v.areas.map(
107 (a) => html`<section id="area-${a.name.replace(/[^A-Za-z0-9_-]/g, "-")}">
108 <h2>${a.name} <span class="muted small">${a.operations.length}</span></h2>
109 ${card(html`<ul class="list">${a.operations.map((o) => operationRow(o, v.base ?? "", v.tryable))}</ul>`, { pad: false })}
110 </section>`,
111 )}`;
112 }
113
114 /**
115 * Upgrades each Try it form: sends the request as the session (or the token
116 * typed), and shows the status, time, Server-Timing and body in place. Static
117 * text only; the page's data arrives in data-* attributes.
118 */
119 export const API_TRY_JS = `
120 for (const form of document.querySelectorAll("form[data-try]")) {
121 const button = form.querySelector("button");
122 if (button) button.disabled = false;
123 form.addEventListener("submit", async (event) => {
124 event.preventDefault();
125 const method = form.dataset.method;
126 let path = form.dataset.path;
127 const query = new URLSearchParams();
128 for (const input of form.querySelectorAll("[data-in]")) {
129 const value = input.value.trim();
130 if (input.dataset.in === "path") path = path.replace("{" + input.dataset.name + "}", encodeURIComponent(value));
131 else if (value !== "") query.set(input.dataset.name, value);
132 }
133 if (/[{}]/.test(path)) return setStatus("fill in every path parameter");
134 if (method !== "GET" && !confirm(method + " " + path + " as you? It changes things.")) return;
135 const headers = {};
136 const token = form.querySelector("[data-token]");
137 if (token && token.value.trim() !== "") headers.authorization = "Bearer " + token.value.trim();
138 const init = { method, headers };
139 const body = form.querySelector("[data-body]");
140 if (method !== "GET" && body) { headers["content-type"] = "application/json"; init.body = body.value; }
141 const url = (form.dataset.base || "") + path + (query.size ? "?" + query : "");
142 const out = form.querySelector("[data-answer]");
143 const started = performance.now();
144 setStatus("…");
145 try {
146 const res = await fetch(url, init);
147 const text = await res.text();
148 let shown = text;
149 try { shown = JSON.stringify(JSON.parse(text), null, 2); } catch {}
150 const timing = res.headers.get("server-timing");
151 setStatus(res.status + " " + res.statusText + " · " + Math.round(performance.now() - started) + " ms" + (timing ? " · " + timing : ""));
152 out.textContent = shown.slice(0, 20000);
153 out.hidden = false;
154 } catch (e) {
155 setStatus("the request failed");
156 }
157 });
158 function setStatus(t) { form.querySelector("[data-status]").textContent = t; }
159 }
160 `;
src/web/chrome.ts +20 −3
@@ -21,7 +21,7 @@ import {
2121 getRepo,
2222 visibleForks,
2323 } from "../core/repos.ts";
24 import type { Repo } from "../core/repos.ts";
24 import { canEnterApp, type Repo } from "../core/repos.ts";
2525 import { parentUrl, shownTrail, sourceLabel } from "../core/provenance.ts";
2626 import type { Database } from "bun:sqlite";
2727 import type { Ctx } from "../http/context.ts";
@@ -39,6 +39,8 @@ import {
3939 type RepoScope,
4040 } from "./scope.ts";
4141 import { changesHref, tellsHref } from "./studio/nouns.ts";
42 import { knownSpec, warmSpec } from "../apps/api_spec.ts";
43 import { appsRuntime } from "../apps/runtime.ts";
4244 import { badge, btn, empty, tabs, type TabItem } from "./ui.ts";
4345
4446 // ---- loading ----
@@ -124,6 +126,7 @@ export type RepoTab =
124126 | "branches"
125127 | "app"
126128 | "runs"
129 | "api"
127130 | "settings"
128131 /** A page that belongs to no tab (waves): nothing is marked current. */
129132 | "none";
@@ -183,8 +186,9 @@ export function scopeHeader(
183186 ? undefined
184187 : forkParentFor(ctx.app.db, repo, ctx.auth.user);
185188
186 const isApp =
187 repo !== undefined && getAppByRepo(ctx.app.db, repo.id) !== undefined;
189 const record =
190 repo === undefined ? undefined : getAppByRepo(ctx.app.db, repo.id);
191 const isApp = record !== undefined;
188192 // An app speaks the studio's words (told, changes), a plain repository
189193 // git's (issues, pulls, commits, branches) — for every viewer (#603).
190194 const studio = isApp;
@@ -290,6 +294,19 @@ export function scopeHeader(
290294 current: active === "runs",
291295 shortcut: "g r",
292296 });
297 // An app that serves /openapi.json gets its API reference one tab over.
298 // The spec is read in the background and remembered, so a header never waits.
299 if (repo !== undefined && record !== undefined && canEnterApp(ctx.app.db, repo, record, ctx.auth.user)) {
300 const port = appsRuntime(ctx.app)?.envStatus(record, "production")?.port;
301 warmSpec(record, port);
302 if (knownSpec(record.id) != null)
303 items.push({
304 label: "API",
305 href: `${base}/api`,
306 icon: "command",
307 current: active === "api",
308 });
sentinelcrew

MEDIUM: the tab shows for anyone seeing the header, but /api requires canEnterApp. Gate the tab on canEnterApp too.
309 }
293310 if (ctx.auth.user !== undefined && sshCommand(ctx.app) !== undefined)
294311 items.push({
295312 label: "Terminal",
src/web/layout.ts +1
@@ -523,6 +523,7 @@ function footer(ctx: Ctx): Html {
523523 >`}
524524 <span class="spacer"></span>
525525 <a href="/changelog">Changelog</a>
526 <a href="/api">API</a>
526527 ${ctx.auth.user !== undefined && ctx.auth.user.kind !== "org"
527528 ? html`<a class="btn" data-variant="link" data-size="sm" href="/#composer">Tell the crew</a>`
528529 : ""}
src/web/pages/api.ts added +83
@@ -0,0 +1,83 @@
1 /**
2 * The API reference: `/api` for the forge's own OpenAPI document, and
3 * `/:owner/:name/api` for the one an app serves at `/openapi.json`. Both are
4 * the same reading (`contract/reference.ts`) drawn the same way
5 * (`web/api_reference.ts`), server-side, with the search as a plain GET.
6 */
7
8 import { knownSpec, refreshSpec } from "../../apps/api_spec.ts";
9 import { appsRuntime } from "../../apps/runtime.ts";
10 import { appView } from "../../actions/apps.ts";
11 import { narrow, reference, type Area } from "../../contract/reference.ts";
12 import { openapiText } from "../../contract/served.ts";
13 import { notePhase, takeSessionTime, timePhase } from "../../core/route_timing.ts";
14 import { canEnterApp } from "../../core/repos.ts";
15 import type { Ctx } from "../../http/context.ts";
16 import { noRoute, prefersJson } from "../../http/respond.ts";
17 import { running } from "../../kernel/version.ts";
18 import { API_TRY_JS, referenceBody } from "../api_reference.ts";
19 import { loadRepo, notFoundPage, repoBase, repoHeader } from "../chrome.ts";
20 import { html } from "../html.ts";
21 import { page } from "../layout.ts";
22
23 const MAX_QUERY = 100;
24 const asked = (ctx: Ctx): string => (ctx.url.searchParams.get("q") ?? "").trim().slice(0, MAX_QUERY);
25 const count = (areas: readonly Area[]): number => areas.reduce((n, a) => n + a.operations.length, 0);
26
27 /** The forge's reading, once per build of the document. */
28 const forgeAreas = (() => {
29 let tag = "";
30 let areas: Area[] = [];
31 return (origin: string): Area[] => {
32 const hit = openapiText(origin, running().version);
33 if (hit.etag !== tag) {
34 areas = reference(JSON.parse(hit.text));
35 tag = hit.etag;
36 }
37 return areas;
38 };
39 })();
40
41 // GET /api
42 export function apiPage(ctx: Ctx): Response {
43 // `/api` is the API's root: a caller that asks for JSON still gets the API's own miss.
44 if (prefersJson(ctx.request)) return noRoute(ctx.request.method, ctx.url.pathname);
45 notePhase(ctx.request, "session", takeSessionTime(ctx.request));
46 const query = asked(ctx);
47 const all = timePhase(ctx.request, "spec", () => forgeAreas(ctx.url.origin));
48 const rendering = performance.now();
49 const body = html`<div class="page-head"><h1>API</h1></div>
50 <p class="muted">
51 Every route the forge names to an agent. <a href="/contract/openapi.json">The OpenAPI document</a>, as JSON.
52 Try it sends the request as you; paste a token to send it as that token instead.
53 </p>
54 ${referenceBody({ areas: narrow(all, query), total: count(all), query, action: "/api", tryable: true })}`;
55 const response = page({ ctx, title: "API", body, scriptJs: API_TRY_JS });
56 notePhase(ctx.request, "render", performance.now() - rendering);
57 return response;
58 }
59
60 // GET /:owner/:name/api
61 export async function appApiPage(ctx: Ctx): Promise<Response> {
62 notePhase(ctx.request, "session", takeSessionTime(ctx.request));
63 const repo = loadRepo(ctx);
64 const view = repo === undefined ? undefined : appView(ctx.app, ctx.auth.user, repo);
65 if (repo === undefined || view === undefined || !view.ok || !canEnterApp(ctx.app.db, repo, view.value.app, ctx.auth.user))
66 return timePhase(ctx.request, "render", () => notFoundPage(ctx));
67 const port = appsRuntime(ctx.app)?.envStatus(view.value.app, "production")?.port;
68 const known = knownSpec(view.value.app.id);
69 const doc = known !== undefined ? known : await refreshSpec(view.value.app, port);
70 const rendering = performance.now();
71 const query = asked(ctx);
72 const all = doc === null ? [] : reference(doc);
73 const base = repoBase(repo);
74 const body = html`${repoHeader(ctx, repo, "api", { compact: true, ownTitle: true })}
75 <div class="page-head"><h1>API</h1></div>
76 ${doc === null
77 ? html`<p class="muted">This app serves no <span class="mono">/openapi.json</span> from its production environment right now.</p>`
78 : html`<p class="muted">What the app describes at <span class="mono">/openapi.json</span>, read live.</p>
79 ${referenceBody({ areas: narrow(all, query), total: count(all), query, action: `${base}/api`, tryable: false })}`}`;
80 const response = page({ ctx, title: `API · ${repo.ownerName}/${repo.name}`, body });
81 notePhase(ctx.request, "render", performance.now() - rendering);
82 return response;
83 }
test/api-reference.test.ts added +146
@@ -0,0 +1,146 @@
1 import { afterAll, beforeAll, describe, expect, test } from "bun:test";
2 import { mkdtemp, rm } from "node:fs/promises";
3 import { tmpdir } from "node:os";
4 import { join } from "node:path";
5 import { newApp } from "../src/actions/apps.ts";
6 import { readSpec, rememberSpec } from "../src/apps/api_spec.ts";
7 import { createApp, type App } from "../src/app.ts";
8 import { narrow, reference, runnable } from "../src/contract/reference.ts";
9 import { ROUTES } from "../src/contract/openapi.ts";
10 import { createUser } from "../src/core/users.ts";
11 import { serve, type Servers } from "../src/http/server.ts";
12 import { createForgeApp } from "../src/tui/forge.ts";
13 import { apiScreen, runGet } from "../src/tui/screens/api.ts";
14 import { headlessTerminal } from "../src/tui/term/terminal.ts";
15 import { quietLogs } from "./helpers/globals.ts";
16 import { settleApp } from "./helpers/tui.ts";
17
18 quietLogs();
19
20 describe("the API reference", () => {
21 let dir = "";
22 let app: App;
23 let servers: Servers;
24 let cookie = "";
25 let appId = "";
26
27 const get = (path: string, headers: Record<string, string> = {}) => fetch(`${servers.url}${path}`, { headers, redirect: "manual" });
28
29 beforeAll(async () => {
30 dir = await mkdtemp(join(tmpdir(), "forge-api-ref-"));
31 app = createApp({ dataDir: dir, port: 0 });
32 servers = await serve(app);
33 const made = await createUser(app.db, { origin: "seed", trusted: true, username: "alice", email: "[email protected]", password: "a-good-passphrase", adminIfFirst: true });
34 if (!made.ok) throw new Error(made.error);
35 const signed = await fetch(`${servers.url}/signin`, {
36 method: "POST",
37 body: new URLSearchParams({ username: "alice", password: "a-good-passphrase" }),
38 redirect: "manual",
39 });
40 cookie = (signed.headers.get("set-cookie") ?? "").split(";")[0]!;
41 const created = await newApp(app, made.value, { name: "notes", visibility: "public" } as never);
42 if (created.ok) appId = created.value.app.id;
43 }, 60_000);
44
45 afterAll(async () => {
46 servers.stop();
47 app.close();
48 await rm(dir, { recursive: true, force: true });
49 });
50
51 test("every route in the contract appears on /api, with no script and no CDN", async () => {
52 const res = await get("/api");
53 expect(res.status).toBe(200);
54 const body = await res.text();
55 for (const r of ROUTES) expect(body).toContain(`id="${r.method}${r.path.replace(/[^A-Za-z0-9]+(.)?/g, (_, c: string | undefined) => (c ?? "").toUpperCase())}"`);
56 expect(body).toContain("<details");
57 expect(body).not.toMatch(/https?:\/\/(?!localhost|127\.0\.0\.1)[a-z.]*cdn/i);
58 });
59
60 test("the search narrows by a word and says how many matched", async () => {
61 const body = await (await get("/api?q=openapi")).text();
62 expect(body).toContain("match");
63 expect(body).toContain("/api/v1/openapi.json");
64 expect(body).not.toContain("/api/v1/repos/{owner}/{name}/issues/{number}/comments");
65 const none = await (await get("/api?q=zzzzqqq")).text();
66 expect(none).toContain("No operation matches");
67 });
68
69 test("an input is escaped in the search box", async () => {
70 const body = await (await get(`/api?q=${encodeURIComponent('"><script>x</script>')}`)).text();
71 expect(body).not.toContain("<script>x</script>");
72 });
73
74 test("Try it on a GET answers 200 for a signed-in person, with Server-Timing", async () => {
75 const page = await (await get("/api", { cookie })).text();
76 expect(page).toContain("data-try");
77 const route = ROUTES.find((r) => r.method === "get" && !r.path.includes("{") && r.path === "/api/v1/user");
78 expect(route).toBeDefined();
79 const res = await get("/api/v1/user", { cookie });
80 expect(res.status).toBe(200);
81 expect(res.headers.get("server-timing")).not.toBeNull();
82 });
83
84 test("an app that serves /openapi.json gets an API tab and a page that renders it", async () => {
85 expect(appId).not.toBe("");
86 const before = await (await get("/alice/notes/api", { cookie })).text();
87 expect(before).toContain("serves no");
88 rememberSpec(appId, {
89 openapi: "3.1.0",
90 paths: { "/notes": { get: { summary: "List <notes>", parameters: [{ name: "q", in: "query", schema: { type: "string" } }] } } },
91 });
92 const home = await (await get("/alice/notes/api", { cookie })).text();
93 expect(home).toContain('href="/alice/notes/api"');
94 expect(home).toContain("List &lt;notes&gt;");
95 expect(home).toContain("/notes");
96 const studio = await (await get("/alice/notes", { cookie })).text();
97 expect(studio).toContain('href="/alice/notes/api"');
98 });
99
100 test("the reading groups by area and tolerates a hostile document", () => {
101 const areas = reference({ paths: { "/api/v1/repos/{o}": { get: { summary: "x" } }, "/health": { post: { tags: ["ops"], requestBody: { content: { "application/json": { schema: { $ref: "#/nope" } } } } } }, "nonsense": 4 } });
102 expect(areas.map((a) => a.name)).toEqual(["ops", "repos"]);
103 expect(reference("junk")).toEqual([]);
104 expect(narrow(areas, "repos").length).toBe(1);
105 expect(runnable(areas[1]!.operations[0]!)).toBe(false);
106 });
107
108 test("the terminal's API screen lists endpoints and runs a GET", async () => {
109 const terminal = headlessTerminal(120, 30);
110 const identity = { sshUsername: "git", remoteAddress: "127.0.0.1", fingerprint: "SHA256:test" };
111 const { app: tui, session } = createForgeApp(app, identity, { terminal, exitHooks: false });
112 tui.start();
113 tui.ctx.push(apiScreen(session, tui.ctx));
114 await settleApp(tui);
115 expect(tui.screen.toText()).toContain("operations");
116 const answer = await runGet(session, "/api/v1/openapi.json");
117 expect(answer.status).toBe(200);
118 expect(answer.body).toContain('"openapi"');
119 });
120
121 test("readSpec stops reading a body past the cap instead of buffering it", async () => {
122 let sent = 0;
123 const big = Bun.serve({
124 port: 0,
125 fetch: () =>
126 new Response(
127 new ReadableStream({
128 async pull(c) {
129 await Bun.sleep(1);
130 sent += 64 * 1024;
131 c.enqueue(new Uint8Array(64 * 1024));
132 },
133 }),
134 ),
135 });
136 const ok = Bun.serve({ port: 0, fetch: () => Response.json({ paths: { "/x": {} } }) });
137 try {
138 expect(await readSpec(big.port!)).toBeNull();
139 expect(sent).toBeLessThan(8 * 1024 * 1024);
140 expect(await readSpec(ok.port!)).toEqual({ paths: { "/x": {} } });
141 } finally {
142 big.stop(true);
143 ok.stop(true);
144 }
145 });
146 });
test/server-timing-sweep.test.ts +3 −3
@@ -12,7 +12,7 @@ import { join } from "node:path";
1212 import { createRepo } from "../src/actions/repos.ts";
1313 import { createApp, type App } from "../src/app.ts";
1414 import { createUser } from "../src/core/users.ts";
15 import { buildRouter, serve, type Servers } from "../src/http/server.ts";
15 import { buildApiRouter, buildRouter, serve, type Servers } from "../src/http/server.ts";
1616 import { proveAll } from "./helpers/verified.ts";
1717 import { quietLogs } from "./helpers/globals.ts";
1818
@@ -58,7 +58,7 @@ function registeredGets(): Set<string> {
5858
5959 /** Parameter-free GET patterns, plus `/:owner/:name/<literal>…` ones. */
6060 function sweptPatterns(): string[] {
61 const registered = new Set(buildRouter().patterns());
61 const registered = new Set([...buildRouter().patterns(), ...buildApiRouter().patterns()]);
6262 return [...registeredGets()]
6363 .filter((p) => registered.has(p) && !NOT_PAGES.has(p) && !p.startsWith("/assets/"))
6464 .filter((p) => !p.replace(/^\/:owner\/:name/, "").includes(":") && !p.includes("*"))
@@ -144,6 +144,6 @@ test("the sweep reads wrapped router.add registrations, so a reformat cannot shr
144144 const found = registeredGets();
145145 expect(found.has("/origins/:id/r/:owner/:name/commit/:oid")).toBe(true); // wrapped over four lines
146146 expect(found.has("/robots.txt")).toBe(true); // one line
147 const registered = new Set(buildRouter().patterns());
147 const registered = new Set([...buildRouter().patterns(), ...buildApiRouter().patterns()]);
148148 expect([...found].filter((p) => !registered.has(p))).toEqual([]);
149149 });

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs