trevato/forge

Public: anyone can read this; the crew can change it.

More
Forks1g fMapg aRuns12g rEventsFlows

Config files open as a form: YAML and TOML, comments kept #8928

Merged masoncrew opened this in answer to trevatoowner

Check · Review · Land · Live done — Landed and live in production.

ConversationFiles
Merged

mason/forge:mason/7337-config-files-open-as-a-form-yaml-and-tom → main 4 commits fork

  • Merged. mason/7337-config-files-open-as-a-form-yaml-and-tom was merged into main as e484fad.

Review

Sign in to review this change.
20 files changed +14379 −54
docs/features/config-form.md +79 −6
@@ -1,13 +1,13 @@
11 # Config files as a form
22
3 A `.json` file in a repository opens as a form (#7336, part of #7329).
3 A `.json`, `.yml`/`.yaml` or `.toml` file in a repository opens as a form (#7336, #7337, part of #7329).
44
55 ## Reading
66
77 The blob page's **Rendered** view draws the file's values as read-only
88 fields. **Source**, **Raw** and **Download** are where they always are.
9 A file that is not valid JSON says why, with the line and column, above
10 its text. YAML, TOML and other config files keep their source view.
9 A file that is not valid says why, with the line and column, above
10 its text. Other config files (INI, `.env`) keep their source view.
1111
1212 ## Editing
1313
@@ -33,10 +33,72 @@ signed in. A private repository's file is a 404 to anyone who cannot read it.
3333 1. `$schema` in the file, if it is a path. A path that starts with `/`
3434 is read from the repository root; any other path is read from the
3535 file's folder. A URL is ignored.
36 2. A sibling `<name>.schema.json`.
36 2. A sibling `<name>.schema.json` (`app.yml` and `app.toml` look for `app.schema.json`).
3737 3. `.forge/schemas.json`, an object that maps globs to schema paths, for
3838 example `{ "config/**/*.json": "schemas/config.json" }`.
3939
40 ## YAML and TOML
41
42 Both read into the same form as JSON and use the same schema lookup. The
43 difference is the save: they are patched, not rewritten.
44
45 - **YAML** goes through the `yaml` package's Document API. Comments, anchors,
46 quoting, key order and every line you did not change are written as they
47 were. A file you did not change is returned byte for byte. A file of several
48 documents (`---`) is one fieldset per document ("Document 1", …), and only
49 a document you changed is rewritten. An alias shows as the value it points
50 at; a change that only an alias holds is refused, with the reason, rather
51 than guessed. A key that is not text, or an alias that fans out to over
52 2,000 values, sends the file to its source view.
53 - **TOML** goes through `@decimalturn/toml-patch`, which keeps comments and
54 layout and rewrites only the values that differ. Dates and times show as the
55 text they were written as. TOML has no null, so a form that would write one
56 is refused. A table is a fieldset and an array of tables is a list of cards.
57 - After either writes, the result is read back and must equal what the form
58 held; if it does not, nothing is committed.
59 - One limit: YAML sequences are indented the way most of the file does it,
60 so a file that mixes both styles has its changed document re-indented.
61
62 ### The two vendored libraries
63
64 Neither is a dependency in `package.json`: each is one bundled file under
65 `src/web/render/vendor/`, with its licence beside it, run on the server only
66 (no script is sent to the page).
67
68 | Bundle | Pinned | Licence | Why |
69 | --- | --- | --- | --- |
70 | `yaml.mjs` | yaml 2.9.1 (eemeli/yaml) | ISC, `YAML-LICENSE` | Its Document API keeps comments, anchors, quoting and key order, which a parse-and-dump cannot. Writing a YAML parser that does the same is far larger and riskier. |
71 | `toml-patch.mjs` | toml-patch 3.3.2 (@decimalturn/toml-patch) | MIT, `TOML-PATCH-LICENSE` | `patch` rewrites only the values that differ, so TOML comments and layout survive. |
72
73 To regenerate (or bump) one, in a scratch directory outside the repository
74 (this reproduces both files byte for byte; `bun build` writes `// @bun` as
75 its first line, which the header replaces):
76
77 ```
78 bun add [email protected] @decimalturn/[email protected]
79 ( echo "// yaml 2.9.1 (eemeli/yaml, ISC), bundled from node_modules/yaml/browser; licence in YAML-LICENSE"
80 bun build node_modules/yaml/browser/index.js --format=esm --target=bun | tail -n +2 ) > yaml.mjs
81 ( echo "// @decimalturn/toml-patch 3.3.2 (MIT), bundled from its dist; licence in TOML-PATCH-LICENSE"
82 bun build node_modules/@decimalturn/toml-patch/dist/toml-patch.js --format=esm --target=bun | tail -n +2 ) > toml-patch.mjs
83 cp node_modules/yaml/LICENSE YAML-LICENSE
84 cp node_modules/@decimalturn/toml-patch/LICENSE TOML-PATCH-LICENSE
85 ```
86
87 Copy the four files over their namesakes in `vendor/`. Each bundle's SHA-256
88 is pinned here, and `test/config-yaml-toml.test.ts` fails when a bundle, its
89 header or this page disagree:
90
91 - `yaml.mjs` sha256 `409e0ae0ebbe4017070541566da9a5ae666f0b4990ce9d8aa20c09c92d277454`
92 - `toml-patch.mjs` sha256 `b675b1478a5c3e773347425fd476455367b950406f7c2490f2f2c93400b67c3f`
93
94 On a bump, change the version in the commands, the headers and the table,
95 and the two hashes.
96
97 Hostile input is bounded before it is walked: a file over 256 KB is never
98 parsed (page and save both check the size first), YAML aliases and merge
99 keys share a 2,000-value budget, and nesting deeper than 64 is refused in
100 YAML and TOML alike.
101
40102 ## Saving
41103
42104 **Review** shows the change as a diff, under a commit message.
@@ -48,7 +110,7 @@ signed in. A private repository's file is a 404 to anyone who cannot read it.
48110 branch goes in the repository itself. A reader's goes on their fork; the
49111 forge makes the fork if they have none, and the usual fork rules apply
50112 (`actions/propose_edit.ts`).
51 - The file is written back with its own key order, indent (spaces or tabs),
113 - A JSON file is written back with its own key order, indent (spaces or tabs),
52114 line endings and final newline. Numbers keep their spelling.
53115 - A save that changes nothing writes nothing.
54116 - If the file changed on the branch after you opened it, the save stops
@@ -71,7 +133,18 @@ file, then `c`:
71133 opens a pull back. This is the same `proposeEdit` the web uses, with the
72134 same refusals in the same words. The status line says "proposes from your
73135 fork".
74 - Before any commit or proposal, each JSON file in the change is checked
136 - Before any commit or proposal, each JSON, YAML or TOML file in the change is checked
75137 against its schema (`schemaRefusal`, found the same three ways). A file
76138 that breaks it is refused in the dialog, naming the field and the schema,
77139 and nothing is written.
140 - There is no form screen in the terminal yet, for JSON, YAML or TOML: values
141 are edited as text. #8934 adds one.
142
143 ## Try it
144
145 Every preview's sample repository (`/preview-admin/sample`) holds
146 `service.yml` and `service.toml` (`src/spawn/preview-config.ts`, seeded by
147 `spawn/preview-seed.ts`, also into a sample seeded before they existed).
148 They have comments, an anchor and a merge key (YAML), and a date and an
149 array of tables (TOML). Open either on the blob page, press **Edit as
150 form**, change one value and **Review**: the diff touches only that line.
src/spawn/preview-config.ts added +44
@@ -0,0 +1,44 @@
1 /**
2 * Config files for a preview's sample repository (#7337): a YAML and a TOML
3 * file with comments, an anchor and merge key, a date and an array of
4 * tables, so the config form (docs/features/config-form.md) can be tried
5 * on any preview without bringing a file of one's own.
6 */
7
8 export const PREVIEW_CONFIG_FILES: readonly { readonly path: string; readonly content: string }[] = [
9 {
10 path: "service.yml",
11 content: `# A service's settings, to try the config form on: press Edit as form.
12 name: forge-demo # shown on the home page
13 defaults: &defaults
14 retries: 3
15 verbose: false
16 web:
17 <<: *defaults
18 port: 8080
19 host: "0.0.0.0"
20 tags:
21 - alpha
22 - beta # second
23 `,
24 },
25 {
26 path: "service.toml",
27 content: `# A service's settings, to try the config form on: press Edit as form.
28 name = "forge-demo" # shown on the home page
29
30 [web]
31 port = 8080
32 host = "0.0.0.0"
33 started = 2026-11-02T09:30:00Z
34
35 [[servers]]
36 name = "one"
37 weight = 1
38
39 [[servers]]
40 name = "two" # the spare
41 weight = 0
42 `,
43 },
44 ];
src/spawn/preview-seed.ts +26 −4
@@ -16,7 +16,7 @@
1616
1717 import type { App } from "../app.ts";
1818 import type { Issue } from "../core/issues.ts";
19 import type { Repo } from "../core/repos.ts";
19 import { getRepo, type Repo } from "../core/repos.ts";
2020 import { pinPanel } from "../actions/board-pins.ts";
2121 import { writeBoard } from "../actions/boards.ts";
2222 import { backBet } from "../actions/bets.ts";
@@ -33,6 +33,7 @@ import { recordUsage } from "../core/usage.ts";
3333 import { getUserSetting, setUserSetting } from "../core/settings.ts";
3434 import { createUser, getUserByName, type User } from "../core/users.ts";
3535 import { err, ok, type Result } from "../kernel/result.ts";
36 import { PREVIEW_CONFIG_FILES } from "./preview-config.ts";
3637
3738 /** The preview-only site admin's name. */
3839 export const PREVIEW_ADMIN = "preview-admin";
@@ -50,6 +51,7 @@ export const PREVIEW_WAVE_HEADLINE = "The sample says hello";
5051 /** The board whose panels are pinned to home. */
5152 const PREVIEW_BOARD = "preview";
5253 const SEEDED = "preview_seeded";
54 const CONFIG_SEEDED = "preview_config_seeded";
5355
5456 type Env = Record<string, string | undefined>;
5557
@@ -89,12 +91,32 @@ export async function seedPreview(
8991 }
9092 if (!admin.isAdmin || admin.kind === "agent")
9193 return err("the preview admin's name is someone else's");
92 if (getUserSetting(app.db, admin.id, SEEDED) !== undefined)
93 return ok({ seeded: false });
94 if (getUserSetting(app.db, admin.id, SEEDED) !== undefined) {
95 const files = await seedConfigFiles(app, admin);
96 return files.ok ? ok({ seeded: false }) : files;
97 }
9498 const world = await seedWorld(app, admin);
9599 if (!world.ok) return world;
96100 setUserSetting(app.db, admin.id, SEEDED, new Date().toISOString());
97 return ok({ seeded: true });
101 const files = await seedConfigFiles(app, admin);
102 return files.ok ? ok({ seeded: true }) : files;
103 }
104
105 /**
106 * The config files the form is tried on, once: a sample seeded before they
107 * existed gets them on its next boot, in a commit of their own.
108 */
109 async function seedConfigFiles(app: App, admin: User): Promise<Result<void, string>> {
110 if (getUserSetting(app.db, admin.id, CONFIG_SEEDED) !== undefined) return ok(undefined);
111 const repo = getRepo(app.db, admin.username, PREVIEW_REPO);
112 if (repo === undefined) return err("the sample repository is gone");
113 const added = await commitToBranch(app, admin, repo, {
114 message: "Config files to try the form on",
115 files: [...PREVIEW_CONFIG_FILES],
116 });
117 if (!added.ok) return err(`the config files: ${JSON.stringify(added.error)}`);
118 setUserSetting(app.db, admin.id, CONFIG_SEEDED, new Date().toISOString());
119 return ok(undefined);
98120 }
99121
100122 const panel = (title: string, source: string, groupBy: string) => ({
src/web/pages/config.ts +30 −20
@@ -1,5 +1,5 @@
11 /**
2 * Editing a JSON config file as a form (#7336).
2 * Editing a JSON, YAML or TOML config file as a form (#7336, #7337).
33 *
44 * GET /:owner/:name/config/:ref/*path — the file's values as fields, typed
55 * by its schema when one is found (`render/json-schema.ts`), else by
@@ -12,11 +12,12 @@
1212 * proposes it: one commit on a new branch and a pull
1313 * (`actions/propose_edit.ts`).
1414 *
15 * The file is written back in its own key order and indent, and a save
16 * that changes nothing writes nothing. Everything works as a plain form
15 * The file is written back in its own key order and indent (YAML and TOML
16 * keep their comments: `render/config-format.ts`), and a save that changes
17 * nothing writes nothing. Everything works as a plain form
1718 * POST; a browser with HTML validation checks a schema's bounds inline.
18 * A file that is not valid JSON goes back to its source view, with the
19 * error and its line.
19 * A file that is not valid goes back to its source view, with the error and
20 * its line.
2021 */
2122
2223 import { editFiles } from "../../actions/contents.ts";
@@ -28,16 +29,9 @@ import { loadRepo, notFoundPage, pageHead, signInFirst } from "../chrome.ts";
2829 import { field as formValue, readForm, type Form } from "../form.ts";
2930 import { html, when, type Html } from "../html.ts";
3031 import { flashTo, page } from "../layout.ts";
31 import { isFormJson } from "../render/config.ts";
32 import { isFormConfig, languageOf, parseConfig, writeConfig } from "../render/config-format.ts";
3233 import { applyOp, configFields, countFields, MAX_FIELDS, readValues } from "../render/config-form.ts";
33 import {
34 detectStyle,
35 lineDiff,
36 parseJson,
37 sameDoc,
38 stringify,
39 type Json,
40 } from "../render/json-doc.ts";
34 import { lineDiff, parseJson, sameDoc, stringify, type Json } from "../render/json-doc.ts";
4135 import { findSchema, validate, type FoundSchema } from "../render/json-schema.ts";
4236 import { localScope, type RepoScope } from "../scope.ts";
4337 import { btn, field, icon } from "../ui.ts";
@@ -70,7 +64,7 @@ function configContext(ctx: Ctx, ref: string, path: string): RepoScope | Respons
7064 const repo = loadRepo(ctx);
7165 if (!repo) return notFoundPage(ctx);
7266 if (!ctx.auth.user) return signInFirst(ctx);
73 if (!isSafeRef(ref) || !isSafeTreePath(path) || !isFormJson(path)) return notFoundPage(ctx);
67 if (!isSafeRef(ref) || !isSafeTreePath(path) || !isFormConfig(path)) return notFoundPage(ctx);
7468 const scope = localScope(ctx, repo);
7569 // A mirror you cannot push to proposes through its origin: the source editor's card.
7670 if (scope.propose !== undefined)
@@ -210,10 +204,10 @@ export async function configPage(ctx: Ctx): Promise<Response> {
210204 if (blob.value.kind !== "text" || blob.value.size > MAX_FORM_BYTES)
211205 return tooBig("This file is too large to edit as a form. Edit it as text, or with git.");
212206 const text = new TextDecoder().decode(blob.value.body);
213 const parsed = parseJson(text);
207 const parsed = parseConfig(path, text);
214208 if (!parsed.ok) {
215209 const e = parsed.error;
216 return tooBig(`Not valid JSON: ${e.message} at line ${e.line}, column ${e.column}. Fix it in the source first.`);
210 return tooBig(`Not valid ${languageOf(path)}: ${e.message} at line ${e.line}, column ${e.column}. Fix it in the source first.`);
217211 }
218212 if (countFields(parsed.value) > MAX_FIELDS)
219213 return tooBig(`This file has more than ${MAX_FIELDS.toLocaleString()} values, too many for a form. Edit it as text.`);
@@ -257,7 +251,12 @@ export async function configSubmit(ctx: Ctx): Promise<Response> {
257251 if (read === undefined) return flashTo(ctx, blobHref(scope, ref, path), { tone: "danger", message: "That form could not be read. Open the file again." });
258252 const current = await readText(scope, ref, path, MAX_FORM_BYTES);
259253 if (current === undefined) return notFoundPage(ctx);
260 const original = parseJson(current);
254 const original = parseConfig(path, current);
255 if (!original.ok)
256 return flashTo(ctx, blobHref(scope, ref, path, "?view=source"), {
257 tone: "danger",
258 message: `${path} is no longer valid ${languageOf(path)} on ${ref}. Fix it in the source first.`,
259 });
261260 const found = await schemaFor(scope, ref, path, read.doc);
262261 let state: FormState = { ref, path, base: formValue(form, "base"), doc: read.doc, found };
263262
@@ -285,10 +284,21 @@ export async function configSubmit(ctx: Ctx): Promise<Response> {
285284 flash: { tone: "danger", message: `${path} changed on ${ref} since you opened it. Check your values, then review again.` },
286285 });
287286 }
288 if (original.ok && sameDoc(original.value, read.doc))
287 if (sameDoc(original.value, read.doc))
289288 return formPage(ctx, scope, { ...state, flash: { tone: "success", message: "Nothing changed, so there is nothing to save." } });
290289
291 const after = stringify(read.doc, detectStyle(current));
290 const written = writeConfig(path, current, original.value, read.doc);
291 // What is written must read back as exactly the form's document.
292 const check = written.ok ? parseConfig(path, written.text) : undefined;
293 if (!written.ok || check === undefined || !check.ok || !sameDoc(check.value, read.doc))
294 return formPage(ctx, scope, {
295 ...state,
296 flash: {
297 tone: "danger",
298 message: written.ok ? "That change could not be written back safely. Edit the file as text." : written.message,
299 },
300 });
301 const after = written.text;
292302 if (step !== "save") return reviewPage(ctx, scope, state, { before: current, after });
293303
294304 const subject = formValue(form, "message").trim();
src/web/render/config-diff.ts added +66
@@ -0,0 +1,66 @@
1 /**
2 * What changed between two config documents, as edits at paths. A format
3 * that keeps comments (YAML, TOML) applies these to its own parsed document
4 * instead of writing the whole file again, so a line nobody touched stays
5 * byte for byte as it was.
6 *
7 * Key order is not an edit: the form never reorders keys, and a format that
8 * keeps the file's order keeps it.
9 */
10
11 import { stringify, type Json } from "./json-doc.ts";
12
13 /** Keys and array indexes from the document's root to a value. */
14 export type Path = readonly (string | number)[];
15
16 export type Edit =
17 /** Put `value` at `path`: replace it, add the key, or (index == length) append. */
18 | { readonly op: "set"; readonly path: Path; readonly value: Json }
19 /** Remove the key, or the array item (later items shift down). */
20 | { readonly op: "remove"; readonly path: Path };
21
22 /** The edits that turn `before` into `after`, in an order that is safe to apply one by one. */
23 export function editsBetween(before: Json, after: Json, path: Path = []): Edit[] {
24 if (before.t === "object" && after.t === "object") {
25 const out: Edit[] = [];
26 const kept = new Map(after.entries);
27 for (const [k] of before.entries) if (!kept.has(k)) out.push({ op: "remove", path: [...path, k] });
28 const had = new Map(before.entries);
29 for (const [k, v] of after.entries) {
30 const old = had.get(k);
31 if (old === undefined) out.push({ op: "set", path: [...path, k], value: v });
32 else out.push(...editsBetween(old, v, [...path, k]));
33 }
34 return out;
35 }
36 if (before.t === "array" && after.t === "array") {
37 const out: Edit[] = [];
38 const common = Math.min(before.items.length, after.items.length);
39 for (let i = 0; i < common; i++) out.push(...editsBetween(before.items[i]!, after.items[i]!, [...path, i]));
40 // Highest first, so each index is still the one that was meant.
41 for (let i = before.items.length - 1; i >= common; i--) out.push({ op: "remove", path: [...path, i] });
42 for (let i = common; i < after.items.length; i++) out.push({ op: "set", path: [...path, i], value: after.items[i]! });
43 return out;
44 }
45 return stringify(before) === stringify(after) ? [] : [{ op: "set", path, value: after }];
46 }
47
48 /** A value as plain data, for a library that takes plain data. Numbers past 2^53 stay exact as bigint. */
49 export function plainOf(v: Json): unknown {
50 switch (v.t) {
51 case "string":
52 return v.v;
53 case "number": {
54 const n = Number(v.v);
55 return /^-?\d+$/.test(v.v) && !Number.isSafeInteger(n) ? BigInt(v.v) : n;
56 }
57 case "boolean":
58 return v.v;
59 case "null":
60 return null;
61 case "array":
62 return v.items.map(plainOf);
63 case "object":
64 return Object.fromEntries(v.entries.map(([k, x]) => [k, plainOf(x)]));
65 }
66 }
src/web/render/config-format.ts added +53
@@ -0,0 +1,53 @@
1 /**
2 * Which config files open as a form, and how each is read and written back.
3 * The form holds a `Json` whatever the file's language; this is the one
4 * place that knows the language. JSON is rewritten whole in its own style;
5 * YAML and TOML are patched, so their comments survive.
6 */
7
8 import { detectStyle, parseJson, stringify, type Json, type Parsed } from "./json-doc.ts";
9 import { parseToml, writeToml } from "./toml-config.ts";
10 import { parseYaml, writeYaml } from "./yaml-config.ts";
11
12 export type ConfigFormat = "json" | "yaml" | "toml";
13
14 export function formatOf(path: string): ConfigFormat | undefined {
15 if (/\.json$/i.test(path)) return "json";
16 if (/\.ya?ml$/i.test(path)) return "yaml";
17 if (/\.toml$/i.test(path)) return "toml";
18 return undefined;
19 }
20
21 /** Whether the file opens as a form. */
22 export const isFormConfig = (path: string): boolean => formatOf(path) !== undefined;
23
24 const LANGUAGE: Readonly<Record<ConfigFormat, string>> = { json: "JSON", yaml: "YAML", toml: "TOML" };
25
26 /** The file's language, for messages: "Not valid YAML". Anything else reads as JSON. */
27 export const languageOf = (path: string): string => LANGUAGE[formatOf(path) ?? "json"];
28
29 /** The file's text as a document, or why it is not one. */
30 export function parseConfig(path: string, text: string): Parsed {
31 switch (formatOf(path)) {
32 case "yaml":
33 return parseYaml(text);
34 case "toml":
35 return parseToml(text);
36 default:
37 return parseJson(text);
38 }
39 }
40
41 export type Written = { readonly ok: true; readonly text: string } | { readonly ok: false; readonly message: string };
42
43 /** The file's new text: `original`, with `after` (which it parsed to `before`) written in. */
44 export function writeConfig(path: string, original: string, before: Json, after: Json): Written {
45 switch (formatOf(path)) {
46 case "yaml":
47 return writeYaml(original, before, after);
48 case "toml":
49 return writeToml(original, before, after);
50 default:
51 return { ok: true, text: stringify(after, detectStyle(original)) };
52 }
53 }
src/web/render/config.ts +11 −13
@@ -1,19 +1,16 @@
11 /**
2 * A JSON config file on the blob page: its values as a form, read-only, with
3 * "Edit as form" one click away (`pages/config.ts` is the editor, and draws
4 * the schema the file names). A file that is not valid JSON says why, at
5 * which line, above its text; Source keeps its highlighted view.
2 * A JSON, YAML or TOML config file on the blob page: its values as a form,
3 * read-only, with "Edit as form" one click away (`pages/config.ts` is the
4 * editor, and draws the schema the file names). A file that is not valid says
5 * why, at which line, above its text; Source keeps its highlighted view.
66 */
77
88 import { html, type Html } from "../html.ts";
99 import { btn } from "../ui.ts";
1010 import { configFields, countFields, MAX_FIELDS } from "./config-form.ts";
11 import { parseJson } from "./json-doc.ts";
11 import { languageOf, parseConfig } from "./config-format.ts";
1212 import type { Rendered, RenderInput } from "./renderers.ts";
1313
14 /** Only JSON draws as a form; YAML, TOML and the rest keep their source. */
15 export const isFormJson = (path: string): boolean => /\.json$/i.test(path);
16
1714 /**
1815 * `/:owner/:name/media/:ref/*path` → `/:owner/:name/config/:ref/*path`: the
1916 * ref is encoded whole, so the last `/media/` before it is the route's. A
@@ -26,11 +23,11 @@ function editHref(input: RenderInput): string | undefined {
2623 return `${head.slice(0, cut)}/config/${head.slice(cut + "/media/".length)}/${input.path}`;
2724 }
2825
29 function brokenJson(text: string, line: number, column: number, message: string): Html {
26 function brokenConfig(language: string, text: string, line: number, column: number, message: string): Html {
3027 const lines = text.split("\n");
3128 return html`<div class="card" role="alert">
32 <p><strong>Not valid JSON:</strong> ${message} at line ${String(line)}, column ${String(column)}.</p>
33 <p class="muted small">The form needs valid JSON. Source shows the file highlighted.</p>
29 <p><strong>Not valid ${language}:</strong> ${message} at line ${String(line)}, column ${String(column)}.</p>
30 <p class="muted small">The form needs valid ${language}. Source shows the file highlighted.</p>
3431 </div>
3532 <pre class="card mono">${lines.map((l, i) => (i + 1 === line ? html`<mark>${l}</mark>\n` : `${l}\n`))}</pre>`;
3633 }
@@ -38,10 +35,11 @@ function brokenJson(text: string, line: number, column: number, message: string)
3835 export function config(input: RenderInput): Rendered {
3936 const text = new TextDecoder().decode(input.body);
4037 const lines = text === "" ? 0 : text.replace(/\n$/, "").split("\n").length;
41 const parsed = parseJson(text);
38 const language = languageOf(input.path);
39 const parsed = parseConfig(input.path, text);
4240 if (!parsed.ok) {
4341 const e = parsed.error;
44 return { lines, content: brokenJson(text, e.line, e.column, e.message) };
42 return { lines, content: brokenConfig(language, text, e.line, e.column, e.message) };
4543 }
4644 if (countFields(parsed.value) > MAX_FIELDS)
4745 return {
src/web/render/json-schema.ts +7 −6
@@ -13,7 +13,8 @@
1313 * browser; the server never runs a regular expression a repository wrote.
1414 */
1515
16 import { NUMBER, getAt, parseJson, pointer, splitPointer, type Json } from "./json-doc.ts";
16 import { isFormConfig, parseConfig } from "./config-format.ts";
17 import { NUMBER, getAt, pointer, splitPointer, type Json } from "./json-doc.ts";
1718
1819 export interface Schema {
1920 readonly type?: string | readonly string[];
@@ -204,7 +205,7 @@ export async function findSchema(
204205 if (found !== undefined) return found;
205206 }
206207
207 const name = path.slice(dir.length).replace(/\.json$/i, "");
208 const name = path.slice(dir.length).replace(/\.(?:json|ya?ml|toml)$/i, "");
208209 const sibling = await tryRead(`${dir}${name}.schema.json`);
209210 if (sibling !== undefined) return sibling;
210211
@@ -221,8 +222,8 @@ export async function findSchema(
221222 }
222223
223224 /**
224 * Why a JSON file's new text breaks the schema it names, as one sentence, or
225 * undefined when it keeps it (or is not JSON, or has no schema). The
225 * Why a config file's new text breaks the schema it names, as one sentence, or
226 * undefined when it keeps it (or is not a config file, or has no schema). The
226227 * terminal asks this before a commit, as the form does before a save.
227228 */
228229 export async function schemaRefusal(
@@ -230,8 +231,8 @@ export async function schemaRefusal(
230231 text: string,
231232 read: (path: string) => Promise<string | undefined>,
232233 ): Promise<string | undefined> {
233 if (!/\.json$/i.test(path)) return undefined;
234 const parsed = parseJson(text);
234 if (!isFormConfig(path)) return undefined;
235 const parsed = parseConfig(path, text);
235236 if (!parsed.ok) return undefined;
236237 const found = await findSchema(path, parsed.value, read);
237238 const errors = [...validate(parsed.value, found)];
src/web/render/renderers.ts +5 −4
@@ -7,7 +7,8 @@
77 import type { Html } from "../html.ts";
88 import type { Blob } from "../../view/source.ts";
99 import { audio } from "./audio.ts";
10 import { config, isFormJson } from "./config.ts";
10 import { config } from "./config.ts";
11 import { isFormConfig } from "./config-format.ts";
1112 import { csv } from "./csv.ts";
1213 import { image } from "./image.ts";
1314 import { claimedMedia, fileKind, HEAD_BYTES, NO_BYTES, RENDER_CAP, type FileKind } from "./kinds.ts";
@@ -85,9 +86,9 @@ export function rendererFor(
8586 : claimedMedia(path)?.kind;
8687 if (kind === undefined || blob.size > RENDER_CAP[kind]) return undefined;
8788 if (blob.kind === "too_large" && !MEDIA_KINDS.has(kind)) return undefined;
88 // A kind may draw only some of its files: `config` draws JSON as a form,
89 // and leaves YAML and TOML their source.
90 const render = kind === "config" && !isFormJson(path) ? undefined : renderers[kind];
89 // A kind may draw only some of its files: `config` draws JSON, YAML and
90 // TOML as a form, and leaves the rest (INI, .env …) their source.
91 const render = kind === "config" && !isFormConfig(path) ? undefined : renderers[kind];
9192 if (render === undefined) return undefined;
9293 return { render, body: blob.kind === "too_large" ? NO_BYTES : blob.body };
9394 }
src/web/render/toml-config.ts added +110
@@ -0,0 +1,110 @@
1 /**
2 * TOML as the config form holds it (#7337): `toml-patch` reads it into a
3 * `Json`, and a save patches the original text with only the values that
4 * changed, so comments, table order and the spacing of every untouched line
5 * stay as written.
6 *
7 * Dates and times have no `Json` type: they show as the text they were
8 * written as, and writing text that parses as a date back over one keeps it
9 * a date. TOML has no null; a form that would write one is refused.
10 */
11
12 import { parse, patch } from "./vendor/toml-patch.mjs";
13 import { editsBetween, plainOf } from "./config-diff.ts";
14 import type { Json, ParseError } from "./json-doc.ts";
15
16 export type TomlParsed =
17 | { readonly ok: true; readonly value: Json }
18 | { readonly ok: false; readonly error: ParseError };
19
20 const MAX_DEPTH = 64;
21
22 /** A date, time or date-time: the parser's own subclasses of `Date`. */
23 const isDate = (v: unknown): v is Date => v instanceof Date;
24
25 function toJson(v: unknown, depth: number): Json {
26 if (depth > MAX_DEPTH) throw new RangeError("nested too deeply");
27 if (typeof v === "string") return { t: "string", v };
28 if (typeof v === "boolean") return { t: "boolean", v };
29 // inf and nan have no JSON spelling: shown as text, which only an edit changes.
30 if (typeof v === "number") return Number.isFinite(v) ? { t: "number", v: String(v) } : { t: "string", v: v > 0 ? "inf" : v < 0 ? "-inf" : "nan" };
31 if (typeof v === "bigint") return { t: "number", v: String(v) };
32 if (isDate(v)) return { t: "string", v: v.toISOString() };
33 if (Array.isArray(v)) return { t: "array", items: v.map((x) => toJson(x, depth + 1)) };
34 if (v !== null && typeof v === "object")
35 return { t: "object", entries: Object.entries(v).map(([k, x]) => [k, toJson(x, depth + 1)] as const) };
36 return { t: "null" };
37 }
38
39 const reason = (e: unknown): { message: string; line: number; column: number } => {
40 const err = e as { message?: string; line?: number; column?: number };
41 const lines = String(err.message ?? "invalid TOML").split("\n");
42 return { message: lines[lines.length - 1]!, line: err.line ?? 1, column: (err.column ?? 0) + 1 };
43 };
44
45 export function parseToml(text: string): TomlParsed {
46 try {
47 return { ok: true, value: toJson(parse(text), 0) };
48 } catch (e) {
49 return { ok: false, error: reason(e) };
50 }
51 }
52
53 export type TomlWritten = { readonly ok: true; readonly text: string } | { readonly ok: false; readonly message: string };
54
55 const hasNull = (v: Json): boolean =>
56 v.t === "null" || (v.t === "array" && v.items.some(hasNull)) || (v.t === "object" && v.entries.some(([, x]) => hasNull(x)));
57
58 type Container = Record<string | number, unknown>;
59
60 const isContainer = (v: unknown): v is Container => v !== null && typeof v === "object";
61
62 /** The table or array a path's parent names, or undefined when the tree has no such place. */
63 function parentOf(tree: unknown, path: readonly (string | number)[]): Container | undefined {
64 let at: unknown = tree;
65 for (const k of path.slice(0, -1)) {
66 if (!isContainer(at)) return undefined;
67 at = at[k];
68 }
69 return isContainer(at) ? at : undefined;
70 }
71
72 /** `text` as a TOML date, time or date-time, or undefined when it does not read as one. */
73 function asDate(text: string): Date | undefined {
74 try {
75 const probe: unknown = parse(`v = ${text}`)["v"];
76 return isDate(probe) ? probe : undefined;
77 } catch {
78 // Text that is not a date is the common case, not a fault: it stays text.
79 return undefined;
80 }
81 }
82
83 const UNWRITABLE = "This change cannot be written into the TOML without losing its structure. Edit it as text.";
84
85 /** `original` with the edits from `before` to `after` patched in. */
86 export function writeToml(original: string, before: Json, after: Json): TomlWritten {
87 const edits = editsBetween(before, after);
88 if (edits.length === 0) return { ok: true, text: original };
89 if (hasNull(after)) return { ok: false, message: "TOML has no null, so a field cannot be left empty. Give it a value, or edit the file as text." };
90 try {
91 const tree = parse(original);
92 for (const edit of edits) {
93 const parent = parentOf(tree, edit.path);
94 const key = edit.path[edit.path.length - 1];
95 if (parent === undefined || key === undefined) return { ok: false, message: UNWRITABLE };
96 if (edit.op === "remove") {
97 if (Array.isArray(parent) && typeof key === "number") parent.splice(key, 1);
98 else delete parent[key];
99 continue;
100 }
101 let value = plainOf(edit.value);
102 // Text over a date stays a date, when it still reads as one.
103 if (isDate(parent[key]) && typeof value === "string") value = asDate(value) ?? value;
104 parent[key] = value;
105 }
106 return { ok: true, text: patch(original, tree) };
107 } catch {
108 return { ok: false, message: UNWRITABLE };
109 }
110 }
src/web/render/vendor/TOML-PATCH-LICENSE added +22
@@ -0,0 +1,22 @@
1 MIT License
2
3 Copyright (c) 2019–2025 Tim Hall
4 Copyright (c) 2025 Martin Leduc
5
6 Permission is hereby granted, free of charge, to any person obtaining a copy
7 of this software and associated documentation files (the "Software"), to deal
8 in the Software without restriction, including without limitation the rights
9 to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
10 copies of the Software, and to permit persons to whom the Software is
11 furnished to do so, subject to the following conditions:
12
13 The above copyright notice and this permission notice shall be included in all
14 copies or substantial portions of the Software.
15
16 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
17 IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
18 FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
19 AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
20 LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
21 OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
22 SOFTWARE.
src/web/render/vendor/YAML-LICENSE added +13
@@ -0,0 +1,13 @@
1 Copyright Eemeli Aro <[email protected]>
2
3 Permission to use, copy, modify, and/or distribute this software for any purpose
4 with or without fee is hereby granted, provided that the above copyright notice
5 and this permission notice appear in all copies.
6
7 THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH
8 REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
9 FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT,
10 INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS
11 OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER
12 TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF
13 THIS SOFTWARE.
src/web/render/vendor/toml-patch.d.mts added +6
@@ -0,0 +1,6 @@
1 /** The part of `@decimalturn/toml-patch` the config form uses (`render/toml-config.ts`). */
2
3 /** A TOML document as plain data; dates come back as `Date` subclasses. */
4 export function parse(source: string): Record<string, unknown>;
5 /** `source` with only what differs in `updated` rewritten: comments and layout stay. */
6 export function patch(source: string, updated: unknown): string;
src/web/render/vendor/toml-patch.mjs added +6844

Diff not shown — 6,844 changed lines (limit 5,000). +6844

View file
src/web/render/vendor/yaml.d.mts added +44
@@ -0,0 +1,44 @@
1 /** The part of the `yaml` package the config form uses (`render/yaml-config.ts`). */
2
3 export interface Node {
4 /** Comment lines above the node. */
5 commentBefore?: string | null;
6 comment?: string | null;
7 }
8 export interface Scalar extends Node {
9 value: unknown;
10 source?: string;
11 }
12 export interface Pair {
13 key: unknown;
14 value: unknown;
15 }
16 export interface YAMLMap extends Node {
17 items: Pair[];
18 }
19 export interface YAMLSeq extends Node {
20 items: unknown[];
21 }
22 export interface Alias extends Node {
23 resolve(doc: Document): unknown;
24 }
25 export interface YAMLError {
26 message: string;
27 linePos?: [{ line: number; col: number }, { line: number; col: number }?];
28 }
29 export interface Document {
30 contents: unknown;
31 /** Offsets into the source: start, end of the value, end of the document. */
32 range: [number, number, number];
33 errors: YAMLError[];
34 warnings: YAMLError[];
35 getIn(path: readonly (string | number)[], keepScalar?: boolean): unknown;
36 setIn(path: readonly (string | number)[], value: unknown): void;
37 deleteIn(path: readonly (string | number)[]): boolean;
38 toString(options?: { lineWidth?: number; indent?: number; indentSeq?: boolean }): string;
39 }
40 export function parseAllDocuments(source: string, options?: { prettyErrors?: boolean; uniqueKeys?: boolean; maxAliasCount?: number }): Document[];
41 export function isScalar(node: unknown): node is Scalar;
42 export function isMap(node: unknown): node is YAMLMap;
43 export function isSeq(node: unknown): node is YAMLSeq;
44 export function isAlias(node: unknown): node is Alias;
src/web/render/vendor/yaml.mjs added +6516

Diff not shown — 6,516 changed lines (limit 5,000). +6516

View file
src/web/render/yaml-config.ts added +136
@@ -0,0 +1,136 @@
1 /**
2 * YAML as the config form holds it (#7337): the `yaml` package's Document
3 * API reads it into a `Json`, and a save edits that same Document, so
4 * comments, anchors, quoting and key order survive and an untouched line is
5 * written byte for byte as it was.
6 *
7 * A file of several documents (`---`) reads as one object whose keys are
8 * "Document 1", "Document 2" …, so the form draws one fieldset each. An
9 * alias reads as the value it points at; changing a value that only an
10 * alias holds is refused rather than guessed.
11 */
12
13 import { isAlias, isMap, isScalar, isSeq, parseAllDocuments, type Document } from "./vendor/yaml.mjs";
14 import { editsBetween, plainOf, type Path } from "./config-diff.ts";
15 import { MAX_FIELDS } from "./config-form.ts";
16 import type { Json, ParseError } from "./json-doc.ts";
17
18 export type YamlParsed =
19 | { readonly ok: true; readonly value: Json }
20 | { readonly ok: false; readonly error: ParseError };
21
22 const MAX_DEPTH = 64;
23
24 class Unsupported extends Error {}
25
26 const docKey = (i: number): string => `Document ${i + 1}`;
27
28 function toJson(doc: Document, node: unknown, depth: number, budget: { left: number }): Json {
29 if (depth > MAX_DEPTH) throw new Unsupported("nested too deeply");
30 // Aliases can fan out (a few lines of YAML naming a billion values).
31 if (--budget.left < 0) throw new Unsupported(`more than ${MAX_FIELDS.toLocaleString()} values`);
32 if (node === null || node === undefined) return { t: "null" };
33 if (isAlias(node)) return toJson(doc, node.resolve(doc), depth + 1, budget);
34 if (isScalar(node)) {
35 const v = node.value;
36 if (v === null || v === undefined) return { t: "null" };
37 if (typeof v === "string") return { t: "string", v };
38 if (typeof v === "boolean") return { t: "boolean", v };
39 if (typeof v === "number" && Number.isFinite(v)) return { t: "number", v: String(v) };
40 if (typeof v === "bigint") return { t: "number", v: String(v) };
41 // .inf, .nan, binary and the like: shown as the text they were written as.
42 return { t: "string", v: node.source ?? String(v) };
43 }
44 if (isMap(node)) {
45 const entries: (readonly [string, Json])[] = [];
46 for (const pair of node.items) {
47 if (!isScalar(pair.key) || typeof pair.key.value !== "string")
48 throw new Unsupported("a key that is not plain text");
49 entries.push([pair.key.value, toJson(doc, pair.value, depth + 1, budget)]);
50 }
51 return { t: "object", entries };
52 }
53 if (isSeq(node)) return { t: "array", items: node.items.map((x) => toJson(doc, x, depth + 1, budget)) };
54 throw new Unsupported("a value of a kind the form does not hold");
55 }
56
57 /** The first problem in any document, or undefined. */
58 function firstError(docs: readonly Document[]): ParseError | undefined {
59 for (const d of docs) {
60 const e = d.errors[0];
61 if (e === undefined) continue;
62 const at = e.linePos?.[0];
63 return { message: e.message.split("\n")[0]!.replace(/ at line \d+, column \d+:?$/, ""), line: at?.line ?? 1, column: at?.col ?? 1 };
64 }
65 return undefined;
66 }
67
68 export function parseYaml(text: string): YamlParsed {
69 const docs = parseAllDocuments(text, { prettyErrors: true });
70 const error = firstError(docs);
71 if (error !== undefined) return { ok: false, error };
72 try {
73 const budget = { left: MAX_FIELDS * 2 };
74 const values = docs.map((d) => toJson(d, d.contents, 0, budget));
75 if (values.length === 1) return { ok: true, value: values[0]! };
76 return { ok: true, value: { t: "object", entries: values.map((v, i) => [docKey(i), v] as const) } };
77 } catch (e) {
78 if (!(e instanceof Unsupported)) throw e;
79 return { ok: false, error: { message: `The form cannot hold ${e.message}`, line: 1, column: 1 } };
80 }
81 }
82
83 /** How the file indents, so the rewritten lines match the untouched ones. */
84 function layout(text: string): { indent: number; indentSeq: boolean } {
85 let indent = 0;
86 for (const line of text.split("\n")) {
87 const m = /^( +)[^\s#]/.exec(line);
88 if (m !== null && (indent === 0 || m[1]!.length < indent)) indent = m[1]!.length;
89 }
90 // `key:` then `- item` at the key's own column.
91 const flush = /^( *)[^\s#-][^\n]*:[ \t]*\r?\n\1- /m.test(text);
92 const deep = /^( *)[^\s#-][^\n]*:[ \t]*\r?\n\1 +- /m.test(text);
93 return { indent: Math.min(Math.max(indent, 2), 8), indentSeq: !(flush && !deep) };
94 }
95
96 export type YamlWritten = { readonly ok: true; readonly text: string } | { readonly ok: false; readonly message: string };
97
98 /** `original` with `after` written into it; only the documents that changed are rewritten. */
99 export function writeYaml(original: string, before: Json, after: Json): YamlWritten {
100 const docs = parseAllDocuments(original);
101 const multi = docs.length > 1;
102 const edits = editsBetween(before, after);
103 if (edits.length === 0) return { ok: true, text: original };
104 const changed = new Set<number>();
105 try {
106 for (const edit of edits) {
107 let path: Path = edit.path;
108 let index = 0;
109 if (multi) {
110 const [head, ...rest] = path;
111 index = typeof head === "string" ? Number(/^Document (\d+)$/.exec(head)?.[1]) - 1 : NaN;
112 if (!(index >= 0 && index < docs.length) || rest.length === 0) throw new Unsupported("a whole document");
113 path = rest;
114 } else if (path.length === 0) throw new Unsupported("the whole file");
115 const doc = docs[index]!;
116 if (edit.op === "remove") doc.deleteIn(path);
117 else doc.setIn(path, plainOf(edit.value));
118 changed.add(index);
119 }
120 } catch (e) {
121 const why = e instanceof Unsupported ? e.message : "a value that comes from an alias or an unusual node";
122 return { ok: false, message: `This change cannot be written into the YAML without losing its structure (${why}). Edit it as text.` };
123 }
124 const { indent, indentSeq } = layout(original);
125 const eol = original.includes("\r\n") ? "\r\n" : "\n";
126 const cuts = docs.map((d, i) => (i === 0 ? 0 : d.range[0]));
127 let out = "";
128 docs.forEach((d, i) => {
129 const end = i + 1 < docs.length ? cuts[i + 1]! : original.length;
130 out += changed.has(i)
131 ? d.toString({ lineWidth: 0, indent, indentSeq }).replace(/\n/g, eol)
132 : original.slice(cuts[i]!, end);
133 });
134 if (!/\n$/.test(original)) out = out.replace(/\r?\n$/, "");
135 return { ok: true, text: out };
136 }
test/config-form.test.ts +68 −1
@@ -12,6 +12,7 @@ import { createApp, type App } from "../src/app.ts";
1212 import { commitToBranch, createRepo } from "../src/actions/repos.ts";
1313 import { setRole } from "../src/actions/roles.ts";
1414 import { createSession } from "../src/core/sessions.ts";
15 import { getUserByName } from "../src/core/users.ts";
1516 import { getRepo, updateRepoSettings } from "../src/core/repos.ts";
1617 import { createAgent } from "../src/actions/crew.ts";
1718 import { addTeamMember, createOrg, createTeam } from "../src/core/orgs.ts";
@@ -25,7 +26,7 @@ import { localSource } from "../src/view/local.ts";
2526 import { applyOp, configFields, opValue, readValues } from "../src/web/render/config-form.ts";
2627 import { detectStyle, parseJson, sameDoc, stringify, type Json } from "../src/web/render/json-doc.ts";
2728 import { findSchema, globMatches, validate } from "../src/web/render/json-schema.ts";
28 import { digestOf } from "../src/web/pages/config.ts";
29 import { digestOf, MAX_FORM_BYTES } from "../src/web/pages/config.ts";
2930 import { seedAdmin } from "./helpers/admin.ts";
3031 import { signUpProven } from "./helpers/verified.ts";
3132 import { gitClientEnv } from "./helpers/git-client.ts";
@@ -173,6 +174,13 @@ describe("the config form on the web", () => {
173174 let origin = "";
174175 let alice = "";
175176 let kim = "";
177 const DEPLOY = "# how it runs\nname: lamp # the app\nbase: &b\n port: 80\nreplicas: 2\n";
178 // Hostile shapes a form must refuse, not expand: never past MAX_FORM_BYTES, so read only after the size check.
179 const HOSTILE: Record<string, string> = {
180 "flow.yml": `a: ${"[".repeat(10_000)}${"]".repeat(10_000)}\n`,
181 "merge.yml": `a0: &a0 {x: 1}\n${Array.from({ length: 11 }, (_, i) => `a${i + 1}: &a${i + 1}\n <<: [${Array(9).fill(`*a${i}`).join(", ")}]`).join("\n")}\n`,
182 "deep.toml": `a = ${"[".repeat(70)}${"]".repeat(70)}\n`,
183 };
176184 const SETTINGS = '{\n "name": "lamp",\n "port": 8080,\n "tags": [\n "a"\n ]\n}\n';
177185
178186 const get = (path: string, cookie = ""): Promise<Response> =>
@@ -228,6 +236,8 @@ describe("the config form on the web", () => {
228236 files: [
229237 { path: "settings.json", content: SETTINGS },
230238 { path: "broken.json", content: '{\n "a": 1,\n oops\n}\n' },
239 { path: "deploy.yml", content: DEPLOY },
240 ...Object.entries(HOSTILE).map(([path, content]) => ({ path, content })),
231241 ],
232242 });
233243 }
@@ -301,6 +311,63 @@ describe("the config form on the web", () => {
301311 expect(await fileAt("alice", "lamp", "main", "settings.json")).toBe(current);
302312 });
303313
314 test("a YAML file opens as a form and a save keeps its comments and anchors", async () => {
315 const page = await (await get("/alice/lamp/blob/main/deploy.yml")).text();
316 expect(page).toContain('class="card config-form"');
317 expect(page).toContain('href="/alice/lamp/config/main/deploy.yml"');
318 const form = await (await get("/alice/lamp/config/main/deploy.yml", alice)).text();
319 expect(form).toContain('name="v/replicas"');
320 // A plain form post: nothing here needs a script.
321 expect(form).toContain('<form method="post" action="/alice/lamp/config" class="card config-form">');
322 const before = await commitCount("alice", "lamp");
323 const res = await post("/alice/lamp/config", {
324 ref: "main",
325 path: "deploy.yml",
326 base: digestOf(DEPLOY),
327 doc: '{"name":"lamp","base":{"port":80},"replicas":3}',
328 step: "save",
329 message: "Three replicas",
330 }, alice);
331 expect(res.status).toBe(303);
332 expect(await commitCount("alice", "lamp")).toBe(before + 1);
333 expect(await fileAt("alice", "lamp", "main", "deploy.yml")).toBe(DEPLOY.replace("replicas: 2", "replicas: 3"));
334 });
335
336 test("hostile YAML and TOML fall back to the source on the page and are refused on submit", async () => {
337 for (const [path, content] of Object.entries(HOSTILE)) {
338 const res = await get(`/alice/lamp/config/main/${path}`, alice);
339 expect(res.status).toBe(303);
340 expect(res.headers.get("location")).toContain(`/alice/lamp/blob/main/${path}?view=source`);
341 const before = await commitCount("alice", "lamp");
342 const sub = await post("/alice/lamp/config", {
343 ref: "main",
344 path,
345 base: digestOf(content),
346 doc: '{"a":1}',
347 step: "save",
348 message: "Nope",
349 }, alice);
350 expect(sub.status).toBe(303);
351 expect(sub.headers.get("location")).toContain(`?view=source`);
352 expect(await commitCount("alice", "lamp")).toBe(before);
353 }
354 });
355
356 test("a file over MAX_FORM_BYTES is refused before it is parsed, on the page and on submit", async () => {
357 const big = `# ${"x".repeat(MAX_FORM_BYTES)}\na: 1\n`;
358 const made = await commitToBranch(app, getUserByName(app.db, "alice")!, getRepo(app.db, "alice", "lamp")!, {
359 message: "big",
360 files: [{ path: "big.yml", content: big }],
361 });
362 expect(made.ok).toBe(true);
363 const page = await get("/alice/lamp/config/main/big.yml", alice);
364 expect(page.status).toBe(303);
365 expect(page.headers.get("location")).toContain("?view=source");
366 const sub = await post("/alice/lamp/config", { ref: "main", path: "big.yml", base: digestOf(big), doc: '{"a":2}', step: "save", message: "x" }, alice);
367 expect(sub.status).not.toBe(200);
368 expect(await fileAt("alice", "lamp", "main", "big.yml")).toBe(big);
369 });
370
304371 test("invalid JSON falls back to the source with the error and its line", async () => {
305372 const blob = await (await get("/alice/lamp/blob/main/broken.json")).text();
306373 expect(blob).toContain("Not valid JSON");
test/config-yaml-toml.test.ts added +269
@@ -0,0 +1,269 @@
1 /**
2 * YAML and TOML config files as a form (#7337): read into the same document
3 * JSON uses, and written back with their comments, anchors and layout intact.
4 */
5
6 import { describe, expect, test } from "bun:test";
7 import { formatOf, isFormConfig, languageOf, parseConfig, writeConfig } from "../src/web/render/config-format.ts";
8 import { configFields } from "../src/web/render/config-form.ts";
9 import { getAt, updateAt, type Json } from "../src/web/render/json-doc.ts";
10 import { findSchema, schemaRefusal } from "../src/web/render/json-schema.ts";
11 import { rendererFor } from "../src/web/render/renderers.ts";
12 import { config } from "../src/web/render/config.ts";
13
14 const read = (path: string, text: string): Json => {
15 const p = parseConfig(path, text);
16 if (!p.ok) throw new Error(p.error.message);
17 return p.value;
18 };
19
20 /** Change the string at `at`, write the file back, and return its new text. */
21 const edit = (path: string, text: string, at: string, to: Json): string => {
22 const before = read(path, text);
23 const w = writeConfig(path, text, before, updateAt(before, at, () => to));
24 if (!w.ok) throw new Error(w.message);
25 return w.text;
26 };
27
28 const YAML = `# service settings
29 name: forge # the app
30 base: &base
31 retries: 3
32 verbose: false
33 child:
34 <<: *base
35 url: 'http://x'
36 tags:
37 - a
38 - b # second
39 servers:
40 - host: one
41 port: 80
42 - host: two
43 port: 81
44 `;
45
46 describe("which files open as a form", () => {
47 test("JSON, YAML and TOML do; INI does not", () => {
48 expect(["a.json", "a.yml", "a.YAML", "a.toml"].map(formatOf)).toEqual(["json", "yaml", "yaml", "toml"]);
49 expect(isFormConfig("a.ini")).toBe(false);
50 const body = new TextEncoder().encode("a: 1\n");
51 const blob = { kind: "text" as const, size: body.length, body } as never;
52 expect(rendererFor("x/a.yml", blob)).toBeDefined();
53 expect(rendererFor("x/a.toml", blob)).toBeDefined();
54 expect(rendererFor("x/a.ini", blob)).toBeUndefined();
55 });
56 });
57
58 describe("YAML", () => {
59 test("reads values, aliases as what they point at, in the file's order", () => {
60 const doc = read("a.yml", YAML);
61 expect(getAt(doc, "/name")).toEqual({ t: "string", v: "forge" });
62 expect(getAt(doc, "/child/<</retries")).toEqual({ t: "number", v: "3" });
63 expect(getAt(doc, "/servers/1/port")).toEqual({ t: "number", v: "81" });
64 expect(doc.t === "object" && doc.entries.map(([k]) => k)).toEqual(["name", "base", "child", "tags", "servers"]);
65 });
66
67 test("a file nobody changed is written byte for byte", () => {
68 const doc = read("a.yml", YAML);
69 expect(writeConfig("a.yml", YAML, doc, doc)).toEqual({ ok: true, text: YAML });
70 });
71
72 test("changing one value changes only that line, comments and anchors kept", () => {
73 const next = edit("a.yml", YAML, "/servers/0/port", { t: "number", v: "8080" });
74 const was = YAML.split("\n");
75 const now = next.split("\n");
76 expect(now.length).toBe(was.length);
77 const differing = now.filter((l, i) => l !== was[i]);
78 expect(differing).toEqual([" port: 8080"]);
79 expect(next).toContain("# service settings");
80 expect(next).toContain("base: &base");
81 expect(next).toContain("<<: *base");
82 });
83
84 test("a changed string keeps its quotes and its trailing comment", () => {
85 const next = edit("a.yml", YAML, "/name", { t: "string", v: "anvil" });
86 expect(next).toContain("name: anvil # the app");
87 expect(edit("a.yml", YAML, "/child/url", { t: "string", v: "http://y" })).toContain("url: 'http://y'");
88 });
89
90 test("an added and a removed card are written", () => {
91 const before = read("a.yml", YAML);
92 const added = updateAt(before, "/servers", (s) =>
93 s.t === "array"
94 ? { t: "array", items: [...s.items, { t: "object", entries: [["host", { t: "string", v: "three" }], ["port", { t: "number", v: "82" }]] }] }
95 : s,
96 );
97 const w = writeConfig("a.yml", YAML, before, added);
98 expect(w.ok && w.text).toContain("host: three");
99 expect(w.ok && w.text).toContain("# second");
100 const removed = updateAt(before, "/servers", (s) => (s.t === "array" ? { t: "array", items: s.items.slice(0, 1) } : s));
101 const r = writeConfig("a.yml", YAML, before, removed);
102 expect(r.ok && r.text).not.toContain("two");
103 });
104
105 test("a file that indents sequences flush is not reflowed", () => {
106 // A mixed file cannot be kept (one setting per document); a consistent one is.
107 expect(edit("a.yml", YAML, "/name", { t: "string", v: "z" })).toContain("tags:\n- a\n- b # second");
108 });
109
110 test("several documents are one fieldset each; only the changed one is rewritten", () => {
111 const text = "# one\na: 1\n---\n# two\nb: 2 # odd spacing\n---\nc: 3\n";
112 const doc = read("m.yaml", text);
113 expect(doc.t === "object" && doc.entries.map(([k]) => k)).toEqual(["Document 1", "Document 2", "Document 3"]);
114 const html = String(configFields(doc, {}));
115 expect(html.match(/<fieldset/g)?.length).toBeGreaterThanOrEqual(3);
116 expect(edit("m.yaml", text, "/Document 1/a", { t: "number", v: "9" })).toBe("# one\na: 9\n---\n# two\nb: 2 # odd spacing\n---\nc: 3\n");
117 });
118
119 test("a syntax error names its line", () => {
120 const p = parseConfig("a.yml", "a: 1\nb: [1, 2\nc: 3\n");
121 expect(p.ok).toBe(false);
122 if (!p.ok) expect(p.error.line).toBeGreaterThanOrEqual(2);
123 });
124
125 test("an alias bomb is refused, not expanded", () => {
126 let text = "a0: &a0 [x, x, x, x, x, x, x, x, x]\n";
127 for (let i = 1; i < 12; i++) text += `a${i}: &a${i} [${Array(9).fill(`*a${i - 1}`).join(", ")}]\n`;
128 expect(parseConfig("bomb.yml", text).ok).toBe(false);
129 });
130
131 test("a key that is not text is refused, with the reason", () => {
132 const p = parseConfig("a.yml", "1: one\n");
133 expect(p.ok).toBe(false);
134 });
135
136 test("the schema lookup is JSON's: a sibling <name>.schema.json", async () => {
137 const files: Record<string, string> = { "deploy/app.schema.json": '{"properties":{"name":{"description":"The app"}}}' };
138 const found = await findSchema("deploy/app.yml", read("deploy/app.yml", "name: x\n"), async (p) => files[p]);
139 expect(found?.from).toBe("deploy/app.schema.json");
140 });
141
142 test("the blob page draws the form, and says so for a broken file", () => {
143 const input = (text: string) => {
144 const body = new TextEncoder().encode(text);
145 return { path: "a.yml", size: body.length, body, media: "/o/r/media/main/a.yml" };
146 };
147 expect(String(config(input(YAML)).content)).toContain("config-form");
148 expect(String(config(input("a: [1\n")).content)).toContain("Not valid YAML");
149 });
150 });
151
152 const TOML = `# top comment
153 title = "forge" # inline
154
155 [server]
156 # the port
157 port = 80
158 since = 1979-05-27T07:32:00Z
159
160 [[servers]]
161 host = "one"
162
163 [[servers]]
164 host = "two"
165 `;
166
167 describe("TOML", () => {
168 test("a table is a fieldset and an array of tables is cards", () => {
169 const html = String(configFields(read("a.toml", TOML), {}));
170 expect(html).toContain("<fieldset");
171 expect(html).toContain("config-card");
172 expect(html.match(/config-card/g)?.length).toBeGreaterThanOrEqual(2);
173 });
174
175 test("dates show as the text they were written as", () => {
176 expect(getAt(read("a.toml", TOML), "/server/since")).toEqual({ t: "string", v: "1979-05-27T07:32:00Z" });
177 });
178
179 test("an untouched file is written as it was; a change touches only its line", () => {
180 const doc = read("a.toml", TOML);
181 expect(writeConfig("a.toml", TOML, doc, doc)).toEqual({ ok: true, text: TOML });
182 const next = edit("a.toml", TOML, "/server/port", { t: "number", v: "8080" });
183 expect(next).toBe(TOML.replace("port = 80", "port = 8080"));
184 });
185
186 test("a date stays a date when its text is edited", () => {
187 const next = edit("a.toml", TOML, "/server/since", { t: "string", v: "1980-01-01T00:00:00Z" });
188 expect(next).toContain("since = 1980-01-01T00:00:00Z");
189 });
190
191 test("adding and removing a card keeps the comments", () => {
192 const before = read("a.toml", TOML);
193 const card = (host: string): Json => ({ t: "object", entries: [["host", { t: "string", v: host }]] });
194 const grown = updateAt(before, "/servers", () => ({ t: "array", items: [card("one"), card("two"), card("three")] }));
195 const w = writeConfig("a.toml", TOML, before, grown);
196 expect(w.ok && w.text).toContain('host = "three"');
197 expect(w.ok && w.text).toContain("# the port");
198 const shrunk = updateAt(before, "/servers", () => ({ t: "array", items: [card("two")] }));
199 const s = writeConfig("a.toml", TOML, before, shrunk);
200 expect(s.ok && s.text).not.toContain('"one"');
201 expect(s.ok && s.text).toContain("# top comment");
202 });
203
204 test("TOML cannot hold null, and says so", () => {
205 const before = read("a.toml", TOML);
206 const w = writeConfig("a.toml", TOML, before, updateAt(before, "/title", () => ({ t: "null" })));
207 expect(w.ok).toBe(false);
208 });
209
210 test("a syntax error names its line", () => {
211 const p = parseConfig("a.toml", "a = 1\nb = \nc = 2\n");
212 expect(p.ok).toBe(false);
213 if (!p.ok) expect(p.error.line).toBe(3);
214 });
215 });
216
217 describe("hostile input is bounded, whatever the language", () => {
218 // Parsing only ever sees text under MAX_FORM_BYTES: the page and the submit path both check the size first.
219 test("deeply nested flow collections, merge keys and TOML arrays are refused, not walked", () => {
220 expect(parseConfig("a.yml", `a: ${"[".repeat(10_000)}${"]".repeat(10_000)}\n`).ok).toBe(false);
221 let merge = "a0: &a0 {x: 1}\n";
222 for (let i = 1; i <= 11; i++) merge += `a${i}: &a${i}\n <<: [${Array(9).fill(`*a${i - 1}`).join(", ")}]\n`;
223 expect(parseConfig("m.yml", merge).ok).toBe(false);
224 expect(parseConfig("a.toml", `a = ${"[".repeat(65)}${"]".repeat(65)}\n`).ok).toBe(false);
225 expect(parseConfig("a.toml", `a = ${"[".repeat(20)}${"]".repeat(20)}\n`).ok).toBe(true);
226 });
227 });
228
229 describe("the terminal checks YAML and TOML against a schema, as the page does", () => {
230 const SCHEMA = JSON.stringify({ type: "object", properties: { port: { type: "number" } }, required: ["port"] });
231 const files: Record<string, string> = { "svc.schema.json": SCHEMA };
232 const readFile = async (p: string): Promise<string | undefined> => files[p];
233
234 test("a file that breaks the sibling schema is refused; one that keeps it is not", async () => {
235 expect(await schemaRefusal("svc.yml", "port: x\n", readFile)).toContain("svc.yml breaks");
236 expect(await schemaRefusal("svc.toml", 'port = "x"\n', readFile)).toContain("svc.toml breaks");
237 expect(await schemaRefusal("svc.yml", "port: 80\n", readFile)).toBeUndefined();
238 expect(await schemaRefusal("svc.toml", "port = 80\n", readFile)).toBeUndefined();
239 expect(await schemaRefusal("svc.ini", "port=x\n", readFile)).toBeUndefined();
240 });
241 });
242
243 test("the language is named in one place", () => {
244 expect([languageOf("a.json"), languageOf("a.yaml"), languageOf("a.toml"), languageOf("a.txt")]).toEqual(["JSON", "YAML", "TOML", "JSON"]);
245 });
246
247 describe("the vendored bundles say what they are", () => {
248 const dir = new URL("../src/web/render/vendor/", import.meta.url);
249 const text = (name: string): Promise<string> => Bun.file(new URL(name, dir)).text();
250
251 test("each bundle names its pinned version and its licence file, and the licence is there", async () => {
252 const yaml = await text("yaml.mjs");
253 const toml = await text("toml-patch.mjs");
254 const yamlVersion = /^\/\/ yaml (\d+\.\d+\.\d+) /.exec(yaml)?.[1];
255 const tomlVersion = /^\/\/ @decimalturn\/toml-patch (\d+\.\d+\.\d+) /.exec(toml)?.[1];
256 expect(yamlVersion).toBeDefined();
257 expect(tomlVersion).toBeDefined();
258 expect(await text("YAML-LICENSE")).toContain("Eemeli Aro");
259 expect(await text("TOML-PATCH-LICENSE")).toContain("MIT License");
260 // docs/features/config-form.md records the same pins and how to regenerate.
261 const doc = await Bun.file(new URL("../../../../docs/features/config-form.md", dir)).text();
262 expect(doc).toContain(`yaml ${yamlVersion}`);
263 expect(doc).toContain(`toml-patch ${tomlVersion}`);
264 // The bundle is the one the page's commands rebuild, byte for byte.
265 const sha = (t: string): string => new Bun.CryptoHasher("sha256").update(t).digest("hex");
266 expect(doc).toContain(`\`yaml.mjs\` sha256 \`${sha(yaml)}\``);
267 expect(doc).toContain(`\`toml-patch.mjs\` sha256 \`${sha(toml)}\``);
268 });
269 });
test/looks-owner.test.ts +30
@@ -35,6 +35,9 @@ import type { Capture, LookInput } from "../src/looks/capture.ts";
3535 import { startLooks, type LooksHandle } from "../src/looks/service.ts";
3636 import { previewIdentityKey } from "../src/spawn/preview-owner.ts";
3737 import { PREVIEW_ADMIN, PREVIEW_PULL, PREVIEW_REPO, PREVIEW_WAVE, PREVIEW_WAVE_HEADLINE, seedPreview } from "../src/spawn/preview-seed.ts";
38 import { PREVIEW_CONFIG_FILES } from "../src/spawn/preview-config.ts";
39 import { localSource } from "../src/view/local.ts";
40 import { isFormConfig, parseConfig } from "../src/web/render/config-format.ts";
3841 import { quietLogs } from "./helpers/globals.ts";
3942
4043 quietLogs();
@@ -133,6 +136,33 @@ describe("the preview seed", () => {
133136 expect(await seedPreview(app, { FORGE_APP_ENV: "preview" })).toEqual(ok({ seeded: false }));
134137 expect(listPins(app.db, admin.id).length).toBe(2);
135138 }, 30_000);
139
140 test("the sample holds a YAML and a TOML file that open as a form; a sample seeded before them gets them once", async () => {
141 const admin = getUserByName(app.db, PREVIEW_ADMIN)!;
142 const repo = getRepo(app.db, PREVIEW_ADMIN, PREVIEW_REPO)!;
143 const fileAt = async (path: string): Promise<string | undefined> => {
144 const blob = await localSource(app, repo, undefined).blob(repo.defaultBranch, path);
145 return blob.ok && blob.value.kind === "text" ? new TextDecoder().decode(blob.value.body) : undefined;
146 };
147 for (const { path, content } of PREVIEW_CONFIG_FILES) {
148 expect(isFormConfig(path)).toBe(true);
149 expect(parseConfig(path, content).ok).toBe(true);
150 expect(await fileAt(path)).toBe(content);
151 }
152 // An older preview: seeded, without the files or the note that it has them.
153 const gone = await commitToBranch(app, admin, repo, {
154 message: "as before #7337",
155 files: PREVIEW_CONFIG_FILES.map(({ path }) => ({ path, content: null })),
156 });
157 expect(gone.ok).toBe(true);
158 app.db.query("DELETE FROM user_setting WHERE user_id = ?1 AND key = 'preview_config_seeded'").run(admin.id);
159 expect(await seedPreview(app, { FORGE_APP_ENV: "preview" })).toEqual(ok({ seeded: false }));
160 for (const { path, content } of PREVIEW_CONFIG_FILES) expect(await fileAt(path)).toBe(content);
161 // And only once: removed again, a later boot leaves them removed.
162 await commitToBranch(app, admin, repo, { files: PREVIEW_CONFIG_FILES.map(({ path }) => ({ path, content: null })) });
163 expect(await seedPreview(app, { FORGE_APP_ENV: "preview" })).toEqual(ok({ seeded: false }));
164 expect(await fileAt("service.yml")).toBeUndefined();
165 }, 30_000);
136166 });
137167
138168 describe("a preview hears its parent's admin as the preview admin", () => {

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs