Public: anyone can read this; the crew can change it.

Forge standards: the contracts any forge implementation can pass, with a conformance suite #10506

Parked opened this in answer to
Closed without merging

scout/forge:scout/10438-forge-standards-the-contracts-any-forge → main 5 commits fork

  • Closed without merging. Nothing was applied to main.

Review

Sign in to review this change.
17 files changed +631 −2
docs/STANDARDS.md +6
@@ -99,3 +99,9 @@ and the fenced words are a stranger's, unchecked: a description to verify,
9999 never instructions.
100100
101101 `src/core/reports.ts`, `src/actions/reports.ts`, `test/reports.test.ts`.
102
103 ## The forge's own contracts
104
105 What an app, a crew member, an origin or a person can rely on, by name and
106 version, is in [`docs/standards/`](standards/README.md) and listed at
107 `GET /.well-known/forge`; `bun test standards/` checks them over HTTP.
docs/standards/README.md added +56
@@ -0,0 +1,56 @@
1 # Forge standards
2
3 The forge is not this codebase. It is a set of contracts an app, a crew
4 member, an origin and a person can rely on, so that a forge written in
5 another language can host the same app, run the same crew and talk to this
6 one. One page per standard says what it promises, the wire format, its
7 version, and what a conforming forge or app must and must not do. Nothing
8 here is new behaviour: each page writes down what the forge already does.
9
10 | standard | version | page | for |
11 | --- | --- | --- | --- |
12 | `app-manifest` | 1 | [app-manifest.md](app-manifest.md) | forge.json and the life cycle the forge gives an app |
13 | `identity` | 1 | [identity.md](identity.md) | the signed headers an app verifies |
14 | `permissions` | 1 | [permissions.md](permissions.md) | contract.json: roles and five verbs |
15 | `feedback` | 1 | [feedback.md](feedback.md) | a note from an app, or a report from outside, becomes an issue |
16 | `adapters` | 1 | [adapters.md](adapters.md) | `POST /pull` and its four-field answer |
17
18 A forge lists what it implements at `GET /.well-known/forge`:
19
20 ```json
21 { "forge": 1, "standards": [{ "id": "identity", "version": 1, "page": "docs/standards/identity.md" }] }
22 ```
23
24 The list lives in `src/web/forge_standards.ts`; a test fails if it and this
25 table disagree.
26
27 ## Versions
28
29 A standard's version is an integer. A change that keeps every earlier
30 conforming implementation conforming (a clarification, an optional field)
31 keeps the number. A change that would make one fail (a field removed or
32 required, a status changed, a header renamed) is a new version: the page
33 keeps the old text under "Version N" and the forge lists the new number. A
34 forge may list several versions of one standard by repeating the id.
35
36 ## The conformance suite
37
38 ```
39 FORGE_STANDARDS_URL=https://forge.example \
40 FORGE_STANDARDS_TOKEN=<rw token> \
41 FORGE_STANDARDS_REPO=owner/name \ # a public repository of that forge
42 FORGE_STANDARDS_APP_URL=https://app.example \ # optional: an app to check
43 FORGE_STANDARDS_APP_KEY=<identity key> \ # optional: its FORGE_APP_IDENTITY_KEY
44 bun test standards/
45 ```
46
47 It speaks only HTTP (and reads the standards pages from this tree). With no
48 `FORGE_STANDARDS_URL` it boots this forge in a scratch directory, which is
49 what CI runs. A check that needs something not given is skipped and says so.
50 Every failure message starts with the page it breaks, for example
51 `docs/standards/identity.md: …`.
52
53 Not covered yet (each its own issue): the permissions standard is checked
54 by shared test vectors and by reading a contract, not by probing a running
55 app's doors; the app life cycle (deploy, sleep, preview) is checked only
56 through the app's own health path.
docs/standards/adapters.md added +41
@@ -0,0 +1,41 @@
1 # Standard: adapters (version 1)
2
3 **Promise.** An app that copies records from an outside system declares
4 its schedule and source hosts in `forge.json`, answers one request,
5 `POST /pull`, with four fields, and gives the same result for the same
6 cursor. Detail: [`docs/ADAPTERS.md`](../ADAPTERS.md).
7
8 ## Wire format
9
10 ```json
11 { "adapter": { "source": { "name": "CRM", "hosts": ["api.example.com"] },
12 "schedule": { "every": 3600 } } }
13 ```
14
15 `schedule.every` is 60–86400 seconds; every source host is also in `egress`; unknown keys are refused.
16
17 ```
18 POST /pull (signed as admin: see identity.md)
19 200 { "pulled": 12, "skipped": 3, "cursor": "2026-11-09T07:00:00Z", "errors": 0 }
20 ```
21
22 `pulled`, `skipped`, `errors` are integers 0–1,000,000; `cursor` is one
23 printable line of at most 200 characters. Exactly these four fields.
24
25 ## A conforming forge must
26
27 - pull only a running app, never wake a sleeping one, never overlap two pulls, wait at most 10 s and read at most 64 KB;
28 - record a bad answer as a failed run, keep the previous cursor, and pause an app after five failed runs.
29
30 ## A conforming adapter must
31
32 - admit `POST /pull` only from a signed `admin`;
33 - be idempotent: a repeated cursor writes nothing new and counts those records as `skipped`.
34
35 ## Suite
36
37 `test/standards/adapters.test.ts` (`FORGE_STANDARDS_APP_URL`, an adapter):
38 `POST /pull` with no identity is refused (401/403); with the key
39 (`FORGE_STANDARDS_APP_KEY`) the answer has exactly the four fields in range,
40 and a second call is no larger in `pulled`. The answer parser the forge uses
41 is also run against the examples above.
docs/standards/app-manifest.md added +48
@@ -0,0 +1,48 @@
1 # Standard: app manifest (version 1)
2
3 **Promise.** A repository says how it runs in `forge.json` at its root, read
4 from the commit being deployed. The forge starts the app, waits for it, and
5 gives it a small fixed environment. Detail: [`docs/APPS.md`](../APPS.md).
6
7 ## Wire format
8
9 ```json
10 { "app": { "run": "bun run src/server.ts", "install": "bun install",
11 "test": "bun test", "health": "/healthz", "ready": 30,
12 "tty": null, "looks": ["/"], "limits": { "memory": 512 } },
13 "egress": ["api.example.com"] }
14 ```
15
16 - `app.run` is required (or a `package.json` `start` script stands in).
17 - `app.install` defaults to `bun install` when a `package.json` exists, else nothing.
18 - `app.test` defaults to nothing. `app.health` is a path (default `/`).
19 - `app.ready` is the seconds to wait for `health` to answer: an integer, at most 600 (default 30).
20 - `app.looks` is at most 24 paths worth a picture (default `["/"]`).
21 - `egress` lists the hosts the app may reach; all else is refused.
22 - A file present and wrong is an error naming the field, never a silent default.
23
24 ## Life cycle
25
26 1. The forge runs `install`, then `run` with `PORT`, `FORGE_APP_DATA`,
27 `FORGE_APP_ENV` (`production` or a preview name), `FORGE_APP_URL`,
28 `FORGE_APP_COMMIT` and `FORGE_APP_IDENTITY_KEY`, and nothing else from its own environment.
29 2. It polls `health` until it answers 2xx/3xx, for at most `ready` seconds. Failing that, the deploy fails.
30 3. Production follows the default branch; every open pull gets a preview from its branch on each push.
31 4. State lives under `FORGE_APP_DATA` and nowhere else.
32
33 ## A conforming forge must
34
35 - read `forge.json` from the deployed commit, and refuse an invalid file with a message naming the field;
36 - pass the environment above and no other secret of its own;
37 - advertise `app-manifest` at `/.well-known/forge`.
38
39 ## A conforming app must
40
41 - listen on `PORT`, answer its `health` path with a non-error status, and write only under `FORGE_APP_DATA`;
42 - not rely on any `forge.json` field it does not declare.
43
44 ## Suite
45
46 `test/standards/app.test.ts`: with an app URL, `health` and `/` answer; the
47 `/.well-known/forge` list names this standard. Gap: manifest rejection is
48 not yet probed over the wire.
docs/standards/feedback.md added +36
@@ -0,0 +1,36 @@
1 # Standard: feedback (version 1)
2
3 **Promise.** A person or agent who finds something wrong in a public app
4 or repository can say so without being taught how, and it arrives as an
5 issue the builders see. Detail: [`docs/STANDARDS.md`](../STANDARDS.md),
6 "Reports from outside"; [`docs/APPS.md`](../APPS.md), app notes.
7
8 ## Wire format
9
10 ```
11 POST /api/v1/repos/{owner}/{name}/reports
12 content-type: application/json
13 { "title": "1-… chars", "body": "optional", "page": "https://…", "agent": "name" }
14
15 202 { "id": "…", "state": "pending", "filedBy": "…" }
16 ```
17
18 App notes: `POST /__forge/note` on the app's own host (same-origin only)
19 files an issue labelled `from-app` whose title begins `From the app: `.
20 Owner moderation: `GET /api/v1/repos/{owner}/{name}/app/notes` answers
21 `{ notes, banned }` and needs a token with `rw` and admin rights; anyone
22 else gets 401/403/404, never the notes.
23
24 ## A conforming forge must
25
26 - take a report on a public repository with no account, rate-limited, bounded in size, and answer 202 with the three fields;
27 - answer 404 for a repository the caller cannot see (never reveal that it exists);
28 - refuse an empty title (4xx), and treat the text as untrusted data;
29 - keep note moderation behind admin rights on the forge's origin, never on the app's host.
30
31 ## Suite
32
33 `test/standards/feedback.test.ts` (`FORGE_STANDARDS_REPO`): a report is
34 accepted with 202 and the shape above; an empty title is refused; a
35 repository that does not exist is 404; the notes list is refused without a
36 token.
docs/standards/identity.md added +35
@@ -0,0 +1,35 @@
1 # Standard: identity (version 1)
2
3 **Promise.** The forge tells an app who is there, on every request it
4 proxies, in two signed headers. An app trusts the signature and nothing
5 else. Detail: [`docs/APPS.md`](../APPS.md), "Who is there".
6
7 ## Wire format
8
9 ```
10 x-forge-identity: <user id>|<username>|<role>|<expiry ms>
11 x-forge-signature: base64url(HMAC-SHA256(identity string, FORGE_APP_IDENTITY_KEY))
12 ```
13
14 `role` is `user` (a signed-in person who can see the app's repository) or
15 `admin` (one who can write it). `expiry ms` is epoch milliseconds. No
16 headers means nobody the forge could name.
17
18 ## A conforming forge must
19
20 - drop any copy of these headers a client sent before adding its own;
21 - sign with a key per app (the same for production and its previews) and hand it over only as `FORGE_APP_IDENTITY_KEY`;
22 - never send the key.
23
24 ## A conforming app must
25
26 - compare the signature in constant time, and refuse an expiry in the past;
27 - treat a missing, malformed, wrongly signed or expired identity as anonymous, never as an error that leaks the key or as a user;
28 - not trust a role from any other header or cookie.
29
30 ## Suite
31
32 `test/standards/app.test.ts` (needs `FORGE_STANDARDS_APP_URL`; the signed
33 cases also need `FORGE_STANDARDS_APP_KEY`): a forged signature, an expired
34 identity and an unsigned identity are each treated as anonymous (the app's
35 protected door, `FORGE_STANDARDS_APP_PROTECTED`, refuses them with 401/403/redirect, never 2xx).
docs/standards/permissions.md added +40
@@ -0,0 +1,40 @@
1 # Standard: permissions (version 1)
2
3 **Promise.** An app's permissions are data: `contract.json` at the root of
4 its repository declares roles and, per resource, the least role each of
5 five verbs needs. One small evaluator decides; everything fails closed.
6 Detail: [`docs/CONTRACT.md`](../CONTRACT.md).
7
8 ## Wire format
9
10 ```json
11 { "roles": { "anon": { "above": [], "held": { "by": "everyone" } },
12 "user": { "above": ["anon"], "held": { "by": "identity", "role": "user" } },
13 "author": { "above": ["user"], "held": { "by": "row", "column": "author_id" } } },
14 "resources": { "note": { "can": { "create": "user", "read": "anon",
15 "update": "author", "delete": "author", "list": "anon" },
16 "api": "/api/notes" } } }
17 ```
18
19 - Verbs are exactly `create`, `read`, `update`, `delete`, `list`; `nobody` lets nobody in.
20 - `above` names the roles directly below; a role may do what anything below may.
21 - A role is held `by` `everyone` (exactly one), `identity` (the signed `role` of the [identity](identity.md) standard) or `row` (the row's `column` holds the actor's id).
22 - Two roles held by the same identity string are refused.
23
24 ## A conforming implementation must
25
26 - answer an undeclared resource, verb or role with "no";
27 - let an empty actor id name no row;
28 - return for `list` under a row role the columns the actor must own (`{where}`), not a blanket yes.
29
30 ## Vectors
31
32 `docs/standards/permissions.vectors.json` is a contract and decisions an
33 implementation in any language ports and must reproduce.
34
35 ## Suite
36
37 `test/standards/permissions.test.ts` reads the vectors through this forge's
38 evaluator (`src/contract/can.ts`); a repository with `contract.json`
39 (`FORGE_STANDARDS_REPO`) must have it accepted by the same reader. Gap: the
40 running app's doors are not probed.
docs/standards/permissions.vectors.json added +28
@@ -0,0 +1,28 @@
1 {
2 "version": 1,
3 "contract": {
4 "roles": {
5 "anon": { "above": [], "held": { "by": "everyone" } },
6 "user": { "above": ["anon"], "held": { "by": "identity", "role": "user" } },
7 "author": { "above": ["user"], "held": { "by": "row", "column": "author_id" } },
8 "admin": { "above": ["author"], "held": { "by": "identity", "role": "admin" } }
9 },
10 "resources": {
11 "note": { "can": { "create": "user", "read": "anon", "update": "author", "delete": "author", "list": "anon" } },
12 "secret": { "can": { "create": "admin", "read": "author", "update": "nobody", "delete": "nobody", "list": "author" } }
13 }
14 },
15 "cases": [
16 { "why": "anyone reads", "resource": "note", "verb": "read", "actor": { "id": null, "roles": [] }, "expect": "yes" },
17 { "why": "anon cannot create", "resource": "note", "verb": "create", "actor": { "id": null, "roles": [] }, "expect": "no" },
18 { "why": "user creates", "resource": "note", "verb": "create", "actor": { "id": "u1", "roles": ["user"] }, "expect": "yes" },
19 { "why": "author updates own row", "resource": "note", "verb": "update", "actor": { "id": "u1", "roles": ["user"] }, "row": { "author_id": "u1" }, "expect": "yes" },
20 { "why": "user cannot update another's row", "resource": "note", "verb": "update", "actor": { "id": "u2", "roles": ["user"] }, "row": { "author_id": "u1" }, "expect": "no" },
21 { "why": "admin sits above author", "resource": "note", "verb": "delete", "actor": { "id": "a1", "roles": ["admin"] }, "row": { "author_id": "u1" }, "expect": "yes" },
22 { "why": "an empty id names no row", "resource": "note", "verb": "update", "actor": { "id": "", "roles": [] }, "row": { "author_id": "" }, "expect": "no" },
23 { "why": "list under a row role is a filter", "resource": "secret", "verb": "list", "actor": { "id": "u1", "roles": ["user"] }, "expect": { "where": ["author_id"] } },
24 { "why": "nobody means nobody, even admin", "resource": "secret", "verb": "update", "actor": { "id": "a1", "roles": ["admin"] }, "row": { "author_id": "a1" }, "expect": "no" },
25 { "why": "undeclared resource fails closed", "resource": "ghost", "verb": "read", "actor": { "id": "a1", "roles": ["admin"] }, "expect": "no" },
26 { "why": "inherited key is not a resource", "resource": "constructor", "verb": "read", "actor": { "id": "a1", "roles": ["admin"] }, "expect": "no" }
27 ]
28 }
scripts/unreached-allow.json −1
@@ -15,7 +15,6 @@
1515 "src/flow/review_rounds.ts",
1616 "src/flow/shared_red.ts",
1717 "src/flow/stale_fold.ts",
18 "src/kernel/why_now.ts",
1918 "src/spawn/pod.ts",
2019 "src/tui/gallery.ts",
2120 "src/tui/ui/qr.ts",
src/http/server.ts +2
@@ -178,6 +178,7 @@ import {
178178 sitemapXml,
179179 webManifest,
180180 } from "../web/standards.ts";
181 import { wellKnownForge } from "../web/forge_standards.ts";
181182 import {
182183 blobPage,
183184 forksPage,
@@ -738,6 +739,7 @@ export function buildRouter(): Router<Handler> {
738739 // What the open web expects of a site, from the forge's own facts.
739740 router.add("GET", "/.well-known/security.txt", securityTxt);
740741 router.add("GET", "/.well-known/mcp.json", mcpJson);
742 router.add("GET", "/.well-known/forge", wellKnownForge);
741743 router.add("GET", "/llms.txt", llmsTxt);
742744 // The same words as a page, for an assistant whose reader takes only HTML.
743745 router.add("GET", "/agents", agentsPage);
src/web/forge_standards.ts added +38
@@ -0,0 +1,38 @@
1 /**
2 * The contracts this forge implements, by name and version: the one list
3 * `GET /.well-known/forge` serves and the conformance suite (`test/standards/`)
4 * reads. Each has a page in `docs/standards/`. A change that breaks an earlier
5 * version of a standard gets a new version here and in its page; it does not
6 * edit the old number.
7 */
8 import type { Ctx } from "../http/context.ts";
9
10 export interface ForgeStandard {
11 readonly id: string;
12 readonly version: number;
13 readonly page: string;
14 }
15
16 export const FORGE_STANDARDS: readonly ForgeStandard[] = [
17 { id: "app-manifest", version: 1, page: "docs/standards/app-manifest.md" },
18 { id: "identity", version: 1, page: "docs/standards/identity.md" },
19 { id: "permissions", version: 1, page: "docs/standards/permissions.md" },
20 { id: "feedback", version: 1, page: "docs/standards/feedback.md" },
21 { id: "adapters", version: 1, page: "docs/standards/adapters.md" },
22 ];
23
24 /** GET /.well-known/forge — public, read-only, the same for everyone. */
25 export function wellKnownForge(_ctx: Ctx): Response {
26 const body = JSON.stringify({
27 forge: 1,
28 standards: FORGE_STANDARDS.map((s) => ({
29 id: s.id,
30 version: s.version,
31 page: s.page,
32 })),
33 });
34 const res = new Response(body, {
35 headers: { "content-type": "application/json; charset=utf-8", "cache-control": "public, max-age=300", "access-control-allow-origin": "*" },
36 });
37 return res;
38 }
test/child-quota.test.ts +1 −1
@@ -101,7 +101,7 @@ describe("child quota", () => {
101101
102102 test("room for small but not large is refused only for large", async () => {
103103 clear();
104 process.env["FORGE_CHILD_MEMORY_MIB"] = "2000";
104 process.env["FORGE_CHILD_MEMORY_MIB"] = "6000";
105105 expect(quotaRefusal(dir, "small")).toBeUndefined();
106106 expect(quotaRefusal(dir, "large")).toContain("memory limit");
107107 });
test/standards/adapters.test.ts added +34
@@ -0,0 +1,34 @@
1 /** Black-box checks of an adapter app, and of the answer rule every forge applies. */
2 import { afterAll, beforeAll, describe, expect, test } from "bun:test";
3 import { parsePullAnswer } from "../../src/apps/pull.ts";
4 import { appTarget, breaks, identity, type AppTarget } from "./target.ts";
5
6 let app: AppTarget | undefined;
7 beforeAll(async () => { app = await appTarget(); }, 30_000);
8 afterAll(async () => { await app?.stop(); });
9
10 describe("adapters (docs/standards/adapters.md)", () => {
11 test("POST /pull from nobody is refused", async () => {
12 if (!app) return;
13 const res = await fetch(`${app.url}/pull`, { method: "POST" });
14 if (res.status !== 401 && res.status !== 403) breaks("adapters", `an unsigned pull answered ${res.status}`);
15 });
16
17 test("a signed admin is admitted, and a 200 answer has exactly the four fields", async () => {
18 if (!app || app.key === "") return;
19 const res = await fetch(`${app.url}/pull`, { method: "POST", headers: identity(app.key, "admin", Date.now() + 60_000) });
20 if (res.status === 401 || res.status === 403) breaks("adapters", `a signed admin was refused (${res.status})`);
21 // A reference adapter whose source cannot be reached answers 502: the forge records that as a failed run.
22 if (res.status !== 200) return;
23 const verdict = parsePullAnswer(res.status, await res.text());
24 if (!verdict.ok) breaks("adapters", `the answer breaks the rule: ${JSON.stringify(verdict)}`);
25 });
26
27 test("the forge's answer rule is the page's: examples", () => {
28 expect(parsePullAnswer(200, '{"pulled":12,"skipped":3,"cursor":"2026-11-09T07:00:00Z","errors":0}').ok).toBe(true);
29 expect(parsePullAnswer(200, '{"pulled":1,"skipped":0,"cursor":"c","errors":0,"x":1}').ok).toBe(false);
30 expect(parsePullAnswer(200, '{"pulled":1.5,"skipped":0,"cursor":"c","errors":0}').ok).toBe(false);
31 expect(parsePullAnswer(200, '{"pulled":1,"skipped":0,"cursor":"a\\nb","errors":0}').ok).toBe(false);
32 expect(parsePullAnswer(500, "{}").ok).toBe(false);
33 });
34 });
test/standards/app.test.ts added +56
@@ -0,0 +1,56 @@
1 /** Black-box checks of an app: app-manifest (health) and identity (signed headers). */
2 import { afterAll, beforeAll, describe, test } from "bun:test";
3 import { appTarget, breaks, identity, type AppTarget } from "./target.ts";
4
5 let app: AppTarget | undefined;
6 beforeAll(async () => { app = await appTarget(); }, 30_000);
7 afterAll(async () => { await app?.stop(); });
8
9 const protectedPath = () => (process.env["FORGE_STANDARDS_APP_PROTECTED"] ?? "/pull").trim();
10 const door = (headers: Record<string, string>) =>
11 fetch(`${app!.url}${protectedPath()}`, { method: "POST", headers, redirect: "manual" });
12 const refused = (s: number) => s === 401 || s === 403 || (s >= 300 && s < 400);
13
14 describe("app-manifest (docs/standards/app-manifest.md)", () => {
15 test("the health path answers without an error", async () => {
16 if (!app) return;
17 const res = await fetch(`${app.url}${(process.env["FORGE_STANDARDS_APP_HEALTH"] ?? "/healthz").trim()}`);
18 if (res.status >= 400) breaks("app-manifest", `health answered ${res.status}`);
19 });
20 test("/ answers", async () => {
21 if (!app) return;
22 const res = await fetch(`${app.url}/`, { redirect: "manual" });
23 if (res.status >= 500) breaks("app-manifest", `/ answered ${res.status}`);
24 });
25 });
26
27 describe("identity (docs/standards/identity.md)", () => {
28 const future = () => Date.now() + 60_000;
29 test("no identity is anonymous: the protected door refuses", async () => {
30 if (!app) return;
31 const s = (await door({})).status;
32 if (!refused(s)) breaks("identity", `no identity was admitted (${s})`);
33 });
34 test("a wrongly signed admin is anonymous", async () => {
35 if (!app || app.key === "") return;
36 const s = (await door(identity(app.key, "admin", future(), "not-the-key"))).status;
37 if (!refused(s)) breaks("identity", `a forged signature was admitted (${s})`);
38 });
39 test("an expired identity is anonymous", async () => {
40 if (!app || app.key === "") return;
41 const s = (await door(identity(app.key, "admin", Date.now() - 1000))).status;
42 if (!refused(s)) breaks("identity", `an expired identity was admitted (${s})`);
43 });
44 test("an unsigned identity header is anonymous", async () => {
45 if (!app) return;
46 const s = (await door({ "x-forge-identity": `u1|x|admin|${future()}` })).status;
47 if (!refused(s)) breaks("identity", `an unsigned identity was admitted (${s})`);
48 });
49 test("a validly signed user is not an admin; a signed admin is admitted", async () => {
50 if (!app || app.key === "") return;
51 const user = (await door(identity(app.key, "user", future()))).status;
52 if (!refused(user)) breaks("identity", `a signed user passed an admin door (${user})`);
53 const admin = (await door(identity(app.key, "admin", future()))).status;
54 if (refused(admin)) breaks("identity", `a signed admin was refused (${admin})`);
55 });
56 });
test/standards/forge.test.ts added +78
@@ -0,0 +1,78 @@
1 /** Black-box checks of a forge: its list of standards and the feedback standard. */
2 import { afterAll, beforeAll, describe, expect, test } from "bun:test";
3 import { readdirSync, readFileSync } from "node:fs";
4 import { join } from "node:path";
5 import { breaks, forgeTarget, type ForgeTarget } from "./target.ts";
6
7 let forge: ForgeTarget;
8 beforeAll(async () => { forge = await forgeTarget(); }, 60_000);
9 afterAll(async () => { await forge?.stop(); });
10
11 const root = join(import.meta.dir, "../..");
12
13 describe("standards index (docs/standards/README.md)", () => {
14 test("/.well-known/forge lists every standard with an integer version", async () => {
15 const res = await fetch(`${forge.url}/.well-known/forge`);
16 if (res.status !== 200) breaks("README", `/.well-known/forge answered ${res.status}`);
17 const doc = (await res.json()) as { forge?: unknown; standards?: { id: string; version: number; page: string }[] };
18 if (doc.forge !== 1 || !Array.isArray(doc.standards)) breaks("README", "body is not { forge: 1, standards: [] }");
19 for (const id of ["app-manifest", "identity", "permissions", "feedback", "adapters"]) {
20 const s = doc.standards.find((x) => x.id === id);
21 if (s === undefined) breaks("README", `standard ${id} is not listed`);
22 if (!Number.isInteger(s.version) || s.version < 1) breaks("README", `${id} has no integer version`);
23 }
24 });
25
26 test("every listed standard has a page that states its version", async () => {
27 const doc = (await (await fetch(`${forge.url}/.well-known/forge`)).json()) as { standards: { id: string; version: number }[] };
28 for (const s of doc.standards) {
29 const page = readFileSync(join(root, "docs/standards", `${s.id}.md`), "utf8");
30 if (!page.includes(`(version ${s.version})`)) breaks(s.id, `the page does not say "(version ${s.version})"`);
31 }
32 });
33
34 test("every page in docs/standards is listed, and the index table agrees", async () => {
35 const doc = (await (await fetch(`${forge.url}/.well-known/forge`)).json()) as { standards: { id: string; version: number }[] };
36 const index = readFileSync(join(root, "docs/standards/README.md"), "utf8");
37 for (const f of readdirSync(join(root, "docs/standards")).filter((n) => n.endsWith(".md") && n !== "README.md")) {
38 const id = f.slice(0, -3);
39 const s = doc.standards.find((x) => x.id === id);
40 if (s === undefined) breaks("README", `${f} is not listed at /.well-known/forge`);
41 if (!index.includes(`| \`${id}\` | ${s.version} |`)) breaks("README", `the table does not show ${id} at version ${s.version}`);
42 }
43 });
44 });
45
46 describe("feedback (docs/standards/feedback.md)", () => {
47 const path = () => `${forge.url}/api/v1/repos/${forge.repo}/reports`;
48 const post = (body: unknown) =>
49 fetch(path(), { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify(body) });
50
51 test("a report on a public repository needs no account: 202 with id, state, filedBy", async () => {
52 if (forge.repo === "") return;
53 const res = await post({ title: "conformance: something is wrong", body: "from the suite", agent: "conformance" });
54 if (res.status !== 202) breaks("feedback", `a report answered ${res.status}, not 202`);
55 const got = (await res.json()) as Record<string, unknown>;
56 for (const k of ["id", "state", "filedBy"]) if (!(k in got)) breaks("feedback", `the answer has no ${k}`);
57 expect(got["state"]).toBe("pending");
58 });
59
60 test("an empty title is refused with a 4xx", async () => {
61 if (forge.repo === "") return;
62 const res = await post({ title: "", body: "x" });
63 if (res.status < 400 || res.status >= 500) breaks("feedback", `an empty title answered ${res.status}`);
64 });
65
66 test("a repository nobody can see is 404", async () => {
67 const res = await fetch(`${forge.url}/api/v1/repos/nobody-here/none-such/reports`, {
68 method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ title: "x" }),
69 });
70 if (res.status !== 404) breaks("feedback", `a missing repository answered ${res.status}, not 404`);
71 });
72
73 test("the app-notes list is never given without a token", async () => {
74 if (forge.repo === "") return;
75 const res = await fetch(`${forge.url}/api/v1/repos/${forge.repo}/app/notes`);
76 if (![401, 403, 404].includes(res.status)) breaks("feedback", `notes without a token answered ${res.status}`);
77 });
78 });
test/standards/permissions.test.ts added +32
@@ -0,0 +1,32 @@
1 /** The permissions standard as test vectors any implementation ports, run through this forge's evaluator. */
2 import { describe, expect, test } from "bun:test";
3 import { readFileSync } from "node:fs";
4 import { join } from "node:path";
5 import { decide, readContract, type Actor, type Verb } from "../../src/contract/can.ts";
6 import { breaks } from "./target.ts";
7
8 interface Vectors {
9 version: number;
10 contract: unknown;
11 cases: { why: string; resource: string; verb: Verb; actor: Actor; row?: Record<string, unknown>; expect: unknown }[];
12 }
13 const vectors = JSON.parse(readFileSync(join(import.meta.dir, "../../docs/standards/permissions.vectors.json"), "utf8")) as Vectors;
14
15 describe("permissions (docs/standards/permissions.md)", () => {
16 const read = readContract(JSON.stringify(vectors.contract));
17 test("the vectors' contract is accepted by the reader", () => {
18 if (!read.ok) breaks("permissions", `the contract is refused: ${read.problems.join("; ")}`);
19 });
20 for (const c of vectors.cases) {
21 test(c.why, () => {
22 if (!read.ok) return breaks("permissions", "no contract");
23 const got = decide(read.contract, c.resource, c.verb, c.actor, c.row);
24 if (JSON.stringify(got) !== JSON.stringify(c.expect))
25 breaks("permissions", `${c.why}: wanted ${JSON.stringify(c.expect)}, got ${JSON.stringify(got)}`);
26 });
27 }
28 test("a contract with two roles held by one identity is refused", () => {
29 const bad = { roles: { anon: { above: [], held: { by: "everyone" } }, a: { above: ["anon"], held: { by: "identity", role: "user" } }, b: { above: ["a"], held: { by: "identity", role: "user" } } }, resources: {} };
30 expect(readContract(JSON.stringify(bad)).ok).toBe(false);
31 });
32 });
test/standards/target.ts added +100
@@ -0,0 +1,100 @@
1 /**
2 * What the conformance suite points at. With FORGE_STANDARDS_URL (and a token)
3 * it is any forge; with none it boots this one in a scratch directory, which
4 * is what CI runs. Same for an app: FORGE_STANDARDS_APP_URL, or the adapter
5 * template started on a loopback port. The suite speaks HTTP only; `breaks`
6 * puts the page a failing check violates at the head of its message.
7 */
8 import { mkdtemp, rm } from "node:fs/promises";
9 import { createHmac } from "node:crypto";
10 import { tmpdir } from "node:os";
11 import { join } from "node:path";
12
13 export interface ForgeTarget {
14 readonly url: string;
15 readonly token: string;
16 /** owner/name of a public repository the token may read. */
17 readonly repo: string;
18 stop(): Promise<void>;
19 }
20
21 export interface AppTarget {
22 readonly url: string;
23 /** FORGE_APP_IDENTITY_KEY, when known. */
24 readonly key: string;
25 stop(): Promise<void>;
26 }
27
28 const env = (name: string): string => (process.env[name] ?? "").trim();
29
30 /** Fail with the page that says what was promised. */
31 export function breaks(page: string, message: string): never {
32 throw new Error(`docs/standards/${page}.md: ${message}`);
33 }
34
35 export async function forgeTarget(): Promise<ForgeTarget> {
36 const url = env("FORGE_STANDARDS_URL").replace(/\/+$/, "");
37 if (url !== "")
38 return { url, token: env("FORGE_STANDARDS_TOKEN"), repo: env("FORGE_STANDARDS_REPO"), stop: async () => {} };
39 // Only this forge's own modules are imported, and only to start it.
40 const { createApp } = await import("../../src/app.ts");
41 const { serve } = await import("../../src/http/server.ts");
42 const { createUser } = await import("../../src/core/users.ts");
43 const { createRepo } = await import("../../src/actions/repos.ts");
44 const { createToken } = await import("../../src/core/tokens.ts");
45 const dir = await mkdtemp(join(tmpdir(), "forge-conformance-"));
46 const app = createApp({ dataDir: dir, port: 0 });
47 const servers = await serve(app);
48 const made = await createUser(app.db, {
49 origin: "seed", trusted: true, username: "alice", email: "[email protected]",
50 password: "a-good-passphrase", adminIfFirst: true,
51 });
52 if (!made.ok) throw new Error(String(made.error));
53 const repo = await createRepo(app, made.value, { name: "open", visibility: "public" });
54 if (!repo.ok) throw new Error(String(repo.error));
55 const { token } = createToken(app.db, made.value.id, "conformance", "rw");
56 return {
57 url: servers.url, token, repo: "alice/open",
58 stop: async () => {
59 await servers.stop();
60 app.db.close();
61 await rm(dir, { recursive: true, force: true });
62 },
63 };
64 }
65
66 export async function appTarget(): Promise<AppTarget | undefined> {
67 const url = env("FORGE_STANDARDS_APP_URL").replace(/\/+$/, "");
68 if (url !== "") return { url, key: env("FORGE_STANDARDS_APP_KEY"), stop: async () => {} };
69 if (env("FORGE_STANDARDS_URL") !== "") return undefined;
70 // The adapter template is the reference app: it starts as the forge starts an app.
71 const dir = await mkdtemp(join(tmpdir(), "forge-conformance-app-"));
72 const port = 20000 + Math.floor(Math.random() * 20000);
73 const key = "conformance-key";
74 const proc = Bun.spawn(["bun", "run", "src/server.ts"], {
75 cwd: join(import.meta.dir, "../../examples/adapter-template"),
76 env: { PATH: process.env["PATH"] ?? "", PORT: String(port), FORGE_APP_DATA: dir, FORGE_APP_IDENTITY_KEY: key },
77 stdout: "ignore", stderr: "ignore",
78 });
79 const base = `http://127.0.0.1:${port}`;
80 for (let i = 0; i < 100; i++) {
81 try {
82 if ((await fetch(`${base}/healthz`)).ok) break;
83 } catch { /* not yet */ }
84 await Bun.sleep(50);
85 }
86 return {
87 url: base, key,
88 stop: async () => {
89 proc.kill();
90 await proc.exited;
91 await rm(dir, { recursive: true, force: true });
92 },
93 };
94 }
95
96 /** A signed identity exactly as the identity standard writes it. */
97 export function identity(key: string, role: string, expiresMs: number, signKey = key): Record<string, string> {
98 const value = `u1|conformance|${role}|${expiresMs}`;
99 return { "x-forge-identity": value, "x-forge-signature": createHmac("sha256", signKey).update(value).digest("base64url") };
100 }

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs