trevato/forge

Public: anyone can read this; the crew can change it.

More
Forks1g fMapg aRuns3g rEventsFlows

Forge standards: the contracts any forge implementation can pass, with a conformance suite #10653

Merged masoncrew opened this in answer to trevatoowner

Check · Review · Land · Live done — Landed and live in production.

ConversationFiles
Merged

mason/forge:mason/10438-forge-standards-the-contracts-any-forge → main 2 commits fork 3 threads · 3 unresolved

  • Merged. mason/10438-forge-standards-the-contracts-any-forge was merged into main as 63138e8.

Review

Sign in to review this change.
14 files changed +736 −0
docs/STANDARDS.md +4
@@ -8,6 +8,7 @@ its own mark. Nothing here is configured, and nothing here is optional.
88 | path | what it is | from |
99 | ------------------------------ | ----------------------------------------------------------------- | ---------------------------------------------------------------------------- |
1010 | `/.well-known/security.txt` | who to tell about a vulnerability (RFC 9116), with a fixed expiry | the sealed new-issue form of `FORGE_REPO` when it is public, else the first account's page here; an address only from `FORGE_SECURITY_CONTACT` |
11 | `/.well-known/forge` | the standards this forge implements, by id and version ([`docs/standards/`](standards/README.md)) | `src/core/standards_index.ts` |
1112 | `/.well-known/change-password` | where a password manager sends a person | `/settings/password` |
1213 | `/robots.txt` | crawlers stay out of accounts, search and the API; the sitemap | — |
1314 | `/sitemap.xml` | the front page and every public repository, never a private one | `listReposVisibleTo(db, nobody)` |
@@ -16,6 +17,9 @@ its own mark. Nothing here is configured, and nothing here is optional.
1617 | `/icons/forge-<size>.png` | the mark at 192, 512, 512-maskable and 180 (iOS), painted at boot | `design/mark.ts`, the favicon's two colours |
1718 | `/:owner/:name/activity.atom` | a repository's activity as Atom: pushes, issues, pulls, forks | a fixed list of kinds, as sentences; never a raw payload |
1819
20 `/.well-known/forge` is deliberately machine-only: it has no page, terminal
21 screen or feature doc, because its reader is a program checking conformance.
22
1923 Every page carries a description, Open Graph and Twitter card metadata, the
2024 manifest link, the touch icon and `viewport-fit=cover`, so a link to a
2125 repository unfurls with its description and the forge installs to a home
docs/standards/README.md added +65
@@ -0,0 +1,65 @@
1 # Forge standards
2
3 The forge is not this codebase; it is the contracts an app, a crew member, an
4 origin and a person can rely on. Two forges in different languages should host
5 the same app, run the same crew and talk to each other. Each page here is one
6 of those contracts: what it promises, the wire format, its version, and what a
7 conforming forge or app **must** and **must not** do.
8
9 These pages write down what the forge does today. They add no behaviour; a gap
10 the conformance suite finds becomes its own issue.
11
12 | standard | version | page |
13 | --- | --- | --- |
14 | App manifest: `forge.json` and its life cycle | 1 | [app-manifest.md](app-manifest.md) |
15 | Identity: the signed headers an app verifies | 1 | [identity.md](identity.md) |
16 | Permissions: `contract.json` | 1 | [permissions.md](permissions.md) |
17 | Feedback: app notes and reports become issues | 1 | [feedback.md](feedback.md) |
18 | Adapters: `POST /pull` | 1 | [adapters.md](adapters.md) |
19
20 The same list, machine-readable, is `GET /.well-known/forge` on every forge:
21
22 ```json
23 {
24 "forge": { "name": "forge", "version": "0.7.300" },
25 "standards": [
26 { "id": "identity", "version": 1, "title": "…", "page": "docs/standards/identity.md" }
27 ]
28 }
29 ```
30
31 `page` is a path in the forge's repository. A forge lists a standard only if it
32 passes that version of the suite.
33
34 ## Versioning
35
36 Each standard has one whole-number version, on its page (`Version: N`) and in
37 `src/core/standards_index.ts`. A change that would make an implementation which
38 passed version N fail is a new version N+1: the page says what changed, the
39 suite keeps testing every version a forge may still claim, and a forge lists
40 the versions it implements. Wording, examples and new optional fields do not
41 change the version. `test/standards-conformance.test.ts` fails when a page and
42 the index disagree.
43
44 ## The conformance suite
45
46 `standards/` is black-box: it speaks HTTP to a URL and nothing else, so it
47 runs against any forge or app, in any language.
48
49 ```sh
50 FORGE_URL=https://forge.example FORGE_TOKEN=<rw token> bun test ./standards/
51 APP_URL=https://notes.forge.example bun test ./standards/ # an app alone
52 ```
53
54 | variable | meaning |
55 | --- | --- |
56 | `FORGE_URL` | a forge's origin. Without it the forge tests are skipped. |
57 | `FORGE_TOKEN` | an `rw` token. The suite creates one scratch public repository (`standards-<random>`) under that account and deletes it at the end. Without it only the token-free checks run. |
58 | `APP_URL` | an app's origin. Without it the app tests are skipped. |
59 | `APP_ADAPTER=1` | the app declares itself an adapter; adds the `POST /pull` checks. |
60
61 (`./` matters: the repository's `bunfig.toml` roots plain filters in `test/`.)
62
63 A failing check begins with `[<standard> v<N>: docs/standards/<page>]`, the page
64 it breaks. The forge's own check runs the suite against itself and against a
65 minimal reference app (`test/standards-conformance.test.ts`).
docs/standards/adapters.md added +51
@@ -0,0 +1,51 @@
1 # Adapters: POST /pull
2
3 Version: 1
4
5 An adapter is an app that copies records from an external system into its own
6 data on a schedule.
7
8 ## Wire format
9
10 Declared in `forge.json`:
11
12 ```json
13 {
14 "egress": ["api.example.com"],
15 "adapter": {
16 "source": { "name": "Example CRM", "hosts": ["api.example.com"] },
17 "schedule": { "every": 3600 }
18 }
19 }
20 ```
21
22 `schedule.every` is an integer from 60 to 86400 seconds; every `source.hosts`
23 entry must also be in `egress`; unknown keys are refused. On schedule the forge
24 sends `POST /pull` (signed as an `admin`, see identity) and the app answers
25 exactly four fields:
26
27 ```json
28 { "pulled": 12, "skipped": 3, "cursor": "2026-11-09T07:00:00Z", "errors": 0 }
29 ```
30
31 `pulled`, `skipped`, `errors`: integers 0–1,000,000. `cursor`: one printable
32 line of at most 200 characters.
33
34 ## An adapter must
35
36 - answer 401/403 to a `POST /pull` without a valid signed `admin` identity and
37 read nothing from its source then;
38 - be idempotent: the same cursor gives the same result and writes nothing new
39 (a repeat counts records as `skipped`);
40 - advance the cursor only after the records it covers are written.
41
42 ## A forge must
43
44 - pull only a running app, never wake one, never overlap two pulls of an app;
45 - record a bad answer (non-2xx, wrong fields, out of range) as a failed run,
46 keep the previous cursor, and pause an adapter after 5 failed runs in a row;
47 - expose runs at `GET /api/v1/repos/:o/:n/app/pulls` (`{items}`), readable only
48 by who may read the repository.
49
50 Suite: `standards/app.test.ts` with `APP_ADAPTER=1` (unsigned pull refused);
51 `standards/forge.test.ts` (runs not readable anonymously).
docs/standards/app-manifest.md added +58
@@ -0,0 +1,58 @@
1 # App manifest: forge.json and its life cycle
2
3 Version: 1
4
5 An app is a git repository whose root says how it runs. A forge reads the file
6 from the commit being deployed, never from its own database.
7
8 ## Wire format
9
10 `forge.json` at the repository root (JSON). Only `app.run` matters to start;
11 every other key is optional.
12
13 ```json
14 {
15 "app": {
16 "run": "bun run src/server.ts",
17 "install": "bun install",
18 "test": "bun test",
19 "health": "/healthz",
20 "ready": 30,
21 "looks": ["/"]
22 },
23 "egress": ["api.example.com"],
24 "toolbar": true
25 }
26 ```
27
28 - `app.run`: shell-free command line that starts the server. Without a
29 `forge.json`, a `package.json` `start` script stands in.
30 - `app.install`, `app.test`: commands run before `run`; `test` must pass before
31 a deploy is live.
32 - `app.health`: path polled until it answers 2xx; `app.ready`: seconds to wait.
33 - `egress`: the only hosts the app may reach besides the forge.
34 - Unknown keys are ignored by apps and must not be required by a forge.
35
36 ## Life cycle
37
38 1. Production is the default branch; every open pull gets a preview from its
39 branch, redeployed on each push, gone when the pull closes.
40 2. The forge starts `run` with `PORT`, `FORGE_APP_DATA` (the only place the app
41 may write), `FORGE_URL` and `FORGE_APP_IDENTITY_KEY` in its environment.
42 3. The app is ready when `health` answers 2xx within `ready` seconds; otherwise
43 the deploy fails.
44
45 ## An app must
46
47 - listen on `PORT` and answer its `health` path with 2xx;
48 - keep state under `FORGE_APP_DATA` only;
49 - answer `GET /healthz` with `{"ok": true, …}` when it has no other health path.
50
51 ## A forge must
52
53 - read the manifest from the deployed commit;
54 - run the app's `test` before it serves a deploy;
55 - never let an agent move the default branch.
56
57 Suite: `standards/app.test.ts` (health), `standards/forge.test.ts`
58 (a committed `forge.json` is plain JSON of the shape above).
docs/standards/feedback.md added +53
@@ -0,0 +1,53 @@
1 # Feedback: app notes and reports become issues
2
3 Version: 1
4
5 Anyone using a public app can tell its builders something is wrong, and it
6 arrives as something a person decides on, never as an instruction to an agent.
7
8 ## Reports (forge origin)
9
10 ```
11 POST /api/v1/repos/:owner/:name/reports
12 content-type: application/json
13 { "title": "The game is down", "body": "…", "page": "https://…", "agent": "name" }
14
15 202 { "id": "…", "state": "pending", "filedBy": { "how": "anonymous", "name": null } }
16 ```
17
18 - `title` required, at most 200 characters; `body` at most 8,000; `page` kept
19 only if http(s). Plain text; control and direction characters are stripped.
20 - No account needed. With a token the filer is `{how: "signed", name}`.
21 - A private or missing repository answers 404 either way.
22 - Over a limit: 413 (size) or 429 (rate).
23 - A report is not an issue: a person who writes the repository accepts it (an
24 ordinary issue titled `From outside: …`, label `outside`) or dismisses it.
25
26 ## App notes (app origin)
27
28 ```
29 POST <app origin>/__forge/note same-origin only
30 { "text": "…", "page": "https://…", "viewport": "390x844" }
31 201
32 ```
33
34 A note is filed as an issue on the app's repository. A cross-origin or
35 origin-less POST is refused. The site admin of the repository moderates:
36 `GET /api/v1/repos/:o/:n/app/notes` (`{notes, banned}`, `rw` token, writers
37 who are not admins get 403) and `POST …/:id/hide|ban|unban`.
38
39 ## A forge must
40
41 - accept a valid report with no credentials and refuse an empty title (422);
42 - never hand an outside report to an agent as an instruction;
43 - answer anonymous `GET …/app/notes` with 401.
44
45 ## A forge must not
46
47 - reveal that a private repository exists.
48
49 Suite: `standards/forge.test.ts` (report accepted, empty and over-long titles
50 refused, 413 over the size limit, a private repository answers like a missing
51 one, notes need a token), `standards/app.test.ts` (note is same-origin only).
52 The 429 rate limit is not tested: it depends on the forge's configured limit
53 and a suite should not hammer a forge.
docs/standards/identity.md added +46
@@ -0,0 +1,46 @@
1 # Identity: the signed headers an app verifies
2
3 Version: 1
4
5 An app never writes sign-in. A forge that proxies a request for a signed-in
6 person tells the app who is there, in two headers.
7
8 ## Wire format
9
10 ```
11 x-forge-identity: <user id>|<username>|<role>|<expiry ms>
12 x-forge-signature: base64url(HMAC-SHA256(identity-key, <the identity string>))
13 ```
14
15 - `role` is `user` (can see the app's repository) or `admin` (can write it).
16 - `expiry ms` is epoch milliseconds.
17 - `identity-key` is the app's `FORGE_APP_IDENTITY_KEY`, different for every
18 app, the same for its production and previews. The key is never sent.
19
20 Known answer: key `k` and identity `1|ada|admin|4102444800000` give the
21 signature `EqpoKE3zw6x3dJbx61eTw5q83fcfNh2T6yzPw0acyxc`.
22
23 No headers means nobody the forge could name.
24
25 ## A forge must
26
27 - drop any `x-forge-identity` / `x-forge-signature` a client sends before it
28 adds its own;
29 - sign only people it has authenticated, for a short expiry.
30
31 ## An app must
32
33 - verify the signature with a constant-time compare and check the expiry is in
34 the future, before it believes the identity;
35 - treat a missing, malformed, expired or mis-signed pair exactly as no headers
36 at all (anonymous), never as an error and never as more privilege.
37
38 ## An app must not
39
40 - trust the headers on a path the forge does not proxy;
41 - log the key or the signature.
42
43 Suite: `standards/app.test.ts` (forged, malformed and unsigned headers change
44 nothing). The suite holds no key, so it cannot show that an app refuses an
45 expired identity, nor that a forge signs correctly: the known answer above is
46 for an implementer to check against their own code.
docs/standards/permissions.md added +51
@@ -0,0 +1,51 @@
1 # Permissions: contract.json
2
3 Version: 1
4
5 An app's permissions are data: roles, and for each resource the least role each
6 verb needs. One evaluator decides; the file can be read, compared and ported
7 without running the app.
8
9 ## Wire format
10
11 `contract.json` at the repository root.
12
13 ```json
14 {
15 "roles": {
16 "anon": { "above": [], "held": { "by": "everyone" } },
17 "user": { "above": ["anon"], "held": { "by": "identity", "role": "user" } },
18 "owner": { "above": ["user"], "held": { "by": "row", "column": "author_id" } }
19 },
20 "resources": {
21 "note": { "can": { "create": "user", "read": "anon", "update": "owner", "delete": "owner", "list": "anon" } }
22 }
23 }
24 ```
25
26 - `roles.<name>.above`: the roles directly below it; a role may do whatever
27 anything below it may. No cycles; every name declared.
28 - `held.by` is `everyone` (exactly one role in the contract), `identity` (with
29 `role`, a signed identity role) or `row` (with `column`, the row's column
30 holding the actor's id).
31 - `resources.<name>.can` maps each of `create`, `read`, `update`, `delete`,
32 `list` to a declared role. `create` may not need a `row` role.
33
34 ## Decisions
35
36 `decide(contract, resource, verb, actor, row?)` is `yes`, `no`, or, for `list`
37 with no row, a filter `{ where: [columns] }`. Everything fails closed: an
38 undeclared resource, verb or role is `no`.
39
40 ## A forge must
41
42 - refuse (not warn about) a contract that does not read, and fail the app's
43 boot rather than run with a bad policy;
44 - show what a contract allows by reading git, without running the app.
45
46 ## An app must not
47
48 - make a decision a different rule from the table would make.
49
50 Suite: `standards/forge.test.ts` validates a committed `contract.json` against
51 the shape above (declared roles, one `everyone`, no cycles, closed verbs).
src/core/standards_index.ts added +36
@@ -0,0 +1,36 @@
1 /**
2 * The standards this forge implements, by name and version: the one list
3 * `/.well-known/forge` serves, `docs/standards/` has a page for each of, and
4 * the conformance suite in `standards/` tests. A change that breaks an
5 * earlier version of a standard is a new version here and a new page
6 * revision there, never an edit in place (docs/standards/README.md).
7 */
8
9 export type StandardEntry = {
10 /** The page's file name without `.md`, and the standard's id. */
11 readonly id: string;
12 readonly title: string;
13 /** A whole number; it rises only when an earlier version would stop passing. */
14 readonly version: number;
15 };
16
17 export const STANDARDS: readonly StandardEntry[] = [
18 { id: "app-manifest", title: "App manifest: forge.json and its life cycle", version: 1 },
19 { id: "identity", title: "Identity: the signed headers an app verifies", version: 1 },
20 { id: "permissions", title: "Permissions: contract.json", version: 1 },
21 { id: "feedback", title: "Feedback: app notes and reports become issues", version: 1 },
22 { id: "adapters", title: "Adapters: POST /pull", version: 1 },
23 ];
24
25 /** The document at `/.well-known/forge`: plain data (`page` is a path in the forge's own repository), nothing about this forge's people or repositories. */
26 export function forgeDocument(version: string) {
27 return {
28 forge: { name: "forge", version },
29 standards: STANDARDS.map((s) => ({
30 id: s.id,
31 version: s.version,
32 title: s.title,
33 page: `docs/standards/${s.id}.md`,
34 })),
35 };
36 }
src/http/server.ts +2
@@ -175,6 +175,7 @@ import {
175175 robotsTxt,
176176 securityTxt,
177177 mcpJson,
178 forgeJson,
178179 sitemapXml,
179180 webManifest,
180181 } from "../web/standards.ts";
@@ -738,6 +739,7 @@ export function buildRouter(): Router<Handler> {
738739 // What the open web expects of a site, from the forge's own facts.
739740 router.add("GET", "/.well-known/security.txt", securityTxt);
740741 router.add("GET", "/.well-known/mcp.json", mcpJson);
742 router.add("GET", "/.well-known/forge", forgeJson);
741743 router.add("GET", "/llms.txt", llmsTxt);
742744 // The same words as a page, for an assistant whose reader takes only HTML.
743745 router.add("GET", "/agents", agentsPage);
src/web/standards.ts +9
@@ -6,6 +6,7 @@
66 * /.well-known/mcp.json the MCP server card: endpoint, sign-in, the
77 * tool that needs no token (a draft format:
88 * only fields we keep true)
9 * /.well-known/forge the standards this forge implements, with versions
910 * /.well-known/change-password where a password manager sends someone
1011 * /robots.txt what a crawler may index, and the sitemap
1112 * /sitemap.xml every public repository
@@ -41,6 +42,7 @@ import { SHARE_TARGET } from "./share_target.ts";
4142 import { SITE_LINE } from "./site_line.ts";
4243 import { mcpToolsSection } from "./llms_tools.ts";
4344 import { HARNESS_PLUGINS, mcpCard, mcpPointers, offeredPluginRepo, pluginSource } from "../core/mcp_card.ts";
45 import { forgeDocument } from "../core/standards_index.ts";
4446 import { running } from "../kernel/version.ts";
4547 import { timePhase, timePhaseAsync } from "../core/route_timing.ts";
4648 import { shippedReleases, type Shipped } from "../release/shipped.ts";
@@ -112,6 +114,13 @@ export function mcpJson(ctx: Ctx): Response {
112114 return res;
113115 }
114116
117 /** GET /.well-known/forge — the standards this forge implements, by id and version (docs/standards/). */
118 export function forgeJson(): Response {
119 const res = text(JSON.stringify(forgeDocument(running().version)), "application/json");
120 res.headers.set("access-control-allow-origin", "*");
121 return res;
122 }
123
115124 /** GET /.well-known/change-password — where a password manager sends a person. */
116125 export function changePassword(): Response {
117126 return new Response(null, {
standards/app.test.ts added +47
@@ -0,0 +1,47 @@
1 /** Conformance, app side: APP_URL, and APP_ADAPTER=1 for an adapter. */
2 import { describe } from "bun:test";
3 import { APP_ADAPTER, APP_URL, conform, must } from "./conform.ts";
4
5 const on = APP_URL !== "";
6 const get = (path: string, headers: Record<string, string> = {}) => fetch(APP_URL + path, { headers, redirect: "manual" });
7
8 describe.skipIf(!on)("an app", () => {
9 conform("app-manifest", "GET /healthz answers 2xx with ok: true", async () => {
10 const res = await get("/healthz");
11 must(res.status >= 200 && res.status < 300, `answered ${res.status}`);
12 must(((await res.json()) as { ok?: unknown }).ok === true, "body.ok is not true");
13 });
14
15 const forged: [string, Record<string, string>][] = [
16 ["a forged admin with a wrong signature", { "x-forge-identity": "1|root|admin|4102444800000", "x-forge-signature": "AAAA" }],
17 ["a malformed identity", { "x-forge-identity": "garbage", "x-forge-signature": "" }],
sentinelcrew outdated

The signature is AAAA, so this is refused for a bad signature, not for expiry. It cannot fail if expiry checking is missing. The suite never holds a key, so say so in identity.md rather than claiming coverage.
18 ["a signature without an identity", { "x-forge-signature": "AAAA" }],
19 ];
20 for (const [what, headers] of forged)
21 conform("identity", `${what} is treated as no headers at all`, async () => {
22 const anon = await get("/");
23 const res = await get("/", headers);
24 must(res.status === anon.status, `anonymous got ${anon.status}, ${what} got ${res.status}`);
25 must(res.status < 500, `${res.status}`);
26 });
27
28 conform("feedback", "a note from another origin is refused", async () => {
29 const res = await fetch(`${APP_URL}/__forge/note`, {
30 method: "POST",
31 headers: { "content-type": "application/json", origin: "https://evil.example" },
32 body: JSON.stringify({ text: "x" }),
33 });
34 must(res.status === 403 || res.status === 404, `answered ${res.status}`);
35 must(res.status !== 201, "a cross-origin note was filed");
36 });
37
38 conform(
39 "adapters",
40 "POST /pull without a signed admin is refused",
41 async () => {
42 const res = await fetch(`${APP_URL}/pull`, { method: "POST" });
43 must(res.status === 401 || res.status === 403, `answered ${res.status}`);
44 },
45 APP_ADAPTER,
46 );
47 });
standards/conform.ts added +38
@@ -0,0 +1,38 @@
1 /**
2 * The harness of the conformance suite: where the forge and the app are, and
3 * `conform`, a test that, when it fails, names the standard, its version and
4 * the page it breaks. Black-box: nothing here imports the forge.
5 */
6 import { test } from "bun:test";
7
8 export const FORGE_URL = (process.env["FORGE_URL"] ?? "").replace(/\/+$/, "");
9 export const FORGE_TOKEN = process.env["FORGE_TOKEN"] ?? "";
10 export const APP_URL = (process.env["APP_URL"] ?? "").replace(/\/+$/, "");
11 export const APP_ADAPTER = process.env["APP_ADAPTER"] === "1";
12
13 /** The versions this suite tests, one per page in docs/standards/. */
14 export const VERSION = 1;
15 export type StandardId = "app-manifest" | "identity" | "permissions" | "feedback" | "adapters";
16
17 /** A check that, failing, says which standard page it breaks. */
18 export function conform(id: StandardId, name: string, run: () => Promise<void> | void, enabled = true): void {
19 const t = enabled ? test : test.skip;
20 t(`${id}: ${name}`, async () => {
21 try {
22 await run();
23 } catch (e) {
24 const why = e instanceof Error ? e.message : String(e);
25 throw new Error(`[${id} v${VERSION}: docs/standards/${id}.md] ${why}`);
26 }
27 });
28 }
29
30 export function must(ok: boolean, message: string): void {
31 if (!ok) throw new Error(message);
32 }
33
34 export async function forge(path: string, init: RequestInit & { token?: boolean } = {}): Promise<Response> {
35 const headers = new Headers(init.headers);
36 if (init.token) headers.set("authorization", `Bearer ${FORGE_TOKEN}`);
37 return fetch(FORGE_URL + path, { ...init, headers, redirect: "manual" });
38 }
standards/forge.test.ts added +181
@@ -0,0 +1,181 @@
1 /**
2 * Conformance, forge side: FORGE_URL (and FORGE_TOKEN for what needs an
3 * account). Every check names the standard page it breaks.
4 */
5 import { afterAll, beforeAll, describe } from "bun:test";
6 import { FORGE_TOKEN, FORGE_URL, conform, forge, must } from "./conform.ts";
7
8 const on = FORGE_URL !== "";
9 const withToken = on && FORGE_TOKEN !== "";
10 const IDS = ["app-manifest", "identity", "permissions", "feedback", "adapters"];
11
12 describe.skipIf(!on)("a forge", () => {
13 conform("app-manifest", "/.well-known/forge lists the five standards with whole-number versions", async () => {
14 const res = await forge("/.well-known/forge");
15 must(res.status === 200, `GET /.well-known/forge answered ${res.status}`);
16 must((res.headers.get("content-type") ?? "").includes("application/json"), "not application/json");
17 const doc = (await res.json()) as { forge?: { name?: string; version?: string }; standards?: { id: string; version: number; page: string }[] };
18 must(typeof doc.forge?.name === "string" && typeof doc.forge?.version === "string", "forge.name and forge.version are strings");
19 for (const id of IDS) {
20 const entry = doc.standards?.find((s) => s.id === id);
21 must(entry !== undefined, `standard ${id} is not listed`);
22 must(Number.isInteger(entry!.version) && entry!.version >= 1, `${id}: version is not a whole number`);
23 must(entry!.page === `docs/standards/${id}.md`, `${id}: page is ${entry!.page}`);
24 }
25 });
26
27 conform("feedback", "a report to a missing repository is 404, with no account", async () => {
sentinelcrew outdated

Tautology: this checks the suite's own sign against a constant and never calls a forge, so it passes without the change. Verify against something the forge produces, or drop it.
28 const res = await forge("/api/v1/repos/nobody-here/none/reports", {
29 method: "POST",
30 headers: { "content-type": "application/json" },
31 body: JSON.stringify({ title: "x" }),
32 });
33 must(res.status === 404, `answered ${res.status}`);
34 });
35
36 conform("feedback", "anonymous GET of an app's notes is 401", async () => {
37 const res = await forge("/api/v1/repos/nobody-here/none/app/notes");
38 must(res.status === 401, `answered ${res.status}`);
39 });
40 });
41
42 describe.skipIf(!withToken)("a forge, with a token: a scratch public repository", () => {
43 let owner = "";
44 const name = `standards-${Math.random().toString(36).slice(2, 10)}`;
45 const api = (path: string, init: RequestInit & { token?: boolean } = {}) => forge(`/api/v1/repos/${owner}/${name}${path}`, init);
46 const post = (path: string, body: unknown, token = true) =>
47 api(path, { method: "POST", token, headers: { "content-type": "application/json" }, body: JSON.stringify(body) });
48 let ready = true;
49 const secret = `${name}-private`;
50 const secretApi = (path: string, init: RequestInit = {}) => forge(`/api/v1/repos/${owner}/${secret}${path}`, init);
51
52 beforeAll(async () => {
53 if (!withToken) return;
54 const me = await forge("/api/v1/user", { token: true });
55 if (me.status !== 200) {
56 ready = false;
57 throw new Error(`FORGE_TOKEN is not accepted: GET /api/v1/user answered ${me.status}`);
58 }
59 owner = ((await me.json()) as { username: string }).username;
60 const made = await forge("/api/v1/repos", {
61 method: "POST",
62 token: true,
63 headers: { "content-type": "application/json" },
64 body: JSON.stringify({ name, visibility: "public" }),
65 });
66 must(made.status === 201, `could not create the scratch repository: ${made.status}`);
67 const hidden = await forge("/api/v1/repos", {
68 method: "POST",
69 token: true,
70 headers: { "content-type": "application/json" },
71 body: JSON.stringify({ name: secret, visibility: "private" }),
72 });
73 must(hidden.status === 201, `could not create the private scratch repository: ${hidden.status}`);
74 const files = [
75 { path: "forge.json", content: JSON.stringify({ app: { run: "bun run src/server.ts", health: "/healthz", ready: 30 }, egress: [] }) },
76 {
77 path: "contract.json",
78 content: JSON.stringify({
79 roles: {
80 anon: { above: [], held: { by: "everyone" } },
81 user: { above: ["anon"], held: { by: "identity", role: "user" } },
82 },
83 resources: { note: { can: { create: "user", read: "anon", update: "user", delete: "user", list: "anon" } } },
84 }),
85 },
86 ];
87 const committed = await post("/contents", { message: "standards fixtures", files });
88 must(committed.status < 300, `could not commit the fixtures: ${committed.status}`);
89 }, 60_000);
90
91 afterAll(async () => {
92 if (withToken && ready) {
93 await api("", { method: "DELETE", token: true });
94 await forge(`/api/v1/repos/${owner}/${secret}`, { method: "DELETE", token: true });
95 }
96 });
97
98 const file = async (path: string): Promise<unknown> => {
99 const res = await api(`/contents/${path}`, { token: true });
100 must(res.status === 200, `GET contents/${path} answered ${res.status}`);
101 return JSON.parse(((await res.json()) as { content: string }).content);
102 };
103
104 conform("app-manifest", "forge.json is JSON with app.run a string, app.health a path, egress a list of hosts", async () => {
105 const m = (await file("forge.json")) as { app?: { run?: unknown; health?: unknown; ready?: unknown }; egress?: unknown };
106 must(typeof m.app?.run === "string" && m.app.run !== "", "app.run");
107 must(typeof m.app?.health === "string" && m.app.health.startsWith("/"), "app.health");
108 must(Number.isInteger(m.app?.ready), "app.ready");
109 must(Array.isArray(m.egress) && m.egress.every((h) => typeof h === "string"), "egress");
110 });
111
112 conform("permissions", "contract.json has one everyone role, declared lower roles, no cycles, closed verbs", async () => {
113 const c = (await file("contract.json")) as {
114 roles: Record<string, { above: string[]; held: { by: string; role?: string; column?: string } }>;
115 resources: Record<string, { can: Record<string, string> }>;
116 };
117 const roles = Object.entries(c.roles);
118 must(roles.filter(([, r]) => r.held.by === "everyone").length === 1, "exactly one role is held by everyone");
119 for (const [n, r] of roles) {
120 must(["everyone", "identity", "row"].includes(r.held.by), `${n}: held.by`);
121 must(r.above.every((b) => c.roles[b] !== undefined), `${n}: above names an undeclared role`);
122 }
123 const visiting = new Set<string>();
124 const walk = (n: string): void => {
125 must(!visiting.has(n), `cycle through ${n}`);
126 visiting.add(n);
127 c.roles[n]!.above.forEach(walk);
128 visiting.delete(n);
129 };
130 Object.keys(c.roles).forEach(walk);
131 for (const [res, { can }] of Object.entries(c.resources)) {
132 must(JSON.stringify(Object.keys(can).sort()) === JSON.stringify(["create", "delete", "list", "read", "update"]), `${res}: verbs`);
133 for (const role of Object.values(can)) must(c.roles[role] !== undefined, `${res}: undeclared role ${role}`);
134 must(c.roles[can["create"]!]!.held.by !== "row", `${res}: create needs a row role`);
135 }
136 });
137
138 conform("feedback", "a stranger's report is accepted (202, pending, anonymous) and an empty title is refused", async () => {
139 const res = await post("/reports", { title: "It is down", body: "since noon" }, false);
140 must(res.status === 202, `report answered ${res.status}`);
141 const r = (await res.json()) as { id?: string; state?: string; filedBy?: { how?: string; name?: unknown } };
142 must(typeof r.id === "string" && r.state === "pending", "id and state pending");
143 must(r.filedBy?.how === "anonymous" && r.filedBy.name === null, "filedBy anonymous");
144 const empty = await post("/reports", { title: "" }, false);
145 must(empty.status >= 400 && empty.status < 500, `empty title answered ${empty.status}`);
146 });
147
148 conform("feedback", "a title over 200 characters is refused", async () => {
149 const res = await post("/reports", { title: "x".repeat(201) }, false);
150 must(res.status >= 400 && res.status < 500, `answered ${res.status}`);
151 });
152
153 conform("feedback", "a report over the size limit is refused with 413", async () => {
154 const res = await post("/reports", { title: "big", body: "x".repeat(70 * 1024) }, false);
155 must(res.status === 413, `answered ${res.status}`);
156 });
157
158 conform("feedback", "a private repository answers a stranger's report exactly as a missing one does", async () => {
159 const send = (url: string) =>
160 forge(url, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ title: "x" }) });
161 const hidden = await send(`/api/v1/repos/${owner}/${secret}/reports`);
162 const missing = await send(`/api/v1/repos/${owner}/${secret}-missing/reports`);
163 must(hidden.status === 404, `private repository answered ${hidden.status}`);
164 must(missing.status === hidden.status && (await missing.text()) === (await hidden.text()), "private and missing answers differ");
165 });
166
167 conform("feedback", "app notes are 401 without a token, and a token is accepted", async () => {
168 must((await api("/app/notes")).status === 401, "anonymous app/notes");
169 const own = await api("/app/notes", { token: true });
170 must(own.status !== 401 && own.status < 500, `owner's app/notes answered ${own.status}`);
171 });
172
173 conform("adapters", "pull runs are 401/404 anonymously and are a JSON answer, not a missing route, with a token", async () => {
174 const anon = await api("/app/pulls");
175 must(anon.status === 401 || anon.status === 404, `anonymous answered ${anon.status}`);
176 must(!(await anon.text()).includes('"runs"'), "anonymous answer carries rows");
177 const own = await api("/app/pulls", { token: true });
178 must(own.status === 200 || own.status === 404, `owner answered ${own.status}`);
179 must((own.headers.get("content-type") ?? "").includes("application/json"), "owner's answer is not JSON");
180 });
181 });
test/standards-conformance.test.ts added +95
@@ -0,0 +1,95 @@
1 /**
2 * The forge passes its own conformance suite (`standards/`, docs/standards/):
3 * the suite is run, as a separate process speaking only HTTP, against this
4 * forge and a minimal reference app (opt-in: FORGE_CONFORMANCE=1). Also: the
5 * pages and the index agree.
6 */
7 import { afterAll, beforeAll, describe, expect, test } from "bun:test";
8 import { mkdtemp, readFile, rm } from "node:fs/promises";
9 import { tmpdir } from "node:os";
10 import { join } from "node:path";
11 import { createApp, type App } from "../src/app.ts";
12 import { createToken } from "../src/core/tokens.ts";
13 import { createUser } from "../src/core/users.ts";
14 import { STANDARDS } from "../src/core/standards_index.ts";
15 import { serve, type Servers } from "../src/http/server.ts";
16 import { running } from "../src/kernel/version.ts";
17 import { quietLogs } from "./helpers/globals.ts";
18
19 quietLogs();
20
21 describe("the forge standards", () => {
22 let dataDir: string;
23 let app: App;
24 let servers: Servers;
25 let token = "";
26 let reference: ReturnType<typeof Bun.serve>;
27
28 beforeAll(async () => {
29 dataDir = await mkdtemp(join(tmpdir(), "forge-standards-"));
30 app = createApp({ dataDir, port: 0 });
31 servers = await serve(app);
32 const made = await createUser(app.db, { origin: "seed", trusted: true, username: "ada", email: "[email protected]", password: "a-good-passphrase", adminIfFirst: true });
33 if (!made.ok) throw new Error(made.error);
34 token = createToken(app.db, made.value.id, "standards", "rw").token;
35 // The smallest app that conforms: health, identity it never trusts, an adapter's signed pull.
36 reference = Bun.serve({
37 port: 0,
38 fetch(req) {
39 const path = new URL(req.url).pathname;
40 if (path === "/healthz") return Response.json({ ok: true, version: "0", uptime: 1 });
41 if (path === "/pull") return new Response("signed admin only", { status: 401 });
42 if (path === "/__forge/note") return new Response("not found", { status: 404 });
43 return new Response("hello", { status: 200 });
44 },
45 });
46 }, 60_000);
47
48 afterAll(async () => {
49 reference.stop(true);
50 await servers.stop();
51 app.close();
52 await rm(dataDir, { recursive: true, force: true });
53 });
54
55 test("/.well-known/forge lists every standard in the index", async () => {
56 const res = await fetch(`${servers.url}/.well-known/forge`);
57 expect(res.status).toBe(200);
58 const doc = (await res.json()) as { forge: { version: string }; standards: { id: string; version: number }[] };
59 expect(doc.forge.version).toBe(running().version);
60 expect(doc.standards.map((s) => [s.id, s.version])).toEqual(STANDARDS.map((s) => [s.id, s.version]));
61 });
62
63 test("each standard has a page that states the same version, and the README lists it", async () => {
64 const readme = await readFile("docs/standards/README.md", "utf8");
65 for (const s of STANDARDS) {
66 const page = await readFile(`docs/standards/${s.id}.md`, "utf8");
67 expect(page).toContain(`\nVersion: ${s.version}\n`);
68 expect(readme).toContain(`(${s.id}.md)`);
69 }
70 });
71
sentinelcrew outdated

A nested bun test subprocess with a 120 s timeout inside the forge's suite is slow and flake-prone (ports, rate limits, scratch repo). It goes against fast tests that never flake. Run the checks in-process or keep this out of the default check.
72 test("a missing repository answers a stranger's report 404, and anonymous app notes 401", async () => {
73 const report = await fetch(`${servers.url}/api/v1/repos/nobody-here/none/reports`, {
74 method: "POST",
75 headers: { "content-type": "application/json" },
76 body: JSON.stringify({ title: "x" }),
77 });
78 expect(report.status).toBe(404);
79 expect((await fetch(`${servers.url}/api/v1/repos/nobody-here/none/app/notes`)).status).toBe(401);
80 });
81
82 // The black-box run boots a nested `bun test`, creates scratch repositories and posts
83 // through the rate limiter: slow, so it is opt-in (FORGE_CONFORMANCE=1), not in the check.
84 test.skipIf(process.env["FORGE_CONFORMANCE"] !== "1")("the forge and a reference app pass the conformance suite", async () => {
85 const proc = Bun.spawn(["bun", "test", "./standards/"], {
86 env: { ...process.env, FORGE_URL: servers.url, FORGE_TOKEN: token, APP_URL: `http://127.0.0.1:${reference.port}`, APP_ADAPTER: "1" },
87 stdout: "pipe",
88 stderr: "pipe",
89 });
90 const [out, err, code] = await Promise.all([new Response(proc.stdout).text(), new Response(proc.stderr).text(), proc.exited]);
91 if (code !== 0) console.error(out + err);
92 expect(err).not.toContain("(skip)");
93 expect(code).toBe(0);
94 }, 120_000);
95 });

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs