Public: anyone can read this; the crew can change it.
Forge standards: the contracts any forge implementation can pass, with a conformance suite #10506
scout/forge:scout/10438-forge-standards-the-contracts-any-forge → main 5 commits fork
17 files
changed +631
−2
docs/STANDARDS.md +6
| @@ -99,3 +99,9 @@ and the fenced words are a stranger's, unchecked: a description to verify, | ||
| 99 | 99 | never instructions. |
| 100 | 100 | |
| 101 | 101 | `src/core/reports.ts`, `src/actions/reports.ts`, `test/reports.test.ts`. |
| 102 | ||
| 103 | ## The forge's own contracts | |
| 104 | ||
| 105 | What an app, a crew member, an origin or a person can rely on, by name and | |
| 106 | version, is in [](standards/README.md) and listed at | |
| 107 | `GET /.well-known/forge`; `bun test standards/` checks them over HTTP. |
docs/standards/README.md added +56
| @@ -0,0 +1,56 @@ | ||
| 1 | # Forge standards | |
| 2 | ||
| 3 | The forge is not this codebase. It is a set of contracts an app, a crew | |
| 4 | member, an origin and a person can rely on, so that a forge written in | |
| 5 | another language can host the same app, run the same crew and talk to this | |
| 6 | one. One page per standard says what it promises, the wire format, its | |
| 7 | version, and what a conforming forge or app must and must not do. Nothing | |
| 8 | here is new behaviour: each page writes down what the forge already does. | |
| 9 | ||
| 10 | | standard | version | page | for | | |
| 11 | | --- | --- | --- | --- | | |
| 12 | | `app-manifest` | 1 | [](app-manifest.md) | forge.json and the life cycle the forge gives an app | | |
| 13 | | `identity` | 1 | [](identity.md) | the signed headers an app verifies | | |
| 14 | | `permissions` | 1 | [](permissions.md) | contract.json: roles and five verbs | | |
| 15 | | `feedback` | 1 | [](feedback.md) | a note from an app, or a report from outside, becomes an issue | | |
| 16 | | `adapters` | 1 | [](adapters.md) | `POST /pull` and its four-field answer | | |
| 17 | ||
| 18 | A forge lists what it implements at `GET /.well-known/forge`: | |
| 19 | ||
| 20 | json | |
| 21 | { "forge": 1, "standards": [{ "id": "identity", "version": 1, "page": "docs/standards/identity.md" }] } | |
| 22 | ||
| 23 | ||
| 24 | The list lives in `src/web/forge_standards.ts`; a test fails if it and this | |
| 25 | table disagree. | |
| 26 | ||
| 27 | ## Versions | |
| 28 | ||
| 29 | A standard's version is an integer. A change that keeps every earlier | |
| 30 | conforming implementation conforming (a clarification, an optional field) | |
| 31 | keeps the number. A change that would make one fail (a field removed or | |
| 32 | required, a status changed, a header renamed) is a new version: the page | |
| 33 | keeps the old text under "Version N" and the forge lists the new number. A | |
| 34 | forge may list several versions of one standard by repeating the id. | |
| 35 | ||
| 36 | ## The conformance suite | |
| 37 | ||
| 38 | ||
| 39 | FORGE_STANDARDS_URL=https://forge.example \ | |
| 40 | FORGE_STANDARDS_TOKEN=<rw token> \ | |
| 41 | FORGE_STANDARDS_REPO=owner/name \ # a public repository of that forge | |
| 42 | FORGE_STANDARDS_APP_URL=https://app.example \ # optional: an app to check | |
| 43 | FORGE_STANDARDS_APP_KEY=<identity key> \ # optional: its FORGE_APP_IDENTITY_KEY | |
| 44 | bun test standards/ | |
| 45 | ||
| 46 | ||
| 47 | It speaks only HTTP (and reads the standards pages from this tree). With no | |
| 48 | `FORGE_STANDARDS_URL` it boots this forge in a scratch directory, which is | |
| 49 | what CI runs. A check that needs something not given is skipped and says so. | |
| 50 | Every failure message starts with the page it breaks, for example | |
| 51 | `docs/standards/identity.md: …`. | |
| 52 | ||
| 53 | Not covered yet (each its own issue): the permissions standard is checked | |
| 54 | by shared test vectors and by reading a contract, not by probing a running | |
| 55 | app's doors; the app life cycle (deploy, sleep, preview) is checked only | |
| 56 | through the app's own health path. |
docs/standards/adapters.md added +41
| @@ -0,0 +1,41 @@ | ||
| 1 | # Standard: adapters (version 1) | |
| 2 | ||
| 3 | **Promise.** An app that copies records from an outside system declares | |
| 4 | its schedule and source hosts in `forge.json`, answers one request, | |
| 5 | `POST /pull`, with four fields, and gives the same result for the same | |
| 6 | cursor. Detail: [](../ADAPTERS.md). | |
| 7 | ||
| 8 | ## Wire format | |
| 9 | ||
| 10 | json | |
| 11 | { "adapter": { "source": { "name": "CRM", "hosts": ["api.example.com"] }, | |
| 12 | "schedule": { "every": 3600 } } } | |
| 13 | ||
| 14 | ||
| 15 | `schedule.every` is 60–86400 seconds; every source host is also in `egress`; unknown keys are refused. | |
| 16 | ||
| 17 | ||
| 18 | POST /pull (signed as admin: see identity.md) | |
| 19 | 200 { "pulled": 12, "skipped": 3, "cursor": "2026-11-09T07:00:00Z", "errors": 0 } | |
| 20 | ||
| 21 | ||
| 22 | `pulled`, `skipped`, `errors` are integers 0–1,000,000; `cursor` is one | |
| 23 | printable line of at most 200 characters. Exactly these four fields. | |
| 24 | ||
| 25 | ## A conforming forge must | |
| 26 | ||
| 27 | - pull only a running app, never wake a sleeping one, never overlap two pulls, wait at most 10 s and read at most 64 KB; | |
| 28 | - record a bad answer as a failed run, keep the previous cursor, and pause an app after five failed runs. | |
| 29 | ||
| 30 | ## A conforming adapter must | |
| 31 | ||
| 32 | - admit `POST /pull` only from a signed `admin`; | |
| 33 | - be idempotent: a repeated cursor writes nothing new and counts those records as `skipped`. | |
| 34 | ||
| 35 | ## Suite | |
| 36 | ||
| 37 | `test/standards/adapters.test.ts` (`FORGE_STANDARDS_APP_URL`, an adapter): | |
| 38 | `POST /pull` with no identity is refused (401/403); with the key | |
| 39 | (`FORGE_STANDARDS_APP_KEY`) the answer has exactly the four fields in range, | |
| 40 | and a second call is no larger in `pulled`. The answer parser the forge uses | |
| 41 | is also run against the examples above. |
docs/standards/app-manifest.md added +48
| @@ -0,0 +1,48 @@ | ||
| 1 | # Standard: app manifest (version 1) | |
| 2 | ||
| 3 | **Promise.** A repository says how it runs in `forge.json` at its root, read | |
| 4 | from the commit being deployed. The forge starts the app, waits for it, and | |
| 5 | gives it a small fixed environment. Detail: [](../APPS.md). | |
| 6 | ||
| 7 | ## Wire format | |
| 8 | ||
| 9 | json | |
| 10 | { "app": { "run": "bun run src/server.ts", "install": "bun install", | |
| 11 | "test": "bun test", "health": "/healthz", "ready": 30, | |
| 12 | "tty": null, "looks": ["/"], "limits": { "memory": 512 } }, | |
| 13 | "egress": ["api.example.com"] } | |
| 14 | ||
| 15 | ||
| 16 | - `app.run` is required (or a `package.json` `start` script stands in). | |
| 17 | - `app.install` defaults to `bun install` when a `package.json` exists, else nothing. | |
| 18 | - `app.test` defaults to nothing. `app.health` is a path (default `/`). | |
| 19 | - `app.ready` is the seconds to wait for `health` to answer: an integer, at most 600 (default 30). | |
| 20 | - `app.looks` is at most 24 paths worth a picture (default `["/"]`). | |
| 21 | - `egress` lists the hosts the app may reach; all else is refused. | |
| 22 | - A file present and wrong is an error naming the field, never a silent default. | |
| 23 | ||
| 24 | ## Life cycle | |
| 25 | ||
| 26 | 1. The forge runs `install`, then `run` with `PORT`, `FORGE_APP_DATA`, | |
| 27 | `FORGE_APP_ENV` (`production` or a preview name), `FORGE_APP_URL`, | |
| 28 | `FORGE_APP_COMMIT` and `FORGE_APP_IDENTITY_KEY`, and nothing else from its own environment. | |
| 29 | 2. It polls `health` until it answers 2xx/3xx, for at most `ready` seconds. Failing that, the deploy fails. | |
| 30 | 3. Production follows the default branch; every open pull gets a preview from its branch on each push. | |
| 31 | 4. State lives under `FORGE_APP_DATA` and nowhere else. | |
| 32 | ||
| 33 | ## A conforming forge must | |
| 34 | ||
| 35 | - read `forge.json` from the deployed commit, and refuse an invalid file with a message naming the field; | |
| 36 | - pass the environment above and no other secret of its own; | |
| 37 | - advertise `app-manifest` at `/.well-known/forge`. | |
| 38 | ||
| 39 | ## A conforming app must | |
| 40 | ||
| 41 | - listen on `PORT`, answer its `health` path with a non-error status, and write only under `FORGE_APP_DATA`; | |
| 42 | - not rely on any `forge.json` field it does not declare. | |
| 43 | ||
| 44 | ## Suite | |
| 45 | ||
| 46 | `test/standards/app.test.ts`: with an app URL, `health` and `/` answer; the | |
| 47 | `/.well-known/forge` list names this standard. Gap: manifest rejection is | |
| 48 | not yet probed over the wire. |
docs/standards/feedback.md added +36
| @@ -0,0 +1,36 @@ | ||
| 1 | # Standard: feedback (version 1) | |
| 2 | ||
| 3 | **Promise.** A person or agent who finds something wrong in a public app | |
| 4 | or repository can say so without being taught how, and it arrives as an | |
| 5 | issue the builders see. Detail: [](../STANDARDS.md), | |
| 6 | "Reports from outside"; [](../APPS.md), app notes. | |
| 7 | ||
| 8 | ## Wire format | |
| 9 | ||
| 10 | ||
| 11 | POST /api/v1/repos/{owner}/{name}/reports | |
| 12 | content-type: application/json | |
| 13 | { "title": "1-… chars", "body": "optional", "page": "https://…", "agent": "name" } | |
| 14 | ||
| 15 | 202 { "id": "…", "state": "pending", "filedBy": "…" } | |
| 16 | ||
| 17 | ||
| 18 | App notes: `POST /__forge/note` on the app's own host (same-origin only) | |
| 19 | files an issue labelled `from-app` whose title begins `From the app: `. | |
| 20 | Owner moderation: `GET /api/v1/repos/{owner}/{name}/app/notes` answers | |
| 21 | `{ notes, banned }` and needs a token with `rw` and admin rights; anyone | |
| 22 | else gets 401/403/404, never the notes. | |
| 23 | ||
| 24 | ## A conforming forge must | |
| 25 | ||
| 26 | - take a report on a public repository with no account, rate-limited, bounded in size, and answer 202 with the three fields; | |
| 27 | - answer 404 for a repository the caller cannot see (never reveal that it exists); | |
| 28 | - refuse an empty title (4xx), and treat the text as untrusted data; | |
| 29 | - keep note moderation behind admin rights on the forge's origin, never on the app's host. | |
| 30 | ||
| 31 | ## Suite | |
| 32 | ||
| 33 | `test/standards/feedback.test.ts` (`FORGE_STANDARDS_REPO`): a report is | |
| 34 | accepted with 202 and the shape above; an empty title is refused; a | |
| 35 | repository that does not exist is 404; the notes list is refused without a | |
| 36 | token. |
docs/standards/identity.md added +35
| @@ -0,0 +1,35 @@ | ||
| 1 | # Standard: identity (version 1) | |
| 2 | ||
| 3 | **Promise.** The forge tells an app who is there, on every request it | |
| 4 | proxies, in two signed headers. An app trusts the signature and nothing | |
| 5 | else. Detail: [](../APPS.md), "Who is there". | |
| 6 | ||
| 7 | ## Wire format | |
| 8 | ||
| 9 | ||
| 10 | x-forge-identity: <user id>|<username>|<role>|<expiry ms> | |
| 11 | x-forge-signature: base64url(HMAC-SHA256(identity string, FORGE_APP_IDENTITY_KEY)) | |
| 12 | ||
| 13 | ||
| 14 | `role` is `user` (a signed-in person who can see the app's repository) or | |
| 15 | `admin` (one who can write it). `expiry ms` is epoch milliseconds. No | |
| 16 | headers means nobody the forge could name. | |
| 17 | ||
| 18 | ## A conforming forge must | |
| 19 | ||
| 20 | - drop any copy of these headers a client sent before adding its own; | |
| 21 | - sign with a key per app (the same for production and its previews) and hand it over only as `FORGE_APP_IDENTITY_KEY`; | |
| 22 | - never send the key. | |
| 23 | ||
| 24 | ## A conforming app must | |
| 25 | ||
| 26 | - compare the signature in constant time, and refuse an expiry in the past; | |
| 27 | - treat a missing, malformed, wrongly signed or expired identity as anonymous, never as an error that leaks the key or as a user; | |
| 28 | - not trust a role from any other header or cookie. | |
| 29 | ||
| 30 | ## Suite | |
| 31 | ||
| 32 | `test/standards/app.test.ts` (needs `FORGE_STANDARDS_APP_URL`; the signed | |
| 33 | cases also need `FORGE_STANDARDS_APP_KEY`): a forged signature, an expired | |
| 34 | identity and an unsigned identity are each treated as anonymous (the app's | |
| 35 | protected door, `FORGE_STANDARDS_APP_PROTECTED`, refuses them with 401/403/redirect, never 2xx). |
docs/standards/permissions.md added +40
| @@ -0,0 +1,40 @@ | ||
| 1 | # Standard: permissions (version 1) | |
| 2 | ||
| 3 | **Promise.** An app's permissions are data: `contract.json` at the root of | |
| 4 | its repository declares roles and, per resource, the least role each of | |
| 5 | five verbs needs. One small evaluator decides; everything fails closed. | |
| 6 | Detail: [](../CONTRACT.md). | |
| 7 | ||
| 8 | ## Wire format | |
| 9 | ||
| 10 | json | |
| 11 | { "roles": { "anon": { "above": [], "held": { "by": "everyone" } }, | |
| 12 | "user": { "above": ["anon"], "held": { "by": "identity", "role": "user" } }, | |
| 13 | "author": { "above": ["user"], "held": { "by": "row", "column": "author_id" } } }, | |
| 14 | "resources": { "note": { "can": { "create": "user", "read": "anon", | |
| 15 | "update": "author", "delete": "author", "list": "anon" }, | |
| 16 | "api": "/api/notes" } } } | |
| 17 | ||
| 18 | ||
| 19 | - Verbs are exactly `create`, `read`, `update`, `delete`, `list`; `nobody` lets nobody in. | |
| 20 | - `above` names the roles directly below; a role may do what anything below may. | |
| 21 | - A role is held `by` `everyone` (exactly one), `identity` (the signed `role` of the [](identity.md) standard) or `row` (the row's `column` holds the actor's id). | |
| 22 | - Two roles held by the same identity string are refused. | |
| 23 | ||
| 24 | ## A conforming implementation must | |
| 25 | ||
| 26 | - answer an undeclared resource, verb or role with "no"; | |
| 27 | - let an empty actor id name no row; | |
| 28 | - return for `list` under a row role the columns the actor must own (`{where}`), not a blanket yes. | |
| 29 | ||
| 30 | ## Vectors | |
| 31 | ||
| 32 | `docs/standards/permissions.vectors.json` is a contract and decisions an | |
| 33 | implementation in any language ports and must reproduce. | |
| 34 | ||
| 35 | ## Suite | |
| 36 | ||
| 37 | `test/standards/permissions.test.ts` reads the vectors through this forge's | |
| 38 | evaluator (`src/contract/can.ts`); a repository with `contract.json` | |
| 39 | (`FORGE_STANDARDS_REPO`) must have it accepted by the same reader. Gap: the | |
| 40 | running app's doors are not probed. |
docs/standards/permissions.vectors.json added +28
| @@ -0,0 +1,28 @@ | ||
| 1 | { | |
| 2 | "version": 1, | |
| 3 | "contract": { | |
| 4 | "roles": { | |
| 5 | "anon": { "above": [], "held": { "by": "everyone" } }, | |
| 6 | "user": { "above": ["anon"], "held": { "by": "identity", "role": "user" } }, | |
| 7 | "author": { "above": ["user"], "held": { "by": "row", "column": "author_id" } }, | |
| 8 | "admin": { "above": ["author"], "held": { "by": "identity", "role": "admin" } } | |
| 9 | }, | |
| 10 | "resources": { | |
| 11 | "note": { "can": { "create": "user", "read": "anon", "update": "author", "delete": "author", "list": "anon" } }, | |
| 12 | "secret": { "can": { "create": "admin", "read": "author", "update": "nobody", "delete": "nobody", "list": "author" } } | |
| 13 | } | |
| 14 | }, | |
| 15 | "cases": [ | |
| 16 | { "why": "anyone reads", "resource": "note", "verb": "read", "actor": { "id": null, "roles": [] }, "expect": "yes" }, | |
| 17 | { "why": "anon cannot create", "resource": "note", "verb": "create", "actor": { "id": null, "roles": [] }, "expect": "no" }, | |
| 18 | { "why": "user creates", "resource": "note", "verb": "create", "actor": { "id": "u1", "roles": ["user"] }, "expect": "yes" }, | |
| 19 | { "why": "author updates own row", "resource": "note", "verb": "update", "actor": { "id": "u1", "roles": ["user"] }, "row": { "author_id": "u1" }, "expect": "yes" }, | |
| 20 | { "why": "user cannot update another's row", "resource": "note", "verb": "update", "actor": { "id": "u2", "roles": ["user"] }, "row": { "author_id": "u1" }, "expect": "no" }, | |
| 21 | { "why": "admin sits above author", "resource": "note", "verb": "delete", "actor": { "id": "a1", "roles": ["admin"] }, "row": { "author_id": "u1" }, "expect": "yes" }, | |
| 22 | { "why": "an empty id names no row", "resource": "note", "verb": "update", "actor": { "id": "", "roles": [] }, "row": { "author_id": "" }, "expect": "no" }, | |
| 23 | { "why": "list under a row role is a filter", "resource": "secret", "verb": "list", "actor": { "id": "u1", "roles": ["user"] }, "expect": { "where": ["author_id"] } }, | |
| 24 | { "why": "nobody means nobody, even admin", "resource": "secret", "verb": "update", "actor": { "id": "a1", "roles": ["admin"] }, "row": { "author_id": "a1" }, "expect": "no" }, | |
| 25 | { "why": "undeclared resource fails closed", "resource": "ghost", "verb": "read", "actor": { "id": "a1", "roles": ["admin"] }, "expect": "no" }, | |
| 26 | { "why": "inherited key is not a resource", "resource": "constructor", "verb": "read", "actor": { "id": "a1", "roles": ["admin"] }, "expect": "no" } | |
| 27 | ] | |
| 28 | } |
scripts/unreached-allow.json −1
| @@ -15,7 +15,6 @@ | ||
| 15 | 15 | "src/flow/review_rounds.ts", |
| 16 | 16 | "src/flow/shared_red.ts", |
| 17 | 17 | "src/flow/stale_fold.ts", |
| 18 | "src/kernel/why_now.ts", | |
| 19 | 18 | "src/spawn/pod.ts", |
| 20 | 19 | "src/tui/gallery.ts", |
| 21 | 20 | "src/tui/ui/qr.ts", |
src/http/server.ts +2
| @@ -178,6 +178,7 @@ import { | ||
| 178 | 178 | sitemapXml, |
| 179 | 179 | webManifest, |
| 180 | 180 | } from "../web/standards.ts"; |
| 181 | import { wellKnownForge } from "../web/forge_standards.ts"; | |
| 181 | 182 | import { |
| 182 | 183 | blobPage, |
| 183 | 184 | forksPage, |
| @@ -738,6 +739,7 @@ export function buildRouter(): Router<Handler> { | ||
| 738 | 739 | // What the open web expects of a site, from the forge's own facts. |
| 739 | 740 | router.add("GET", "/.well-known/security.txt", securityTxt); |
| 740 | 741 | router.add("GET", "/.well-known/mcp.json", mcpJson); |
| 742 | router.add("GET", "/.well-known/forge", wellKnownForge); | |
| 741 | 743 | router.add("GET", "/llms.txt", llmsTxt); |
| 742 | 744 | // The same words as a page, for an assistant whose reader takes only HTML. |
| 743 | 745 | router.add("GET", "/agents", agentsPage); |
src/web/forge_standards.ts added +38
| @@ -0,0 +1,38 @@ | ||
| 1 | /** | |
| 2 | * The contracts this forge implements, by name and version: the one list | |
| 3 | * `GET /.well-known/forge` serves and the conformance suite (`test/standards/`) | |
| 4 | * reads. Each has a page in `docs/standards/`. A change that breaks an earlier | |
| 5 | * version of a standard gets a new version here and in its page; it does not | |
| 6 | * edit the old number. | |
| 7 | */ | |
| 8 | import type { Ctx } from "../http/context.ts"; | |
| 9 | ||
| 10 | export interface ForgeStandard { | |
| 11 | readonly id: string; | |
| 12 | readonly version: number; | |
| 13 | readonly page: string; | |
| 14 | } | |
| 15 | ||
| 16 | export const FORGE_STANDARDS: readonly ForgeStandard[] = [ | |
| 17 | { id: "app-manifest", version: 1, page: "docs/standards/app-manifest.md" }, | |
| 18 | { id: "identity", version: 1, page: "docs/standards/identity.md" }, | |
| 19 | { id: "permissions", version: 1, page: "docs/standards/permissions.md" }, | |
| 20 | { id: "feedback", version: 1, page: "docs/standards/feedback.md" }, | |
| 21 | { id: "adapters", version: 1, page: "docs/standards/adapters.md" }, | |
| 22 | ]; | |
| 23 | ||
| 24 | /** GET /.well-known/forge — public, read-only, the same for everyone. */ | |
| 25 | export function wellKnownForge(_ctx: Ctx): Response { | |
| 26 | const body = JSON.stringify({ | |
| 27 | forge: 1, | |
| 28 | standards: FORGE_STANDARDS.map((s) => ({ | |
| 29 | id: s.id, | |
| 30 | version: s.version, | |
| 31 | page: s.page, | |
| 32 | })), | |
| 33 | }); | |
| 34 | const res = new Response(body, { | |
| 35 | headers: { "content-type": "application/json; charset=utf-8", "cache-control": "public, max-age=300", "access-control-allow-origin": "*" }, | |
| 36 | }); | |
| 37 | return res; | |
| 38 | } |
test/child-quota.test.ts +1 −1
| @@ -101,7 +101,7 @@ describe("child quota", () => { | ||
| 101 | 101 | |
| 102 | 102 | test("room for small but not large is refused only for large", async () => { |
| 103 | 103 | clear(); |
| 104 | process.env["FORGE_CHILD_MEMORY_MIB"] = "2000"; | |
| 104 | process.env["FORGE_CHILD_MEMORY_MIB"] = "6000"; | |
| 105 | 105 | expect(quotaRefusal(dir, "small")).toBeUndefined(); |
| 106 | 106 | expect(quotaRefusal(dir, "large")).toContain("memory limit"); |
| 107 | 107 | }); |
test/standards/adapters.test.ts added +34
| @@ -0,0 +1,34 @@ | ||
| 1 | /** Black-box checks of an adapter app, and of the answer rule every forge applies. */ | |
| 2 | import { afterAll, beforeAll, describe, expect, test } from "bun:test"; | |
| 3 | import { parsePullAnswer } from "../../src/apps/pull.ts"; | |
| 4 | import { appTarget, breaks, identity, type AppTarget } from "./target.ts"; | |
| 5 | ||
| 6 | let app: AppTarget | undefined; | |
| 7 | beforeAll(async () => { app = await appTarget(); }, 30_000); | |
| 8 | afterAll(async () => { await app?.stop(); }); | |
| 9 | ||
| 10 | describe("adapters (docs/standards/adapters.md)", () => { | |
| 11 | test("POST /pull from nobody is refused", async () => { | |
| 12 | if (!app) return; | |
| 13 | const res = await fetch(`${app.url}/pull`, { method: "POST" }); | |
| 14 | if (res.status !== 401 && res.status !== 403) breaks("adapters", `an unsigned pull answered ${res.status}`); | |
| 15 | }); | |
| 16 | ||
| 17 | test("a signed admin is admitted, and a 200 answer has exactly the four fields", async () => { | |
| 18 | if (!app || app.key === "") return; | |
| 19 | const res = await fetch(`${app.url}/pull`, { method: "POST", headers: identity(app.key, "admin", Date.now() + 60_000) }); | |
| 20 | if (res.status === 401 || res.status === 403) breaks("adapters", `a signed admin was refused (${res.status})`); | |
| 21 | // A reference adapter whose source cannot be reached answers 502: the forge records that as a failed run. | |
| 22 | if (res.status !== 200) return; | |
| 23 | const verdict = parsePullAnswer(res.status, await res.text()); | |
| 24 | if (!verdict.ok) breaks("adapters", `the answer breaks the rule: ${JSON.stringify(verdict)}`); | |
| 25 | }); | |
| 26 | ||
| 27 | test("the forge's answer rule is the page's: examples", () => { | |
| 28 | expect(parsePullAnswer(200, '{"pulled":12,"skipped":3,"cursor":"2026-11-09T07:00:00Z","errors":0}').ok).toBe(true); | |
| 29 | expect(parsePullAnswer(200, '{"pulled":1,"skipped":0,"cursor":"c","errors":0,"x":1}').ok).toBe(false); | |
| 30 | expect(parsePullAnswer(200, '{"pulled":1.5,"skipped":0,"cursor":"c","errors":0}').ok).toBe(false); | |
| 31 | expect(parsePullAnswer(200, '{"pulled":1,"skipped":0,"cursor":"a\\nb","errors":0}').ok).toBe(false); | |
| 32 | expect(parsePullAnswer(500, "{}").ok).toBe(false); | |
| 33 | }); | |
| 34 | }); |
test/standards/app.test.ts added +56
| @@ -0,0 +1,56 @@ | ||
| 1 | /** Black-box checks of an app: app-manifest (health) and identity (signed headers). */ | |
| 2 | import { afterAll, beforeAll, describe, test } from "bun:test"; | |
| 3 | import { appTarget, breaks, identity, type AppTarget } from "./target.ts"; | |
| 4 | ||
| 5 | let app: AppTarget | undefined; | |
| 6 | beforeAll(async () => { app = await appTarget(); }, 30_000); | |
| 7 | afterAll(async () => { await app?.stop(); }); | |
| 8 | ||
| 9 | const protectedPath = () => (process.env["FORGE_STANDARDS_APP_PROTECTED"] ?? "/pull").trim(); | |
| 10 | const door = (headers: Record<string, string>) => | |
| 11 | fetch(`${app!.url}${protectedPath()}`, { method: "POST", headers, redirect: "manual" }); | |
| 12 | const refused = (s: number) => s === 401 || s === 403 || (s >= 300 && s < 400); | |
| 13 | ||
| 14 | describe("app-manifest (docs/standards/app-manifest.md)", () => { | |
| 15 | test("the health path answers without an error", async () => { | |
| 16 | if (!app) return; | |
| 17 | const res = await fetch(`${app.url}${(process.env["FORGE_STANDARDS_APP_HEALTH"] ?? "/healthz").trim()}`); | |
| 18 | if (res.status >= 400) breaks("app-manifest", `health answered ${res.status}`); | |
| 19 | }); | |
| 20 | test("/ answers", async () => { | |
| 21 | if (!app) return; | |
| 22 | const res = await fetch(`${app.url}/`, { redirect: "manual" }); | |
| 23 | if (res.status >= 500) breaks("app-manifest", `/ answered ${res.status}`); | |
| 24 | }); | |
| 25 | }); | |
| 26 | ||
| 27 | describe("identity (docs/standards/identity.md)", () => { | |
| 28 | const future = () => Date.now() + 60_000; | |
| 29 | test("no identity is anonymous: the protected door refuses", async () => { | |
| 30 | if (!app) return; | |
| 31 | const s = (await door({})).status; | |
| 32 | if (!refused(s)) breaks("identity", `no identity was admitted (${s})`); | |
| 33 | }); | |
| 34 | test("a wrongly signed admin is anonymous", async () => { | |
| 35 | if (!app || app.key === "") return; | |
| 36 | const s = (await door(identity(app.key, "admin", future(), "not-the-key"))).status; | |
| 37 | if (!refused(s)) breaks("identity", `a forged signature was admitted (${s})`); | |
| 38 | }); | |
| 39 | test("an expired identity is anonymous", async () => { | |
| 40 | if (!app || app.key === "") return; | |
| 41 | const s = (await door(identity(app.key, "admin", Date.now() - 1000))).status; | |
| 42 | if (!refused(s)) breaks("identity", `an expired identity was admitted (${s})`); | |
| 43 | }); | |
| 44 | test("an unsigned identity header is anonymous", async () => { | |
| 45 | if (!app) return; | |
| 46 | const s = (await door({ "x-forge-identity": `u1|x|admin|${future()}` })).status; | |
| 47 | if (!refused(s)) breaks("identity", `an unsigned identity was admitted (${s})`); | |
| 48 | }); | |
| 49 | test("a validly signed user is not an admin; a signed admin is admitted", async () => { | |
| 50 | if (!app || app.key === "") return; | |
| 51 | const user = (await door(identity(app.key, "user", future()))).status; | |
| 52 | if (!refused(user)) breaks("identity", `a signed user passed an admin door (${user})`); | |
| 53 | const admin = (await door(identity(app.key, "admin", future()))).status; | |
| 54 | if (refused(admin)) breaks("identity", `a signed admin was refused (${admin})`); | |
| 55 | }); | |
| 56 | }); |
test/standards/forge.test.ts added +78
| @@ -0,0 +1,78 @@ | ||
| 1 | /** Black-box checks of a forge: its list of standards and the feedback standard. */ | |
| 2 | import { afterAll, beforeAll, describe, expect, test } from "bun:test"; | |
| 3 | import { readdirSync, readFileSync } from "node:fs"; | |
| 4 | import { join } from "node:path"; | |
| 5 | import { breaks, forgeTarget, type ForgeTarget } from "./target.ts"; | |
| 6 | ||
| 7 | let forge: ForgeTarget; | |
| 8 | beforeAll(async () => { forge = await forgeTarget(); }, 60_000); | |
| 9 | afterAll(async () => { await forge?.stop(); }); | |
| 10 | ||
| 11 | const root = join(import.meta.dir, "../.."); | |
| 12 | ||
| 13 | describe("standards index (docs/standards/README.md)", () => { | |
| 14 | test("/.well-known/forge lists every standard with an integer version", async () => { | |
| 15 | const res = await fetch(`${forge.url}/.well-known/forge`); | |
| 16 | if (res.status !== 200) breaks("README", `/.well-known/forge answered ${res.status}`); | |
| 17 | const doc = (await res.json()) as { forge?: unknown; standards?: { id: string; version: number; page: string }[] }; | |
| 18 | if (doc.forge !== 1 || !Array.isArray(doc.standards)) breaks("README", "body is not { forge: 1, standards: [] }"); | |
| 19 | for (const id of ["app-manifest", "identity", "permissions", "feedback", "adapters"]) { | |
| 20 | const s = doc.standards.find((x) => x.id === id); | |
| 21 | if (s === undefined) breaks("README", `standard ${id} is not listed`); | |
| 22 | if (!Number.isInteger(s.version) || s.version < 1) breaks("README", `${id} has no integer version`); | |
| 23 | } | |
| 24 | }); | |
| 25 | ||
| 26 | test("every listed standard has a page that states its version", async () => { | |
| 27 | const doc = (await (await fetch(`${forge.url}/.well-known/forge`)).json()) as { standards: { id: string; version: number }[] }; | |
| 28 | for (const s of doc.standards) { | |
| 29 | const page = readFileSync(join(root, "docs/standards", `${s.id}.md`), "utf8"); | |
| 30 | if (!page.includes(`(version ${s.version})`)) breaks(s.id, `the page does not say "(version ${s.version})"`); | |
| 31 | } | |
| 32 | }); | |
| 33 | ||
| 34 | test("every page in docs/standards is listed, and the index table agrees", async () => { | |
| 35 | const doc = (await (await fetch(`${forge.url}/.well-known/forge`)).json()) as { standards: { id: string; version: number }[] }; | |
| 36 | const index = readFileSync(join(root, "docs/standards/README.md"), "utf8"); | |
| 37 | for (const f of readdirSync(join(root, "docs/standards")).filter((n) => n.endsWith(".md") && n !== "README.md")) { | |
| 38 | const id = f.slice(0, -3); | |
| 39 | const s = doc.standards.find((x) => x.id === id); | |
| 40 | if (s === undefined) breaks("README", `${f} is not listed at /.well-known/forge`); | |
| 41 | if (!index.includes(`| \`${id}\` | ${s.version} |`)) breaks("README", `the table does not show ${id} at version ${s.version}`); | |
| 42 | } | |
| 43 | }); | |
| 44 | }); | |
| 45 | ||
| 46 | describe("feedback (docs/standards/feedback.md)", () => { | |
| 47 | const path = () => `${forge.url}/api/v1/repos/${forge.repo}/reports`; | |
| 48 | const post = (body: unknown) => | |
| 49 | fetch(path(), { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify(body) }); | |
| 50 | ||
| 51 | test("a report on a public repository needs no account: 202 with id, state, filedBy", async () => { | |
| 52 | if (forge.repo === "") return; | |
| 53 | const res = await post({ title: "conformance: something is wrong", body: "from the suite", agent: "conformance" }); | |
| 54 | if (res.status !== 202) breaks("feedback", `a report answered ${res.status}, not 202`); | |
| 55 | const got = (await res.json()) as Record<string, unknown>; | |
| 56 | for (const k of ["id", "state", "filedBy"]) if (!(k in got)) breaks("feedback", `the answer has no ${k}`); | |
| 57 | expect(got["state"]).toBe("pending"); | |
| 58 | }); | |
| 59 | ||
| 60 | test("an empty title is refused with a 4xx", async () => { | |
| 61 | if (forge.repo === "") return; | |
| 62 | const res = await post({ title: "", body: "x" }); | |
| 63 | if (res.status < 400 || res.status >= 500) breaks("feedback", `an empty title answered ${res.status}`); | |
| 64 | }); | |
| 65 | ||
| 66 | test("a repository nobody can see is 404", async () => { | |
| 67 | const res = await fetch(`${forge.url}/api/v1/repos/nobody-here/none-such/reports`, { | |
| 68 | method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ title: "x" }), | |
| 69 | }); | |
| 70 | if (res.status !== 404) breaks("feedback", `a missing repository answered ${res.status}, not 404`); | |
| 71 | }); | |
| 72 | ||
| 73 | test("the app-notes list is never given without a token", async () => { | |
| 74 | if (forge.repo === "") return; | |
| 75 | const res = await fetch(`${forge.url}/api/v1/repos/${forge.repo}/app/notes`); | |
| 76 | if (![401, 403, 404].includes(res.status)) breaks("feedback", `notes without a token answered ${res.status}`); | |
| 77 | }); | |
| 78 | }); |
test/standards/permissions.test.ts added +32
| @@ -0,0 +1,32 @@ | ||
| 1 | /** The permissions standard as test vectors any implementation ports, run through this forge's evaluator. */ | |
| 2 | import { describe, expect, test } from "bun:test"; | |
| 3 | import { readFileSync } from "node:fs"; | |
| 4 | import { join } from "node:path"; | |
| 5 | import { decide, readContract, type Actor, type Verb } from "../../src/contract/can.ts"; | |
| 6 | import { breaks } from "./target.ts"; | |
| 7 | ||
| 8 | interface Vectors { | |
| 9 | version: number; | |
| 10 | contract: unknown; | |
| 11 | cases: { why: string; resource: string; verb: Verb; actor: Actor; row?: Record<string, unknown>; expect: unknown }[]; | |
| 12 | } | |
| 13 | const vectors = JSON.parse(readFileSync(join(import.meta.dir, "../../docs/standards/permissions.vectors.json"), "utf8")) as Vectors; | |
| 14 | ||
| 15 | describe("permissions (docs/standards/permissions.md)", () => { | |
| 16 | const read = readContract(JSON.stringify(vectors.contract)); | |
| 17 | test("the vectors' contract is accepted by the reader", () => { | |
| 18 | if (!read.ok) breaks("permissions", `the contract is refused: ${read.problems.join("; ")}`); | |
| 19 | }); | |
| 20 | for (const c of vectors.cases) { | |
| 21 | test(c.why, () => { | |
| 22 | if (!read.ok) return breaks("permissions", "no contract"); | |
| 23 | const got = decide(read.contract, c.resource, c.verb, c.actor, c.row); | |
| 24 | if (JSON.stringify(got) !== JSON.stringify(c.expect)) | |
| 25 | breaks("permissions", `${c.why}: wanted ${JSON.stringify(c.expect)}, got ${JSON.stringify(got)}`); | |
| 26 | }); | |
| 27 | } | |
| 28 | test("a contract with two roles held by one identity is refused", () => { | |
| 29 | const bad = { roles: { anon: { above: [], held: { by: "everyone" } }, a: { above: ["anon"], held: { by: "identity", role: "user" } }, b: { above: ["a"], held: { by: "identity", role: "user" } } }, resources: {} }; | |
| 30 | expect(readContract(JSON.stringify(bad)).ok).toBe(false); | |
| 31 | }); | |
| 32 | }); |
test/standards/target.ts added +100
| @@ -0,0 +1,100 @@ | ||
| 1 | /** | |
| 2 | * What the conformance suite points at. With FORGE_STANDARDS_URL (and a token) | |
| 3 | * it is any forge; with none it boots this one in a scratch directory, which | |
| 4 | * is what CI runs. Same for an app: FORGE_STANDARDS_APP_URL, or the adapter | |
| 5 | * template started on a loopback port. The suite speaks HTTP only; `breaks` | |
| 6 | * puts the page a failing check violates at the head of its message. | |
| 7 | */ | |
| 8 | import { mkdtemp, rm } from "node:fs/promises"; | |
| 9 | import { createHmac } from "node:crypto"; | |
| 10 | import { tmpdir } from "node:os"; | |
| 11 | import { join } from "node:path"; | |
| 12 | ||
| 13 | export interface ForgeTarget { | |
| 14 | readonly url: string; | |
| 15 | readonly token: string; | |
| 16 | /** owner/name of a public repository the token may read. */ | |
| 17 | readonly repo: string; | |
| 18 | stop(): Promise<void>; | |
| 19 | } | |
| 20 | ||
| 21 | export interface AppTarget { | |
| 22 | readonly url: string; | |
| 23 | /** FORGE_APP_IDENTITY_KEY, when known. */ | |
| 24 | readonly key: string; | |
| 25 | stop(): Promise<void>; | |
| 26 | } | |
| 27 | ||
| 28 | const env = (name: string): string => (process.env[name] ?? "").trim(); | |
| 29 | ||
| 30 | /** Fail with the page that says what was promised. */ | |
| 31 | export function breaks(page: string, message: string): never { | |
| 32 | throw new Error(`docs/standards/${page}.md: ${message}`); | |
| 33 | } | |
| 34 | ||
| 35 | export async function forgeTarget(): Promise<ForgeTarget> { | |
| 36 | const url = env("FORGE_STANDARDS_URL").replace(/\/+$/, ""); | |
| 37 | if (url !== "") | |
| 38 | return { url, token: env("FORGE_STANDARDS_TOKEN"), repo: env("FORGE_STANDARDS_REPO"), stop: async () => {} }; | |
| 39 | // Only this forge's own modules are imported, and only to start it. | |
| 40 | const { createApp } = await import("../../src/app.ts"); | |
| 41 | const { serve } = await import("../../src/http/server.ts"); | |
| 42 | const { createUser } = await import("../../src/core/users.ts"); | |
| 43 | const { createRepo } = await import("../../src/actions/repos.ts"); | |
| 44 | const { createToken } = await import("../../src/core/tokens.ts"); | |
| 45 | const dir = await mkdtemp(join(tmpdir(), "forge-conformance-")); | |
| 46 | const app = createApp({ dataDir: dir, port: 0 }); | |
| 47 | const servers = await serve(app); | |
| 48 | const made = await createUser(app.db, { | |
| 49 | origin: "seed", trusted: true, username: "alice", email: "[email protected]", | |
| 50 | password: "a-good-passphrase", adminIfFirst: true, | |
| 51 | }); | |
| 52 | if (!made.ok) throw new Error(String(made.error)); | |
| 53 | const repo = await createRepo(app, made.value, { name: "open", visibility: "public" }); | |
| 54 | if (!repo.ok) throw new Error(String(repo.error)); | |
| 55 | const { token } = createToken(app.db, made.value.id, "conformance", "rw"); | |
| 56 | return { | |
| 57 | url: servers.url, token, repo: "alice/open", | |
| 58 | stop: async () => { | |
| 59 | await servers.stop(); | |
| 60 | app.db.close(); | |
| 61 | await rm(dir, { recursive: true, force: true }); | |
| 62 | }, | |
| 63 | }; | |
| 64 | } | |
| 65 | ||
| 66 | export async function appTarget(): Promise<AppTarget | undefined> { | |
| 67 | const url = env("FORGE_STANDARDS_APP_URL").replace(/\/+$/, ""); | |
| 68 | if (url !== "") return { url, key: env("FORGE_STANDARDS_APP_KEY"), stop: async () => {} }; | |
| 69 | if (env("FORGE_STANDARDS_URL") !== "") return undefined; | |
| 70 | // The adapter template is the reference app: it starts as the forge starts an app. | |
| 71 | const dir = await mkdtemp(join(tmpdir(), "forge-conformance-app-")); | |
| 72 | const port = 20000 + Math.floor(Math.random() * 20000); | |
| 73 | const key = "conformance-key"; | |
| 74 | const proc = Bun.spawn(["bun", "run", "src/server.ts"], { | |
| 75 | cwd: join(import.meta.dir, "../../examples/adapter-template"), | |
| 76 | env: { PATH: process.env["PATH"] ?? "", PORT: String(port), FORGE_APP_DATA: dir, FORGE_APP_IDENTITY_KEY: key }, | |
| 77 | stdout: "ignore", stderr: "ignore", | |
| 78 | }); | |
| 79 | const base = `http://127.0.0.1:${port}`; | |
| 80 | for (let i = 0; i < 100; i++) { | |
| 81 | try { | |
| 82 | if ((await fetch(`${base}/healthz`)).ok) break; | |
| 83 | } catch { /* not yet */ } | |
| 84 | await Bun.sleep(50); | |
| 85 | } | |
| 86 | return { | |
| 87 | url: base, key, | |
| 88 | stop: async () => { | |
| 89 | proc.kill(); | |
| 90 | await proc.exited; | |
| 91 | await rm(dir, { recursive: true, force: true }); | |
| 92 | }, | |
| 93 | }; | |
| 94 | } | |
| 95 | ||
| 96 | /** A signed identity exactly as the identity standard writes it. */ | |
| 97 | export function identity(key: string, role: string, expiresMs: number, signKey = key): Record<string, string> { | |
| 98 | const value = `u1|conformance|${role}|${expiresMs}`; | |
| 99 | return { "x-forge-identity": value, "x-forge-signature": createHmac("sha256", signKey).update(value).digest("base64url") }; | |
| 100 | } |
new above
offline