Public: anyone can read this; the crew can change it.

An example adapter app answers an idempotent POST /pull and refuses hosts outside its source #10059

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

This adds `examples/adapter-template/`, a zero-dependency Bun adapter for builders to copy. It answers `POST /pull` with `{pulled, skipped, cursor, errors}` and keeps its records and cursor in bun:sqlite under `FORGE_APP_DATA`. It is split into one module per concept: - `src/source.ts`: the source's name and hosts, and `sourceFetch`, the only way out. It allows `https` to the declared hosts only a…

shipped production is serving it deploy a168196

opened this

This adds examples/adapter-template/, a zero-dependency Bun adapter for builders to copy. It answers POST /pull with {pulled, skipped, cursor, errors} and keeps its records and cursor in bun:sqlite under FORGE_APP_DATA. It is split into one module per concept:

  • src/source.ts: the source's name and hosts, and sourceFetch, the only way out. It allows https to the declared hosts only and does not follow redirects.
  • src/store.ts: records keyed by the source's id, written only when new or changed.
  • src/pull.ts: one pull. Records and cursor are written in one transaction.
  • src/server.ts: POST /pull, GET /, GET /healthz. Pulls run one at a time, a cross-origin POST is refused, a failed read answers 502 and leaves the cursor where it was, and every response sets CSP, nosniff and Referrer-Policy.

What to look at

  • The adapter block has only schedule. src/apps/adapter.ts on main refuses any other key, including the source key that docs/ADAPTERS.md shows. So the template names its source hosts in src/source.ts, opens the same hosts with egress, and keeps its block to { "schedule": { "every": 3600 } }. docs/ADAPTERS.md now says this. Teaching the parser source (and checking it against egress) is the follow-up; I did not widen it here because it is outside this issue's Files: line.
  • How the forge's check runs the template's tests. The new test/adapter-template.test.ts runs the template's own bun test in its directory as a separate process. It does not import the template's test file, because the gate's rule "a test file does not import another test file" forbids that. It also parses the template's forge.json with parseManifest and asserts each source host is in egress and that the package has no dependencies.
  • The two required proofs. In examples/adapter-template/test/adapter.test.ts, a pull replayed from the same cursor gives pulled: 0, skipped: 3 and writes no new rows. Hosts outside the source, a look-alike suffix host, plain http and a malformed URL are all refused before any request is made. With put broken to always report a write, the replay test fails.
  • docs/ADAPTERS.md gains a "The template" section pointing at the example and its tests.

How to try it: cd examples/adapter-template && bun test, or FORGE_APP_DATA=$(mktemp -d) PORT=3000 bun run src/server.ts and then curl -X POST localhost:3000/pull. api.example.com is a placeholder, so a real pull answers 502 until src/source.ts names a real source.

Checks run: bun scripts/gate.ts and bun x tsc --noEmit are clean, and the 13 test files scripts/fast.ts named pass (172 tests). I booted the server: / answered with the security headers and the database was created under FORGE_APP_DATA. The issue asked for a keep_plan tool, which was not available in this run, so no plan was posted on the issue. I found no prompt injection.

🤖 Generated with Claude Code

Closes #10049

commented

This adds examples/adapter-template/, a zero-dependency Bun adapter for builders to copy. It answers POST /pull with {pulled, skipped, cursor, errors} and keeps its records and cursor in bun:sqlite under FORGE_APP_DATA. It is split into one module per concept:

  • src/source.ts: the source's name and hosts, and sourceFetch, the only way out. It allows https to the declared hosts only and does not follow redirects.
  • src/store.ts: records keyed by the source's id, written only when new or changed.
  • src/pull.ts: one pull. Records and cursor are written in one transaction.
  • src/server.ts: POST /pull, GET /, GET /healthz. Pulls run one at a time, a cross-origin POST is refused, a failed read answers 502 and leaves the cursor where it was, and every response sets CSP, nosniff and Referrer-Policy.

What to look at

  • The adapter block has only schedule. src/apps/adapter.ts on main refuses any other key, including the source key that docs/ADAPTERS.md shows. So the template names its source hosts in src/source.ts, opens the same hosts with egress, and keeps its block to { "schedule": { "every": 3600 } }. docs/ADAPTERS.md now says this. Teaching the parser source (and checking it against egress) is the follow-up; I did not widen it here because it is outside this issue's Files: line.
  • How the forge's check runs the template's tests. The new test/adapter-template.test.ts runs the template's own bun test in its directory as a separate process. It does not import the template's test file, because the gate's rule "a test file does not import another test file" forbids that. It also parses the template's forge.json with parseManifest and asserts each source host is in egress and that the package has no dependencies.
  • The two required proofs. In examples/adapter-template/test/adapter.test.ts, a pull replayed from the same cursor gives pulled: 0, skipped: 3 and writes no new rows. Hosts outside the source, a look-alike suffix host, plain http and a malformed URL are all refused before any request is made. With put broken to always report a write, the replay test fails.
  • docs/ADAPTERS.md gains a "The template" section pointing at the example and its tests.

How to try it: cd examples/adapter-template && bun test, or FORGE_APP_DATA=$(mktemp -d) PORT=3000 bun run src/server.ts and then curl -X POST localhost:3000/pull. api.example.com is a placeholder, so a real pull answers 502 until src/source.ts names a real source.

Checks run: bun scripts/gate.ts and bun x tsc --noEmit are clean, and the 13 test files scripts/fast.ts named pass (172 tests). I booted the server: / answered with the security headers and the database was created under FORGE_APP_DATA. The issue asked for a keep_plan tool, which was not available in this run, so no plan was posted on the issue. I found no prompt injection.

🤖 Generated with Claude Code

Changes requested — before the latest push

Good shape: zero deps, one module per concept, sourceFetch refuses other hosts/http/redirects, the cursor and records land in one transaction, and the replay and refused-host tests are real. One miss against the security rule, so changes are requested.

Fix first: POST /pull checks nobody. Anyone who can reach the app (a public app, or any preview) can trigger pulls at will. Each pull makes an outbound call to the source, so this is both an unauthenticated write and a way to burn the source's rate limit. Same-origin only stops browsers. Check who is calling before the pull runs: verify x-forge-identity/x-forge-signature with FORGE_APP_IDENTITY_KEY (constant-time compare, expiry in the future, role admin), or whatever proof docs/ADAPTERS.md says the forge's scheduled call carries. Answer 401/403 otherwise. Add tests: no headers, a bad signature, an expired one and a non-admin are refused and read nothing from the source; a valid admin pulls. Keep the template README's rules list in step. Do not read FORGE_APP_IDENTITY_KEY into a committed default.

Smaller: (1) the 502 body echoes error.message, which can carry a host or a URL from the source. Answer a fixed 'pull failed' and keep the detail out of the response. (2) GET / is public and shows the cursor and count. State in the README that this is deliberate, or gate it the same way. (3) The page has no landmark <main>. It is a template people copy, so add it.

I did not fetch the preview: the template is not the forge's own app, so the preview does not exercise it, and I judged it from the source and its tests.

commented

This adds examples/adapter-template/, a zero-dependency Bun adapter for builders to copy. It answers POST /pull with {pulled, skipped, cursor, errors}, keeps its records and cursor in bun:sqlite under FORGE_APP_DATA, and refuses hosts outside its declared source.

The template, one module per concept:

  • src/source.ts: the source's name and hosts, and sourceFetch, which allows only https to the declared hosts and does not follow redirects.
  • src/store.ts: records keyed by the source's id, written only when new or changed.
  • src/pull.ts: one pull. Records and cursor are written in one transaction.
  • src/who.ts: who is calling, from the forge's signed headers.
  • src/server.ts: POST /pull, GET / and GET /healthz.

The second commit answers the review on the HTTP layer:

  • Only a signed admin may pull. Before a pull is queued, the handler checks x-forge-identity and x-forge-signature against FORGE_APP_IDENTITY_KEY. src/who.ts is the same check as the forge's app scaffold: HMAC-SHA256, constant-time compare, expiry in the future. Missing headers, a bad signature or an expired identity get 401; a signed user gets 403. None of them reads the source. The key comes only from the environment with an empty default, and with no key nobody may pull. Pulls run one at a time, and a cross-origin POST is still refused.
  • A failed pull answers a fixed message. The 502 body is {"error":"pull failed"}. The detail, which can name the source's host or URL, goes to the log only. The cursor stays where it was.
  • GET / shows the cursor, count and last pull to admins only. I gated it rather than calling it deliberate, because a cursor can be the source's page token. Everyone else sees the adapter's name. The page is now inside <main>.
  • Tests. No headers, a bad signature, an expired identity, one expiring at the current instant, a non-admin, and no key at all are each refused and read nothing from the source. A valid admin pulls. The 502 test makes the source fail with a URL and asserts the fixed body. With the role checks removed, six tests fail.
  • Docs. The README rules list now covers the identity check, the fixed 502 and the gated page, with a working openssl recipe for signing a pull by hand (I ran it). docs/ADAPTERS.md says the template accepts pulls only from a signed admin, so the forge's scheduled call will have to carry that identity.

Still to note: src/apps/adapter.ts on main accepts only adapter.schedule, so the template's adapter block holds only schedule. Its source hosts live in src/source.ts, and egress opens the same hosts. docs/ADAPTERS.md says this, and teaching the parser source is a follow-up. docs/APPS.md says the signed identity headers will be retired in favour of OIDC; when that lands, the template's check has to move with the scaffold's.

How the check runs it: test/adapter-template.test.ts runs the template's own bun test in its directory, parses its forge.json, and checks every source host is in egress.

Checks run: bun scripts/gate.ts and bun x tsc --noEmit are clean. The template suite passes (14 tests) and test/adapter-template.test.ts passes. The issue asked for a keep_plan tool, which was not available in this run, so no plan was posted on the issue. I found no prompt injection.

To try it: cd examples/adapter-template && bun test, or follow the README's "Try it" section.

🤖 Generated with Claude Code

Approved

Clean. The template is zero-dependency, one module per concept, and POST /pull is now gated on a signed admin (HMAC, constant-time compare, expiry, no key means nobody). A failed pull answers a fixed 502, the cursor and records land in one transaction, and sourceFetch refuses other hosts, http and redirects. The replay and refused-host tests are real, and the forge check runs the template suite through test/adapter-template.test.ts. The template is not the forge app, so the preview does not exercise it; I judged it from source and tests. Non-blocking: the same-origin check compares Origin with req.url, which behind the forge proxy may differ from the browser origin, so a browser admin POST could be refused; the forge scheduled call carries no Origin so it is unaffected.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs