Public: anyone can read this; the crew can change it.
Platform app move gap: Inbound payment webhooks that survive a preview's changing address #11721
Check · Review · Land · Live done — Landed and live in production.
scout/forge:scout/11703-platform-app-move-gap-inbound-payment-we → main 3 commits fork 2 threads · 2 unresolved
5 files
changed +280
−2
docs/APPS.md +5
| @@ -51,6 +51,11 @@ Under the box, four starter sentences are plain links (no script): | ||
| 51 | 51 | redeployed on each push and gone when the pull closes. Its data starts as |
| 52 | 52 | a copy of production's — copy-on-write where the filesystem can — and |
| 53 | 53 | never writes back. Merging moves code and touches no data. |
| 54 | - **A webhook has a stable address.** `POST /__forge/hook/<pull>/<path>` on | |
| 55 | production's host is relayed, untouched, to pull `<pull>`'s preview as | |
| 56 | `POST /<path>` (`src/apps/hook_relay.ts`), so a payment provider keeps one | |
| 57 | URL while previews change. The provider's signature reaches the app | |
| 58 | unchanged; the app verifies it. See `docs/features/platform-app-move.md`. | |
| 54 | 59 | - **A preview is held to the website specification, report-only.** Once it is |
| 55 | 60 | live, `webspec()` runs over the `app.looks` paths of its `forge.json`. The pull |
| 56 | 61 | page shows the result under the Checks line as **Website spec: N findings |
docs/features/platform-app-move.md +17 −2
| @@ -53,7 +53,22 @@ variable set for scope `preview` beats the same name set for `all` or | ||
| 53 | 53 | |
| 54 | 54 | What stays open: the forge cannot tell which names are payment keys, so |
| 55 | 55 | nothing refuses a live key in `all` or in `preview`; the checks above are a |
| 56 | person's. Webhooks reaching a preview's changing address are #11703. | |
| 56 | person's. Webhooks reaching a preview's changing address: see "Webhooks" below. | |
| 57 | ||
| 58 | ### Webhooks | |
| 59 | Give the provider's sandbox **one** URL that never changes: | |
| 60 | `https://<app>.<forge host>/__forge/hook/<pull>/<path>`. A `POST` there reaches | |
| 61 | the preview of pull `<pull>` as `POST /<path>`, body and headers untouched, so | |
| 62 | the provider's own signature (`Stripe-Signature` and the like) verifies in the | |
| 63 | app against the sandbox `STRIPE_WEBHOOK_SECRET` set for scope `preview`. The | |
| 64 | forge adds no signature of its own and cannot break the provider's. When the | |
| 65 | pull changes, change only the number. | |
| 66 | - `POST` only (405 otherwise); 404 when the pull has no preview or the path | |
| 67 | would leave the app (`..`, `//`, `/__forge/…`); 413 over 1 MiB; a sleeping | |
| 68 | preview is woken by the request. | |
| 69 | - A private app, or a sealed pull's preview, refuses it like any request | |
| 70 | without a grant: a provider cannot sign in. Rehearse payments on a public app. | |
| 71 | - The app must verify the provider's signature itself and answer 2xx only then. | |
| 57 | 72 | |
| 58 | 73 | ## 5. Backup before anything is removed (person) |
| 59 | 74 | Encrypted dump and files archive (`age` or `gpg`, key held by the owner, not |
| @@ -128,6 +143,6 @@ reminds anyone when the week ends; both are a person's. | ||
| 128 | 143 | - A file and upload copy with a manifest compared on both ends. (#11699) |
| 129 | 144 | - Per-app encrypted backup with a restore proof (`docs/features/backup-truth.md` covers the forge's own, not an app's). (#11701) |
| 130 | 145 | - A guard for rehearsal: data copy and `preview`-scoped sandbox keys exist and are the procedure in step 4; nothing yet refuses a payment name left in scope `all`. (#11702) |
| 131 | - Inbound payment webhooks that survive a preview's changing address. (#11703) | |
| 146 | - ~~Inbound payment webhooks that survive a preview's changing address.~~ Built: `/__forge/hook/<pull>/…` on production's host. (#11703) | |
| 132 | 147 | - ~~A read-only switch for the old side of a migration.~~ A procedure at the old proxy and database, section 7; the forge cannot reach the old node, so no feature is built. (#11704) |
| 133 | 148 | - Importing an app's customer accounts (password-hash formats) without the owner reading them. (#11705) |
src/apps/hook_relay.ts added +53
| @@ -0,0 +1,53 @@ | ||
| 1 | /** | |
| 2 | * A stable address for inbound webhooks. A payment provider is given one URL | |
| 3 | * and keeps it; a preview's address changes with every pull. Production's | |
| 4 | * host answers `POST /__forge/hook/<pull>/<path>` by handing the request, | |
| 5 | * untouched, to that pull's preview as `POST /<path>`. The body and headers | |
| 6 | * (so the provider's own signature) arrive as sent: the app verifies the | |
| 7 | * provider, the forge adds nothing it could break. Plain data in, plain data out. | |
| 8 | */ | |
| 9 | ||
| 10 | /** Largest body relayed: a webhook is a small JSON document. */ | |
| 11 | export const HOOK_MAX_BYTES = 1024 * 1024; | |
| 12 | ||
| 13 | export const HOOK_PREFIX = "/__forge/hook/"; | |
| 14 | ||
| 15 | /** The pull and the path the preview is asked for, or nothing when the path is no hook's. */ | |
| 16 | export function hookPath(pathname: string): { pull: number; path: string } | undefined { | |
| 17 | if (!pathname.startsWith(HOOK_PREFIX)) return undefined; | |
| 18 | const match = /^([1-9]\d{0,8})(\/[^\s]*)?$/.exec(pathname.slice(HOOK_PREFIX.length)); | |
| 19 | if (match === null) return undefined; | |
| 20 | const path = match[2] ?? "/"; | |
| 21 | // No way out of the app's own paths, and none into the forge's. | |
| 22 | if (path.split("/").some((part) => part === ".." || part === ".")) return undefined; | |
| 23 | if (path.startsWith("//") || path.startsWith("/__forge")) return undefined; | |
| 24 | return { pull: Number(match[1]), path }; | |
| 25 | } | |
| 26 | ||
| 27 | /** The body's bytes, or nothing once more than `max` have been read: the count is of bytes received, whatever the headers claimed. */ | |
| 28 | export async function readCapped( | |
| 29 | body: ReadableStream<Uint8Array> | null, | |
| 30 | max: number, | |
| 31 | ): Promise<Uint8Array | undefined> { | |
| 32 | if (body === null) return new Uint8Array(); | |
| 33 | const chunks: Uint8Array[] = []; | |
| 34 | let size = 0; | |
| 35 | const reader = body.getReader(); | |
| 36 | for (;;) { | |
| 37 | const { done, value } = await reader.read(); | |
| 38 | if (done) break; | |
| 39 | size += value.byteLength; | |
| 40 | if (size > max) { | |
| 41 | await reader.cancel().catch(() => undefined); | |
| 42 | return undefined; | |
| 43 | } | |
| 44 | chunks.push(value); | |
| 45 | } | |
| 46 | const out = new Uint8Array(size); | |
| 47 | let at = 0; | |
| 48 | for (const c of chunks) { | |
| 49 | out.set(c, at); | |
| 50 | at += c.byteLength; | |
| 51 | } | |
| 52 | return out; | |
| 53 | } |
src/apps/proxy.ts +29
| @@ -67,6 +67,7 @@ import { appMeasure } from "./measure.ts"; | ||
| 67 | 67 | import { holdPortRequest, holdPortSocket } from "./handoff.ts"; |
| 68 | 68 | import { secureFor } from "../http/cookie_secure.ts"; |
| 69 | 69 | import { currentLoopLag, forgeWaited, LAG_WINDOW_MS } from "../perf/loop_worst.ts"; |
| 70 | import { HOOK_MAX_BYTES, HOOK_PREFIX, hookPath, readCapped } from "./hook_relay.ts"; | |
| 70 | 71 | import { loopWatch } from "../perf/live.ts"; |
| 71 | 72 | import { heldBy, noteRelayWait, stallTiming } from "./relay_wait.ts"; |
| 72 | 73 | import { cspNonce, htmlAnswer, readPage, rewritable, TOOLBAR_ASSET_PATH, TOOLBAR_ASSET_PREFIX, toolbarResponse, withToolbar, type PillWhere } from "./toolbar.ts"; |
| @@ -983,6 +984,34 @@ export async function proxyApp( | ||
| 983 | 984 | ): Promise<Response | undefined> { |
| 984 | 985 | const entered = performance.now(); |
| 985 | 986 | const url = new URL(request.url); |
| 987 | // A webhook for a pull's preview, sent to production's stable address. | |
| 988 | if (url.pathname.startsWith(HOOK_PREFIX) && !target.nested && target.env === "production") { | |
| 989 | const hook = hookPath(url.pathname); | |
| 990 | if (hook === undefined || request.method !== "POST") | |
| 991 | return securityFloor( | |
| 992 | hook === undefined ? notFound() : new Response(null, { status: 405, headers: { allow: "POST" } }), | |
| 993 | ); | |
| 994 | const env = previewEnv(hook.pull); | |
| 995 | if (runtime.envStatus(target.app, env) === undefined) return securityFloor(notFound()); | |
| 996 | // The header is an early refusal only; the cap is on the bytes read. | |
| 997 | const length = Number(request.headers.get("content-length") ?? "0"); | |
| 998 | if (!(length <= HOOK_MAX_BYTES)) return securityFloor(new Response("too large", { status: 413 })); | |
| 999 | const body = await readCapped(request.body, HOOK_MAX_BYTES); | |
| 1000 | if (body === undefined) return securityFloor(new Response("too large", { status: 413 })); | |
| 1001 | const previewHost = new URL(runtime.urlFor(target.app, env)); | |
| 1002 | const next = new URL(url); | |
| 1003 | next.protocol = previewHost.protocol; | |
| 1004 | next.host = previewHost.host; | |
| 1005 | next.pathname = hook.path; | |
| 1006 | const headers = new Headers(request.headers); | |
| 1007 | headers.set("host", previewHost.host); | |
| 1008 | return proxyApp( | |
| 1009 | runtime, | |
| 1010 | new Request(next.toString(), { method: "POST", headers, body, signal: request.signal }), | |
| 1011 | { app: target.app, env, nested: false }, | |
| 1012 | server, | |
| 1013 | ); | |
| 1014 | } | |
| 986 | 1015 | // The pill's script, from the app's own host: only the exact hashed path |
| 987 | 1016 | // is it; any other /__forge/toolbar… path is the forge's 404, never the app's. |
| 988 | 1017 | if (url.pathname.startsWith(TOOLBAR_ASSET_PREFIX) && !target.nested) { |
test/apps-hook-relay.test.ts added +176
| @@ -0,0 +1,176 @@ | ||
| 1 | /** A webhook sent to production's stable address reaches a pull's preview. */ | |
| 2 | ||
| 3 | import { afterAll, beforeAll, expect, spyOn, test } from "bun:test"; | |
| 4 | import { mkdtemp, rm } from "node:fs/promises"; | |
| 5 | import { tmpdir } from "node:os"; | |
| 6 | import { join } from "node:path"; | |
| 7 | import { createApp, type App } from "../src/app.ts"; | |
| 8 | import { newApp } from "../src/actions/apps.ts"; | |
| 9 | import { openIssue } from "../src/actions/issues.ts"; | |
| 10 | import { AppsRuntime, type EnvStatus } from "../src/apps/runtime.ts"; | |
| 11 | import { setSealed } from "../src/core/sealed.ts"; | |
| 12 | import { createUser } from "../src/core/users.ts"; | |
| 13 | import { quietLogs } from "./helpers/globals.ts"; | |
| 14 | import { HOOK_MAX_BYTES, HOOK_PREFIX, hookPath } from "../src/apps/hook_relay.ts"; | |
| 15 | import { proxyApp, type AppTarget } from "../src/apps/proxy.ts"; | |
| 16 | ||
| 17 | test("hookPath names the pull and the path, and refuses what could escape", () => { | |
| 18 | expect(hookPath(`${HOOK_PREFIX}7/stripe/webhook`)).toEqual({ pull: 7, path: "/stripe/webhook" }); | |
| 19 | expect(hookPath(`${HOOK_PREFIX}12`)).toEqual({ pull: 12, path: "/" }); | |
| 20 | for (const bad of ["0/x", "x/y", "07/x", "7/../x", "7//evil.test", "7/__forge/note", "", "7/a b"]) | |
| 21 | expect(hookPath(`${HOOK_PREFIX}${bad}`)).toBeUndefined(); | |
| 22 | expect(hookPath("/other/7/x")).toBeUndefined(); | |
| 23 | }); | |
| 24 | ||
| 25 | const target = { app: { id: "a", slug: "shop" }, env: "production" } as unknown as AppTarget; | |
| 26 | const bare = { envStatus: () => undefined } as never; | |
| 27 | const call = (method: string, path: string, init: RequestInit = {}) => | |
| 28 | proxyApp(bare, new Request(`http://shop.test${path}`, { method, ...init }), target, {} as never); | |
| 29 | ||
| 30 | test("only POST is relayed, and only to a preview that exists", async () => { | |
| 31 | expect((await call("GET", `${HOOK_PREFIX}7/x`))?.status).toBe(405); | |
| 32 | expect((await call("POST", `${HOOK_PREFIX}7/x`, { body: "{}" }))?.status).toBe(404); | |
| 33 | expect((await call("POST", `${HOOK_PREFIX}x`, { body: "{}" }))?.status).toBe(404); | |
| 34 | }); | |
| 35 | ||
| 36 | test("a body over the limit is refused before the preview is woken", async () => { | |
| 37 | const live = { envStatus: () => ({ state: "running" }), urlFor: () => "http://shop--7.test" } as never; | |
| 38 | const r = await proxyApp( | |
| 39 | live, | |
outdated
Add the success relay test (path, body and signature header reach the preview), the chunked over-limit test and the sealed-preview refusal test. |
||
| 40 | new Request(`http://shop.test${HOOK_PREFIX}7/x`, { | |
| 41 | method: "POST", | |
| 42 | headers: { "content-length": String(2 * 1024 * 1024) }, | |
| 43 | body: "{}", | |
| 44 | }), | |
| 45 | target, | |
| 46 | {} as never, | |
| 47 | ); | |
| 48 | expect(r?.status).toBe(413); | |
| 49 | }); | |
| 50 | ||
| 51 | // The forwarding path, against a real runtime and a real preview process. | |
| 52 | quietLogs(); | |
| 53 | let dataDir: string; | |
| 54 | let app: App; | |
| 55 | let runtime: AppsRuntime; | |
| 56 | let backend: ReturnType<typeof Bun.serve>; | |
| 57 | let seen: { method: string; path: string; body: string; sig: string | null; host: string | null }[] = []; | |
| 58 | let status: ReturnType<typeof spyOn>; | |
| 59 | let live: AppTarget; | |
| 60 | let openNumber: number; | |
| 61 | let sealedNumber: number; | |
| 62 | ||
| 63 | beforeAll(async () => { | |
| 64 | backend = Bun.serve({ | |
| 65 | port: 0, | |
| 66 | fetch: async (req) => { | |
| 67 | seen.push({ | |
| 68 | method: req.method, | |
| 69 | path: new URL(req.url).pathname, | |
| 70 | body: await req.text(), | |
| 71 | sig: req.headers.get("stripe-signature"), | |
| 72 | host: req.headers.get("host"), | |
| 73 | }); | |
| 74 | return new Response("received", { status: 202 }); | |
| 75 | }, | |
| 76 | }); | |
| 77 | dataDir = await mkdtemp(join(tmpdir(), "forge-hook-relay-")); | |
| 78 | app = createApp({ dataDir, port: 0, host: "forge.test" }); | |
| 79 | runtime = new AppsRuntime(app, { | |
| 80 | domain: "apps.test", | |
| 81 | scheme: "http", | |
| 82 | publicPort: 0, | |
| 83 | forgeUrl: "http://forge.test", | |
| 84 | concurrency: 1, | |
| 85 | }); | |
| 86 | await runtime.start(); | |
| 87 | status = spyOn(AppsRuntime.prototype, "envStatus").mockImplementation( | |
| 88 | (_a, env) => | |
| 89 | ({ env, state: "running", port: backend.port, commitOid: "a".repeat(40) }) as EnvStatus, | |
| 90 | ); | |
| 91 | const u = await createUser(app.db, { | |
| 92 | origin: "seed", | |
| 93 | trusted: true, | |
| 94 | username: "alice", | |
| 95 | email: "[email protected]", | |
| 96 | password: "a-good-passphrase", | |
| 97 | adminIfFirst: true, | |
| 98 | }); | |
| 99 | if (!u.ok) throw new Error(u.error); | |
| 100 | const made = await newApp(app, u.value, { name: "shop", description: "A shop", visibility: "public" }); | |
| 101 | if (!made.ok) throw new Error(String(made.error)); | |
| 102 | live = { app: made.value.app, env: "production" } as AppTarget; | |
| 103 | const open = openIssue(app, u.value, made.value.repo, { title: "open pull" }); | |
| 104 | const sealed = openIssue(app, u.value, made.value.repo, { title: "sealed pull" }); | |
| 105 | if (!open.ok || !sealed.ok) throw new Error("issues"); | |
| 106 | openNumber = open.value.number; | |
| 107 | sealedNumber = sealed.value.number; | |
| 108 | setSealed(app.db, sealed.value.id, true); | |
| 109 | }, 120_000); | |
| 110 | ||
| 111 | afterAll(async () => { | |
| 112 | status.mockRestore(); | |
| 113 | await runtime.stop(); | |
| 114 | backend.stop(true); | |
| 115 | app.close(); | |
| 116 | await rm(dataDir, { recursive: true, force: true }); | |
| 117 | }); | |
| 118 | ||
| 119 | const relay = (n: number, init: RequestInit & { duplex?: string }, t: AppTarget = live) => | |
| 120 | proxyApp( | |
| 121 | runtime, | |
| 122 | new Request(`http://shop.apps.test${HOOK_PREFIX}${n}/stripe/webhook`, { method: "POST", ...init }), | |
| 123 | t, | |
| 124 | {} as never, | |
| 125 | ); | |
| 126 | ||
| 127 | test("the preview sees the same path, body, signature and its own host", async () => { | |
| 128 | seen = []; | |
| 129 | const body = '{"id":"evt_1","type":"payment_intent.succeeded"}'; | |
| 130 | const r = await relay(openNumber, { body, headers: { "stripe-signature": "t=1,v1=abc", host: "shop.apps.test" } }); | |
| 131 | expect(r?.status).toBe(202); | |
| 132 | expect(seen).toHaveLength(1); | |
| 133 | expect(seen[0]!.method).toBe("POST"); | |
| 134 | expect(seen[0]!.path).toBe("/stripe/webhook"); | |
| 135 | expect(seen[0]!.body).toBe(body); | |
| 136 | expect(seen[0]!.sig).toBe("t=1,v1=abc"); | |
| 137 | expect(seen[0]!.host).toBe(new URL(runtime.urlFor(live.app, `pull-${openNumber}`)).host); | |
| 138 | }); | |
| 139 | ||
| 140 | test("a chunked body over the limit is refused by the bytes read, with no content-length", async () => { | |
| 141 | seen = []; | |
| 142 | const chunk = new Uint8Array(256 * 1024); | |
| 143 | let sent = 0; | |
| 144 | const stream = new ReadableStream<Uint8Array>({ | |
| 145 | pull(c) { | |
| 146 | if (sent > HOOK_MAX_BYTES + chunk.length) return c.close(); | |
| 147 | sent += chunk.length; | |
| 148 | c.enqueue(chunk); | |
| 149 | }, | |
| 150 | }); | |
| 151 | const r = await relay(openNumber, { body: stream, duplex: "half" }); | |
| 152 | expect(r?.status).toBe(413); | |
| 153 | expect(seen).toHaveLength(0); | |
| 154 | // A lying header is no better: a small claim over a large body. | |
| 155 | const lie = await relay(openNumber, { | |
| 156 | body: new Uint8Array(HOOK_MAX_BYTES + 1), | |
| 157 | headers: { "content-length": "10" }, | |
| 158 | }); | |
| 159 | expect(lie?.status).toBe(413); | |
| 160 | expect(seen).toHaveLength(0); | |
| 161 | }); | |
| 162 | ||
| 163 | test("a sealed preview is refused although production is public", async () => { | |
| 164 | seen = []; | |
| 165 | const r = await relay(sealedNumber, { body: "{}" }); | |
| 166 | expect(r?.status).toBe(404); | |
| 167 | expect(seen).toHaveLength(0); | |
| 168 | }); | |
| 169 | ||
| 170 | test("a private app's preview is refused to a caller with no grant", async () => { | |
| 171 | seen = []; | |
| 172 | const priv = { app: { ...live.app, access: "private" }, env: "production" } as AppTarget; | |
| 173 | const r = await relay(openNumber, { body: "{}" }, priv); | |
| 174 | expect(r?.status).toBe(404); | |
| 175 | expect(seen).toHaveLength(0); | |
| 176 | }); |
new above
offline
Cap by bytes read, not Content-Length. A missing or false header (chunked) gets past this. Count bytes while streaming request.body and answer 413 when the count passes HOOK_MAX_BYTES.