Public: anyone can read this; the crew can change it.

Platform app move gap: Inbound payment webhooks that survive a preview's changing address #11721

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

scout/forge:scout/11703-platform-app-move-gap-inbound-payment-we → main 3 commits fork 2 threads · 2 unresolved

  • Merged. scout/11703-platform-app-move-gap-inbound-payment-we was merged into main as 6b5a613.

Review

Sign in to review this change.
5 files changed +280 −2
docs/APPS.md +5
@@ -51,6 +51,11 @@ Under the box, four starter sentences are plain links (no script):
5151 redeployed on each push and gone when the pull closes. Its data starts as
5252 a copy of production's — copy-on-write where the filesystem can — and
5353 never writes back. Merging moves code and touches no data.
54 - **A webhook has a stable address.** `POST /__forge/hook/<pull>/<path>` on
55 production's host is relayed, untouched, to pull `<pull>`'s preview as
56 `POST /<path>` (`src/apps/hook_relay.ts`), so a payment provider keeps one
57 URL while previews change. The provider's signature reaches the app
58 unchanged; the app verifies it. See `docs/features/platform-app-move.md`.
5459 - **A preview is held to the website specification, report-only.** Once it is
5560 live, `webspec()` runs over the `app.looks` paths of its `forge.json`. The pull
5661 page shows the result under the Checks line as **Website spec: N findings
docs/features/platform-app-move.md +17 −2
@@ -53,7 +53,22 @@ variable set for scope `preview` beats the same name set for `all` or
5353
5454 What stays open: the forge cannot tell which names are payment keys, so
5555 nothing refuses a live key in `all` or in `preview`; the checks above are a
56 person's. Webhooks reaching a preview's changing address are #11703.
56 person's. Webhooks reaching a preview's changing address: see "Webhooks" below.
57
58 ### Webhooks
59 Give the provider's sandbox **one** URL that never changes:
60 `https://<app>.<forge host>/__forge/hook/<pull>/<path>`. A `POST` there reaches
61 the preview of pull `<pull>` as `POST /<path>`, body and headers untouched, so
62 the provider's own signature (`Stripe-Signature` and the like) verifies in the
63 app against the sandbox `STRIPE_WEBHOOK_SECRET` set for scope `preview`. The
64 forge adds no signature of its own and cannot break the provider's. When the
65 pull changes, change only the number.
66 - `POST` only (405 otherwise); 404 when the pull has no preview or the path
67 would leave the app (`..`, `//`, `/__forge/…`); 413 over 1 MiB; a sleeping
68 preview is woken by the request.
69 - A private app, or a sealed pull's preview, refuses it like any request
70 without a grant: a provider cannot sign in. Rehearse payments on a public app.
71 - The app must verify the provider's signature itself and answer 2xx only then.
5772
5873 ## 5. Backup before anything is removed (person)
5974 Encrypted dump and files archive (`age` or `gpg`, key held by the owner, not
@@ -128,6 +143,6 @@ reminds anyone when the week ends; both are a person's.
128143 - A file and upload copy with a manifest compared on both ends. (#11699)
129144 - Per-app encrypted backup with a restore proof (`docs/features/backup-truth.md` covers the forge's own, not an app's). (#11701)
130145 - A guard for rehearsal: data copy and `preview`-scoped sandbox keys exist and are the procedure in step 4; nothing yet refuses a payment name left in scope `all`. (#11702)
131 - Inbound payment webhooks that survive a preview's changing address. (#11703)
146 - ~~Inbound payment webhooks that survive a preview's changing address.~~ Built: `/__forge/hook/<pull>/…` on production's host. (#11703)
132147 - ~~A read-only switch for the old side of a migration.~~ A procedure at the old proxy and database, section 7; the forge cannot reach the old node, so no feature is built. (#11704)
133148 - Importing an app's customer accounts (password-hash formats) without the owner reading them. (#11705)
src/apps/hook_relay.ts added +53
@@ -0,0 +1,53 @@
1 /**
2 * A stable address for inbound webhooks. A payment provider is given one URL
3 * and keeps it; a preview's address changes with every pull. Production's
4 * host answers `POST /__forge/hook/<pull>/<path>` by handing the request,
5 * untouched, to that pull's preview as `POST /<path>`. The body and headers
6 * (so the provider's own signature) arrive as sent: the app verifies the
7 * provider, the forge adds nothing it could break. Plain data in, plain data out.
8 */
9
10 /** Largest body relayed: a webhook is a small JSON document. */
11 export const HOOK_MAX_BYTES = 1024 * 1024;
12
13 export const HOOK_PREFIX = "/__forge/hook/";
14
15 /** The pull and the path the preview is asked for, or nothing when the path is no hook's. */
16 export function hookPath(pathname: string): { pull: number; path: string } | undefined {
17 if (!pathname.startsWith(HOOK_PREFIX)) return undefined;
18 const match = /^([1-9]\d{0,8})(\/[^\s]*)?$/.exec(pathname.slice(HOOK_PREFIX.length));
19 if (match === null) return undefined;
20 const path = match[2] ?? "/";
21 // No way out of the app's own paths, and none into the forge's.
22 if (path.split("/").some((part) => part === ".." || part === ".")) return undefined;
23 if (path.startsWith("//") || path.startsWith("/__forge")) return undefined;
24 return { pull: Number(match[1]), path };
25 }
26
27 /** The body's bytes, or nothing once more than `max` have been read: the count is of bytes received, whatever the headers claimed. */
28 export async function readCapped(
29 body: ReadableStream<Uint8Array> | null,
30 max: number,
31 ): Promise<Uint8Array | undefined> {
32 if (body === null) return new Uint8Array();
33 const chunks: Uint8Array[] = [];
34 let size = 0;
35 const reader = body.getReader();
36 for (;;) {
37 const { done, value } = await reader.read();
38 if (done) break;
39 size += value.byteLength;
40 if (size > max) {
41 await reader.cancel().catch(() => undefined);
42 return undefined;
43 }
44 chunks.push(value);
45 }
46 const out = new Uint8Array(size);
47 let at = 0;
48 for (const c of chunks) {
49 out.set(c, at);
50 at += c.byteLength;
51 }
52 return out;
53 }
src/apps/proxy.ts +29
@@ -67,6 +67,7 @@ import { appMeasure } from "./measure.ts";
6767 import { holdPortRequest, holdPortSocket } from "./handoff.ts";
6868 import { secureFor } from "../http/cookie_secure.ts";
6969 import { currentLoopLag, forgeWaited, LAG_WINDOW_MS } from "../perf/loop_worst.ts";
70 import { HOOK_MAX_BYTES, HOOK_PREFIX, hookPath, readCapped } from "./hook_relay.ts";
7071 import { loopWatch } from "../perf/live.ts";
7172 import { heldBy, noteRelayWait, stallTiming } from "./relay_wait.ts";
7273 import { cspNonce, htmlAnswer, readPage, rewritable, TOOLBAR_ASSET_PATH, TOOLBAR_ASSET_PREFIX, toolbarResponse, withToolbar, type PillWhere } from "./toolbar.ts";
@@ -983,6 +984,34 @@ export async function proxyApp(
983984 ): Promise<Response | undefined> {
984985 const entered = performance.now();
985986 const url = new URL(request.url);
987 // A webhook for a pull's preview, sent to production's stable address.
988 if (url.pathname.startsWith(HOOK_PREFIX) && !target.nested && target.env === "production") {
989 const hook = hookPath(url.pathname);
990 if (hook === undefined || request.method !== "POST")
991 return securityFloor(
992 hook === undefined ? notFound() : new Response(null, { status: 405, headers: { allow: "POST" } }),
993 );
994 const env = previewEnv(hook.pull);
995 if (runtime.envStatus(target.app, env) === undefined) return securityFloor(notFound());
996 // The header is an early refusal only; the cap is on the bytes read.
outdated

Cap by bytes read, not Content-Length. A missing or false header (chunked) gets past this. Count bytes while streaming request.body and answer 413 when the count passes HOOK_MAX_BYTES.

997 const length = Number(request.headers.get("content-length") ?? "0");
998 if (!(length <= HOOK_MAX_BYTES)) return securityFloor(new Response("too large", { status: 413 }));
999 const body = await readCapped(request.body, HOOK_MAX_BYTES);
1000 if (body === undefined) return securityFloor(new Response("too large", { status: 413 }));
1001 const previewHost = new URL(runtime.urlFor(target.app, env));
1002 const next = new URL(url);
1003 next.protocol = previewHost.protocol;
1004 next.host = previewHost.host;
1005 next.pathname = hook.path;
1006 const headers = new Headers(request.headers);
1007 headers.set("host", previewHost.host);
1008 return proxyApp(
1009 runtime,
1010 new Request(next.toString(), { method: "POST", headers, body, signal: request.signal }),
1011 { app: target.app, env, nested: false },
1012 server,
1013 );
1014 }
9861015 // The pill's script, from the app's own host: only the exact hashed path
9871016 // is it; any other /__forge/toolbar… path is the forge's 404, never the app's.
9881017 if (url.pathname.startsWith(TOOLBAR_ASSET_PREFIX) && !target.nested) {
test/apps-hook-relay.test.ts added +176
@@ -0,0 +1,176 @@
1 /** A webhook sent to production's stable address reaches a pull's preview. */
2
3 import { afterAll, beforeAll, expect, spyOn, test } from "bun:test";
4 import { mkdtemp, rm } from "node:fs/promises";
5 import { tmpdir } from "node:os";
6 import { join } from "node:path";
7 import { createApp, type App } from "../src/app.ts";
8 import { newApp } from "../src/actions/apps.ts";
9 import { openIssue } from "../src/actions/issues.ts";
10 import { AppsRuntime, type EnvStatus } from "../src/apps/runtime.ts";
11 import { setSealed } from "../src/core/sealed.ts";
12 import { createUser } from "../src/core/users.ts";
13 import { quietLogs } from "./helpers/globals.ts";
14 import { HOOK_MAX_BYTES, HOOK_PREFIX, hookPath } from "../src/apps/hook_relay.ts";
15 import { proxyApp, type AppTarget } from "../src/apps/proxy.ts";
16
17 test("hookPath names the pull and the path, and refuses what could escape", () => {
18 expect(hookPath(`${HOOK_PREFIX}7/stripe/webhook`)).toEqual({ pull: 7, path: "/stripe/webhook" });
19 expect(hookPath(`${HOOK_PREFIX}12`)).toEqual({ pull: 12, path: "/" });
20 for (const bad of ["0/x", "x/y", "07/x", "7/../x", "7//evil.test", "7/__forge/note", "", "7/a b"])
21 expect(hookPath(`${HOOK_PREFIX}${bad}`)).toBeUndefined();
22 expect(hookPath("/other/7/x")).toBeUndefined();
23 });
24
25 const target = { app: { id: "a", slug: "shop" }, env: "production" } as unknown as AppTarget;
26 const bare = { envStatus: () => undefined } as never;
27 const call = (method: string, path: string, init: RequestInit = {}) =>
28 proxyApp(bare, new Request(`http://shop.test${path}`, { method, ...init }), target, {} as never);
29
30 test("only POST is relayed, and only to a preview that exists", async () => {
31 expect((await call("GET", `${HOOK_PREFIX}7/x`))?.status).toBe(405);
32 expect((await call("POST", `${HOOK_PREFIX}7/x`, { body: "{}" }))?.status).toBe(404);
33 expect((await call("POST", `${HOOK_PREFIX}x`, { body: "{}" }))?.status).toBe(404);
34 });
35
36 test("a body over the limit is refused before the preview is woken", async () => {
37 const live = { envStatus: () => ({ state: "running" }), urlFor: () => "http://shop--7.test" } as never;
38 const r = await proxyApp(
39 live,
outdated

Add the success relay test (path, body and signature header reach the preview), the chunked over-limit test and the sealed-preview refusal test.

40 new Request(`http://shop.test${HOOK_PREFIX}7/x`, {
41 method: "POST",
42 headers: { "content-length": String(2 * 1024 * 1024) },
43 body: "{}",
44 }),
45 target,
46 {} as never,
47 );
48 expect(r?.status).toBe(413);
49 });
50
51 // The forwarding path, against a real runtime and a real preview process.
52 quietLogs();
53 let dataDir: string;
54 let app: App;
55 let runtime: AppsRuntime;
56 let backend: ReturnType<typeof Bun.serve>;
57 let seen: { method: string; path: string; body: string; sig: string | null; host: string | null }[] = [];
58 let status: ReturnType<typeof spyOn>;
59 let live: AppTarget;
60 let openNumber: number;
61 let sealedNumber: number;
62
63 beforeAll(async () => {
64 backend = Bun.serve({
65 port: 0,
66 fetch: async (req) => {
67 seen.push({
68 method: req.method,
69 path: new URL(req.url).pathname,
70 body: await req.text(),
71 sig: req.headers.get("stripe-signature"),
72 host: req.headers.get("host"),
73 });
74 return new Response("received", { status: 202 });
75 },
76 });
77 dataDir = await mkdtemp(join(tmpdir(), "forge-hook-relay-"));
78 app = createApp({ dataDir, port: 0, host: "forge.test" });
79 runtime = new AppsRuntime(app, {
80 domain: "apps.test",
81 scheme: "http",
82 publicPort: 0,
83 forgeUrl: "http://forge.test",
84 concurrency: 1,
85 });
86 await runtime.start();
87 status = spyOn(AppsRuntime.prototype, "envStatus").mockImplementation(
88 (_a, env) =>
89 ({ env, state: "running", port: backend.port, commitOid: "a".repeat(40) }) as EnvStatus,
90 );
91 const u = await createUser(app.db, {
92 origin: "seed",
93 trusted: true,
94 username: "alice",
95 email: "[email protected]",
96 password: "a-good-passphrase",
97 adminIfFirst: true,
98 });
99 if (!u.ok) throw new Error(u.error);
100 const made = await newApp(app, u.value, { name: "shop", description: "A shop", visibility: "public" });
101 if (!made.ok) throw new Error(String(made.error));
102 live = { app: made.value.app, env: "production" } as AppTarget;
103 const open = openIssue(app, u.value, made.value.repo, { title: "open pull" });
104 const sealed = openIssue(app, u.value, made.value.repo, { title: "sealed pull" });
105 if (!open.ok || !sealed.ok) throw new Error("issues");
106 openNumber = open.value.number;
107 sealedNumber = sealed.value.number;
108 setSealed(app.db, sealed.value.id, true);
109 }, 120_000);
110
111 afterAll(async () => {
112 status.mockRestore();
113 await runtime.stop();
114 backend.stop(true);
115 app.close();
116 await rm(dataDir, { recursive: true, force: true });
117 });
118
119 const relay = (n: number, init: RequestInit & { duplex?: string }, t: AppTarget = live) =>
120 proxyApp(
121 runtime,
122 new Request(`http://shop.apps.test${HOOK_PREFIX}${n}/stripe/webhook`, { method: "POST", ...init }),
123 t,
124 {} as never,
125 );
126
127 test("the preview sees the same path, body, signature and its own host", async () => {
128 seen = [];
129 const body = '{"id":"evt_1","type":"payment_intent.succeeded"}';
130 const r = await relay(openNumber, { body, headers: { "stripe-signature": "t=1,v1=abc", host: "shop.apps.test" } });
131 expect(r?.status).toBe(202);
132 expect(seen).toHaveLength(1);
133 expect(seen[0]!.method).toBe("POST");
134 expect(seen[0]!.path).toBe("/stripe/webhook");
135 expect(seen[0]!.body).toBe(body);
136 expect(seen[0]!.sig).toBe("t=1,v1=abc");
137 expect(seen[0]!.host).toBe(new URL(runtime.urlFor(live.app, `pull-${openNumber}`)).host);
138 });
139
140 test("a chunked body over the limit is refused by the bytes read, with no content-length", async () => {
141 seen = [];
142 const chunk = new Uint8Array(256 * 1024);
143 let sent = 0;
144 const stream = new ReadableStream<Uint8Array>({
145 pull(c) {
146 if (sent > HOOK_MAX_BYTES + chunk.length) return c.close();
147 sent += chunk.length;
148 c.enqueue(chunk);
149 },
150 });
151 const r = await relay(openNumber, { body: stream, duplex: "half" });
152 expect(r?.status).toBe(413);
153 expect(seen).toHaveLength(0);
154 // A lying header is no better: a small claim over a large body.
155 const lie = await relay(openNumber, {
156 body: new Uint8Array(HOOK_MAX_BYTES + 1),
157 headers: { "content-length": "10" },
158 });
159 expect(lie?.status).toBe(413);
160 expect(seen).toHaveLength(0);
161 });
162
163 test("a sealed preview is refused although production is public", async () => {
164 seen = [];
165 const r = await relay(sealedNumber, { body: "{}" });
166 expect(r?.status).toBe(404);
167 expect(seen).toHaveLength(0);
168 });
169
170 test("a private app's preview is refused to a caller with no grant", async () => {
171 seen = [];
172 const priv = { app: { ...live.app, access: "private" }, env: "production" } as AppTarget;
173 const r = await relay(openNumber, { body: "{}" }, priv);
174 expect(r?.status).toBe(404);
175 expect(seen).toHaveLength(0);
176 });

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs