Public: anyone can read this; the crew can change it.
An app's fetch to an undeclared host fails with the forge's own words, and a test shows the text names the host and forge.json egress #13552
Check · Review · Land · Live done — Landed and live in production.
scout/forge:scout/13537-an-app-s-fetch-to-an-undeclared-host-fai → main 3 commits fork 3 threads · 3 unresolved
8 files
changed +130
−18
docs/features/app-egress-refused.md +2 −2
| @@ -2,8 +2,8 @@ | ||
| 2 | 2 | |
| 3 | 3 | The forge's gate refuses every outside host that a repository's `forge.json` `egress` does not declare. The refusal says whose no it was, so an app never blames the service it was calling. |
| 4 | 4 | |
| 5 | - **The words.** The status line and body are `403 refused by this forge: <host>:<port> is not in this app's forge.json egress`, with `X-Forge-Egress: refused` (`REFUSED_PREFIX`, `src/apps/egress.ts`). The template's `src/egress.ts` reads the same words. | |
| 6 | - **The log.** The first refusal of a host and port in an hour is one line in the forge's log, naming the app (`cellHooks`, `src/apps/egress_refused.ts`). The line is printable ASCII, at most 200 bytes (`logSafe`), because the host is the app's own choice. The memory behind "once an hour" holds at most 2,000 keys (`SAID_MAX`); the oldest goes first. | |
| 5 | - **The words.** The status line and body are `403 refused by this forge: <host>:<port> is not in this app's forge.json egress`, with `X-Forge-Egress: refused` (`refusedWords`, `src/apps/egress.ts`, the one source: #13537). The log line and the owner's Blocked row, on the page and in the terminal, print the same sentence, and `test/egress-words.test.ts` pins it literally. The template's `src/egress.ts` reads the same words. | |
| 6 | - **The log.** The first refusal of a host and port in an hour is one line in the forge's log, `[egress] <app>: refused by this forge: <host>:<port> is not in this app's forge.json egress` (a private destination: `[egress] <app> was refused <host>:<port>: a private destination`) (`cellHooks`, `src/apps/egress_refused.ts`). The line is printable ASCII, at most 200 bytes (`logSafe`), because the host is the app's own choice. The memory behind "once an hour" holds at most 2,000 keys (`SAID_MAX`); the oldest goes first. | |
| 7 | 7 | - **Needs you.** An admin of the repository sees one line per refused host: "game tried to reach huggingface.co and was refused: Allow it (3 tries, last just now)", linking to the app's settings (`needsYouLine`, one wording for `blockedLines`). The page's needs-you stack and the terminal's both read `blockedLines` through `src/crew/notify.ts`; Allow on the settings page lands the host live and in `forge.json`. |
| 8 | 8 | - **The crew.** The brief tells an agent to add a host its change calls to `forge.json` `egress` in the same change. |
| 9 | 9 | - **The check.** A new app's `forge.json` carries `check.run: bun scripts/egress-check.ts && bun test`, which fails when a literal host in the code is not declared. |
docs/features/blocked-hosts.md +1 −1
| @@ -3,7 +3,7 @@ | ||
| 3 | 3 | When a production cell is refused a host, the forge says so: `apps/egress_refused.ts` counts one row per host and port (first and last time, a count) in memory and writes `<data>/egress/<repo>.refused.json` at most every few seconds. The first refusal of a host appends `app.egress.refused`, which wakes the notifier. Preview cells (any pull's, a fork's too) neither report here nor read the owner's yes. |
| 4 | 4 | |
| 5 | 5 | - **Validated:** the host comes from app code, so only a hostname or IP of at most 253 characters is kept; the rest is dropped. At 50 hosts new ones are dropped, never old ones. |
| 6 | - **Where it shows:** the app's settings page (`Blocked: huggingface.co (14 tries, last 2 min ago)` with **Allow** and **Ignore**), the terminal (`tui/screens/blocked.ts`, the palette while the repository is open: `a`, `i`), the owner's notifications (`blockedLines`) and the crew's brief (`ownAppLines`). All are for the repository's admins; a write from one is an event, so the others update live. | |
| 6 | - **Where it shows:** the app's settings page (`refused by this forge: huggingface.co:443 is not in this app's forge.json egress (14 tries, last 2 min ago)` with **Allow** and **Ignore**: the gate's own sentence, said once, from `blockedRowLine`; a private destination reads `Blocked: 10.0.0.5:5432 (…)`, since the gate says other words to it, #13537), the terminal, the same line (`blockedListRows`) (`tui/screens/blocked.ts`, the palette while the repository is open: `a`, `i`), the owner's notifications (`blockedLines`) and the crew's brief (`ownAppLines`). All are for the repository's admins; a write from one is an event, so the others update live. | |
| 7 | 7 | - **Allow:** adds the entry to `<data>/egress/<repo>.allowed.json`, which the production cell's proxy asks on each connection (cached 2 s), so it passes within seconds with no redeploy, and commits it to `forge.json` `egress` on the default branch as the owner. An agent admin cannot allow (`needs_human`): an opening no one reads in the repository is refused, and the agent proposes the `forge.json` change in a pull. An entry allowed live that `forge.json` does not declare (the commit failed, or the owner later removed it) is listed on the page as "Allowed now, not in forge.json", so the repository's silence is never hidden. |
| 8 | 8 | - **Ignore:** keeps the row and its count, stops saying it. |
| 9 | 9 | - **No wildcards** are offered; an owner who wants `*.hf.co` writes it in `forge.json`. |
src/actions/egress_refused.ts +10 −4
| @@ -20,7 +20,7 @@ import { | ||
| 20 | 20 | blockedHosts, |
| 21 | 21 | entryFor, |
| 22 | 22 | needsYouLine, |
| 23 | refusalLine, | |
| 23 | blockedRowLine, | |
| 24 | 24 | ignoreRefused, |
| 25 | 25 | type Refusal, |
| 26 | 26 | } from "../apps/egress_refused.ts"; |
| @@ -37,7 +37,10 @@ import { editFiles } from "./contents.ts"; | ||
| 37 | 37 | export interface BlockedRow { |
| 38 | 38 | readonly host: string; |
| 39 | 39 | readonly port: number; |
| 40 | /** "Blocked: huggingface.co (14 tries, last 2 min ago)" */ | |
| 40 | /** | |
| 41 | * "refused by this forge: huggingface.co:443 is not in this app's forge.json egress (14 tries, | |
| 42 | * last 2 min ago)"; a private one "Blocked: 10.0.0.5:5432 (…)". | |
| 43 | */ | |
| 41 | 44 | readonly line: string; |
| 42 | 45 | /** `host:port`, what the forms post. */ |
| 43 | 46 | readonly destination: string; |
| @@ -64,7 +67,7 @@ const siteAdmin = (actor: User): boolean => actor.isAdmin && actor.kind !== "age | ||
| 64 | 67 | const rowOf = (r: Refusal, now: number): BlockedRow => ({ |
| 65 | 68 | host: r.host, |
| 66 | 69 | port: r.port, |
| 67 | line: refusalLine(r, now), | |
| 70 | line: blockedRowLine(r, now), | |
| 68 | 71 | destination: `${r.host}:${r.port}`, |
| 69 | 72 | private: r.private, |
| 70 | 73 | }); |
| @@ -104,7 +107,10 @@ async function declaredEgress(app: App, repo: Repo): Promise<unknown[] | undefin | ||
| 104 | 107 | return (current ?? []) as unknown[]; |
| 105 | 108 | } |
| 106 | 109 | |
| 107 | /** `view` with the entries allowed live that forge.json does not declare (shown so none stays hidden). */ | |
| 110 | /** | |
| 111 | * `view` with the entries allowed live that forge.json does not declare (shown so none stays | |
| 112 | * hidden). | |
| 113 | */ | |
| 108 | 114 | export async function withLiveOnly(app: App, repo: Repo, view: BlockedView): Promise<BlockedView> { |
| 109 | 115 | const declared = await declaredEgress(app, repo); |
| 110 | 116 | const live = allowedHosts(app.config.dataDir, repo.id); |
src/apps/egress.ts +9 −1
| @@ -477,6 +477,14 @@ export const UNRESOLVED_WORDS = "does not resolve"; | ||
| 477 | 477 | */ |
| 478 | 478 | export const REFUSED_PREFIX = "refused by this forge:"; |
| 479 | 479 | |
| 480 | /** | |
| 481 | * The forge's own no, in the one wording every door says it (#13537): the | |
| 482 | * gate's 403, the parent's log and the owner's Blocked row. Only the host and | |
| 483 | * port are named, never anything else of the request. | |
| 484 | */ | |
| 485 | export const refusedWords = (host: string, port: number): string => | |
| 486 | `${REFUSED_PREFIX} ${host}:${port} is not in this app's forge.json egress`; | |
| 487 | ||
| 480 | 488 | export type Verdict = |
| 481 | 489 | | { ok: true; address: string; undeclared: boolean } |
| 482 | 490 | | { ok: false; why: string; policy?: { private: boolean } }; |
| @@ -509,7 +517,7 @@ export async function decide( | ||
| 509 | 517 | if (!declared && (policy.report === undefined || payment || !WEB_PORTS.includes(port))) |
| 510 | 518 | return { |
| 511 | 519 | ok: false, |
| 512 | why: `${REFUSED_PREFIX} ${name}:${port} is not in this app's forge.json egress`, | |
| 520 | why: refusedWords(name, port), | |
| 513 | 521 | policy: { private: false }, |
| 514 | 522 | }; |
| 515 | 523 | // Inside a cell, the cell's proxy resolves and checks the name itself. |
src/apps/egress_refused.ts +21 −4
| @@ -15,7 +15,7 @@ import { during } from "../perf/jobs.ts"; | ||
| 15 | 15 | import { mkdirSync, readFileSync, renameSync, writeFileSync } from "node:fs"; |
| 16 | 16 | import { isIP } from "node:net"; |
| 17 | 17 | import { join } from "node:path"; |
| 18 | import { hostAllowed } from "./egress.ts"; | |
| 18 | import { hostAllowed, refusedWords } from "./egress.ts"; | |
| 19 | 19 | import { defect } from "../kernel/error.ts"; |
| 20 | 20 | import { sinceWords } from "../kernel/vitals.ts"; |
| 21 | 21 | |
| @@ -31,7 +31,10 @@ export interface Refusal { | ||
| 31 | 31 | readonly private: boolean; |
| 32 | 32 | } |
| 33 | 33 | |
| 34 | /** At most this many hosts are kept per repository; once full, new hosts are dropped, never old ones. */ | |
| 34 | /** | |
| 35 | * At most this many hosts are kept per repository; once full, new hosts are dropped, never old | |
| 36 | * ones. | |
| 37 | */ | |
| 35 | 38 | export const ROWS_MAX = 50; |
| 36 | 39 | /** An allowed list read is reused this long ("within seconds"). */ |
| 37 | 40 | const ALLOWED_TTL_MS = 2_000; |
| @@ -247,7 +250,9 @@ export function cellHooks( | ||
| 247 | 250 | onRefused: (host, port, isPrivate) => { |
| 248 | 251 | if (noteRefused(dataDir, repoId, host, port, isPrivate)) first(host, port); |
| 249 | 252 | if (say !== undefined && saidNow(`${repoId} ${host}:${port}`, Date.now())) |
| 250 | say(logSafe(`[egress] ${name} was refused ${entryFor(host, port)}: not in its forge.json egress`)); | |
| 253 | say(logSafe(isPrivate | |
| 254 | ? `[egress] ${name} was refused ${entryFor(host, port)}: a private destination` | |
| 255 | : `[egress] ${name}: ${refusedWords(host, port)}`)); | |
| 251 | 256 | }, |
| 252 | 257 | }; |
| 253 | 258 | } |
| @@ -263,8 +268,20 @@ export function needsYouLine(appName: string, r: Refusal, now = Date.now()): str | ||
| 263 | 268 | return `${appName} tried to reach ${entryFor(r.host, r.port)} and was refused: Allow it (${r.count} ${r.count === 1 ? "try" : "tries"}, last ${sinceWords(r.last, now)})`; |
| 264 | 269 | } |
| 265 | 270 | |
| 271 | const triesWords = (r: Refusal, now: number): string => | |
| 272 | `(${r.count} ${r.count === 1 ? "try" : "tries"}, last ${sinceWords(r.last, now)})`; | |
| 273 | ||
| 266 | 274 | /** "Blocked: huggingface.co (14 tries, last 2 min ago)" */ |
| 267 | 275 | export function refusalLine(r: Refusal, now = Date.now()): string { |
| 268 | 276 | const name = r.port === 80 || r.port === 443 ? r.host : `${r.host}:${r.port}`; |
| 269 | return `Blocked: ${name} (${r.count} ${r.count === 1 ? "try" : "tries"}, last ${sinceWords(r.last, now)})`; | |
| 277 | return `Blocked: ${name} ${triesWords(r, now)}`; | |
| 278 | } | |
| 279 | ||
| 280 | /** | |
| 281 | * The owner's Blocked row, on the page and in the terminal (#13537): an | |
| 282 | * undeclared host is said once, in the gate's own words, then the tries; a | |
| 283 | * private destination keeps `refusalLine`, since the gate says other words. | |
| 284 | */ | |
| 285 | export function blockedRowLine(r: Refusal, now = Date.now()): string { | |
Non-blocking: two builders ( |
||
| 286 | return r.private ? refusalLine(r, now) : `${refusedWords(r.host, r.port)} ${triesWords(r, now)}`; | |
| 270 | 287 | } |
src/tui/screens/blocked.ts +10 −3
| @@ -7,7 +7,7 @@ | ||
| 7 | 7 | * updates, and this screen follows a write made there. |
| 8 | 8 | */ |
| 9 | 9 | |
| 10 | import { allowBlocked, blockedView, ignoreBlocked, type BlockedError } from "../../actions/egress_refused.ts"; | |
| 10 | import { allowBlocked, blockedView, ignoreBlocked, type BlockedError, type BlockedRow } from "../../actions/egress_refused.ts"; | |
| 11 | 11 | import { getAppByRepo } from "../../core/apps.ts"; |
| 12 | 12 | import { canAdmin, type Repo } from "../../core/repos.ts"; |
| 13 | 13 | import type { Ctx, Screen } from "../app/app.ts"; |
| @@ -30,6 +30,14 @@ const WORDS: Record<BlockedError, string> = { | ||
| 30 | 30 | const mayOpen = (f: Session, repo: Repo | undefined): repo is Repo => |
| 31 | 31 | repo !== undefined && f.user !== undefined && canAdmin(f.db, repo, f.user) && getAppByRepo(f.db, repo.id) !== undefined; |
| 32 | 32 | |
| 33 | /** | |
| 34 | * The rows as the terminal lists them: the page's line, so both doors say the gate's words | |
| 35 | * (#13537). | |
| 36 | */ | |
| 37 | export function blockedListRows(rows: readonly BlockedRow[]): ListRow[] { | |
| 38 | return rows.map((r) => ({ title: r.line, ...(r.private ? { meta: "private: a site admin allows it" } : {}) })); | |
| 39 | } | |
| 40 | ||
| 33 | 41 | export function blockedScreen(f: Session, ctx: Ctx, repo: Repo): Screen { |
| 34 | 42 | const state = new ListState(); |
| 35 | 43 | const rows = () => { |
| @@ -53,8 +61,7 @@ export function blockedScreen(f: Session, ctx: Ctx, repo: Repo): Screen { | ||
| 53 | 61 | style(c.mutedFg, c.bg), |
| 54 | 62 | head!.x + head!.w, |
| 55 | 63 | ); |
| 56 | const lines: ListRow[] = all.map((r) => ({ title: r.line, ...(r.private ? { meta: "private: a site admin allows it" } : {}) })); | |
| 57 | drawList(s, list!, c, state, lines); | |
| 64 | drawList(s, list!, c, state, blockedListRows(all)); | |
| 58 | 65 | }, |
| 59 | 66 | key(e) { |
| 60 | 67 | const all = rows(); |
test/egress-refused.test.ts +3 −3
| @@ -139,7 +139,7 @@ describe("refused hosts", () => { | ||
| 139 | 139 | test("the app's page says it, with an Allow beside it", async () => { |
| 140 | 140 | const cookie = await signIn("cy"); |
| 141 | 141 | const page = await (await fetch(`${servers.url}/cy/game/settings/app`, { headers: { cookie } })).text(); |
| 142 | expect(page.replace(/\s+/g, " ")).toMatch(/Blocked: huggingface\.co \(3 tries, last just now\)/); | |
| 142 | expect(page.replace(/\s+/g, " ")).toMatch(/refused by this forge: huggingface\.co:443 is not in this app's forge\.json egress \(3 tries, last just now\)/); | |
| 143 | 143 | expect(page).toContain("/cy/game/settings/app/blocked/huggingface.co%3A443/allow"); |
| 144 | 144 | }); |
| 145 | 145 | |
| @@ -309,7 +309,7 @@ describe("an app told in the forge's own words (#12182)", () => { | ||
| 309 | 309 | const lines: string[] = []; |
| 310 | 310 | const hooks = cellHooks(dataDir, "walk1", "production", () => {}, (l) => lines.push(l)); |
| 311 | 311 | for (let i = 0; i < 3; i++) hooks.onRefused?.("beta-bridge.simplefin.org", 443, false); |
| 312 | expect(lines).toEqual(["[egress] walk1 was refused beta-bridge.simplefin.org: not in its forge.json egress"]); | |
| 312 | expect(lines).toEqual(["[egress] walk1: refused by this forge: beta-bridge.simplefin.org:443 is not in this app's forge.json egress"]); | |
| 313 | 313 | const policy = { hosts: [] as string[], origins: [], allowed: () => [] as string[] }; |
| 314 | 314 | const probe = Bun.listen({ hostname: "127.0.0.1", port: 0, socket: { data() {} } }); |
| 315 | 315 | const port = probe.port; |
| @@ -402,7 +402,7 @@ describe("the needs-you line at every door, and the log and status line it rides | ||
| 402 | 402 | const hooks = cellHooks(dataDir, "slug1", "production", () => {}, (l) => lines.push(l), slug); |
| 403 | 403 | hooks.onRefused?.(`${"h".repeat(120)}.example.org`, 443, false); |
| 404 | 404 | expect(lines).toHaveLength(1); |
| 405 | expect(lines[0]).toStartWith(`[egress] ${slug} was refused hhh`); | |
| 405 | expect(lines[0]).toStartWith(`[egress] ${slug}: refused by this forge: hhh`); | |
| 406 | 406 | expect(lines[0]).not.toContain("slug1"); |
| 407 | 407 | expect(lines[0]!.length).toBeLessThanOrEqual(200); |
| 408 | 408 | expect(lines[0]).not.toContain("\n"); |
test/egress-words.test.ts added +74
| @@ -0,0 +1,74 @@ | ||
| 1 | /** | |
| 2 | * An app's fetch to an undeclared host is told the forge's own no, and every | |
| 3 | * door that shows the refusal says the same sentence (#13537): the gate's | |
| 4 | * answer, the parent's log line, and the owner's Blocked row on the page and | |
| 5 | * in the terminal. The sentence is | |
| 6 | * spelled out literally here so a change to it, or a dropped host, fails. | |
| 7 | */ | |
| 8 | ||
| 9 | import { afterAll, describe, expect, test } from "bun:test"; | |
| 10 | import { mkdtempSync, rmSync } from "node:fs"; | |
| 11 | import { tmpdir } from "node:os"; | |
| 12 | import { join } from "node:path"; | |
| 13 | import { decide, refusedWords } from "../src/apps/egress.ts"; | |
| 14 | import { blockedRowLine, cellHooks, type Refusal } from "../src/apps/egress_refused.ts"; | |
| 15 | import { blockedListRows } from "../src/tui/screens/blocked.ts"; | |
| 16 | import { blockedNote } from "../src/web/studio/blocked.ts"; | |
| 17 | import type { BlockedRow, BlockedView } from "../src/actions/egress_refused.ts"; | |
| 18 | import { html } from "../src/web/html.ts"; | |
| 19 | ||
| 20 | const SENTENCE = "refused by this forge: api.undeclared.example:443 is not in this app's forge.json egress"; | |
| 21 | const dataDir = mkdtempSync(join(tmpdir(), "egress-words-")); | |
| 22 | afterAll(() => rmSync(dataDir, { recursive: true, force: true })); | |
| 23 | ||
| 24 | describe("the forge's own words for an undeclared host (#13537)", () => { | |
| 25 | test("the words name the host and forge.json egress", () => { | |
| 26 | expect(refusedWords("api.undeclared.example", 443)).toBe(SENTENCE); | |
| 27 | }); | |
| 28 | ||
| 29 | test("the gate refuses a fetch to it with that sentence", async () => { | |
| 30 | const verdict = await decide({ hosts: ["declared.example"], origins: [] }, "api.undeclared.example", 443); | |
| 31 | expect(verdict.ok).toBe(false); | |
| 32 | expect(verdict.ok ? "" : verdict.why).toBe(SENTENCE); | |
| 33 | }); | |
| 34 | ||
| 35 | test("the log line prints the same sentence", () => { | |
| 36 | const lines: string[] = []; | |
| 37 | const hooks = cellHooks(dataDir, "words1", "production", () => {}, (l) => lines.push(l)); | |
| 38 | hooks.onRefused?.("api.undeclared.example", 443, false); | |
| 39 | expect(lines).toEqual([`[egress] words1: ${SENTENCE}`]); | |
| 40 | }); | |
| 41 | ||
| 42 | test("the private-destination log line keeps its own words, which the gate does not say", () => { | |
| 43 | const lines: string[] = []; | |
| 44 | const hooks = cellHooks(dataDir, "words2", "production", () => {}, (l) => lines.push(l)); | |
| 45 | hooks.onRefused?.("10.0.0.5", 5432, true); | |
| 46 | expect(lines).toEqual(["[egress] words2 was refused 10.0.0.5:5432: a private destination"]); | |
| 47 | }); | |
| 48 | ||
| 49 | const refusal: Refusal = { host: "api.undeclared.example", port: 443, count: 2, first: new Date().toISOString(), last: new Date().toISOString(), denied: false, private: false }; | |
| 50 | const row: BlockedRow = { | |
| 51 | host: refusal.host, | |
| 52 | port: refusal.port, | |
| 53 | line: blockedRowLine(refusal), | |
| 54 | destination: "api.undeclared.example:443", | |
| 55 | private: false, | |
| 56 | }; | |
| 57 | ||
| 58 | test("the Blocked row says the sentence once, then the tries", () => { | |
| 59 | expect(row.line).toBe(`${SENTENCE} (2 tries, last just now)`); | |
| 60 | expect(blockedRowLine({ ...refusal, host: "10.0.0.5", port: 5432, private: true })).toStartWith("Blocked: 10.0.0.5:5432 ("); | |
| 61 | }); | |
| 62 | ||
| 63 | test("the owner's page prints the same sentence once, escaped", () => { | |
| 64 | const view: BlockedView = { rows: [row], liveOnly: [], canAllowPrivate: false }; | |
| 65 | const page = String(blockedNote("/o/r", view, () => html``)); | |
| 66 | const escaped = SENTENCE.replace("'", "'"); | |
| 67 | expect(page).toContain(escaped); | |
| 68 | expect(page.split(escaped)).toHaveLength(2); | |
| 69 | }); | |
| 70 | ||
| 71 | test("the terminal's Blocked screen prints the same sentence", () => { | |
| 72 | expect(blockedListRows([row]).map((r) => r.title)).toEqual([`${SENTENCE} (2 tries, last just now)`]); | |
| 73 | }); | |
| 74 | }); |
new above
offline
Fine as the single source. Note the private-destination log branch in egress_refused.ts still uses
entryForwording, which the gate does not say; that is deliberate, but add a test line pinning it so it does not drift.