Public: anyone can read this; the crew can change it.

An app's fetch to an undeclared host fails with the forge's own words, and a test shows the text names the host and forge.json egress #13552

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

scout/forge:scout/13537-an-app-s-fetch-to-an-undeclared-host-fai → main 3 commits fork 3 threads · 3 unresolved

  • Merged. scout/13537-an-app-s-fetch-to-an-undeclared-host-fai was merged into main as 13d731e.

Review

Sign in to review this change.
8 files changed +130 −18
docs/features/app-egress-refused.md +2 −2
@@ -2,8 +2,8 @@
22
33 The forge's gate refuses every outside host that a repository's `forge.json` `egress` does not declare. The refusal says whose no it was, so an app never blames the service it was calling.
44
5 - **The words.** The status line and body are `403 refused by this forge: <host>:<port> is not in this app's forge.json egress`, with `X-Forge-Egress: refused` (`REFUSED_PREFIX`, `src/apps/egress.ts`). The template's `src/egress.ts` reads the same words.
6 - **The log.** The first refusal of a host and port in an hour is one line in the forge's log, naming the app (`cellHooks`, `src/apps/egress_refused.ts`). The line is printable ASCII, at most 200 bytes (`logSafe`), because the host is the app's own choice. The memory behind "once an hour" holds at most 2,000 keys (`SAID_MAX`); the oldest goes first.
5 - **The words.** The status line and body are `403 refused by this forge: <host>:<port> is not in this app's forge.json egress`, with `X-Forge-Egress: refused` (`refusedWords`, `src/apps/egress.ts`, the one source: #13537). The log line and the owner's Blocked row, on the page and in the terminal, print the same sentence, and `test/egress-words.test.ts` pins it literally. The template's `src/egress.ts` reads the same words.
6 - **The log.** The first refusal of a host and port in an hour is one line in the forge's log, `[egress] <app>: refused by this forge: <host>:<port> is not in this app's forge.json egress` (a private destination: `[egress] <app> was refused <host>:<port>: a private destination`) (`cellHooks`, `src/apps/egress_refused.ts`). The line is printable ASCII, at most 200 bytes (`logSafe`), because the host is the app's own choice. The memory behind "once an hour" holds at most 2,000 keys (`SAID_MAX`); the oldest goes first.
77 - **Needs you.** An admin of the repository sees one line per refused host: "game tried to reach huggingface.co and was refused: Allow it (3 tries, last just now)", linking to the app's settings (`needsYouLine`, one wording for `blockedLines`). The page's needs-you stack and the terminal's both read `blockedLines` through `src/crew/notify.ts`; Allow on the settings page lands the host live and in `forge.json`.
88 - **The crew.** The brief tells an agent to add a host its change calls to `forge.json` `egress` in the same change.
99 - **The check.** A new app's `forge.json` carries `check.run: bun scripts/egress-check.ts && bun test`, which fails when a literal host in the code is not declared.
docs/features/blocked-hosts.md +1 −1
@@ -3,7 +3,7 @@
33 When a production cell is refused a host, the forge says so: `apps/egress_refused.ts` counts one row per host and port (first and last time, a count) in memory and writes `<data>/egress/<repo>.refused.json` at most every few seconds. The first refusal of a host appends `app.egress.refused`, which wakes the notifier. Preview cells (any pull's, a fork's too) neither report here nor read the owner's yes.
44
55 - **Validated:** the host comes from app code, so only a hostname or IP of at most 253 characters is kept; the rest is dropped. At 50 hosts new ones are dropped, never old ones.
6 - **Where it shows:** the app's settings page (`Blocked: huggingface.co (14 tries, last 2 min ago)` with **Allow** and **Ignore**), the terminal (`tui/screens/blocked.ts`, the palette while the repository is open: `a`, `i`), the owner's notifications (`blockedLines`) and the crew's brief (`ownAppLines`). All are for the repository's admins; a write from one is an event, so the others update live.
6 - **Where it shows:** the app's settings page (`refused by this forge: huggingface.co:443 is not in this app's forge.json egress (14 tries, last 2 min ago)` with **Allow** and **Ignore**: the gate's own sentence, said once, from `blockedRowLine`; a private destination reads `Blocked: 10.0.0.5:5432 (…)`, since the gate says other words to it, #13537), the terminal, the same line (`blockedListRows`) (`tui/screens/blocked.ts`, the palette while the repository is open: `a`, `i`), the owner's notifications (`blockedLines`) and the crew's brief (`ownAppLines`). All are for the repository's admins; a write from one is an event, so the others update live.
77 - **Allow:** adds the entry to `<data>/egress/<repo>.allowed.json`, which the production cell's proxy asks on each connection (cached 2 s), so it passes within seconds with no redeploy, and commits it to `forge.json` `egress` on the default branch as the owner. An agent admin cannot allow (`needs_human`): an opening no one reads in the repository is refused, and the agent proposes the `forge.json` change in a pull. An entry allowed live that `forge.json` does not declare (the commit failed, or the owner later removed it) is listed on the page as "Allowed now, not in forge.json", so the repository's silence is never hidden.
88 - **Ignore:** keeps the row and its count, stops saying it.
99 - **No wildcards** are offered; an owner who wants `*.hf.co` writes it in `forge.json`.
src/actions/egress_refused.ts +10 −4
@@ -20,7 +20,7 @@ import {
2020 blockedHosts,
2121 entryFor,
2222 needsYouLine,
23 refusalLine,
23 blockedRowLine,
2424 ignoreRefused,
2525 type Refusal,
2626 } from "../apps/egress_refused.ts";
@@ -37,7 +37,10 @@ import { editFiles } from "./contents.ts";
3737 export interface BlockedRow {
3838 readonly host: string;
3939 readonly port: number;
40 /** "Blocked: huggingface.co (14 tries, last 2 min ago)" */
40 /**
41 * "refused by this forge: huggingface.co:443 is not in this app's forge.json egress (14 tries,
42 * last 2 min ago)"; a private one "Blocked: 10.0.0.5:5432 (…)".
43 */
4144 readonly line: string;
4245 /** `host:port`, what the forms post. */
4346 readonly destination: string;
@@ -64,7 +67,7 @@ const siteAdmin = (actor: User): boolean => actor.isAdmin && actor.kind !== "age
6467 const rowOf = (r: Refusal, now: number): BlockedRow => ({
6568 host: r.host,
6669 port: r.port,
67 line: refusalLine(r, now),
70 line: blockedRowLine(r, now),
6871 destination: `${r.host}:${r.port}`,
6972 private: r.private,
7073 });
@@ -104,7 +107,10 @@ async function declaredEgress(app: App, repo: Repo): Promise<unknown[] | undefin
104107 return (current ?? []) as unknown[];
105108 }
106109
107 /** `view` with the entries allowed live that forge.json does not declare (shown so none stays hidden). */
110 /**
111 * `view` with the entries allowed live that forge.json does not declare (shown so none stays
112 * hidden).
113 */
108114 export async function withLiveOnly(app: App, repo: Repo, view: BlockedView): Promise<BlockedView> {
109115 const declared = await declaredEgress(app, repo);
110116 const live = allowedHosts(app.config.dataDir, repo.id);
src/apps/egress.ts +9 −1
@@ -477,6 +477,14 @@ export const UNRESOLVED_WORDS = "does not resolve";
477477 */
478478 export const REFUSED_PREFIX = "refused by this forge:";
479479
480 /**
481 * The forge's own no, in the one wording every door says it (#13537): the
482 * gate's 403, the parent's log and the owner's Blocked row. Only the host and
483 * port are named, never anything else of the request.
484 */

Fine as the single source. Note the private-destination log branch in egress_refused.ts still uses entryFor wording, which the gate does not say; that is deliberate, but add a test line pinning it so it does not drift.

485 export const refusedWords = (host: string, port: number): string =>
486 `${REFUSED_PREFIX} ${host}:${port} is not in this app's forge.json egress`;
487
480488 export type Verdict =
481489 | { ok: true; address: string; undeclared: boolean }
482490 | { ok: false; why: string; policy?: { private: boolean } };
@@ -509,7 +517,7 @@ export async function decide(
509517 if (!declared && (policy.report === undefined || payment || !WEB_PORTS.includes(port)))
510518 return {
511519 ok: false,
512 why: `${REFUSED_PREFIX} ${name}:${port} is not in this app's forge.json egress`,
520 why: refusedWords(name, port),
513521 policy: { private: false },
514522 };
515523 // Inside a cell, the cell's proxy resolves and checks the name itself.
src/apps/egress_refused.ts +21 −4
@@ -15,7 +15,7 @@ import { during } from "../perf/jobs.ts";
1515 import { mkdirSync, readFileSync, renameSync, writeFileSync } from "node:fs";
1616 import { isIP } from "node:net";
1717 import { join } from "node:path";
18 import { hostAllowed } from "./egress.ts";
18 import { hostAllowed, refusedWords } from "./egress.ts";
1919 import { defect } from "../kernel/error.ts";
2020 import { sinceWords } from "../kernel/vitals.ts";
2121
@@ -31,7 +31,10 @@ export interface Refusal {
3131 readonly private: boolean;
3232 }
3333
34 /** At most this many hosts are kept per repository; once full, new hosts are dropped, never old ones. */
34 /**
35 * At most this many hosts are kept per repository; once full, new hosts are dropped, never old
36 * ones.
37 */
3538 export const ROWS_MAX = 50;
3639 /** An allowed list read is reused this long ("within seconds"). */
3740 const ALLOWED_TTL_MS = 2_000;
@@ -247,7 +250,9 @@ export function cellHooks(
247250 onRefused: (host, port, isPrivate) => {
248251 if (noteRefused(dataDir, repoId, host, port, isPrivate)) first(host, port);
249252 if (say !== undefined && saidNow(`${repoId} ${host}:${port}`, Date.now()))
250 say(logSafe(`[egress] ${name} was refused ${entryFor(host, port)}: not in its forge.json egress`));
253 say(logSafe(isPrivate
254 ? `[egress] ${name} was refused ${entryFor(host, port)}: a private destination`
255 : `[egress] ${name}: ${refusedWords(host, port)}`));
251256 },
252257 };
253258 }
@@ -263,8 +268,20 @@ export function needsYouLine(appName: string, r: Refusal, now = Date.now()): str
263268 return `${appName} tried to reach ${entryFor(r.host, r.port)} and was refused: Allow it (${r.count} ${r.count === 1 ? "try" : "tries"}, last ${sinceWords(r.last, now)})`;
264269 }
265270
271 const triesWords = (r: Refusal, now: number): string =>
272 `(${r.count} ${r.count === 1 ? "try" : "tries"}, last ${sinceWords(r.last, now)})`;
273
266274 /** "Blocked: huggingface.co (14 tries, last 2 min ago)" */
267275 export function refusalLine(r: Refusal, now = Date.now()): string {
268276 const name = r.port === 80 || r.port === 443 ? r.host : `${r.host}:${r.port}`;
269 return `Blocked: ${name} (${r.count} ${r.count === 1 ? "try" : "tries"}, last ${sinceWords(r.last, now)})`;
277 return `Blocked: ${name} ${triesWords(r, now)}`;
278 }
279
280 /**
281 * The owner's Blocked row, on the page and in the terminal (#13537): an
282 * undeclared host is said once, in the gate's own words, then the tries; a
283 * private destination keeps `refusalLine`, since the gate says other words.
284 */
285 export function blockedRowLine(r: Refusal, now = Date.now()): string {

Non-blocking: two builders (refusalLine, blockedRowLine) now share one row. Once the needs-you stack and brief move to the same sentence, drop refusalLine.

286 return r.private ? refusalLine(r, now) : `${refusedWords(r.host, r.port)} ${triesWords(r, now)}`;
270287 }
src/tui/screens/blocked.ts +10 −3
@@ -7,7 +7,7 @@
77 * updates, and this screen follows a write made there.
88 */
99
10 import { allowBlocked, blockedView, ignoreBlocked, type BlockedError } from "../../actions/egress_refused.ts";
10 import { allowBlocked, blockedView, ignoreBlocked, type BlockedError, type BlockedRow } from "../../actions/egress_refused.ts";
1111 import { getAppByRepo } from "../../core/apps.ts";
1212 import { canAdmin, type Repo } from "../../core/repos.ts";
1313 import type { Ctx, Screen } from "../app/app.ts";
@@ -30,6 +30,14 @@ const WORDS: Record<BlockedError, string> = {
3030 const mayOpen = (f: Session, repo: Repo | undefined): repo is Repo =>
3131 repo !== undefined && f.user !== undefined && canAdmin(f.db, repo, f.user) && getAppByRepo(f.db, repo.id) !== undefined;
3232
33 /**
34 * The rows as the terminal lists them: the page's line, so both doors say the gate's words
35 * (#13537).
36 */
37 export function blockedListRows(rows: readonly BlockedRow[]): ListRow[] {
38 return rows.map((r) => ({ title: r.line, ...(r.private ? { meta: "private: a site admin allows it" } : {}) }));
39 }
40
3341 export function blockedScreen(f: Session, ctx: Ctx, repo: Repo): Screen {
3442 const state = new ListState();
3543 const rows = () => {
@@ -53,8 +61,7 @@ export function blockedScreen(f: Session, ctx: Ctx, repo: Repo): Screen {
5361 style(c.mutedFg, c.bg),
5462 head!.x + head!.w,
5563 );
56 const lines: ListRow[] = all.map((r) => ({ title: r.line, ...(r.private ? { meta: "private: a site admin allows it" } : {}) }));
57 drawList(s, list!, c, state, lines);
64 drawList(s, list!, c, state, blockedListRows(all));
5865 },
5966 key(e) {
6067 const all = rows();
test/egress-refused.test.ts +3 −3
@@ -139,7 +139,7 @@ describe("refused hosts", () => {
139139 test("the app's page says it, with an Allow beside it", async () => {
140140 const cookie = await signIn("cy");
141141 const page = await (await fetch(`${servers.url}/cy/game/settings/app`, { headers: { cookie } })).text();
142 expect(page.replace(/\s+/g, " ")).toMatch(/Blocked: huggingface\.co \(3 tries, last just now\)/);
142 expect(page.replace(/\s+/g, " ")).toMatch(/refused by this forge: huggingface\.co:443 is not in this app&#39;s forge\.json egress \(3 tries, last just now\)/);
143143 expect(page).toContain("/cy/game/settings/app/blocked/huggingface.co%3A443/allow");
144144 });
145145
@@ -309,7 +309,7 @@ describe("an app told in the forge's own words (#12182)", () => {
309309 const lines: string[] = [];
310310 const hooks = cellHooks(dataDir, "walk1", "production", () => {}, (l) => lines.push(l));
311311 for (let i = 0; i < 3; i++) hooks.onRefused?.("beta-bridge.simplefin.org", 443, false);
312 expect(lines).toEqual(["[egress] walk1 was refused beta-bridge.simplefin.org: not in its forge.json egress"]);
312 expect(lines).toEqual(["[egress] walk1: refused by this forge: beta-bridge.simplefin.org:443 is not in this app's forge.json egress"]);
313313 const policy = { hosts: [] as string[], origins: [], allowed: () => [] as string[] };
314314 const probe = Bun.listen({ hostname: "127.0.0.1", port: 0, socket: { data() {} } });
315315 const port = probe.port;
@@ -402,7 +402,7 @@ describe("the needs-you line at every door, and the log and status line it rides
402402 const hooks = cellHooks(dataDir, "slug1", "production", () => {}, (l) => lines.push(l), slug);
403403 hooks.onRefused?.(`${"h".repeat(120)}.example.org`, 443, false);
404404 expect(lines).toHaveLength(1);
405 expect(lines[0]).toStartWith(`[egress] ${slug} was refused hhh`);
405 expect(lines[0]).toStartWith(`[egress] ${slug}: refused by this forge: hhh`);
406406 expect(lines[0]).not.toContain("slug1");
407407 expect(lines[0]!.length).toBeLessThanOrEqual(200);
408408 expect(lines[0]).not.toContain("\n");
test/egress-words.test.ts added +74
@@ -0,0 +1,74 @@
1 /**
2 * An app's fetch to an undeclared host is told the forge's own no, and every
3 * door that shows the refusal says the same sentence (#13537): the gate's
4 * answer, the parent's log line, and the owner's Blocked row on the page and
5 * in the terminal. The sentence is
6 * spelled out literally here so a change to it, or a dropped host, fails.
7 */
8
9 import { afterAll, describe, expect, test } from "bun:test";
10 import { mkdtempSync, rmSync } from "node:fs";
11 import { tmpdir } from "node:os";
12 import { join } from "node:path";
13 import { decide, refusedWords } from "../src/apps/egress.ts";
14 import { blockedRowLine, cellHooks, type Refusal } from "../src/apps/egress_refused.ts";
15 import { blockedListRows } from "../src/tui/screens/blocked.ts";
16 import { blockedNote } from "../src/web/studio/blocked.ts";
17 import type { BlockedRow, BlockedView } from "../src/actions/egress_refused.ts";
18 import { html } from "../src/web/html.ts";
19
20 const SENTENCE = "refused by this forge: api.undeclared.example:443 is not in this app's forge.json egress";
21 const dataDir = mkdtempSync(join(tmpdir(), "egress-words-"));
22 afterAll(() => rmSync(dataDir, { recursive: true, force: true }));
23
24 describe("the forge's own words for an undeclared host (#13537)", () => {
25 test("the words name the host and forge.json egress", () => {
26 expect(refusedWords("api.undeclared.example", 443)).toBe(SENTENCE);
27 });
28
29 test("the gate refuses a fetch to it with that sentence", async () => {
30 const verdict = await decide({ hosts: ["declared.example"], origins: [] }, "api.undeclared.example", 443);
31 expect(verdict.ok).toBe(false);
32 expect(verdict.ok ? "" : verdict.why).toBe(SENTENCE);
33 });
34
35 test("the log line prints the same sentence", () => {
36 const lines: string[] = [];
37 const hooks = cellHooks(dataDir, "words1", "production", () => {}, (l) => lines.push(l));
38 hooks.onRefused?.("api.undeclared.example", 443, false);
39 expect(lines).toEqual([`[egress] words1: ${SENTENCE}`]);
40 });
41
42 test("the private-destination log line keeps its own words, which the gate does not say", () => {
43 const lines: string[] = [];
44 const hooks = cellHooks(dataDir, "words2", "production", () => {}, (l) => lines.push(l));
45 hooks.onRefused?.("10.0.0.5", 5432, true);
46 expect(lines).toEqual(["[egress] words2 was refused 10.0.0.5:5432: a private destination"]);
47 });
48
49 const refusal: Refusal = { host: "api.undeclared.example", port: 443, count: 2, first: new Date().toISOString(), last: new Date().toISOString(), denied: false, private: false };
50 const row: BlockedRow = {
51 host: refusal.host,
52 port: refusal.port,
53 line: blockedRowLine(refusal),
54 destination: "api.undeclared.example:443",
55 private: false,
56 };
57
58 test("the Blocked row says the sentence once, then the tries", () => {
59 expect(row.line).toBe(`${SENTENCE} (2 tries, last just now)`);
60 expect(blockedRowLine({ ...refusal, host: "10.0.0.5", port: 5432, private: true })).toStartWith("Blocked: 10.0.0.5:5432 (");
61 });
62
63 test("the owner's page prints the same sentence once, escaped", () => {
64 const view: BlockedView = { rows: [row], liveOnly: [], canAllowPrivate: false };
65 const page = String(blockedNote("/o/r", view, () => html``));
66 const escaped = SENTENCE.replace("'", "&#39;");
67 expect(page).toContain(escaped);
68 expect(page.split(escaped)).toHaveLength(2);
69 });
70
71 test("the terminal's Blocked screen prints the same sentence", () => {
72 expect(blockedListRows([row]).map((r) => r.title)).toEqual([`${SENTENCE} (2 tries, last just now)`]);
73 });
74 });

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs