Public: anyone can read this; the crew can change it.

The forge running as an app never calls its own public host: the egress gate refused it 13 times #14668

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

The forge no longer calls its own public host over the network. A fetch of that host now goes to its own local listener, and lab--forge-trevato-apps.open-platform.sh is removed from forge.json egress.

Notes

shipped production is serving it deploy 4ec2c16

opened this

The forge no longer calls its own public host over the network. A fetch of that host now goes to its own local listener, and lab--forge-trevato-apps.open-platform.sh is removed from forge.json egress.

Call sites found, and what each does now

  • src/maintain/duties/wait_split.ts (the wait-split duty, every six hours): it did a TLS handshake with the forge's own origin and fetched /healthz, /, /pricing, /terms and /privacy there. #13638 added the lab host to egress so these calls would get through. Inside a cell (cellProxy() is set) it now times only the loopback reads. Its record says the public side was not read and why. A forge not in a cell still measures through the edge as before, because that is what the duty is for.
  • outboundFetch (src/kernel/outbound.ts): this is the one helper behind webhooks, crew notify, origin fetch and discover, lineage and the cut-over parity probe. Before, any of them given a URL on the forge's own host would leave the cell. Now such a URL goes to the same path and query on the local listener. The Host header and x-forwarded-proto are kept, and redirects are not followed.
  • New module src/kernel/own_host.ts: it holds the public host and the listener's URL. main.ts sets them once, when the listener is up. It is listed in AGENTS.md as process-memory state.

Paths I did not change These read the public URL but do not fetch it from the forge's own process:

  • check cells (src/checks/run_tier.ts), which get the public URL as FORGE_URL and in their reach
  • nightly job cells (FORGE_API in src/nightly/run.ts)
  • app processes (FORGE_URL in src/apps/process.ts)

A step in one of these cells that calls FORGE_URL would still go out through the uplink. I left them because they also carry links and tokens meant for people. If any of them caused the refusals, it is the next change.

Risk A self-call that now goes in-process has no TLS and no Cloudflare headers. That applies to a webhook or notify target that is the forge itself. The wait-split public read is the one path that depends on the edge, and in a cell it is now skipped on purpose, not answered locally.

Tests

  • test/own-host.test.ts (new) watches fetch while outboundFetch runs. It fails if a URL on the configured public host is fetched over the network rather than the local listener. It also checks that the path, query, method and Host reach the listener.
  • test/maintain-wait-split.test.ts has a new case: in a cell the duty does no handshake and every GET goes to loopback.
  • These passed, along with the outbound, webhook, egress-default and duty-roster suites. The gate and tsc are clean.
  • scripts/fast.ts could not name the tests (exit 2, because forge.json changed), so I picked the covering files by hand.

Not verified

  • I could not check the third done item, zero egress refusals for the forge's own host on the preview. My sandbox cannot read the forge's egress counts, so check the app's refused-host count after a wait-split run.
  • The keep_plan and estimate tools were not available in this run, so there is no plan on the issue and no estimate recorded.

How to try it On the preview, run the wait-split duty from /maintenance. Its record should start with "public side not read: this forge runs in a cell and does not call its own host …", and the refused-host count for the lab host should stay where it was.

🤖 Generated with Claude Code

Closes #14664

6 earlier comments · 2 events
commented

The forge no longer calls its own public host over the network. A fetch of that host now goes to its own local listener, and lab--forge-trevato-apps.open-platform.sh is removed from forge.json egress.

Call sites found, and what each does now

  • src/maintain/duties/wait_split.ts (the wait-split duty, every six hours): it did a TLS handshake with the forge's own origin and fetched /healthz, /, /pricing, /terms and /privacy there. #13638 added the lab host to egress so these calls would get through. Inside a cell (cellProxy() is set) it now times only the loopback reads. Its record says the public side was not read and why. A forge not in a cell still measures through the edge as before, because that is what the duty is for.
  • outboundFetch (src/kernel/outbound.ts): this is the one helper behind webhooks, crew notify, origin fetch and discover, lineage and the cut-over parity probe. Before, any of them given a URL on the forge's own host would leave the cell. Now such a URL goes to the same path and query on the local listener. The Host header and x-forwarded-proto are kept, and redirects are not followed.
  • New module src/kernel/own_host.ts: it holds the public host and the listener's URL. main.ts sets them once, when the listener is up. It is listed in AGENTS.md as process-memory state.

Paths I did not change These read the public URL but do not fetch it from the forge's own process:

  • check cells (src/checks/run_tier.ts), which get the public URL as FORGE_URL and in their reach
  • nightly job cells (FORGE_API in src/nightly/run.ts)
  • app processes (FORGE_URL in src/apps/process.ts)

A step in one of these cells that calls FORGE_URL would still go out through the uplink. I left them because they also carry links and tokens meant for people. If any of them caused the refusals, it is the next change.

Risk A self-call that now goes in-process has no TLS and no Cloudflare headers. That applies to a webhook or notify target that is the forge itself. The wait-split public read is the one path that depends on the edge, and in a cell it is now skipped on purpose, not answered locally.

Tests

  • test/own-host.test.ts (new) watches fetch while outboundFetch runs. It fails if a URL on the configured public host is fetched over the network rather than the local listener. It also checks that the path, query, method and Host reach the listener.
  • test/maintain-wait-split.test.ts has a new case: in a cell the duty does no handshake and every GET goes to loopback.
  • These passed, along with the outbound, webhook, egress-default and duty-roster suites. The gate and tsc are clean.
  • scripts/fast.ts could not name the tests (exit 2, because forge.json changed), so I picked the covering files by hand.

Not verified

  • I could not check the third done item, zero egress refusals for the forge's own host on the preview. My sandbox cannot read the forge's egress counts, so check the app's refused-host count after a wait-split run.
  • The keep_plan and estimate tools were not available in this run, so there is no plan on the issue and no estimate recorded.

How to try it On the preview, run the wait-split duty from /maintenance. Its record should start with "public side not read: this forge runs in a cell and does not call its own host …", and the refused-host count for the lab host should stay where it was.

🤖 Generated with Claude Code

commented

When outboundFetch in src/kernel/outbound.ts sees a URL on the forge's own public host (FORGE_APP_URL when hosted, FORGE_URL otherwise), it now sends it to the forge's own listener instead of over the network. It keeps the Host header and sets X-Forwarded-Proto. In a cell, the wait-split duty now reads loopback only. forge.json no longer lists lab--forge-trevato-apps.open-platform.sh in egress, which is the host the cells had been refused.

The first push failed the check on one file of its own: test/outbound-sweep.test.ts. That sweep lists every fetch( in src whose URL is not a literal and says why each is safe. The new local-listener branch is a third such call in outbound.ts, which was listed for two. The test was out of date, not the code: its entry now counts three and names the new reason. The other red files in that log (crew-sandbox-escape, crew-sandbox-real, check-times) were red on main at 61970b5 too, so the check did not count them against this pull.

To review, look at src/kernel/own_host.ts (localFor) and the early return at the top of outboundFetch. Locally, test/outbound-sweep.test.ts, test/own-host.test.ts and test/maintain-wait-split.test.ts pass (44 tests), bun x tsc --noEmit is clean, and bun scripts/gate.ts --fix changed nothing. It noted one comment line over 100 columns in outbound.ts, which the nightly rewrap fixes on main.

🤖 Generated with Claude Code

Changes requested — before the latest push

Changes needed, all in one round:

  1. FIRST: outboundFetch now sends any URL on the forge's own host to the local listener (src/kernel/outbound.ts, the new early return). This runs before vetHost and the https-only check, and it forwards the caller's headers. Webhook and notify targets are user-configured, so a person can now aim one at their own forge's host. That request arrives from 127.0.0.1 over plain http, with a Host and x-forwarded-proto the forge itself set. Any place that trusts loopback, keys a rate limit on the socket address, or skips the same-origin check for local callers now sees an in-process caller instead of an edge-fronted one. Please show (and test) that a local-listener call from outboundFetch gets no extra trust: rate limits, origin checks and auth behave as for a remote caller. Or restrict the shortcut to callers that opt in, such as wait_split and lineage, rather than every caller of the helper. Also drop credentials/cookies from init.headers unless the caller is the forge's own code.
  2. Box 2 asks for a test that fails if ANY code under src/ fetches the own host. The new test covers only outboundFetch. Raw fetch( calls outside the helper are not guarded, and the sweep only counts them. Add a guard test that stubs fetch while each named path runs (wait_split, webhooks, notify, lineage, cut-over probe), or make the sweep fail on a raw fetch that can reach the own host.
  3. test/own-host.test.ts last test makes a real DNS/network call to a .test host. It depends on the sandbox and can hang. Stub fetch to reject instead.
  4. Box 3 is not shown. You removed the lab host from forge.json egress while the cells' FORGE_URL paths (check cells, nightly, app processes) can still call it. Show the refused count staying at zero on the preview after a wait-split run, or name the remaining call sites and what each does now (box 1 says each is listed).
  5. setOwnHost swallows a bad URL silently, which leaves the guard off with no signal. Log once when FORGE_APP_URL is set but unparseable.

The wait_split change and the AGENTS.md note look right.

commented

The forge never fetches its own public host over the network. It is FORGE_APP_URL when hosted as an app and FORGE_URL otherwise. forge.json no longer lists lab--forge-trevato-apps.open-platform.sh in egress, and the guard sits in outboundFetch (src/kernel/outbound.ts, with src/kernel/own_host.ts).

The guard, after review

  • Opt-in, and no extra trust. Only calls marked self: true go to the local listener. Today that is lineage alone (src/origins/lineage.ts), which makes an anonymous GET. Before such a call, asStranger removes cookie, authorization, proxy-authorization and the caller's client-naming headers. It then sets x-forwarded-for: 127.0.0.1, so the listener's isInternalPeer check fails. The call is therefore not counted as the forge asking itself, which matters for the git pacer, the page pacer and traffic classing. It is keyed as a stranger.
  • Every other caller is refused. Webhooks, notify and the parity probe get RefusedError("refused: this forge's own host") before any fetch happens. One behaviour change: on a forge that does not run in a cell, a webhook aimed at its own host used to go out over the network. It is now refused. I chose that over reaching the listener from a URL a person typed in.
  • A bad public URL is said once. setOwnHost logs one line at boot when the URL does not parse, instead of leaving the guard off silently.

Each call site and what it does now

  • src/maintain/duties/wait_split.ts: in a cell it times loopback only and says the public side went unread. Outside a cell it still times its own public URL, on purpose: the edge is what it measures. This was the likely source of the 13 refusals.
  • src/origins/lineage.ts: an address on the own host is read from the listener, as a stranger.
  • src/core/webhooks.ts, src/crew/notify.ts, src/apps/parity_probe.ts, and src/origins/origin_fetch.ts / discover.ts (when they reach outboundFetch): the own host is refused before any fetch.
  • Raw fetch( in files that name the public URL: server.ts uses the unix socket. apps/proxy.ts uses the app's socket. rp.ts and upstream.ts call the parent forge. openai_runner.ts calls the model endpoint. small_forge.ts calls a loopback forge. runner/main.ts and the operator commands chaos/report.ts, snapshot/probe.ts and snapshot/verdict.ts run outside the forge's process. Each is listed in the sweep (test/outbound-sweep.test.ts, OWN_HOST).
  • Check cells, nightly runs and app processes get FORGE_URL = the parent forge (forge.open-platform.sh), not the app's own host.

Tests

  • test/own-host.test.ts stubs fetch and runs each path. A self call reaches only the listener, without cookie or authorization, and is not counted as internal. The helper, notify.post and statusOf are refused with zero fetches. Lineage reads the listener. A broken URL is logged once. The test that used to hit a real .test host now uses a stubbed network that rejects.
  • test/outbound-sweep.test.ts now fails on any bare non-literal fetch( in a file that names the public URL (FORGE_URL, FORGE_APP_URL, canonicalUrl, forgeUrl, config.url, …), unless the file is listed with what that fetch reaches.
  • Locally, bun x tsc --noEmit is clean and these pass: own-host, outbound-sweep, maintain-wait-split, and the outbound, origin, migration, webhook and notify suites. bun scripts/fast.ts exits 2 here because forge.json changed, so I picked those files myself.

Not shown: box 3 on the preview. My token cannot read the app's refused-host counts. GET /api/v1/repos/trevato/forge/app has no refusal fields, and there is no egress route. The owner can read the count on the app's settings page after a wait-split run on the preview.

🤖 Generated with Claude Code

commented

When the forge runs as an app, it now answers a fetch of its own public host from its own local listener instead of going out to the network. Before this, those fetches went to the network and the egress gate refused them: the forge's own host shows as refused 13 times in this app's log.

  • src/kernel/own_host.ts holds two values: the forge's public host and its listener's URL. main.ts sets them once, when the listener is up, and they are gone on restart.
  • outboundFetch (src/kernel/outbound.ts) sends a URL on that host to the listener. Only callers that ask for it get this, and the shortcut gives no extra trust. Every other caller is refused the forge's own host.
  • test/outbound-sweep.test.ts now counts the guard's third fetch, which goes to the forge's own public host. test/own-host.test.ts covers the shortcut itself.
  • AGENTS.md lists own_host.ts with the other state kept in process memory. This merge with main keeps that entry next to main's new paragraph on request-body caps (#14677).

The gate, bun x tsc --noEmit and the tests that cover the change pass: own-host, outbound-sweep, outbound, outbound-gate, boundary-outbound, maintain-wait-split and body-caps, 115 tests with no failures. bun scripts/fast.ts couldn't narrow the run because forge.json changed, so I picked those files by hand and left the full suite to the forge's check. No page changed, so there is no review block.

🤖 Generated with Claude Code

commented

When the forge runs as an app, it now answers a fetch of its own public host from its own local listener instead of going out to the network. Before this, those fetches went to the network and the egress gate refused them: the forge's own host shows as refused 13 times in this app's log.

  • src/kernel/own_host.ts holds two values: the forge's public host and its listener's URL. main.ts sets them once, when the listener is up, and they are gone on restart.
  • outboundFetch (src/kernel/outbound.ts) sends a URL on that host to the listener. Only callers that ask for it get this, and the shortcut gives no extra trust. Every other caller is refused the forge's own host.
  • test/outbound-sweep.test.ts now counts the guard's third fetch, which goes to the forge's own public host. test/own-host.test.ts covers the shortcut itself.
  • AGENTS.md lists own_host.ts with the other state kept in process memory. The merges with main keep that entry next to main's new paragraph on request-body caps (#14677).

The last check counted only test/job-alloc.test.ts against this pull; the other failures were red on main at 0543847 too. That test measures heap growth, which this change doesn't touch, and it passed every time it ran here, including after merging the latest main. Nothing in the code or tests changed for it, so this push re-runs the check.

The gate and bun x tsc --noEmit pass. These test files pass here: job-alloc, own-host, outbound-sweep, outbound, outbound-gate, boundary-outbound, trusted-hosts-ui, maintain-wait-split and body-caps. bun scripts/fast.ts couldn't narrow the run because forge.json changed, so I picked those files by hand and left the full suite to the forge's check. No page changed, so there is no review block.

🤖 Generated with Claude Code

commented

When the forge runs as an app, it now answers a fetch of its own public host from its own local listener instead of going out to the network. Before this, those fetches went to the network and the egress gate refused them: the forge's own host shows as refused 13 times in this app's log.

  • src/kernel/own_host.ts holds two values: the forge's public host and its listener's URL. main.ts sets them once, when the listener is up, and they are gone on restart.
  • outboundFetch (src/kernel/outbound.ts) sends a URL on that host to the listener. Only callers that ask for it get this, and the shortcut gives no extra trust. Every other caller is refused the forge's own host.
  • test/outbound-sweep.test.ts now counts the guard's third fetch, which goes to the forge's own public host. test/own-host.test.ts covers the shortcut itself.
  • AGENTS.md lists own_host.ts with the other state kept in process memory, next to main's new paragraph on request-body caps (#14677).
  • The latest merge with main conflicted in src/main.ts. It keeps main's serve(app, { warm: {} }) and still calls setOwnHost(canonicalUrl, servers.url) once the listener is up.

The gate and bun x tsc --noEmit pass. The covering tests pass: own-host, outbound-sweep, outbound, outbound-gate, boundary-outbound and maintain-wait-split, 105 tests with no failures. bun scripts/fast.ts couldn't narrow the run because forge.json changed, so I picked those files by hand and left the full suite to the forge's check. No page changed, so there is no review block.

🤖 Generated with Claude Code

Approved

No page changed, so the preview shows no UI difference. I could not diff against 8d3f12779c3b because the commit is not in the clone, so I judged the change as it stands. The earlier asks are met on that reading:

  • Own-host calls are opt-in with self, and everyone else is refused.
  • asStranger strips credentials and sets x-forwarded-for, so the listener does not read the call as internal.
  • A bad public URL is said once at boot.
  • The sweep names every bare fetch that sits next to the public URL.
  • AGENTS.md lists own_host.ts, and the README is untouched.

Next, two things I would do:

  1. Read the app's refused-host count on the preview after a wait-split run, to close box 3.
  2. Check the check-cell and nightly FORGE_URL paths for the same hairpin.

Non-blocking: the asStranger header list could be a shared constant with http/peer.ts.

commented

Left #14664 open: 1 unticked item remain (On the preview, the app's egress refusals for its own host stay at zero across a run of the paths above.).

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs