Public: anyone can read this; the crew can change it.

The egress proxy and relay() bound what they buffer: a flood during a dial or a slow reader is paused, then cut #15209

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

The egress proxy and relay() now cap how much they hold for a side that isn't reading. A sender that floods is paused once its queue passes a quarter of the limit, and cut once it passes the whole limit. src/apps/egress.ts, bytes sent while the upstream is being dialled.

Notes

shipped production is serving it deploy cfe12e4

opened this

The egress proxy and relay() now cap how much they hold for a side that isn't reading. A sender that floods is paused once its queue passes a quarter of the limit, and cut once it passes the whole limit.

src/apps/egress.ts, bytes sent while the upstream is being dialled. These used to be copied onto conn.head again with each chunk, which has no bound and copies O(n²) bytes. They now go on a separate conn.early list that keeps a running byte count:

  • Past queueMax / PAUSE_AT the client is paused.
  • Past queueMax the client is cut and the cut is logged.
  • When the connection is established, the held bytes go upstream in order after the head, and the client resumes. If what was sent is itself the backlog, the peer's drain resumes the client instead.

relay() (forge → app in.sock, cell → forge loopback). It had no backpressure in either direction. It now follows the same pattern as the egress data path:

  • Whichever side fills its queue (the client's early bytes, toPeer or toClient) is paused past QUEUE_MAX / PAUSE_AT.
  • That side resumes when the other side drains.
  • Both sides are cut past QUEUE_MAX.

relay() takes an optional fourth argument, queueMax (default QUEUE_MAX), which the tests use to shrink the limit. Existing callers don't change.

test/egress-bounded.test.ts has four tests:

  • Dial that never completes: a proxy that accepts CONNECT and never answers, with a client flooding 32 MiB.
  • Relay target faster than its client: a target writing 32 MiB to a client that never reads.
  • Relay client faster than its target: a client writing 32 MiB to a target that never reads.
  • Ordinary exchange: 8 MiB of random bytes, through the proxy (sent right after the head, so most of it arrives during the dial) and through relay(), with a reader that keeps pausing. The echo must come back byte-identical.

I checked the three flood tests against main's egress.ts: each fails there (all 32 MiB accepted) and passes with this change. The byte-identical test passes on both. scripts/gate.ts takes every test/*.test.ts, so there was no list to add the file to.

bun test test/egress-bounded.test.ts test/egress*.test.ts: 172 pass, 0 fail, 34 skipped. The skipped ones are the unix-socket and cell suites, which this sandbox can't run (it can't bind a unix socket); the forge's check runs them on push. bun scripts/gate.ts --fix rewrapped comments in the two changed files, and that is committed. bun x tsc --noEmit passes.

Left out:

  • The cut itself isn't tested. A pause always held in these tests, so a flood stalls and never reaches the limit; the flood tests accept either a stall or a cut.
  • Unlike the existing egress data path, the cut fires as soon as a queue passes the limit; it doesn't wait for the pause to take effect first. With the default 4 MiB limit, the pause comes at 1 MiB, which leaves 3 MiB for a read already under way.

Not done:

  • The issue asks for the head branch to be in trevato/forge. git push sent it to the fork scout/forge, and git configuration is read-only here, so a person needs to push or mirror the branch to trevato/forge before asking for review.
  • The keep_plan tool the issue asks for wasn't available in this run, so the issue has no plan.

🤖 Generated with Claude Code

Closes #15207

2 earlier comments · 2 events
commented

The egress proxy and relay() now cap how much they hold for a side that isn't reading. A sender that floods is paused once its queue passes a quarter of the limit, and cut once it passes the whole limit.

src/apps/egress.ts, bytes sent while the upstream is being dialled. These used to be copied onto conn.head again with each chunk, which has no bound and copies O(n²) bytes. They now go on a separate conn.early list that keeps a running byte count:

  • Past queueMax / PAUSE_AT the client is paused.
  • Past queueMax the client is cut and the cut is logged.
  • When the connection is established, the held bytes go upstream in order after the head, and the client resumes. If what was sent is itself the backlog, the peer's drain resumes the client instead.

relay() (forge → app in.sock, cell → forge loopback). It had no backpressure in either direction. It now follows the same pattern as the egress data path:

  • Whichever side fills its queue (the client's early bytes, toPeer or toClient) is paused past QUEUE_MAX / PAUSE_AT.
  • That side resumes when the other side drains.
  • Both sides are cut past QUEUE_MAX.

relay() takes an optional fourth argument, queueMax (default QUEUE_MAX), which the tests use to shrink the limit. Existing callers don't change.

test/egress-bounded.test.ts has four tests:

  • Dial that never completes: a proxy that accepts CONNECT and never answers, with a client flooding 32 MiB.
  • Relay target faster than its client: a target writing 32 MiB to a client that never reads.
  • Relay client faster than its target: a client writing 32 MiB to a target that never reads.
  • Ordinary exchange: 8 MiB of random bytes, through the proxy (sent right after the head, so most of it arrives during the dial) and through relay(), with a reader that keeps pausing. The echo must come back byte-identical.

I checked the three flood tests against main's egress.ts: each fails there (all 32 MiB accepted) and passes with this change. The byte-identical test passes on both. scripts/gate.ts takes every test/*.test.ts, so there was no list to add the file to.

bun test test/egress-bounded.test.ts test/egress*.test.ts: 172 pass, 0 fail, 34 skipped. The skipped ones are the unix-socket and cell suites, which this sandbox can't run (it can't bind a unix socket); the forge's check runs them on push. bun scripts/gate.ts --fix rewrapped comments in the two changed files, and that is committed. bun x tsc --noEmit passes.

Left out:

  • The cut itself isn't tested. A pause always held in these tests, so a flood stalls and never reaches the limit; the flood tests accept either a stall or a cut.
  • Unlike the existing egress data path, the cut fires as soon as a queue passes the limit; it doesn't wait for the pause to take effect first. With the default 4 MiB limit, the pause comes at 1 MiB, which leaves 3 MiB for a read already under way.

Not done:

  • The issue asks for the head branch to be in trevato/forge. git push sent it to the fork scout/forge, and git configuration is read-only here, so a person needs to push or mirror the branch to trevato/forge before asking for review.
  • The keep_plan tool the issue asks for wasn't available in this run, so the issue has no plan.

🤖 Generated with Claude Code

Changes requested — before the latest push

Changes needed, one round:

  1. FIRST: the cut is untested. The change adds two cut paths (egress conn.earlyBytes > queueMax in data(), and relay bound() > queueMax, both sides terminated) and no test would fail if they were deleted; the PR text admits it. Every behaviour added needs a test that fails without it. Add tests that reach the cut deterministically: e.g. one data callback delivering a single chunk larger than queueMax (a chunk > SMALL written in one write to the silent-dial proxy, and to the relay with a never-reading target), then assert the client socket closes and the relay target sees close. Also assert the log line for the egress cut.
  2. The three flood tests accept either a stall or a cut (took < FLOOD) and gate on a 500 ms no-progress timer inside flood(). A slow CI machine can see a 500 ms gap on a healthy pump and pass/stall for the wrong reason, and a stall proves only that the kernel buffers filled, not that the proxy paused. Assert what the issue asks: the proxy-held bytes stay under queueMax (e.g. the relay/egress sent total is bounded by a small multiple of SMALL plus socket buffers) or that after resuming the reader the bytes all arrive in order. Direction: fast tests that never flake.
  3. Cut timing differs from the egress data path the issue says to copy (that one waits for the pause to take effect); you note it. Fine with the 4 MiB default, but add the sentence to the code comment at bound() so the next reader knows it is deliberate.
  4. Box 5 (diff visible in trevato/forge): the PR text says the branch went to the fork scout/forge, not trevato/forge. Confirm the head branch is on trevato/forge.

The rest reads well: early list with running count, resume on established/drain in both paths, optional queueMax arg leaves callers unchanged, no new dependency, no README change, no web/TUI surface. Boxes 1, 2 and 4 look met by the code; box 3 is not met until the cut is tested; box 5 needs confirmation.

commented

The egress proxy and relay() now limit how much they hold for a side that isn't reading. A sender that floods is paused once its queue passes a quarter of queueMax, and cut past all of it.

src/apps/egress.ts

  • During the dial: bytes that arrive after a complete head while the upstream is still being dialled go on their own list, conn.early, with a running byte count. They used to be glued onto the head again with each chunk, with no limit. They go upstream in order once the connection is up, and the client resumes then.
  • In relay(): whichever side fills its queue is paused, resumes when the other side drains, and both sides are cut past queueMax. relay() takes an optional queueMax argument (default 4 MiB), so existing callers don't change.
  • New held() on Egress: an optional method that sums every queue a listener holds right now. The tests use it to measure what the proxy holds. It is optional so supervisor.ts's HeldDoor and the other implementations stay as they are.
  • Comment at bound(): it now says that the relay cuts at the chunk that passes queueMax, unlike the egress data path, which waits for its pause to take effect. That is deliberate: with the 4 MiB default, three quarters of the limit is left for the read already under way.

test/egress-bounded.test.ts

  • Cut tests (new):
    • Egress: one 64 KiB write while the dial never completes, with queueMax 1024. The client is cut, the egress cut: log line is checked, and held() returns to 0.
    • Relay, both directions, with queueMax 0, so the first byte the relay has to keep is already past the limit. The flooding side is cut and the far end (a target or client that never reads) sees the close.
  • Flood tests (rewritten): the 500 ms no-progress timer is gone. Each test:
    • keeps the reader stopped until the proxy holds past the pause line and the writer's own socket is full;
    • then lets the reader go and checks that the proxy never held more than queueMax (2 MiB), nothing was cut, and all 16 MiB arrive in order.
    • The egress version uses a fake upstream proxy that answers CONNECT only when told to.
  • The file now uses freePort from src/apps/process.ts instead of its own copy.

Checks

  • Removing each pause or cut line, one at a time, fails exactly the tests that cover it. A removed pause fails in about 50 ms, because the cut fires. A removed cut fails on the 30 s test timeout, because the pause then holds for ever.
  • Six runs of this file in a row passed, about 3 s each.
  • scripts/gate.ts --fix and tsc --noEmit are clean.
  • The 29 egress and relay test files pass (195 pass, 4 skipped). The skipped ones are the unix-socket suites, which can't run in this sandbox; the forge's check runs them.

Head branch (box 5): the pull's head is on the fork scout/forge (crossRepo: true). This clone's push URL is that fork and I can't push to trevato/forge, so I can't move it. The diff does show in trevato/forge#15209: the pull's API record lists both files.

No new dependency, no README change, no page or terminal surface.

🤖 Generated with Claude Code

Approved

Approved. The pull is a bounded-memory change with no page or terminal surface, so the experience checks do not apply. The early list with its byte count, the pause at a quarter of queueMax and the cut past queueMax are all there, and relay() now has the same backpressure. The new cut tests cover what the earlier round left untested. held() is optional, so other Egress implementations are untouched. I could not diff against a0e6535c9e92 because the clone does not have that commit, so I read the change as shown.

Next, both non-blocking:

  1. Non-blocking: the test file has some oddly wrapped inline socket handler objects. Running the formatter over it would tidy them.
  2. Non-blocking: move the head branch into trevato/forge so box 5 can be ticked.
commented

Left #15207 open: 1 unticked item remain (The pull's diff is visible on the forge (head branch in trevato/forge) before review is asked.).

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs