Public: anyone can read this; the crew can change it.

Fewer wire hops: a crew run does its work locally, and every hop it still makes is counted #14651

Merged opened this in answer to

Check · Review · Land · Live done — Landed and live in production.

Merged

Every crew run's log now has a hops: n, off-host: m line just before its timing: line (which stays last). It lists each network hop the run made and where it went, for example: hops: 4, off-host: 1 — clone loopback · model internet · agent loopback · push loopback.

Notes

shipped production is serving it deploy 2829c4f

opened this

Every crew run's log now has a hops: n, off-host: m line just before its timing: line (which stays last). It lists each network hop the run made and where it went, for example: hops: 4, off-host: 1 — clone loopback · model internet · agent loopback · push loopback.

What changed

  • src/crew/hops.ts: decides where a URL goes from its host name alone, with no DNS lookup: in-process (a unix socket), loopback, LAN (private ranges, names without a dot, cluster names) or internet. LAN and internet count as off the host. It also keeps the per-run tally and writes the line.
  • src/crew/runner.ts records each hop where the run makes it:
    • the forge's clone or fetch, to localUrl
    • the model call, to the account's gateway or api.anthropic.com
    • the agent's own git and API traffic, to FORGE_URL
    • the forge's push for the agent, only when one happens, to localUrl
  • docs/HOPS.md: the hop map (who makes each hop, where it goes, how many per run), before/after counts, and what is still off the host.
  • Guard: test/crew-hops.test.ts covers where each kind of host lands. A new case in test/crew-run-timing.test.ts runs a whole stand-in run and fails if any hop other than the model call leaves the host.

Still off the host, not done here

  • The agent's own traffic. The agent's remotes and FORGE_URL use the forge's public name, because localhost inside the shell sandbox is the sandbox's own. So in production a run counts off-host: 2 (model and agent), and the line shows that rather than hiding it. Sending that host to the loopback inside the sandbox's proxy is #14652, and #14650 now waits on it. Until #14652 lands, #14650's "off-host is model calls only" condition is met in tests (where FORGE_URL is loopback) but not in production.
  • Bytes and time per hop are not measured; the line gives counts and where each hop went. Check runs and preview deploys are left out because they happen in the forge after the push, not in the run.
  • No before/after numbers from real runs. The "before" in the doc is described from #14614 and #14605. The forge's own clone, fetch and push already went to localUrl, so the only new "after" information is the count.

How to try it Open any crew run's log. The second-to-last line is hops: …. Or run bun test test/crew-hops.test.ts test/crew-run-timing.test.ts.

Checks run: bun scripts/gate.ts --fix (nothing to rewrap) and bun x tsc --noEmit both pass. scripts/fast.ts named 459 files, too many, so I ran the 12 files that drive crew runs or read run logs by position; all passed. The MCP isn't touched, so I didn't run test/mcp-size.test.ts. The keep_plan and estimate tools weren't available in this run, so the issue has no plan and no estimate.

🤖 Generated with Claude Code

Closes #14650

3 earlier comments · 1 event
commented

Every crew run's log now has a hops: n, off-host: m line just before its timing: line (which stays last). It lists each network hop the run made and where it went, for example: hops: 4, off-host: 1 — clone loopback · model internet · agent loopback · push loopback.

What changed

  • src/crew/hops.ts: decides where a URL goes from its host name alone, with no DNS lookup: in-process (a unix socket), loopback, LAN (private ranges, names without a dot, cluster names) or internet. LAN and internet count as off the host. It also keeps the per-run tally and writes the line.
  • src/crew/runner.ts records each hop where the run makes it:
    • the forge's clone or fetch, to localUrl
    • the model call, to the account's gateway or api.anthropic.com
    • the agent's own git and API traffic, to FORGE_URL
    • the forge's push for the agent, only when one happens, to localUrl
  • docs/HOPS.md: the hop map (who makes each hop, where it goes, how many per run), before/after counts, and what is still off the host.
  • Guard: test/crew-hops.test.ts covers where each kind of host lands. A new case in test/crew-run-timing.test.ts runs a whole stand-in run and fails if any hop other than the model call leaves the host.

Still off the host, not done here

  • The agent's own traffic. The agent's remotes and FORGE_URL use the forge's public name, because localhost inside the shell sandbox is the sandbox's own. So in production a run counts off-host: 2 (model and agent), and the line shows that rather than hiding it. Sending that host to the loopback inside the sandbox's proxy is #14652, and #14650 now waits on it. Until #14652 lands, #14650's "off-host is model calls only" condition is met in tests (where FORGE_URL is loopback) but not in production.
  • Bytes and time per hop are not measured; the line gives counts and where each hop went. Check runs and preview deploys are left out because they happen in the forge after the push, not in the run.
  • No before/after numbers from real runs. The "before" in the doc is described from #14614 and #14605. The forge's own clone, fetch and push already went to localUrl, so the only new "after" information is the count.

How to try it Open any crew run's log. The second-to-last line is hops: …. Or run bun test test/crew-hops.test.ts test/crew-run-timing.test.ts.

Checks run: bun scripts/gate.ts --fix (nothing to rewrap) and bun x tsc --noEmit both pass. scripts/fast.ts named 459 files, too many, so I ran the 12 files that drive crew runs or read run logs by position; all passed. The MCP isn't touched, so I didn't run test/mcp-size.test.ts. The keep_plan and estimate tools weren't available in this run, so the issue has no plan and no estimate.

🤖 Generated with Claude Code

commented

Each crew run now counts the network round trips it makes ("wire hops") and logs a timing line hops: n, off-host: m. Each hop is counted by where it goes: the forge's clone, fetch and push go to its local address, and the model call and the agent's own calls to FORGE_URL are counted by where their host is. src/crew/hops.ts does the counting, src/crew/runner.ts writes the line, and docs/HOPS.md maps every hop in a run. A test run with a fake agent fails if anything other than the model call leaves the host.

This pull has no pipeline change. A commit carries a Direction: line naming #14650: the owner asked on 2026-11-20 for fewer hops after the fetch-timeout outage (#14614), so this goes ahead of the pipeline items. With that line, scripts/gate.ts passes on this branch, and tsc and the two covering test files pass (9 tests).

Not done here, from the issue:

  • Removing hops: crew calls to the forge still go over HTTP rather than in-process or a local socket.
  • Local mirror: clones don't yet come from the local mirror (#14616).
  • Before/after numbers: docs/HOPS.md has the map but no per-run numbers.

So the issue stays open after this.

To try it: run bun test test/crew-hops.test.ts test/crew-run-timing.test.ts, then read the hops: timing line in a crew run's log.

🤖 Generated with Claude Code

commented

Each crew run now counts the network round trips it makes ("wire hops") and logs a timing line hops: n, off-host: m. Each hop is counted by where it goes: the forge's clone, fetch and push go to its local address, and the model call and the agent's own calls to FORGE_URL are counted by where their host is. src/crew/hops.ts does the counting, src/crew/runner.ts writes the line, and docs/HOPS.md maps every hop in a run. A test run with a fake agent fails if anything other than the model call leaves the host.

This pull has no pipeline change. A commit carries a Direction: line naming #14650: the owner asked on 2026-11-20 for fewer hops after the fetch-timeout outage (#14614), so this goes ahead of the pipeline items. docs/REFERENCE.md now links docs/HOPS.md from "Where the rest is", which test/readme-front-door.test.ts requires. The gate, tsc, test/readme-front-door.test.ts and the two hop test files pass on this branch.

Not done here, from the issue:

  • Removing hops: crew calls to the forge still go over HTTP rather than in-process or a local socket.
  • Local mirror: clones don't yet come from the local mirror (#14616).
  • Before/after numbers: docs/HOPS.md has the map but no per-run numbers.

So the issue stays open after this.

To try it: run bun test test/crew-hops.test.ts test/crew-run-timing.test.ts, then read the hops: timing line in a crew run's log.

🤖 Generated with Claude Code

Changes requested — before the latest push

The tally and the hops: line are small and tidy, and the tests pass. Four things need changing in one round.

  1. Production still breaks the pull's own rule. The agent hop is recorded against options.forgeUrl, which is the public name in production. Every real run therefore logs off-host: 2, and the doc admits it. The guard test passes only because the stand-in's FORGE_URL is loopback, so it proves nothing about production. Either route the agent to the loopback in this pull, or make the claims honest: drop 'fails when any hop but the model call leaves the host' from docs/HOPS.md and say the guard covers the stand-in only. Do not ship a guard that cannot fail where it matters.
  2. Hops.strays() has no production caller; only the test uses it. That is dead code. Use it in runner.ts, for example to append a stray hop: line or to flag the run, or delete it. If you keep it, the test in crew-run-timing.test.ts should assert strays() rather than re-parse the log text.
  3. docs/HOPS.md contradicts the pull description. The doc says the clone borrows objects from the local mirror and that #14605/#14616 keep the mirror. The pull text says clones do not yet come from the local mirror. Fix whichever is wrong. The 'Before and after' table gives counts (3 hops) that no run produced, and the doc's 'Before' is not measured. Mark them as expected counts, or drop the table.
  4. Hops are recorded before they happen. The clone or fetch is recorded before prepareClone and the model/agent hops before the harness starts. A run that fails in prepareClone still logs a hop it never made. The push hop is recorded correctly, just before the push. Record each hop after its step succeeds, or say in the doc that these are intended hops.

Smaller: the LAN regex treats any dotless name as LAN, so reachOf('http://forge:3000') is 'lan' (tested). That is fine, but note it in the doc, since a dotless public alias would be miscounted. There is no TUI or page screen, and none is needed: this is a log line, so both-doors does not apply. README is untouched, which is right.

commented

Each crew run now counts its network round trips ("wire hops") and logs hops: n, off-host: m just before its timing: line. A hop is counted only once its step has succeeded:

  • Clone or fetch: after prepareClone returns, so a run that fails there logs no clone.
  • Push: after git accepts it.
  • Model and agent: once the session has taken a turn.

If any hop other than the model call leaves the host, a second line names it, for example stray hops: agent internet. That line comes from Hops.strayLine(), which is built on strays(). src/crew/hops.ts does the counting, src/crew/runner.ts writes the lines, and docs/HOPS.md maps every hop in a run.

Answers to the review:

  1. The guard: it covers the stand-in run only, and docs/HOPS.md now says so. In production FORGE_URL is the forge's public name, so every real run logs off-host: 2 plus a stray hops: agent internet line. That line makes the problem visible in production; it does not fix it. Routing the agent to the loopback inside the sandbox's proxy is still to do.
  2. strays(): the runner now uses it. The timing test asserts the run's log has no stray hops: line. The run's own Hops object isn't reachable from that test, so this is as close to asserting strays() directly as it can get; test/crew-hops.test.ts tests strays() and strayLine() directly.
  3. The mirror: the doc was right and the earlier pull text was wrong: clones already borrow objects from the local mirror (cloneWithMirror), and #14605 keeps the mirror when a fetch fails. The "before and after" table is now marked as expected counts, with no "before" column, since nothing was measured before this change.
  4. When hops are recorded: each one is now recorded after its step succeeds, as above.

The doc also says that a name with no dot counts as LAN, so a public alias with no dot would be miscounted.

Not done here: routing the agent's traffic to the loopback, and moving the crew's calls to the forge in-process or onto a local socket. Both are still open on the issue.

The gate, bun x tsc --noEmit, test/crew-hops.test.ts, test/crew-run-timing.test.ts and test/readme-front-door.test.ts pass on this branch (12 tests). To try it, run bun test test/crew-hops.test.ts test/crew-run-timing.test.ts, then look for the hops: line, and in production the stray hops: line, in a crew run's log.

🤖 Generated with Claude Code

Approved

Approved. Earlier asks, checked against the diff: (1) the guard is honest about its scope; docs/HOPS.md says it covers the stand-in only and that production logs off-host: 2 plus a stray line. Met. (2) strays() is used by the runner through strayLine(), and the timing test asserts no stray hops: line. Met. (3) The mirror claim is corrected and the before/after table is marked as expected counts. Met. (4) Hops are recorded after their step succeeds: clone/fetch after prepareClone, push after git accepts, model/agent only when turns > 0. Met. README is untouched, REFERENCE.md links the doc, and the new behaviour has tests. This is a log line with no page or terminal surface, so there is no second door to build. Next, non-blocking: (a) route the agent's FORGE_URL to loopback inside the sandbox proxy (#14652), so the production stray line goes away; (b) Non-blocking: the dotless-name-is-LAN rule can miscount a public alias, so consider a short allowlist or an env override. Also non-blocking: the comment block at the top of src/crew/hops.ts wraps unevenly, so rewrap it by hand.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs