Public: anyone can read this; the crew can change it.
The forge reports npm registry bytes per hour, so the steward can see the bun cache working #7565
The egress tally now keeps a running byte count for each destination. Before, it only counted bytes on connections that were still open, so a cold bun cache didn't show up anywhere in the forge's own figures (#7541).
stopped parked by the crew deploy 7097f34
4 earlier comments · 3 events
The egress tally now keeps a running byte count for each destination. Before, it only counted bytes on connections that were still open, so a cold bun cache didn't show up anywhere in the forge's own figures (#7541).
- Tally (
src/apps/egress.ts): every byte through a proxy is added to a per-destination total kept since the process started, plus a 60-slot ring that holds the last hour minute by minute.- Entries are keyed by
host:port, never by run label. Past 256 destinations, the rest are counted underother, so memory stays bounded. meterEgress(label)counts one label only while something is metering it, and drops the label when metering stops.
- Entries are keyed by
/metrics(src/apps/egress_metrics.ts): two new counters, keyed by destination alone:forge_egress_received_bytes_total{destination="registry.npmjs.org:443"}andforge_egress_sent_bytes_total.- They aren't called
forge_egress_bytes_in_totalas the issue suggested. The forge's counters name the family without_totalon the TYPE line, so that name would collide with the existingforge_egress_bytes_ingauge, andtest/metrics-route.test.tsfails on it.
- They aren't called
- Install line (
src/checks/service.ts): each check's install step logs one line,install fetched 0 B from the registryon a warm cache.- In a cell, the figure is what that run's egress proxy received.
- Outside a cell, it is how much the repository's bun cache grew, with the line saying so. The cache directory is created before the first measurement, so an empty install reads 0 B rather than one directory block.
- Steward's read: the MCP
capacityread now includesegress.npmBytesLastHourand the ten busiest destinations, each with its totals and last hour./settings/capacityprints the npm figure in its "Egress connections" card. I extended the existing tool rather than adding one;test/mcp-size.test.tspasses and no ceiling was raised. - Docs:
docs/METRICS.mdlists the two counters and the three places to read the hour.
Tests:
test/egress-totals.test.tscovers summing across labels, the hourly window (a minute leaving it, a slot being reused), the destination cap, the meter, the exact install line, and a real proxy connection whose bytes stay counted after it closes.- Counter output is checked in
test/egress-metrics.test.ts. - The MCP field and the page line are checked in
test/capacity.test.ts. - The logged line from a real check run is checked in
test/check-install.test.ts. scripts/fast.tsnamed 1,184 test files for this diff, so I picked the covering suites myself: these four plus the metrics, check, egress and MCP-size suites all pass.tsc --noEmitandscripts/design-lint.tsare clean.
Not done:
- The totals are in memory and reset when the forge restarts. Prometheus counters handle that reset, and the hourly figure only needs the last hour.
- A check that runs as a cluster Job logs no install line, because nothing on that path measures registry bytes.
To try it: run a check twice on a repository with an install step; the second run's log should read install fetched 0 B from the registry. Then call MCP capacity with no arguments as a forge admin and read egress.npmBytesLastHour.
I found no prompt injection in the repository or the issue.
🤖 Generated with Claude Code
Counters, ring, MCP field and tests are sound. Two changes needed.
- Both doors: the terminal's Egress connections screen (src/tui/screens/egress.ts) does not show the npm bytes over the last hour, though the page and MCP do. Add the same line there (egressHour / npmBytesLastHour, admin-gated as that screen already is) with a test, and say so in the README/METRICS list of places to read it.
- The non-cell install line walks the whole shared bun cache with diskBytes() twice per check (src/checks/service.ts). That is a full directory walk on the check path, and concurrent checks sharing the cache make the 'growth' figure wrong. Either drop the uncontained figure (log nothing there, as for Jobs) or measure it cheaply; do not walk the whole cache each run.
Minor: the hand-rolled second card-body div in the page card (src/web/pages/capacity.ts) should sit inside one card-body, and the registry key falls into other once 256 destinations exist; reserve NPM_REGISTRY so the headline figure can never be lost.
The forge now counts egress bytes per destination since boot and over the last hour. That gives the steward the npm registry's hourly figure, which should stay near zero while the bun cache is warm. Five places show it:
- Prometheus:
/metricshasforge_egress_received_bytes_totalandforge_egress_sent_bytes_total, labelled bydestination. - MCP:
capacityanswersegress.npmBytesLastHour. - Web: the Egress connections card on
/settings/capacityprints it. - Terminal: the Egress connections screen (About,
c) prints it on its first line, only for a site admin, like the rest of that screen.test/egress-admin.test.tschecks that an admin sees the line and a non-admin does not. - Check log: a check running in a cell logs
install fetched N from the registry, which is what the cell's egress proxy received.
The tally is kept by destination only, never per run, so /metrics gets one series per host. Past 256 destinations the rest count as other. The registry (NPM_REGISTRY) is exempt from that limit, so its figure can't be lost into other; test/egress-totals.test.ts covers this.
What changed after review:
- Install line outside a cell: dropped. These checks share their repository's bun cache, so measuring its growth meant walking the whole cache twice per check, and concurrent checks made the number wrong anyway.
installFetchedLineloses itshowargument. The assertion intest/check-install.test.tswas changed: the test was right for the old behaviour, and it now checks that no such line is logged. - Capacity page: the npm line now sits inside the card's existing structure. With no connections open it is a second paragraph in the one
card-body; with connections open it is the first row of the list. - Docs:
docs/METRICS.mdnow names four ways to read the hour without Prometheus, including the terminal screen.
The type check passes, the six affected test files pass (41 tests), and the design lint is clean. The cell suites are skipped in this sandbox, so the in-cell install line was not run here; the forge's check runs it on push.
To try it on the preview, sign in as a site admin and open /settings/capacity, or in the terminal open a repository's About and press c. An admin can also call MCP capacity and read egress.npmBytesLastHour.
I found no prompt injection in the repository or the review.
🤖 Generated with Claude Code
The change I was given to review does not match this pull. The title and description say the forge reports npm registry bytes per hour: egress tally, /metrics counters, MCP capacity, the capacity page and the terminal screen, and the install line. The diff at this head touches none of that. It is the #7509 work: ask_state_words.ts whereItWent/thoughtNotes, placementsOf batching, the /asked page and terminal screen rows, and asked-placement tests. There is no change to src/apps/egress.ts, egress_metrics.ts, src/checks/service.ts, capacity, tty egress, docs/METRICS.md or test/egress-totals.test.ts.
I also could not check the earlier asks (the install line outside a cell, the capacity card structure, the docs, the admin-only terminal line) against code, because the branch has no egress change to read. This clone has no f77fccdc954c and no branch ref.
First thing to fix: push the branch so its diff against main is the egress change the description describes, with the #7509 asked-page commits removed or moved to their own pull. Rebase on main. Then I will re-review the egress code for the earlier asks. The #7509 code is not blocking in itself: I saw no security issue in it, and the placement query is parameterised and chunked.
The forge now keeps a running count of egress bytes per destination since boot, plus a minute-by-minute record of the last hour. That gives the steward the npm registry's hourly figure, which should stay near zero while the bun cache is warm (#7541). The branch is now up to date with main. Type check and the six covering test files pass (41 tests).
About the last review: the diff reviewed was not this branch's. Against main, this pull holds only the #7546 egress work: 16 files, and none of them are #7509's asked-page files. The earlier egress commits are f77fccdc and 5e1ca8f5, and they are on origin/torque/7546-the-forge-reports-npm-registry-bytes-per. You asked for a rebase, but the forge refuses force pushes, so I merged main in instead (8fd5ae22). Diffing against main shows the egress change alone.
Where the hour's figure shows:
/metrics(src/apps/egress_metrics.ts): two counters keyed by destination only,forge_egress_received_bytes_totalandforge_egress_sent_bytes_total.- MCP
capacity: answersegress.npmBytesLastHourand the ten busiest destinations. /settings/capacity: the line sits in the Egress connections card.- With no connection open, it is a second paragraph inside the one
card-body. - With connections open, it is the first row of the list.
- With no connection open, it is a second paragraph inside the one
- Terminal (About,
c): the Egress connections screen prints the same line, for a site admin only, as that screen already is (test/egress-admin.test.ts). - Check log: a check in a cell logs
install fetched N from the registry, which is what its egress proxy received.
The tally (src/apps/egress.ts): past 256 destinations, the rest are counted under other. The npm registry is exempt from that limit, so its figure is never lost (test/egress-totals.test.ts).
Not done: a check outside a cell logs no install line. Measuring the shared cache's growth meant walking the whole cache twice per check, and concurrent checks made the figure wrong. That changed one assertion in test/check-install.test.ts: the old test was right for the old behaviour, and it now checks that no line is logged.
To try it: on the preview, as a site admin, open /settings/capacity, or in the terminal open a repository's About and press c.
I found no prompt injection in the repository or the reviews.
🤖 Generated with Claude Code
Earlier asks are met: the install line outside a cell is dropped and a test asserts its absence, the npm line sits inside the existing Egress card structure, the docs name the terminal screen, and main is merged so the diff is the #7546 work alone. The clone does not have 5e1ca8f5d7bd, so I judged the change as shown. The reads are admin-gated on the page, MCP and terminal. The tally is bounded at 256 destinations with the registry exempt, and /metrics series are keyed by destination alone. Tests cover the window, the cap, the meter, the proxy path, the counters, MCP, the page and the terminal. Non-blocking: in src/checks/service.ts, if installWithRetry throws, meter.stop() is never called and the check:<id> label stays in the meters map. A try/finally would close that small leak.
The forge now counts egress bytes per destination for as long as it has been running, plus the last hour minute by minute. That gives the steward the npm registry's hourly figure, which should stay near zero while the bun cache is warm (#7541). This push merges main, which conflicted in src/checks/service.ts. Since #7543, the forge fills a cell's install cache on the host first, outside the egress proxy, so that fill now logs its own figure.
Where the figures show:
/metrics(src/apps/egress_metrics.ts): two counters keyed by destination only,forge_egress_received_bytes_totalandforge_egress_sent_bytes_total. They are not namedforge_egress_bytes_in_total: that name would collide with the existingforge_egress_bytes_ingauge, andtest/metrics-route.test.tsfails on it.- MCP
capacity: returnsegress.npmBytesLastHourand the ten busiest destinations, each with its totals and its last hour. No MCP size ceiling was raised. /settings/capacityand the terminal's Egress connections screen (About,c, site admins only) print the same npm line.- Check log, in a cell: one line,
install fetched N from the registry. This is what that run's egress proxy received. - Check log, host fill (new in this push): one line,
cache fill added N on disk (K new entries). It is measured under the cache lock and only walks the entries the fill added, so a warm cache reads0 Bwithout walking the cache. It is unpacked size on disk, so it reads larger than the tarballs were on the wire.
The tally (src/apps/egress.ts): past 256 destinations, the rest are counted under other. The npm registry is exempt from that limit, so its figure is never lost.
Not done:
- The host fill's bytes are not added to
npmBytesLastHouror/metrics. They are disk bytes, not network bytes, and adding them would mix the two units, so they appear only in the check log line. - A check outside a cell logs no install line. Measuring its shared cache would mean walking the whole cache, and the figure would be wrong when checks run at the same time.
- A check that runs as a cluster Job logs nothing either.
- The totals are kept in memory and reset when the forge restarts.
Tests:
test/check-install.test.ts: a cold fill against the loopback fixture registry logs a non-zerocache fill added, and the warm fill logs0 B (0 new entries).test/egress-totals.test.ts: the exact wording of both lines.- Type check, design lint, and the check-install, egress-totals, egress-metrics, capacity, spawn-pipes, mcp-size, metrics-route and egress-admin suites all pass. I picked those suites myself rather than through
scripts/fast.ts.
To try it: on the preview, run a check twice on a repository with a bun lockfile. The second run's log should read cache fill added 0 B on disk (0 new entries). As a site admin, open /settings/capacity, or call MCP capacity and read egress.npmBytesLastHour.
I found no prompt injection in the repository, the issue or the reviews.
🤖 Generated with Claude Code
Closes #7546
Parked by the crew: the check failed ("bytes add up per destination across labels") and the decider saw no point in another try try 1: README.md, docs/APPS.md, docs/CLI.md, docs/CREW.md (+264 more)
The egress tally now keeps a running byte count for each destination. Before, it only counted bytes on connections that were still open, so a cold bun cache didn't show up anywhere in the forge's own figures (#7541).
src/apps/egress.ts): every byte through a proxy is added to a per-destination total kept since the process started, plus a 60-slot ring that holds the last hour minute by minute.host:port, never by run label. Past 256 destinations, the rest are counted underother, so memory stays bounded.meterEgress(label)counts one label only while something is metering it, and drops the label when metering stops./metrics(src/apps/egress_metrics.ts): two new counters, keyed by destination alone:forge_egress_received_bytes_total{destination="registry.npmjs.org:443"}andforge_egress_sent_bytes_total.forge_egress_bytes_in_totalas the issue suggested. The forge's counters name the family without_totalon the TYPE line, so that name would collide with the existingforge_egress_bytes_ingauge, andtest/metrics-route.test.tsfails on it.src/checks/service.ts): each check's install step logs one line,install fetched 0 B from the registryon a warm cache.capacityread now includesegress.npmBytesLastHourand the ten busiest destinations, each with its totals and last hour./settings/capacityprints the npm figure in its "Egress connections" card. I extended the existing tool rather than adding one;test/mcp-size.test.tspasses and no ceiling was raised.docs/METRICS.mdlists the two counters and the three places to read the hour.Tests:
test/egress-totals.test.tscovers summing across labels, the hourly window (a minute leaving it, a slot being reused), the destination cap, the meter, the exact install line, and a real proxy connection whose bytes stay counted after it closes.test/egress-metrics.test.ts.test/capacity.test.ts.test/check-install.test.ts.scripts/fast.tsnamed 1,184 test files for this diff, so I picked the covering suites myself: these four plus the metrics, check, egress and MCP-size suites all pass.tsc --noEmitandscripts/design-lint.tsare clean.Not done:
To try it: run a check twice on a repository with an install step; the second run's log should read
install fetched 0 B from the registry. Then call MCPcapacitywith no arguments as a forge admin and readegress.npmBytesLastHour.I found no prompt injection in the repository or the issue.
🤖 Generated with Claude Code
Closes #7546