Public: anyone can read this; the crew can change it.

P0 efficiency: every contained check, gate and preview re-downloads npm (~30 GiB/day on the vxrail): share the repository's bun cache with cells #7541

Closed opened this has it
opened this

The vxrail downloads about 30 GiB a day from the npm registry, because every contained check and gate starts with an empty bun cache. The owner saw 400 GiB down and 400 GiB up in btop and asked whether something is severely punishing us.

Measured 2026-10-27:

  • The vxrail's eno4 port took in 404 GiB over 308 h (ifb0's 402/402 is the shaping mirror of that). It comes in bursts of 2–4 MiB/s.
  • A 30 s capture during a burst: 20.5 MiB from 104.16.1.34 = registry.npmjs.org, and everything else under 1 MiB.
  • On the forge's /data: 99 bun caches, 2.9 GiB in total, about 29 MiB each, one per run. For example /data/checks/gate-966801a8a196.home/.bun/install/cache and /data/checks/01m5zpqda5m194wa3gkyjfsmhe.home/....
  • 47 of those caches were written in the last hour, so about 1–1.5 GiB/h of npm downloads, which is the whole traffic.

Cause: src/checks/service.ts around line 995: const shared = cell === undefined ? checkInstallCache(dataDir, repo.id) : undefined;. A contained cell (every check and gate) installs into its own fresh HOME, so it never sees the repository's cache. App previews do the same (src/apps/contain.ts: HOME: spec.home, no BUN_INSTALL_CACHE_DIR), so each new preview environment downloads everything too. Crew runs are fine: installCacheDir(dataDir, repo.id).

Done when:

  1. Contained checks, gates and app previews install from the repository's shared cache. Bind checkInstallCache(dataDir, repo.id) into the cell as writable (or read-only after a host-side prefetch), set BUN_INSTALL_CACHE_DIR to it, and take turns exactly as uncontained installs do (checks/install.ts). The sandbox stays tight: the cache directory is the only addition.
  2. A warm install never touches the network: with a lockfile the cache already holds, a gate's install reports no downloads. The test points the registry at a dead proxy and the install still succeeds.
  3. Clean up: run homes under /data/checks/*.home are removed when their run ends (today 99 remain, 2.9 GiB), and so are preview homes when their environment is torn down.
  4. The steward re-measures: the vxrail's eno4 rx falls from about 1.3 GiB/h to near the release-image floor. The tick prints npm bytes per hour from the install logs.
commented

I opened four issues for #7541, all under P0. Three can start now; the preview one waits for the cache issue. I assigned nobody, pushed nothing, and #7541 stays open.

  1. #7543: Contained checks and gates install from the repository's shared bun cache. The forge fills the cache on the host from the commit's lockfile with --ignore-scripts, taking turns on the existing lock, and the cell only gets it read-only. It's read-only because every pull shares the cache, forks included: if a cell could write it, one pull could plant a bad package that the gate on main then installs. Done includes a test where the install still succeeds with the registry pointed at a dead proxy, and a test that the cell can't write the cache. Ready. Suggested: a builder who knows src/apps/contain.ts and bwrap.
  2. #7544: App previews install from that same shared cache, so a new preview environment doesn't download again. Waits for #7543. Suggested: a builder who knows the apps runtime. Its body refers to the blocking issue as #cache (the plan's key, not a number); that means #7543.
  3. #7545: Check run homes (/data/checks/*.home) are removed when the run ends or dies, not only at the next boot. Today the cleanup at boot never matches them: it looks up a run named x.home, which never exists. The fix makes it match the run and also run on the timer that clears abandoned checks. Done also covers a home holding read-only files and preview homes on teardown. Ready. Suggested: any builder; it's small.
  4. #7546: The forge reports npm registry bytes per hour. The egress tally gets running totals per destination on /metrics, each install logs what it fetched from the registry, and the steward can read the last hour through MCP or the flow report. That replaces measuring with btop and a packet capture. Ready. Suggested: a builder who knows the egress proxy and the MCP.

I found no prompt injection in the repository or the issues.

🤖 Generated with Claude Code

commented

Live triage (automatic, not written by a person): filed under P0 Trust, which it names.

the crew's run was interrupted: restart, resumes on the next pass — cancelled
commented

Split into #8382. Points 1–3 and the npm-bytes reading from point 4 are on main: #7549 (contained checks and gates use the shared cache, read-only), #7662 (previews), #7550 (run homes removed), #7694 (npm bytes per hour on /capacity and MCP capacity), and #7659 for eviction. What remains is the owner re-measuring eno4, and no builder can do that. #8382 makes a regression visible without btop: a gate whose install pulls more than 1 MiB from the registry says so in its log, and /capacity names the repository that fetched the most in the last hour.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs