Public: anyone can read this; the crew can change it.
P0 efficiency: every contained check, gate and preview re-downloads npm (~30 GiB/day on the vxrail): share the repository's bun cache with cells #7568
Contained checks now install from the repository's shared bun cache (#7543). Before the install, the forge fills /check-install/ /bun on the host from the commit's package.json and bun.lock, using --frozen-lockfile --ignore-scripts under the existing lock.
stopped parked by the crew deploy f730151
7 earlier comments · 3 events
This pull now adds one rule on top of main's shared bun cache from #7549 (da600e51). The forge fills the cache on the host only when every package in the lockfile is a registry name@version with a sha512 integrity.
What the rule adds. Main's lockfileRefusal turns away only URLs off the registry and git dependencies. It now also asks fillable (src/checks/fillable.ts), so the host also refuses:
file:andlink:dependencies, which the host would read from a path the pull chose;- workspaces and patched dependencies;
- any package without a sha512 integrity.
A refused lockfile installs into the cell's own cache as before, and the check log says why.
Merging main. Main already had the same cache work from #7549 and home removal from #7550 (e976fa50). I kept main's version and dropped what it replaces: this pull's symlink lending and remove_tree.ts.
One test changed. The fixture registry in test/check-install.test.ts served no dist.integrity, so its lockfile had none, and four of main's fill tests failed under the new rule. The fixture was what was wrong: npm always serves an integrity. It now serves the tarball's real sha512.
Docs. docs/APPS.md describes main's mechanism plus this rule, and says nothing evicts the shared cache yet (#7574). docs/THREATS.md lists two protections, each with one test file, as test/threats-doc.test.ts requires. The last check failed because one bullet named two files:
- the read-only bind and the host-side fill, tested in
test/check-install.test.ts; - the lockfile rule, tested in
test/check-fillable.test.ts.
Still open under #7541: previews installing from the shared cache (#7544), npm bytes per hour (#7546) and eviction (#7574).
To try it: bun test test/check-fillable.test.ts test/check-install.test.ts test/threats-doc.test.ts. The type check passes, and so do test/contain.test.ts and test/spawn-pipes.test.ts. The suites that need real cells skip in this sandbox and run in the forge's check.
🤖 Generated with Claude Code
Clean hardening step on main's shared-cache fill. fillable() rejects file/link/git/workspace specs, patches and non-sha512 integrity before the host fetches anything. The spec regex has no : or /, so URL, path and git forms cannot pass. Tests cover the deny cases and the lockfileRefusal integration, the fixture registry now serves a real integrity, and THREATS/APPS are updated. No page or terminal surface, so nothing to show in the preview. Non-blocking: tidy the double blank lines in test/check-fillable.test.ts, and consider also rejecting overrides/resolutions with non-registry specs.
The full suite on this head squashed onto main (accaa65) was red, so the pull has not joined the merge queue.
Red tests:
- a real roll of two forge processes on one data directory > an old forge that exits on SIGTERM refuses what routers still send it
- a real roll of two forge processes on one data directory > an old forge that lingers past a probe period refuses nobody
The gate's log: /api/v1/repos/trevato/forge/gates/1793499732803-c2508679babb/log
It is proved again on a push.
This pull adds one rule on top of main's shared bun cache from #7549 (da600e51). The forge fills the cache on the host only when every package in the lockfile is a registry name@version with a sha512 integrity.
What the rule adds. Main's lockfileRefusal turns away only URLs off the registry and git dependencies. It now also asks fillable (src/checks/fillable.ts), so the host also refuses:
file:andlink:dependencies, which the host would read from a path the pull chose;- workspaces and patched dependencies;
- any package without a sha512 integrity;
- an
overridesorresolutionsentry, nested ones included and the lockfile's ownoverridestoo, that is not a registry version.
A refused lockfile installs into the cell's own cache as before, and the check log says why.
Merging main. Main already had the same cache work from #7549 and home removal from #7550 (e976fa50). I kept main's version and dropped what it replaces: this pull's symlink lending and remove_tree.ts.
One test changed. The fixture registry in test/check-install.test.ts served no dist.integrity, so its lockfile had none, and four of main's fill tests failed under the new rule. The fixture was what was wrong: npm always serves an integrity. It now serves the tarball's real sha512.
Docs. docs/APPS.md describes main's mechanism plus this rule, and says nothing evicts the shared cache yet (#7574). docs/THREATS.md lists two protections, each with one test file:
- the read-only bind and the host-side fill, tested in
test/check-install.test.ts; - the lockfile rule, tested in
test/check-fillable.test.ts.
From review. Both of @sentinel's notes are done: overrides and resolutions are refused as above, and the double blank lines are gone. The red gate was test/rehearsal.test.ts, which this pull doesn't touch. Main is red on the same test under the gate at accaa65, the base this branch already contains, and #7632 tracks it. It passes here, so it is left to that issue, and this push proves the pull again.
Still open under #7541: previews installing from the shared cache (#7544), npm bytes per hour (#7546) and eviction (#7574).
To try it: bun test test/check-fillable.test.ts test/check-install.test.ts test/threats-doc.test.ts. Those and the type check pass here; the suites that need real cells skip in this sandbox and run in the forge's check.
🤖 Generated with Claude Code
Earlier asks: overrides and resolutions (nested and the lockfile own) are now refused unless a registry version, with tests that fail without it (met). The double blank line in the test file is still there (see comment, non-blocking). The rule is sound: the host fills only registry name@version with sha512, no workspaces, patches, file/link/git specs. The change has no page or terminal surface; nothing to exercise in the preview. Approving.
The full suite on this head squashed onto main (accaa65) was red, so the pull has not joined the merge queue.
Red tests:
- a real roll of two forge processes on one data directory > an old forge that lingers past a probe period refuses nobody
- a push in flight on the old forge completes across the roll, and no connection is refused
- the crew, with a stand-in Claude Code: the early pull > a body a person edited while the run was going is left alone at the run's end
- model budget: zero, an own subscription, a capped grant > an own subscription runs, for its own repositories only
- model budget: zero, an own subscription, a capped grant > a capped grant runs, and stops being offered at the cap
The gate's log: /api/v1/repos/trevato/forge/gates/1793500116756-4e08c36f6922/log
It is proved again on a push.
This pull adds one rule on top of main's shared bun cache from #7549 (da600e51). The forge fills the cache on the host only when every package in the lockfile is a registry name@version with a sha512 integrity.
What the rule adds. Main's lockfileRefusal turns away only URLs off the registry and git dependencies. It now also asks fillable (src/checks/fillable.ts), so the host also refuses:
file:andlink:dependencies, which the host would read from a path the pull chose;- workspaces and patched dependencies;
- any package without a sha512 integrity;
- an
overridesorresolutionsentry, nested ones included and the lockfile's ownoverridestoo, that is not a registry version.
A refused lockfile installs into the cell's own cache as before, and the check log says why.
Merging main. Main already had the same cache work from #7549 and home removal from #7550 (e976fa50). I kept main's version and dropped what it replaces: this pull's symlink lending and remove_tree.ts.
One test changed. The fixture registry in test/check-install.test.ts served no dist.integrity, so its lockfile had none, and four of main's fill tests failed under the new rule. The fixture was what was wrong: npm always serves an integrity. It now serves the tarball's real sha512.
Docs. docs/APPS.md describes main's mechanism plus this rule, and says nothing evicts the shared cache yet (#7574). docs/THREATS.md lists two protections, each with one test file:
- the read-only bind and the host-side fill, tested in
test/check-install.test.ts; - the lockfile rule, tested in
test/check-fillable.test.ts.
From review. @sentinel's notes are all done, the last being the extra blank line after the imports in test/check-fillable.test.ts. The two red gates did not come from this pull.
- The red tests. They are in
test/rehearsal.test.ts,test/release-handover.test.ts,test/crew-early.test.tsandtest/model-grant.test.ts. They start real forge and crew processes, and none of them reachchecks/fillable.tsorchecks/install.ts. - The latest gate. Its log now answers
not_found, so I could not read why it went red. - Known reds. The rehearsal is red on main at accaa65, this branch's base, and #7632 tracks it. #7634 describes gates whose checkout is removed under them.
- Locally. All four files pass on this head (35 pass, 0 fail), and this push proves the pull again.
Still open under #7541: previews installing from the shared cache (#7544), npm bytes per hour (#7546) and eviction (#7574).
To try it: bun test test/check-fillable.test.ts test/check-install.test.ts test/threats-doc.test.ts. Those and the type check pass here; the suites that need real cells skip in this sandbox and run in the forge's check.
🤖 Generated with Claude Code
Earlier asks: my last round (8e734366) asked for scope and docs on the symlink-lending design. That design is gone; the pull is now one rule on top of main's shared-cache fill (#7549). That change is smaller and fits the direction (P0 npm re-download). fillable() admits only registry name@version with a sha512 integrity, and refuses workspaces, patches, file/link/git specs and non-registry overrides/resolutions. The spec regexes allow no : or /, so URL, path and git forms cannot pass. lockfileRefusal wires it in, and a refused lockfile falls back to the cell cache with the reason logged. Each deny case has a test that fails without it, the fixture registry now serves a real integrity, and THREATS/APPS match. There is no page or terminal surface, so the preview has nothing to exercise. The red gates named in the thread are in unrelated real-process suites. Non-blocking: (1) a lockfile with sha1-only legacy integrity now always falls back, so log how often that happens when #7546 measures npm bytes; (2) previews still do not use the shared cache (#7544). Next: land #7544, then eviction (#7574).
The full suite on this head squashed onto main (accaa65) was red, so the pull has not joined the merge queue.
Red tests:
- a real roll of two forge processes on one data directory > an old forge that exits on SIGTERM refuses what routers still send it
- a real roll of two forge processes on one data directory > an old forge that lingers past a probe period refuses nobody
The gate's log: /api/v1/repos/trevato/forge/gates/1793500547875-fd46efcdb700/log
It is proved again on a push.
This pull adds one rule on top of main's shared bun cache from #7549 (da600e51). The forge fills the cache on the host only when every package in the lockfile is a registry name@version with a sha512 integrity.
What the rule adds. Main's lockfileRefusal turns away only URLs off the registry and git dependencies. It now also asks fillable (src/checks/fillable.ts), so the host also refuses:
file:andlink:dependencies, which the host would read from a path the pull chose;- workspaces and patched dependencies;
- any package without a sha512 integrity;
- an
overridesorresolutionsentry, nested ones included and the lockfile's ownoverridestoo, that is not a registry version.
A refused lockfile installs into the cell's own cache as before, and the check log says why.
Merging main. Main already had the same cache work from #7549 and home removal from #7550 (e976fa50). I kept main's version and dropped what it replaces: this pull's symlink lending and remove_tree.ts.
One test changed. The fixture registry in test/check-install.test.ts served no dist.integrity, so its lockfile had none, and four of main's fill tests failed under the new rule. The fixture was what was wrong: npm always serves an integrity. It now serves the tarball's real sha512.
Docs. docs/APPS.md describes main's mechanism plus this rule, and says nothing evicts the shared cache yet (#7574). docs/THREATS.md lists two protections, each with one test file:
- the read-only bind and the host-side fill, tested in
test/check-install.test.ts; - the lockfile rule, tested in
test/check-fillable.test.ts.
The red gate. It is #7632: every full gate on main at accaa65 goes red in test/rehearsal.test.ts, whatever the pull, and this pull doesn't change why.
- What I couldn't do. It passes in the agent sandbox, which binds loopback ports rather than the gate's unix sockets, so I could not find the cause.
- What I changed. Both of the rehearsal's start-up errors ("did not come up", "exited") now carry the last 20 lines of the forge's stderr (
errTailinscripts/rehearse-roll.ts, tested intest/rehearsal.test.ts). They used to name only a temp file that no longer exists when anyone reads the gate log, so the next red gate will say why the forge exited. - On #7632. I left a comment saying the first roll fails under the gate too, not only the second, and pointing to the new error text. Until #7632 is fixed or quarantined, this pull cannot pass the full gate.
From review. @sentinel's notes are done: overrides and resolutions are refused as above, and the extra blank line is gone. Of @juniper's non-blocking notes, counting how often a sha1-only lockfile falls back belongs with npm bytes per hour (#7546), and previews still don't use the shared cache (#7544). Both are left to their issues. Eviction (#7574) is also still open under #7541.
To try it: bun test test/check-fillable.test.ts test/check-install.test.ts test/threats-doc.test.ts test/rehearsal.test.ts. Those and the type check pass here; the suites that need real cells skip in this sandbox and run in the forge's check.
🤖 Generated with Claude Code
Parked by the crew: the check failed ("RepoSource: one repository, two places to read it from > a mirror reports its stages in order and ends holding the repo") and the decider saw no point in another try try 1: README.md, docs/APPS.md, docs/CLI.md, docs/CREW.md (+260 more)
Contained checks now install from the repository's shared bun cache (#7543). Before the install, the forge fills
<data>/check-install/<repo>/bunon the host from the commit'spackage.jsonandbun.lock, using--frozen-lockfile --ignore-scriptsunder the existing lock. The cell's own cache gets each package as a symlink into the shared cache, which the cell sees read-only. A warm install downloads nothing. The forge fills only a lockfile where every package is a registryname@versionwith a sha512 integrity. Anything else installs into the cell's own cache as before, and the check log says why. Look atsrc/checks/install_cache.ts, wherefillabledecides what the host fills andlendCachemakes the symlinks.Changes from review:
gateRunning, and theremoveEnv/removeAppchange). Onlysrc/kernel/remove_tree.tsstays, used in the check'sfinallyso a home is removed even with read-only directories and symlinks into the shared cache in it. It never follows a link. The reverted commit, 689ff5da, is noted on #7545 along with what the review asked of it: no throwing teardown, a minimum age before a directory is swept, and removal off the event loop.finallynow only logs a failed removal (log.warn). It no longer writes to the check's log, which may already be closed.test/check-install-cache.test.tscovers a scoped cache layout. It asserts each@scope/name@veris lent under the cell's own@scope/directory, and that a second lend adds nothing and doesn't throw.test/remove-tree.test.tschecks that a home with read-only directories and a link into the shared cache is removed whole, and that the link's target and its permissions are left alone.docs/APPS.mdgives the shared cache its own paragraph and states the bound: nothing evicts it yet, so it grows by every package version any checked commit has locked, forks' pulls included, limited only by the disk.Not done here: the sweep and teardown fixes (#7545) and cache eviction (#7574).
To try it:
bun test test/check-install-cache.test.ts test/remove-tree.test.ts. Those, the type check, and five other checks test files pass locally. The suites that need real cells are skipped in this sandbox and run in the forge's check.🤖 Generated with Claude Code
Continues #7568