Builders can run tsc and the socket tests inside their sandbox: two builders shipped untested code tonight because they could not #1369

Builders' sandboxes now allow unix sockets, so tests that boot the forge (checks, api, web…) can run before a push. The brief says in two lines what the shell can and cannot run. The last checkbox is not proven yet: this run's sandbox was set up by the old settings, so no enforced crew run has run the probe so far.

What blocked the builders. Inside this run's sandbox, listening on a unix socket fails with EPERM, both under the clone and under $TMPDIR. Loopback TCP works. The refusal comes from Claude Code's Linux seccomp filter, not from bubblewrap. Seccomp cannot allow some socket paths and refuse others, so the only switch is sandbox.network.allowAllUnixSockets. scripts/standby/host/forge-sandbox.seccomp.json does not exist in this repository, so nothing changed there.

What changed

• src/crew/claude.ts: the settings now set allowAllUnixSockets: true.
• src/crew/sandbox.ts: with sockets allowed, the shell can only reach a socket whose path it can see. hostRunDirs() adds the machine's real runtime directory to denyRead: /run on Linux, /private/var/run on macOS. That is where a container runtime's socket, the system bus and a pod's service-account token live. Everything else is unchanged: the data directory, the system temp directory (with the cells' sockets) and ~/.ssh stay denied, and the network stays separate from the host's.
• Type check: no install step is needed in the runner. In the sandbox, bun install --frozen-lockfile fetched @types/bun and typescript from the registry in 0.6 s, and bun x tsc --noEmit then passed. The image has no package cache, so this needs the registry rather than working offline. The brief names the command.
• src/crew/brief.ts: SANDBOX_LINES is the two-line can/cannot statement in the system prompt, replacing the old one-line note.
• scripts/sandbox-probe.ts: binds a server the way test/checks.test.ts does, on a unix socket under $TMPDIR, one under the clone, and a loopback port. It prints one verdict per line and exits 1 if any is refused. In this run it printed two refusals and loopback ok.
• test/crew-sandbox.test.ts: new tests check the settings flag, that /run is denied, the two brief lines in the prompt, and the probe's three verdicts.
• scripts/gate.ts: separate commit. #1298 left test/chaos.test.ts, test/fuzz-found.test.ts and test/fuzz-parsers.test.ts in neither GATE nor EXCLUDED, so test/check-list.test.ts fails on main. They are now in EXCLUDED under #1244, because a file joins the gate only through a pull whose check is green.

Test results here. The type check passes, and so do check-list and shards. These fail here only with the EPERM this pull removes: the new probe test, plus studio, reserved-names and crew-brief-audience. The check lane allows unix sockets, so it can run them all. hollow was not run on its own; it was run with studio, and its result was not separated out. Commit 11c2e2f2 imports SANDBOX_LINES from the next commit, so that one commit does not type-check by itself.

Not done

• No loopback-port version of the tests: the forge's hook socket is a production unix socket, and changing it to TCP would change the product.
• The stray dotfiles the sandbox leaves in a clone (.bashrc, .gitconfig, .claude/…) could be picked up by the wrap-up's git add -A. They belong in CLONE_EXCLUDES in another issue.

To try it: once this is deployed, a crew run on this repository runs bun install && bun x tsc --noEmit && bun scripts/sandbox-probe.ts. The run log should show three ok lines, which ticks the last checkbox. After that, bun test test/checks.test.ts should reach a verdict.

🤖 Generated with Claude Code

Closes #1319

The builder's brief now has two lines on what its sandboxed shell can and cannot run. It no longer has to report "I could not run" for something it could have run.

The conflict. While this pull was open, main changed how the sandbox handles the socket tests (#1293 and the PRs before it). The sandbox no longer lets unix sockets through. Instead every crew run sets FORGE_TEST_SOCKET=port, and the forge listens on a loopback port (src/git/hook-address.ts). This branch's first approach went the other way: it turned the unix-socket filter off and denied /run and /var/run by name. Main's design replaces that, so main's src/crew/sandbox.ts is taken unchanged and that part of this branch is dropped.

What the pull carries now:

• src/crew/brief.ts: SANDBOX_LINES puts main's paragraph into two lines.

- What the shell can run: bun install (first, when node_modules is missing), bun x tsc --noEmit, and bun test with FORGE_TEST_SOCKET=port already set. It also names which suites are skipped in that mode and are run by the check instead. - What it cannot do: reach hosts other than the forge and the registry, write outside the clone and $TMPDIR, read the forge's or the machine's data, bind a unix socket, or signal a process started in an earlier command. It also says the sandbox cannot be left, and how to start the app and stop it in one command. - The wording is main's, including "the forge runs the whole check on your push and sends you its log".

• scripts/sandbox-probe.ts: listens on a unix socket under $TMPDIR, one under the clone, and a loopback port, connects to each, and prints a verdict per listener plus the value of FORGE_TEST_SOCKET. It exits 1 only when the loopback port is refused, because a sandboxed run is expected to refuse the unix sockets.
• test/crew-sandbox.test.ts: two tests. One checks that the probe reports all three listeners and the loopback one answers. The other checks that the brief carries both lines next to each other in the system prompt.
• README.md: the Sandbox paragraph says what a builder can run inside the sandbox and points to the probe.

bun x tsc --noEmit is clean. With FORGE_TEST_SOCKET=port, test/crew-sandbox.test.ts and test/crew-loop.test.ts pass 33 of 33. A first run alongside the type check failed 2 of them; I did not record which, and a run on its own passed all 33.

To try it: bun scripts/sandbox-probe.ts from a crew shell.

🤖 Generated with Claude Code

The full gate on land was red; the pull is open again. Red tests:

• the gate's list > every test/*.test.ts is in exactly one list, and every entry exists
• the gate's list > every test/*.test.ts is in exactly one list, and every entry exists [0.60ms] [shard 182]

The crew brief now tells builders in two lines what their sandboxed shell can run and what it cannot. This repository also gets a probe script that checks which kinds of listener work, so a builder doesn't write "I could not run" about something they could. The sandbox itself is unchanged: it still lets no unix sockets through. An earlier description of this pull said otherwise, and this one replaces it.

• src/crew/brief.ts: the sandbox paragraph is now SANDBOX_LINES. The first line says what the shell can run: bun install, bun x tsc --noEmit, and bun test with FORGE_TEST_SOCKET=port. The suites about unix sockets and cells are skipped in that mode and named as skipped, and the forge's check runs them. The second line says what the shell cannot do. Every repository's runs get this brief, so the mention of scripts/sandbox-probe.ts, which only this repository has, is gone (sentinel's line comment). The README still points to the script.
• scripts/sandbox-probe.ts: prints whether a listener answers on a loopback port and on a unix socket in a fresh temporary directory, then removes the directory. The README's Sandbox paragraph says what a builder can run.
• test/crew-sandbox.test.ts: covers the two brief lines and the probe's verdicts.
• scripts/gate.ts: fixes the red gate. test/plan-route.test.ts landed on main in #1170 without an entry in GATE or EXCLUDED, so test/check-list.test.ts failed. It now sits in EXCLUDED under #261. It waits up to 30 s for a fake planner run under the crew sandbox, well over the gate's limit of 1 s a test. Here it times out because that run can't start inside a sandbox. The test was right to go red, so I left it unchanged. Getting plan-route into the gate is #261's job, not this pull's.

Not done: sentinel's nit that the probe, when run inside bun test, makes its forge-probe-* directory in the working tree. finally removes it; only a test killed partway through would leave it behind.

To check: bun test test/check-list.test.ts test/crew-sandbox.test.ts test/crew.test.ts test/crew-brief-audience.test.ts passed here (54 tests), and so did bun x tsc --noEmit. To see the verdicts yourself, run bun scripts/sandbox-probe.ts inside a crew sandbox.

🤖 Generated with Claude Code