Public: anyone can read this; the crew can change it.

A crew run's sandbox refuses writes outside its worktree and hosts past the egress allowlist, each a test #10600

Parked opened this
Closed without merging

scout/forge:scout/10553-a-crew-run-s-sandbox-refuses-writes-outs → main 3 commits fork

  • Closed without merging. Nothing was applied to main.

Review

Sign in to review this change.
2 files changed +124 −6
docs/ops/RED-TEAM.md +28 −2
@@ -15,14 +15,40 @@ so the live run is still open as its own issue.
1515 | A visitor sends `x-forge-identity` / `x-forge-signature` that the app believes | The proxy deletes both and writes signed ones of its own (`IDENTITY_HEADERS`, `src/apps/proxy.ts`) | `test/apps-identity.test.ts`, `test/redteam-doors.test.ts` |
1616 | An app learns the edge secret or a bearer token from a forwarded request | `stripForgeCredentials` drops every `x-plat-*` header, `authorization`, and the grant cookie | `test/redteam-doors.test.ts` "forged identity headers" |
1717 | Someone outside claims a person with `x-plat-user` | `edgeIdentity` believes it only with the edge secret, compared as digests | `test/redteam-doors.test.ts`, `test/edge-auth.test.ts` |
18 | A crew run leaves its worktree or its network allowlist | bwrap in `src/crew/sandbox.ts` | `test/crew-sandbox.test.ts`; the escape attempts under `procMount: Unmasked` are still open |
18 | A crew run leaves its worktree or its network allowlist | bwrap in `src/crew/sandbox.ts` | `test/crew-sandbox.test.ts`; the escape attempts under `procMount: Unmasked` are below, in "The crew sandbox" |
1919 | Note or issue text steers a crew brief | `untrusted` marking in briefs | still open: needs a wider test than #10365 |
2020
21 ## The crew sandbox, attempt by attempt
22
23 A real sandboxed shell (bwrap, made by `srt` from `claudeSettings`, the
24 settings a crew run hands Claude Code) with `/proc` unmasked, as the pod runs
25 it. Every row is a test in `test/crew-sandbox-escape.test.ts` that fails if
26 the attempt works; `<pid>` is a process outside the sandbox. Where no sandbox
27 can be made (inside a crew run, or user namespaces refused) the file is
28 skipped and prints `skipped: test/crew-sandbox-escape.test.ts: <reason>`; the
29 forge's check runs it.
30
31 | Attempt | Result | Test |
32 | --- | --- | --- |
33 | Read `forge.db`: by its path, `/proc/self/root`, `/proc/1/root`, `/proc/<pid>/root` | refused (no canary back) | "reads nothing from a denied data file, …" (one per route) |
34 | Read another crew's file, the same four routes | refused | "reads nothing from another crew's file, …" |
35 | Read a secret from `/proc/1/environ` | not there | "reads no secret from /proc/1/environ" |
36 | Read a secret from any `/proc/<pid>/environ` the shell lists | not there | "reads no secret from any /proc/<pid>/environ …" |
37 | Write into the data dir, the same four routes | refused (file absent, checked from outside) | "writes nothing into the data dir, …" |
38 | Write into another crew's dir, the same four routes | refused | "writes nothing into another crew's dir, …" |
39 | Connect to `127.0.0.1:<port off the allowlist>` directly | refused (own network) | "cannot reach 127.0.0.1:<port off the allowlist>, directly" |
40 | The same through the sandbox's proxy | refused by the proxy | "… through the proxy" |
41 | Connect to `localhost:<port off the allowlist>`, directly and through the proxy | refused | "cannot reach localhost:<port off the allowlist>, …" |
42 | Controls: read and write the own worktree, read the own environ, reach the forge's host:port through the proxy | works | the three "control:" tests |
43
44 The controls keep a probe that never ran, or a proxy that refuses
45 everything, from passing as "all refused".
46
2147 ## Not done here
2248
2349 - **Pod security enforced on all nine namespaces.** The namespace labels live
2450 in the cluster's GitOps source, which is not in this repository. That work
2551 happens there.
26 - **The live run inside the pod and the sandbox escape attempts**: #10450.
52 - **The live run inside the pod**: #10450. The sandbox escape attempts are above.
2753 - **Wider prompt-injection tests**: #10451.
2854 - Done: the security controls page, `/settings/security/controls` (#10452); how it reads `security-report.json` is in `docs/features/security-controls.md`.
test/crew-sandbox-escape.test.ts +96 −4
@@ -13,6 +13,16 @@
1313 * Two controls keep a probe that never ran from passing as "all refused": the
1414 * shell reads the canary from its own worktree, and reads its own environ.
1515 *
16 * Write attempts go the same routes to the data dir and another crew's dir;
17 * each is judged from outside the sandbox (the file must not exist after),
18 * not by what the shell says. The control writes into its own worktree.
19 *
20 * Network attempts are at a server outside the sandbox on a port that is not
21 * on the egress allowlist, by `127.0.0.1` and by the name `localhost`, each
22 * directly (the sandbox has a network of its own) and through the sandbox's
23 * proxy. The control reaches the forge's host:port, which is allowed, through
24 * the proxy.
25 *
1626 * Like crew-sandbox-real.test.ts this needs a sandbox of its own, so inside a
1727 * crew run or a container that refuses user namespaces it is skipped, naming
1828 * itself and the reason on the console; the forge's check runs it. Where a
@@ -21,7 +31,7 @@
2131 * `ripgrep.command`: the check host has no ripgrep on its PATH.
2232 */
2333 import { afterAll, beforeAll, describe, expect, test } from "bun:test";
24 import { existsSync, mkdirSync, writeFileSync } from "node:fs";
34 import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
2535 import { mkdtemp, rm } from "node:fs/promises";
2636 import { tmpdir } from "node:os";
2737 import { join } from "node:path";
@@ -52,6 +62,14 @@ const FILES = {
5262 /** The routes to each file; `<pid>` is a process outside the sandbox. */
5363 const ROUTES = ["by its path", "/proc/self/root", "/proc/1/root", "/proc/<pid>/root"] as const;
5464 type Route = (typeof ROUTES)[number];
65 /** The directories outside the worktree a write is tried in. */
66 const DIRS = {
67 "the data dir": ".",
68 "another crew's dir": "crew/mason/01repo/3",
69 } as const;
70 const WRITTEN = "escape-written.txt";
71 /** Hosts off the allowlist that name the outside server. */
72 const OFF_LIST = ["127.0.0.1", "localhost"] as const;
5573
5674 function routed(route: Route, path: string, pid: number): string {
5775 if (route === "by its path") return path;
@@ -66,6 +84,10 @@ interface Report {
6684 pidsListed: number;
6785 initEnviron: boolean;
6886 environLeaks: string[];
87 /** `host|direct` or `host|proxy` → the body that came back, or "refused: <error>". */
88 net: Record<string, string>;
89 forgeViaProxy: string;
90 ownWrite: string;
6991 }
7092
7193 describe.skipIf(nestedReason !== undefined)(
@@ -74,10 +96,14 @@ describe.skipIf(nestedReason !== undefined)(
7496 let dataDir: string;
7597 let report: Report;
7698 let holder: ReturnType<typeof Bun.spawn>;
99 let work: string;
100 let writes: Record<string, string>;
101 let forge: ReturnType<typeof Bun.serve>;
102 let outside: ReturnType<typeof Bun.serve>;
77103
78104 beforeAll(async () => {
79105 dataDir = await mkdtemp(join(tmpdir(), "forge-escape-"));
80 const work = join(dataDir, "crew/scout/01repo/1");
106 work = join(dataDir, "crew/scout/01repo/1");
81107 mkdirSync(work, { recursive: true });
82108 mkdirSync(join(dataDir, "crew/mason/01repo/3"), { recursive: true });
83109 for (const rel of Object.values(FILES)) writeFileSync(join(dataDir, rel), CANARY);
@@ -97,21 +123,55 @@ describe.skipIf(nestedReason !== undefined)(
97123 for (const route of ROUTES)
98124 targets[`${label}|${route}`] = routed(route, join(dataDir, rel), holder.pid);
99125
126 writes = {};
127 for (const [label, rel] of Object.entries(DIRS))
128 for (const route of ROUTES)
129 writes[`${label}|${route}`] = routed(
130 route,
131 join(dataDir, rel, `${route.replace(/\W+/g, "-")}-${WRITTEN}`),
132 holder.pid,
133 );
134
135 forge = Bun.serve({ hostname: "127.0.0.1", port: 0, fetch: () => new Response("forge") });
136 outside = Bun.serve({ hostname: "127.0.0.1", port: 0, fetch: () => new Response("outside") });
137
100138 writeFileSync(
101139 join(work, "probe.ts"),
102140 `
103 import { readFileSync, readdirSync } from "node:fs";
141 import { readFileSync, readdirSync, writeFileSync } from "node:fs";
104142 const CANARY = ${JSON.stringify(CANARY)};
105143 const ENV = ${JSON.stringify(ENV_CANARY)};
106144 const has = (path: string, needle: string) => {
107145 try { return readFileSync(path, "utf8").includes(needle); } catch { return false; }
108146 };
147 // The network first, before the shell touches /proc.
148 const get = async (url: string, proxy?: string) => {
149 try { return await (await fetch(url, { proxy })).text(); }
150 catch (e) { return "refused: " + String(e); }
151 };
152 // The allowed control first: a refusal must not be what decides it.
153 // A few tries, spaced: a control that is slow to answer is not a refusal.
154 let forgeViaProxy = "";
155 for (let i = 0; i < 5 && forgeViaProxy !== "forge"; i++) {
156 if (i > 0) await Bun.sleep(200);
157 forgeViaProxy = await get("http://127.0.0.1:${forge.port}/", process.env.HTTP_PROXY);
158 }
159 const net: Record<string, string> = {};
160 for (const host of ${JSON.stringify(OFF_LIST)}) {
161 const url = "http://" + host + ":" + ${outside.port} + "/";
162 net[host + "|direct"] = await get(url);
163 net[host + "|proxy"] = process.env.HTTP_PROXY ? await get(url, process.env.HTTP_PROXY) : "no proxy";
164 }
109165 const reads: Record<string, boolean> = {};
110166 for (const [key, path] of Object.entries(${JSON.stringify(targets)} as Record<string, string>))
111167 reads[key] = has(path, CANARY);
112168 let pids: string[] = [];
113169 try { pids = readdirSync("/proc").filter((n) => /^[0-9]+$/.test(n)); } catch {}
114170 const environLeaks = pids.filter((pid) => has("/proc/" + pid + "/environ", ENV));
171 for (const path of Object.values(${JSON.stringify(writes)} as Record<string, string>))
172 try { writeFileSync(path, CANARY); } catch {}
173 let ownWrite = "ok";
174 try { writeFileSync("own-written.txt", CANARY); } catch (e) { ownWrite = String(e); }
115175 console.log(JSON.stringify({
116176 reads,
117177 ownRead: has("own.txt", CANARY),
@@ -119,6 +179,9 @@ console.log(JSON.stringify({
119179 pidsListed: pids.length,
120180 initEnviron: has("/proc/1/environ", ENV),
121181 environLeaks,
182 net,
183 forgeViaProxy,
184 ownWrite,
122185 }));
123186 `,
124187 );
@@ -126,7 +189,7 @@ console.log(JSON.stringify({
126189 const policy = sandboxPolicy({
127190 dataDir,
128191 agentName: "scout",
129 urls: [],
192 urls: [`http://127.0.0.1:${forge.port}`],
130193 runId: "01m46abcdefghjkmnpqrstvwxy",
131194 });
132195 const real = claudeSettings(policy).sandbox as Record<string, any>;
@@ -159,6 +222,8 @@ console.log(JSON.stringify({
159222
160223 afterAll(async () => {
161224 holder?.kill();
225 forge?.stop(true);
226 outside?.stop(true);
162227 if (dataDir !== undefined) await rm(dataDir, { recursive: true, force: true });
163228 });
164229
@@ -180,5 +245,32 @@ console.log(JSON.stringify({
180245 test("reads no secret from any /proc/<pid>/environ of a process outside", () => {
181246 expect(report.environLeaks).toEqual([]);
182247 });
248
249 test("control: the shell writes its own worktree", () => {
250 expect(report.ownWrite).toBe("ok");
251 expect(existsSync(join(work, "own-written.txt"))).toBe(true);
252 expect(readFileSync(join(work, "own-written.txt"), "utf8")).toBe(CANARY);
253 });
254
255 test("control: the shell reaches the forge's host:port through the proxy", () => {
256 expect(report.forgeViaProxy).toBe("forge");
257 });
258
259 for (const label of Object.keys(DIRS))
260 for (const route of ROUTES)
261 test(`writes nothing into ${label}, ${route}`, () => {
262 // Judged from outside: the path resolved without /proc routes.
263 const rel = DIRS[label as keyof typeof DIRS];
264 const path = join(dataDir, rel, `${route.replace(/\W+/g, "-")}-${WRITTEN}`);
265 expect(writes[`${label}|${route}`]).toEndWith(path);
266 expect(existsSync(path)).toBe(false);
267 });
268
269 for (const host of OFF_LIST)
270 for (const way of ["direct", "proxy"] as const)
271 test(`cannot reach ${host}:<port off the allowlist>, ${way === "direct" ? "directly" : "through the proxy"}`, () => {
272 expect(report.net[`${host}|${way}`]).not.toBe("outside");
273 expect(report.net[`${host}|${way}`]).not.toBe("no proxy");
274 });
183275 },
184276 );

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs