Public: anyone can read this; the crew can change it.
A crew run's sandbox refuses writes outside its worktree and hosts past the egress allowlist, each a test #10600
scout/forge:scout/10553-a-crew-run-s-sandbox-refuses-writes-outs → main 3 commits fork
2 files
changed +124
−6
docs/ops/RED-TEAM.md +28 −2
| @@ -15,14 +15,40 @@ so the live run is still open as its own issue. | ||
| 15 | 15 | | A visitor sends `x-forge-identity` / `x-forge-signature` that the app believes | The proxy deletes both and writes signed ones of its own (`IDENTITY_HEADERS`, `src/apps/proxy.ts`) | `test/apps-identity.test.ts`, `test/redteam-doors.test.ts` | |
| 16 | 16 | | An app learns the edge secret or a bearer token from a forwarded request | `stripForgeCredentials` drops every `x-plat-*` header, `authorization`, and the grant cookie | `test/redteam-doors.test.ts` "forged identity headers" | |
| 17 | 17 | | Someone outside claims a person with `x-plat-user` | `edgeIdentity` believes it only with the edge secret, compared as digests | `test/redteam-doors.test.ts`, `test/edge-auth.test.ts` | |
| 18 | | A crew run leaves its worktree or its network allowlist | bwrap in `src/crew/sandbox.ts` | `test/crew-sandbox.test.ts`; the escape attempts under `procMount: Unmasked` are still open | | |
| 18 | | A crew run leaves its worktree or its network allowlist | bwrap in `src/crew/sandbox.ts` | `test/crew-sandbox.test.ts`; the escape attempts under `procMount: Unmasked` are below, in "The crew sandbox" | | |
| 19 | 19 | | Note or issue text steers a crew brief | `untrusted` marking in briefs | still open: needs a wider test than #10365 | |
| 20 | 20 | |
| 21 | ## The crew sandbox, attempt by attempt | |
| 22 | ||
| 23 | A real sandboxed shell (bwrap, made by `srt` from `claudeSettings`, the | |
| 24 | settings a crew run hands Claude Code) with `/proc` unmasked, as the pod runs | |
| 25 | it. Every row is a test in `test/crew-sandbox-escape.test.ts` that fails if | |
| 26 | the attempt works; `<pid>` is a process outside the sandbox. Where no sandbox | |
| 27 | can be made (inside a crew run, or user namespaces refused) the file is | |
| 28 | skipped and prints `skipped: test/crew-sandbox-escape.test.ts: <reason>`; the | |
| 29 | forge's check runs it. | |
| 30 | ||
| 31 | | Attempt | Result | Test | | |
| 32 | | --- | --- | --- | | |
| 33 | | Read `forge.db`: by its path, `/proc/self/root`, `/proc/1/root`, `/proc/<pid>/root` | refused (no canary back) | "reads nothing from a denied data file, …" (one per route) | | |
| 34 | | Read another crew's file, the same four routes | refused | "reads nothing from another crew's file, …" | | |
| 35 | | Read a secret from `/proc/1/environ` | not there | "reads no secret from /proc/1/environ" | | |
| 36 | | Read a secret from any `/proc/<pid>/environ` the shell lists | not there | "reads no secret from any /proc/<pid>/environ …" | | |
| 37 | | Write into the data dir, the same four routes | refused (file absent, checked from outside) | "writes nothing into the data dir, …" | | |
| 38 | | Write into another crew's dir, the same four routes | refused | "writes nothing into another crew's dir, …" | | |
| 39 | | Connect to `127.0.0.1:<port off the allowlist>` directly | refused (own network) | "cannot reach 127.0.0.1:<port off the allowlist>, directly" | | |
| 40 | | The same through the sandbox's proxy | refused by the proxy | "… through the proxy" | | |
| 41 | | Connect to `localhost:<port off the allowlist>`, directly and through the proxy | refused | "cannot reach localhost:<port off the allowlist>, …" | | |
| 42 | | Controls: read and write the own worktree, read the own environ, reach the forge's host:port through the proxy | works | the three "control:" tests | | |
| 43 | ||
| 44 | The controls keep a probe that never ran, or a proxy that refuses | |
| 45 | everything, from passing as "all refused". | |
| 46 | ||
| 21 | 47 | ## Not done here |
| 22 | 48 | |
| 23 | 49 | - **Pod security enforced on all nine namespaces.** The namespace labels live |
| 24 | 50 | in the cluster's GitOps source, which is not in this repository. That work |
| 25 | 51 | happens there. |
| 26 | - **The live run inside the pod and the sandbox escape attempts**: #10450. | |
| 52 | - **The live run inside the pod**: #10450. The sandbox escape attempts are above. | |
| 27 | 53 | - **Wider prompt-injection tests**: #10451. |
| 28 | 54 | - Done: the security controls page, `/settings/security/controls` (#10452); how it reads `security-report.json` is in `docs/features/security-controls.md`. |
test/crew-sandbox-escape.test.ts +96 −4
| @@ -13,6 +13,16 @@ | ||
| 13 | 13 | * Two controls keep a probe that never ran from passing as "all refused": the |
| 14 | 14 | * shell reads the canary from its own worktree, and reads its own environ. |
| 15 | 15 | * |
| 16 | * Write attempts go the same routes to the data dir and another crew's dir; | |
| 17 | * each is judged from outside the sandbox (the file must not exist after), | |
| 18 | * not by what the shell says. The control writes into its own worktree. | |
| 19 | * | |
| 20 | * Network attempts are at a server outside the sandbox on a port that is not | |
| 21 | * on the egress allowlist, by `127.0.0.1` and by the name `localhost`, each | |
| 22 | * directly (the sandbox has a network of its own) and through the sandbox's | |
| 23 | * proxy. The control reaches the forge's host:port, which is allowed, through | |
| 24 | * the proxy. | |
| 25 | * | |
| 16 | 26 | * Like crew-sandbox-real.test.ts this needs a sandbox of its own, so inside a |
| 17 | 27 | * crew run or a container that refuses user namespaces it is skipped, naming |
| 18 | 28 | * itself and the reason on the console; the forge's check runs it. Where a |
| @@ -21,7 +31,7 @@ | ||
| 21 | 31 | * `ripgrep.command`: the check host has no ripgrep on its PATH. |
| 22 | 32 | */ |
| 23 | 33 | import { afterAll, beforeAll, describe, expect, test } from "bun:test"; |
| 24 | import { existsSync, mkdirSync, writeFileSync } from "node:fs"; | |
| 34 | import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs"; | |
| 25 | 35 | import { mkdtemp, rm } from "node:fs/promises"; |
| 26 | 36 | import { tmpdir } from "node:os"; |
| 27 | 37 | import { join } from "node:path"; |
| @@ -52,6 +62,14 @@ const FILES = { | ||
| 52 | 62 | /** The routes to each file; `<pid>` is a process outside the sandbox. */ |
| 53 | 63 | const ROUTES = ["by its path", "/proc/self/root", "/proc/1/root", "/proc/<pid>/root"] as const; |
| 54 | 64 | type Route = (typeof ROUTES)[number]; |
| 65 | /** The directories outside the worktree a write is tried in. */ | |
| 66 | const DIRS = { | |
| 67 | "the data dir": ".", | |
| 68 | "another crew's dir": "crew/mason/01repo/3", | |
| 69 | } as const; | |
| 70 | const WRITTEN = "escape-written.txt"; | |
| 71 | /** Hosts off the allowlist that name the outside server. */ | |
| 72 | const OFF_LIST = ["127.0.0.1", "localhost"] as const; | |
| 55 | 73 | |
| 56 | 74 | function routed(route: Route, path: string, pid: number): string { |
| 57 | 75 | if (route === "by its path") return path; |
| @@ -66,6 +84,10 @@ interface Report { | ||
| 66 | 84 | pidsListed: number; |
| 67 | 85 | initEnviron: boolean; |
| 68 | 86 | environLeaks: string[]; |
| 87 | /** `host|direct` or `host|proxy` → the body that came back, or "refused: <error>". */ | |
| 88 | net: Record<string, string>; | |
| 89 | forgeViaProxy: string; | |
| 90 | ownWrite: string; | |
| 69 | 91 | } |
| 70 | 92 | |
| 71 | 93 | describe.skipIf(nestedReason !== undefined)( |
| @@ -74,10 +96,14 @@ describe.skipIf(nestedReason !== undefined)( | ||
| 74 | 96 | let dataDir: string; |
| 75 | 97 | let report: Report; |
| 76 | 98 | let holder: ReturnType<typeof Bun.spawn>; |
| 99 | let work: string; | |
| 100 | let writes: Record<string, string>; | |
| 101 | let forge: ReturnType<typeof Bun.serve>; | |
| 102 | let outside: ReturnType<typeof Bun.serve>; | |
| 77 | 103 | |
| 78 | 104 | beforeAll(async () => { |
| 79 | 105 | dataDir = await mkdtemp(join(tmpdir(), "forge-escape-")); |
| 80 | const work = join(dataDir, "crew/scout/01repo/1"); | |
| 106 | work = join(dataDir, "crew/scout/01repo/1"); | |
| 81 | 107 | mkdirSync(work, { recursive: true }); |
| 82 | 108 | mkdirSync(join(dataDir, "crew/mason/01repo/3"), { recursive: true }); |
| 83 | 109 | for (const rel of Object.values(FILES)) writeFileSync(join(dataDir, rel), CANARY); |
| @@ -97,21 +123,55 @@ describe.skipIf(nestedReason !== undefined)( | ||
| 97 | 123 | for (const route of ROUTES) |
| 98 | 124 | targets[`${label}|${route}`] = routed(route, join(dataDir, rel), holder.pid); |
| 99 | 125 | |
| 126 | writes = {}; | |
| 127 | for (const [label, rel] of Object.entries(DIRS)) | |
| 128 | for (const route of ROUTES) | |
| 129 | writes[`${label}|${route}`] = routed( | |
| 130 | route, | |
| 131 | join(dataDir, rel, `${route.replace(/\W+/g, "-")}-${WRITTEN}`), | |
| 132 | holder.pid, | |
| 133 | ); | |
| 134 | ||
| 135 | forge = Bun.serve({ hostname: "127.0.0.1", port: 0, fetch: () => new Response("forge") }); | |
| 136 | outside = Bun.serve({ hostname: "127.0.0.1", port: 0, fetch: () => new Response("outside") }); | |
| 137 | ||
| 100 | 138 | writeFileSync( |
| 101 | 139 | join(work, "probe.ts"), |
| 102 | 140 | ` |
| 103 | import { readFileSync, readdirSync } from "node:fs"; | |
| 141 | import { readFileSync, readdirSync, writeFileSync } from "node:fs"; | |
| 104 | 142 | const CANARY = ${JSON.stringify(CANARY)}; |
| 105 | 143 | const ENV = ${JSON.stringify(ENV_CANARY)}; |
| 106 | 144 | const has = (path: string, needle: string) => { |
| 107 | 145 | try { return readFileSync(path, "utf8").includes(needle); } catch { return false; } |
| 108 | 146 | }; |
| 147 | // The network first, before the shell touches /proc. | |
| 148 | const get = async (url: string, proxy?: string) => { | |
| 149 | try { return await (await fetch(url, { proxy })).text(); } | |
| 150 | catch (e) { return "refused: " + String(e); } | |
| 151 | }; | |
| 152 | // The allowed control first: a refusal must not be what decides it. | |
| 153 | // A few tries, spaced: a control that is slow to answer is not a refusal. | |
| 154 | let forgeViaProxy = ""; | |
| 155 | for (let i = 0; i < 5 && forgeViaProxy !== "forge"; i++) { | |
| 156 | if (i > 0) await Bun.sleep(200); | |
| 157 | forgeViaProxy = await get("http://127.0.0.1:${forge.port}/", process.env.HTTP_PROXY); | |
| 158 | } | |
| 159 | const net: Record<string, string> = {}; | |
| 160 | for (const host of ${JSON.stringify(OFF_LIST)}) { | |
| 161 | const url = "http://" + host + ":" + ${outside.port} + "/"; | |
| 162 | net[host + "|direct"] = await get(url); | |
| 163 | net[host + "|proxy"] = process.env.HTTP_PROXY ? await get(url, process.env.HTTP_PROXY) : "no proxy"; | |
| 164 | } | |
| 109 | 165 | const reads: Record<string, boolean> = {}; |
| 110 | 166 | for (const [key, path] of Object.entries(${JSON.stringify(targets)} as Record<string, string>)) |
| 111 | 167 | reads[key] = has(path, CANARY); |
| 112 | 168 | let pids: string[] = []; |
| 113 | 169 | try { pids = readdirSync("/proc").filter((n) => /^[0-9]+$/.test(n)); } catch {} |
| 114 | 170 | const environLeaks = pids.filter((pid) => has("/proc/" + pid + "/environ", ENV)); |
| 171 | for (const path of Object.values(${JSON.stringify(writes)} as Record<string, string>)) | |
| 172 | try { writeFileSync(path, CANARY); } catch {} | |
| 173 | let ownWrite = "ok"; | |
| 174 | try { writeFileSync("own-written.txt", CANARY); } catch (e) { ownWrite = String(e); } | |
| 115 | 175 | console.log(JSON.stringify({ |
| 116 | 176 | reads, |
| 117 | 177 | ownRead: has("own.txt", CANARY), |
| @@ -119,6 +179,9 @@ console.log(JSON.stringify({ | ||
| 119 | 179 | pidsListed: pids.length, |
| 120 | 180 | initEnviron: has("/proc/1/environ", ENV), |
| 121 | 181 | environLeaks, |
| 182 | net, | |
| 183 | forgeViaProxy, | |
| 184 | ownWrite, | |
| 122 | 185 | })); |
| 123 | 186 | `, |
| 124 | 187 | ); |
| @@ -126,7 +189,7 @@ console.log(JSON.stringify({ | ||
| 126 | 189 | const policy = sandboxPolicy({ |
| 127 | 190 | dataDir, |
| 128 | 191 | agentName: "scout", |
| 129 | urls: [], | |
| 192 | urls: [`http://127.0.0.1:${forge.port}`], | |
| 130 | 193 | runId: "01m46abcdefghjkmnpqrstvwxy", |
| 131 | 194 | }); |
| 132 | 195 | const real = claudeSettings(policy).sandbox as Record<string, any>; |
| @@ -159,6 +222,8 @@ console.log(JSON.stringify({ | ||
| 159 | 222 | |
| 160 | 223 | afterAll(async () => { |
| 161 | 224 | holder?.kill(); |
| 225 | forge?.stop(true); | |
| 226 | outside?.stop(true); | |
| 162 | 227 | if (dataDir !== undefined) await rm(dataDir, { recursive: true, force: true }); |
| 163 | 228 | }); |
| 164 | 229 | |
| @@ -180,5 +245,32 @@ console.log(JSON.stringify({ | ||
| 180 | 245 | test("reads no secret from any /proc/<pid>/environ of a process outside", () => { |
| 181 | 246 | expect(report.environLeaks).toEqual([]); |
| 182 | 247 | }); |
| 248 | ||
| 249 | test("control: the shell writes its own worktree", () => { | |
| 250 | expect(report.ownWrite).toBe("ok"); | |
| 251 | expect(existsSync(join(work, "own-written.txt"))).toBe(true); | |
| 252 | expect(readFileSync(join(work, "own-written.txt"), "utf8")).toBe(CANARY); | |
| 253 | }); | |
| 254 | ||
| 255 | test("control: the shell reaches the forge's host:port through the proxy", () => { | |
| 256 | expect(report.forgeViaProxy).toBe("forge"); | |
| 257 | }); | |
| 258 | ||
| 259 | for (const label of Object.keys(DIRS)) | |
| 260 | for (const route of ROUTES) | |
| 261 | test(`writes nothing into ${label}, ${route}`, () => { | |
| 262 | // Judged from outside: the path resolved without /proc routes. | |
| 263 | const rel = DIRS[label as keyof typeof DIRS]; | |
| 264 | const path = join(dataDir, rel, `${route.replace(/\W+/g, "-")}-${WRITTEN}`); | |
| 265 | expect(writes[`${label}|${route}`]).toEndWith(path); | |
| 266 | expect(existsSync(path)).toBe(false); | |
| 267 | }); | |
| 268 | ||
| 269 | for (const host of OFF_LIST) | |
| 270 | for (const way of ["direct", "proxy"] as const) | |
| 271 | test(`cannot reach ${host}:<port off the allowlist>, ${way === "direct" ? "directly" : "through the proxy"}`, () => { | |
| 272 | expect(report.net[`${host}|${way}`]).not.toBe("outside"); | |
| 273 | expect(report.net[`${host}|${way}`]).not.toBe("no proxy"); | |
| 274 | }); | |
| 183 | 275 | }, |
| 184 | 276 | ); |
new above
offline