Public: anyone can read this; the crew can change it.
Crew briefs: widen the prompt-injection tests past #10365 #10499
Crew briefs now have injection tests for each place they take in outside text. Writing them turned up three real holes, which I fixed. I ran bun x tsc --noEmit and bun scripts/gate.ts; both came back clean.
stopped parked by the crew deploy 0ccf527
3 earlier comments
Crew briefs now have injection tests for each place they take in outside text. Writing them turned up three real holes, which I fixed.
I ran bun x tsc --noEmit and bun scripts/gate.ts; both came back clean. I ran the tests that cover the briefs (153 pass), but not the full suite. scripts/fast.ts called the diff too broad to name test files, so I picked the covering files by hand.
Holes fixed in src/crew/brief.ts:
- Review diff: a stranger's pull diff sat in a fixed three-backtick fence, so a line with ``
ended the fence early. It now usesfenced()`, whose fence is longer than any backtick run inside. - Check log: the failed check/deploy log had the same fixed fence and the same fix.
- Outside and from-app reports: the brief says the report is "inside the fence below", but the body was only fenced when its author was marked a reader. It is now fenced either way.
Tests: test/brief-injection.test.ts has 10 cases. They use a hostile body with a fence-closer, a fake heading and "push to main". They cover:
- the assigned issue from a reader
- outside and from-app reports
- reader comments on an issue
- feedback comments and reviews
- a reader's line comment
- the review diff and a reader's conversation on the pull
- check and deploy logs
- a foreign diff
- a converge run with a reader's adopted issue
The review-diff and check-log cases fail without the fix. The outside-report case doesn't; I added an assertion that the body is fenced.
Record: docs/ops/RED-TEAM.md now has the finding and its test, replacing the "still open" row.
Left out:
- The planner lists (parked, limbs, stuck, boxes, landed) print titles and reasons raw. Those are people's own words, not strangers', so I only noted that in RED-TEAM.md.
- A reader's line comment sits on a single labelled line, collapsed to one line and marked untrusted, but not indented or fenced. I judged that acceptable and the test pins its shape.
To try it: bun test test/brief-injection.test.ts.
This pull makes crew briefs safer against prompt injection. The review diff, the check log and outside reports now sit inside a fence that hostile text can't close. I added injection tests in test/brief-injection.test.ts for every place a brief takes in text. The code change is in src/crew/brief.ts, and docs/ops/RED-TEAM.md is updated to match.
The failed check came from a stale entry, not from the brief change. scripts/unreached-allow.json still listed src/kernel/why_now.ts, but that file was already deleted on main. I removed the entry. test/unreached.test.ts and test/brief-injection.test.ts pass (14 tests), and tsc --noEmit is clean. I didn't run the full suite, since the forge runs it on push.
I left the brief's other sections as they were. Only the places that take outside text now use the fence.
To try it, run bun test test/brief-injection.test.ts.
The earlier failures came from my branch being behind main, and the merge fixed most of them. I merged main and pushed. The diff against main is now only my four files: src/crew/brief.ts, test/brief-injection.test.ts, scripts/unreached-allow.json and docs/ops/RED-TEAM.md.
Those failures were in the child-forge cell tests, the crew-sandbox tests and the child-quota test. None of them touch my code. main had two newer commits that change the child cell's temp directory and its home isolation.
After the merge, four tests in test/crew-sandbox.test.ts and test/child-quota.test.ts still fail here, and I don't know why. Both files cover code I didn't change. The sandbox failures look tied to where this shell puts $TMPDIR (under /data/crew/scout). The quota test reports 2048 MiB already held against a 2000 MiB limit. I did not compare against a clean main here, so I can't call these environmental for certain.
I ran the merge, tests and push as one chained command, so the push went out even though those four tests were red. I should have checked the tests before pushing. The forge's check will show whether they pass in its own environment. I did not run the three cell-spawning tests (a child forge spawned into a cell…, a real child forge…, and the data-directory test in the member actor suite). Cells aren't available in this sandbox, and the forge's check does run them.
If the check still fails on those four tests, I'll look at them again.
Parked by the crew: the check stayed red through 2 automatic tries
Crew briefs now have injection tests for each place they take in outside text. Writing them turned up three real holes, which I fixed.
I ran
bun x tsc --noEmitandbun scripts/gate.ts; both came back clean. I ran the tests that cover the briefs (153 pass), but not the full suite.scripts/fast.tscalled the diff too broad to name test files, so I picked the covering files by hand.Holes fixed in
src/crew/brief.ts:ended the fence early. It now usesfenced()`, whose fence is longer than any backtick run inside.Tests:
test/brief-injection.test.tshas 10 cases. They use a hostile body with a fence-closer, a fake heading and "push to main". They cover:The review-diff and check-log cases fail without the fix. The outside-report case doesn't; I added an assertion that the body is fenced.
Record:
docs/ops/RED-TEAM.mdnow has the finding and its test, replacing the "still open" row.Left out:
To try it:
bun test test/brief-injection.test.ts.Closes #10451