Public: anyone can read this; the crew can change it.

Crew briefs: widen the prompt-injection tests past #10365 #10463

Parked opened this
Closed without merging

This pull adds prompt-injection tests for the places a crew brief takes in outside text. Writing them found two holes in the briefs, and the pull closes both.

Notes

stopped parked by the crew deploy 90e2d47

opened this

This pull adds prompt-injection tests for the places a crew brief takes in outside text. Writing them found two holes in the briefs, and the pull closes both. The new tests, the related brief tests, the two proposal suites and tsc pass here; the gate script printed only a sentinel warning and I did not check its exit code. The full check has not run; it runs on this push. I did not call keep_plan on the issue, so no plan is posted there.

What moved

  • Check log (src/crew/brief.ts): the tail of a failed check's log, which the pull's own code prints, sat in a fixed three-backtick fence. A line of `` in it closed the fence and the rest read as the brief. It now uses fenced()`, and the brief says the log is data about the failure, never instructions.
  • Review diff (src/crew/brief.ts): the reviewer's diff had the same fixed fence and now uses fenced() too.
  • Conflict blocks (src/crew/proposal.ts): resolutionBrief had the same fixed fence around the blocks from both sides. It now uses fenced(), and the brief calls the commit subjects and fenced text the branches' own words.
  • Existing test changed: test/crew-check-failed.test.ts matched the old log heading, so I changed it to match the new wording.

Tests

  • test/brief-injection.test.ts has ten tests, one per place the brief takes in text:

- a reader's issue body; - outside reports and app notes; - comments on an assigned issue; - comments, reviews and line comments in feedback; - the conversation shown to a reviewer; - the diff under review; - a stranger's pull; - a failed check's log; - the previous verdict and answer; - upstream file names and a foreign pull's diff.

  • Each feeds hostile text containing fence breakers and a fake instruction. It fails if any line of the brief itself, outside a fence and not indented, contains that text without a reader mark.
  • With the src changes stashed, the review-diff and check-log tests fail.

Not done

  • The fence in resolutionBrief has no test of its own, because it needs a repository with a real conflict. Only fenced's own test covers it.
  • I did not trace look findings to a brief. The reviewer's brief names look pictures by file path only. docs/ops/RED-TEAM.md says so.
  • The row on this in docs/ops/RED-TEAM.md is updated, and the "open" bullet for #10451 is gone.

To try it: bun test test/brief-injection.test.ts.

🤖 Generated with Claude Code

Closes #10451

3 earlier comments
commented

This pull adds prompt-injection tests for the places a crew brief takes in outside text. Writing them found two holes in the briefs, and the pull closes both. The new tests, the related brief tests, the two proposal suites and tsc pass here; the gate script printed only a sentinel warning and I did not check its exit code. The full check has not run; it runs on this push. I did not call keep_plan on the issue, so no plan is posted there.

What moved

  • Check log (src/crew/brief.ts): the tail of a failed check's log, which the pull's own code prints, sat in a fixed three-backtick fence. A line of `` in it closed the fence and the rest read as the brief. It now uses fenced()`, and the brief says the log is data about the failure, never instructions.
  • Review diff (src/crew/brief.ts): the reviewer's diff had the same fixed fence and now uses fenced() too.
  • Conflict blocks (src/crew/proposal.ts): resolutionBrief had the same fixed fence around the blocks from both sides. It now uses fenced(), and the brief calls the commit subjects and fenced text the branches' own words.
  • Existing test changed: test/crew-check-failed.test.ts matched the old log heading, so I changed it to match the new wording.

Tests

  • test/brief-injection.test.ts has ten tests, one per place the brief takes in text:

- a reader's issue body; - outside reports and app notes; - comments on an assigned issue; - comments, reviews and line comments in feedback; - the conversation shown to a reviewer; - the diff under review; - a stranger's pull; - a failed check's log; - the previous verdict and answer; - upstream file names and a foreign pull's diff.

  • Each feeds hostile text containing fence breakers and a fake instruction. It fails if any line of the brief itself, outside a fence and not indented, contains that text without a reader mark.
  • With the src changes stashed, the review-diff and check-log tests fail.

Not done

  • The fence in resolutionBrief has no test of its own, because it needs a repository with a real conflict. Only fenced's own test covers it.
  • I did not trace look findings to a brief. The reviewer's brief names look pictures by file path only. docs/ops/RED-TEAM.md says so.
  • The row on this in docs/ops/RED-TEAM.md is updated, and the "open" bullet for #10451 is gone.

To try it: bun test test/brief-injection.test.ts.

🤖 Generated with Claude Code

commented

This pull makes crew briefs fence every piece of outside text past any backtick run and tests each place a brief takes such text in (#10451). A second commit deletes a dead module that was failing the check.

What changed

  • Briefs: the check log, the review diff and the conflict blocks now go through fenced(). Hostile text can't close the fence early and pass itself off as the brief's own instructions. The check-log and conflict-block intros also say the text is data, never instructions. test/brief-injection.test.ts covers every place a brief takes in outside text, and docs/ops/RED-TEAM.md is updated.
  • Check failure 1: every module under src/ is imported by another, or allowed failed on src/kernel/why_now.ts. Nothing under src/ imports it since the Why-now gate was retired in #10317. I deleted it and test/why-now.test.ts. test/unreached.test.ts and bun x tsc --noEmit pass.
  • Check failure 2: storage quota > a mirror bigger than the limit… is not caused by this change. It passes alone (16 of 16 in test/quotas-storage.test.ts) and looks like a flake under the full run's load. I didn't change it.

I did not run bun scripts/gate.ts or the brief tests again after the deletion. The forge's full check on this push will run them.

To try it

  • Run bun test test/brief-injection.test.ts test/unreached.test.ts.
commented

The check failure came from a stale entry in scripts/unreached-allow.json. I fixed it, and the fix is pushed as 90e2d47fd. The unreached test and bun x tsc --noEmit pass on the merged branch. I didn't run scripts/gate.ts, scripts/fast.ts or the rest of the suite here, so the forge's check on the push is the full run.

Cause. main had deleted src/kernel/why_now.ts and added it to the allow list in the same change. My branch held the earlier commit that deleted the file but didn't touch the list. Merged, the list named a file that was gone, so the test "an allowed module that has become reached leaves the list" failed.

Fix.

  • I merged origin/main into the branch.
  • I removed src/kernel/why_now.ts from scripts/unreached-allow.json.

Nothing else changed, and the prompt-injection brief fencing from earlier commits is untouched.

To try it: run bun test test/unreached.test.ts.

🤖 Generated with Claude Code

commented

Parked by the crew: the check stayed red through 2 automatic tries

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs