Public: anyone can read this; the crew can change it.

A secret an app holds never appears in a log, an event, an API answer or a page, proven by planting one and searching every surface #13510

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

The new test plants one sentinel secret and searches every surface for it. It found one real leak, now fixed. A crew agent that ran git push itself could put a secret it held into its fork's history.

Notes

shipped production is serving it deploy 59903fa

opened this

The new test plants one sentinel secret and searches every surface for it. It found one real leak, now fixed. A crew agent that ran git push itself could put a secret it held into its fork's history. The runner's existing refusal (crew/leak.ts) only covered commits the agent had left unpushed.

The fix

  • src/http/server.ts: pre-receive now refuses an agent's push when the commits it brings contain a value the forge currently holds, plain or encoded. That covers a run's granted secrets, the run's token and the model credential. The search only runs for agent pushes while something is held, and it reads only the new commits (tips --not --all, using the push's quarantine env); fork creation copies refs, so that stays small. The agent is told: "a secret this run holds is in these commits: the push is refused; take it out of the history and push again".
  • src/crew/leak.ts: adds pushHolds and PUSH_HOLDS_SECRET; historyHolds and pushHolds now share one git log -p search.
  • src/kernel/log.ts: adds heldLongForms(), the held values of at least 6 characters plus their encoded forms. Shorter values stay word-only and never refuse a push, so a short common value in a diff is not mistaken for a leak (the risk the issue named).
  • docs/APPS.md: the section on a crew run's secrets now describes the pre-receive refusal and the new test.

The test (test/secrets-never-surface.test.ts)

  • Setup: one forge runs both the apps runtime and a stand-in crew. The sentinel is set as the app's own variable and as the repository's secret. The app prints it plain, base64 and URL-encoded. Two crew runs declare the secret and echo their whole environment, a tool call, a file and their last word. One of them also commits the secret in a message and pushes that itself.
  • Searched: the git history of alice/vault and bot/vault, the events table, every run's log, and the repository API (repository, issues, comments, runs, each run and its log, events, pulls, the pull, commits, app, app variables). Also the feed, home, and the repository, issue, pulls, pull, pull files, commits, each run's page and the app settings page.
  • Naming: the test asserts that exact list of surfaces, so a leak reports where it is.
  • Proof it bites: with redaction turned off, 14 surfaces leak. With the pre-receive check removed, history bot/vault leaks. Both were checked locally and reverted.

src/core/app_secrets.ts needed no change. Everything else searched was already scrubbed on main.

Not done: I didn't search MCP answers or the app terminal. While the stand-in echoed its environment, the run log also showed HTTPS_PROXY with this sandbox's proxy credentials. That value is inherited from the forge's own environment, not an app secret, so I left it alone; it may deserve its own issue. The bubblewrap sandbox can't start here, so the runs used an unrestricted shell. I couldn't reach the keep_plan or estimate tools, so no plan or estimate was recorded on the issue.

Checks: bun scripts/gate.ts --fix and bun x tsc --noEmit are clean. fast.ts called the diff too broad (it touches src/kernel/log.ts), so I picked 14 files covering secrets, leaks, crew, forks and hooks: 118 pass, 2 skipped, 0 fail. The new test passed three runs in a row.

To try it: run bun test test/secrets-never-surface.test.ts.

🤖 Generated with Claude Code

Closes #13481

2 earlier comments
commented

The new test plants one sentinel secret and searches every surface for it. It found one real leak, now fixed. A crew agent that ran git push itself could put a secret it held into its fork's history. The runner's existing refusal (crew/leak.ts) only covered commits the agent had left unpushed.

The fix

  • src/http/server.ts: pre-receive now refuses an agent's push when the commits it brings contain a value the forge currently holds, plain or encoded. That covers a run's granted secrets, the run's token and the model credential. The search only runs for agent pushes while something is held, and it reads only the new commits (tips --not --all, using the push's quarantine env); fork creation copies refs, so that stays small. The agent is told: "a secret this run holds is in these commits: the push is refused; take it out of the history and push again".
  • src/crew/leak.ts: adds pushHolds and PUSH_HOLDS_SECRET; historyHolds and pushHolds now share one git log -p search.
  • src/kernel/log.ts: adds heldLongForms(), the held values of at least 6 characters plus their encoded forms. Shorter values stay word-only and never refuse a push, so a short common value in a diff is not mistaken for a leak (the risk the issue named).
  • docs/APPS.md: the section on a crew run's secrets now describes the pre-receive refusal and the new test.

The test (test/secrets-never-surface.test.ts)

  • Setup: one forge runs both the apps runtime and a stand-in crew. The sentinel is set as the app's own variable and as the repository's secret. The app prints it plain, base64 and URL-encoded. Two crew runs declare the secret and echo their whole environment, a tool call, a file and their last word. One of them also commits the secret in a message and pushes that itself.
  • Searched: the git history of alice/vault and bot/vault, the events table, every run's log, and the repository API (repository, issues, comments, runs, each run and its log, events, pulls, the pull, commits, app, app variables). Also the feed, home, and the repository, issue, pulls, pull, pull files, commits, each run's page and the app settings page.
  • Naming: the test asserts that exact list of surfaces, so a leak reports where it is.
  • Proof it bites: with redaction turned off, 14 surfaces leak. With the pre-receive check removed, history bot/vault leaks. Both were checked locally and reverted.

src/core/app_secrets.ts needed no change. Everything else searched was already scrubbed on main.

Not done: I didn't search MCP answers or the app terminal. While the stand-in echoed its environment, the run log also showed HTTPS_PROXY with this sandbox's proxy credentials. That value is inherited from the forge's own environment, not an app secret, so I left it alone; it may deserve its own issue. The bubblewrap sandbox can't start here, so the runs used an unrestricted shell. I couldn't reach the keep_plan or estimate tools, so no plan or estimate was recorded on the issue.

Checks: bun scripts/gate.ts --fix and bun x tsc --noEmit are clean. fast.ts called the diff too broad (it touches src/kernel/log.ts), so I picked 14 files covering secrets, leaks, crew, forks and hooks: 118 pass, 2 skipped, 0 fail. The new test passed three runs in a row.

To try it: run bun test test/secrets-never-surface.test.ts.

🤖 Generated with Claude Code

commented

This pull adds test/secrets-never-surface.test.ts, which plants one secret and searches every surface the forge writes for it (#13481). It also adds a rule to pre-receive: an agent's own push that carries a secret it holds is refused.

The check failure. The gate rejected two Bun.sleep(20) polls in the test's setup (lines 172 and 204) under the rule "a test does not wait on a real timer". Both loops wait on real processes, so a fake clock can't make them finish sooner:

  • line 172 waits for the deployed app to write its first lines to its deploy log;
  • line 204 waits for the crew's run to open its pull.

Each line now ends with a // real-timer: <why> comment saying so, which is how the rule allows a real wait. No other code changed. bun scripts/gate.ts, bun x tsc --noEmit and the test file all pass: 3 tests, about 8 s.

Not done: I didn't replace these waits with an injected clock. The deploy and the crew run are separate processes, and the test's point is to read what they really write.

To try it: bun test test/secrets-never-surface.test.ts. It names every surface it searched. When an agent tries to git push a commit holding its granted secret, the push is refused.

🤖 Generated with Claude Code

commented

This pull adds test/secrets-never-surface.test.ts, which plants one secret and searches every surface the forge writes for it (#13481). It also adds a rule to pre-receive: an agent's own push that carries a secret it holds is refused.

The check failures. The test's setup waits in two loops, and each wait line is marked // real-timer: <why>, because both wait on real processes:

  • one waits for the deployed app to write its first log lines;
  • the other waits for the crew's run to open its pull.

The comment at the end of the pull-wait line confused the fall-through-wait detector that test/flake.test.ts runs. That detector takes a loop's body to end at the first line ending in ;, so it read past the if (pull === undefined) throw check and reported the wait as unchecked. Each wait now sits in braces, and each loop is followed by its own check. The deploy-log wait now throws "the deployed app never wrote its first lines" where it used to carry on silently. The detector reports nothing for the file. bun scripts/gate.ts, bun x tsc --noEmit, the secrets test and test/flake.test.ts all pass here.

Not changed: the other failing test, test/talk-page.test.ts ("the page's own bytes are held"), fails the same way on main at 18b9f9c. The check doesn't count it against this pull, and this pull doesn't touch it.

To try it: bun test test/secrets-never-surface.test.ts. It names every surface it searched. When an agent tries to git push a commit holding its granted secret, the push is refused.

🤖 Generated with Claude Code

Approved

No high findings. Pre-receive refuses an agent push whose new commits (tips --not --all, quarantine env) carry a held value of 6+ chars in any encoded form. Delete pushes are skipped and human pushes are untouched. The test plants one sentinel, runs the app and two echoing crew runs, searches named surfaces, and covers the push refusal. Low notes, not blocking: (1) the search reads git log -p output for every agent push while anything is held, so a very large push costs memory, so consider a cap that refuses over the cap. (2) heldLongForms is process-wide, so a push in one repo is refused for another run's secret; that is safe, just broad. No README change, no new dependency.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs