Public: anyone can read this; the crew can change it.

A test proves a guest pull's check runs with no secrets, and has them after a person approves #9038

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

A new end-to-end test checks the secrets a guest pull's check and preview actually hold. It covers the issue except one clause, which I left out on purpose: the test does **not** show the check getting the secrets after a writer approves. A check holds none of the repository's secrets, for any pull, guest or not (`docs/features/guest-pulls.md`, `src/checks/service.ts`). The issue's "after a writer…

shipped production is serving it deploy d5155e2

opened this

A new end-to-end test checks the secrets a guest pull's check and preview actually hold. It covers the issue except one clause, which I left out on purpose: the test does not show the check getting the secrets after a writer approves. A check holds none of the repository's secrets, for any pull, guest or not (docs/features/guest-pulls.md, src/checks/service.ts). The issue's "after a writer approves, the same check has them" would mean starting to pass the app's variables into checks. That is a trust decision for a person to make, not something to slip into a test pull. If it is ever made, the check log would also need secrets hidden from readers, and the "already proven green" cache would need the secrets in its key: today a green tree is not re-run.

test/guest-pull-env.test.ts starts a forge with its apps runtime and checks running, and an app with API_KEY set. The app's check is env, and its server answers with whether it holds API_KEY and a FORGE_TOKEN. It opens a pull that is a guest from birth: a temporary SQL trigger names an origin author on the pull's issue before anything is queued, the same marking a pull brought in from another forge carries. Then it reads what the real processes hold:

  • Check while the guest waits: prints its environment, with no API_KEY, no secret value and no FORGE_TOKEN.
  • Preview while the guest waits: answers with no key and no token.
  • Preview after a writer approves through approveGuestPull: restarts and answers with both.
  • Check after the guest pushes again: still none of the secrets.
  • Preview after that push: keeps the approval.

The existing test/guest-pull.test.ts only asked secretsFor what it would hand out; this reads what the running processes actually hold. No runtime code changed, and docs/features/guest-pulls.md now names the test.

I checked that the test catches regressions by breaking the code twice. Turning off the guest gate in AppsRuntime.secretsFor makes it fail on the preview. Adding API_KEY to the check environment makes it fail on the check log.

To try it: bun test test/guest-pull-env.test.ts. The gate, bun x tsc --noEmit and the files scripts/fast.ts named plus test/guest-pull.test.ts all pass. In this sandbox (FORGE_TEST_SOCKET) the check runs outside a cell, as its log says, so the forge's own check will run it contained. The keep_plan tool the issue asks for was not available in this run, so the issue has no plan.

🤖 Generated with Claude Code

Closes #9018

commented

A new end-to-end test checks the secrets a guest pull's check and preview actually hold. It covers the issue except one clause, which I left out on purpose: the test does not show the check getting the secrets after a writer approves. A check holds none of the repository's secrets, for any pull, guest or not (docs/features/guest-pulls.md, src/checks/service.ts). The issue's "after a writer approves, the same check has them" would mean starting to pass the app's variables into checks. That is a trust decision for a person to make, not something to slip into a test pull. If it is ever made, the check log would also need secrets hidden from readers, and the "already proven green" cache would need the secrets in its key: today a green tree is not re-run.

test/guest-pull-env.test.ts starts a forge with its apps runtime and checks running, and an app with API_KEY set. The app's check is env, and its server answers with whether it holds API_KEY and a FORGE_TOKEN. It opens a pull that is a guest from birth: a temporary SQL trigger names an origin author on the pull's issue before anything is queued, the same marking a pull brought in from another forge carries. Then it reads what the real processes hold:

  • Check while the guest waits: prints its environment, with no API_KEY, no secret value and no FORGE_TOKEN.
  • Preview while the guest waits: answers with no key and no token.
  • Preview after a writer approves through approveGuestPull: restarts and answers with both.
  • Check after the guest pushes again: still none of the secrets.
  • Preview after that push: keeps the approval.

The existing test/guest-pull.test.ts only asked secretsFor what it would hand out; this reads what the running processes actually hold. No runtime code changed, and docs/features/guest-pulls.md now names the test.

I checked that the test catches regressions by breaking the code twice. Turning off the guest gate in AppsRuntime.secretsFor makes it fail on the preview. Adding API_KEY to the check environment makes it fail on the check log.

To try it: bun test test/guest-pull-env.test.ts. The gate, bun x tsc --noEmit and the files scripts/fast.ts named plus test/guest-pull.test.ts all pass. In this sandbox (FORGE_TEST_SOCKET) the check runs outside a cell, as its log says, so the forge's own check will run it contained. The keep_plan tool the issue asks for was not available in this run, so the issue has no plan.

🤖 Generated with Claude Code

Approved

Approved. The test reads what the real check and preview processes hold, not what secretsFor would hand out, and it covers waiting, approval and a later push. It changes no runtime code, adds no dependency, and the doc is in docs/features/guest-pulls.md rather than the README. There is no UI change, so the screenshots do not apply. The two things I would do next: (1) decide, as a person, whether checks should ever get app secrets; the issue box 1 clause wants that, and torque rightly did not slip it into a test pull. If yes, it needs log redaction and the secrets in the green-cache key. Otherwise reword the issue. (2) Replace the temporary SQL trigger that makes the pull a guest with a small helper in test/helpers that opens a pull as an origin guest, so other guest tests can share it.

commented

Left #9018 open: 1 unticked item remain (A test opens a pull from another forge, runs its check and finds none of the repository's secrets in the environment; after a writer approves, the same check has them.).

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs