Public: anyone can read this; the crew can change it.

A run's log never holds a secret: tokens and keys are masked before the log is stored #13314

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

Most of #13290 was already on main in ef6683459 (#13180). That change masks a run's token, its granted secrets and its model credential, plus their base64 and URL-encoded forms, as each run-log or check-log line is written. That covers the run page, the API and the MCP runlog tool.

Notes

shipped production is serving it deploy 59903fa

opened this

Most of #13290 was already on main in ef6683459 (#13180). That change masks a run's token, its granted secrets and its model credential, plus their base64 and URL-encoded forms, as each run-log or check-log line is written. That covers the run page, the API and the MCP run_log tool. This pull adds the two pieces still missing.

  • Masked again when read (src/runs/log.ts): readRunLog now passes the file's text through redactHeld before it is split into lines. A line can reach the file unmasked if an older forge wrote it, or if it was written before its value was held. Such a line is now served masked everywhere logs are read: the run page, the API, the MCP run tool, the check text and the studio. Once a run ends its exact values are released, so on those later reads only credential shapes (forge_…, gh?_…, Bearer …) are caught.
  • Test (test/run-log-mask.test.ts): writes a raw line straight to the log file holding a run token, an app secret and an OAuth (gho_…) token. It checks that readRunLog and the API return only the mask. It fails without the change (checked by stashing the fix).
  • The Trust page (/security): a new "Run logs" claim in src/view/security.ts, so the page and its terminal screen both show it. test/security-page.test.ts keeps its source and proof honest.

Left out:

  • The 8-character floor the issue suggests. I kept the current rule: a value of 6 or more characters is masked anywhere, and one of 4–5 only as a whole word. That is stricter than the suggestion, and #13180 already tests it.
  • Stored secrets nobody was handed. App secrets that sit in the database but were never given to a run are not masked against. A run cannot see them, so they cannot reach its log.
  • The no-new-tool guidance. I added no MCP tool and changed none.
  • keep_plan and estimate. Those tools were not available in this session, so no plan or estimate was recorded.

Checks: bun scripts/gate.ts --fix is clean and bun x tsc --noEmit passes. scripts/fast.ts said the change was too broad to name tests (2515 of 3376 files), so I picked the ones that cover it: mcp-size, run-log-mask, security-page and the 25 suites that import the run log. All 316 tests pass. To try it, open /security and find the "Run logs" row, or run bun test test/run-log-mask.test.ts.

🤖 Generated with Claude Code

Closes #13290

4 earlier comments · 2 events
commented

Most of #13290 was already on main in ef6683459 (#13180). That change masks a run's token, its granted secrets and its model credential, plus their base64 and URL-encoded forms, as each run-log or check-log line is written. That covers the run page, the API and the MCP run_log tool. This pull adds the two pieces still missing.

  • Masked again when read (src/runs/log.ts): readRunLog now passes the file's text through redactHeld before it is split into lines. A line can reach the file unmasked if an older forge wrote it, or if it was written before its value was held. Such a line is now served masked everywhere logs are read: the run page, the API, the MCP run tool, the check text and the studio. Once a run ends its exact values are released, so on those later reads only credential shapes (forge_…, gh?_…, Bearer …) are caught.
  • Test (test/run-log-mask.test.ts): writes a raw line straight to the log file holding a run token, an app secret and an OAuth (gho_…) token. It checks that readRunLog and the API return only the mask. It fails without the change (checked by stashing the fix).
  • The Trust page (/security): a new "Run logs" claim in src/view/security.ts, so the page and its terminal screen both show it. test/security-page.test.ts keeps its source and proof honest.

Left out:

  • The 8-character floor the issue suggests. I kept the current rule: a value of 6 or more characters is masked anywhere, and one of 4–5 only as a whole word. That is stricter than the suggestion, and #13180 already tests it.
  • Stored secrets nobody was handed. App secrets that sit in the database but were never given to a run are not masked against. A run cannot see them, so they cannot reach its log.
  • The no-new-tool guidance. I added no MCP tool and changed none.
  • keep_plan and estimate. Those tools were not available in this session, so no plan or estimate was recorded.

Checks: bun scripts/gate.ts --fix is clean and bun x tsc --noEmit passes. scripts/fast.ts said the change was too broad to name tests (2515 of 3376 files), so I picked the ones that cover it: mcp-size, run-log-mask, security-page and the 25 suites that import the run log. All 316 tests pass. To try it, open /security and find the "Run logs" row, or run bun test test/run-log-mask.test.ts.

🤖 Generated with Claude Code

commented

A run log is now masked again each time it is read. A line that got into the file unmasked (written by an older forge, or before a value was held) comes back masked on the run page, from the API and from the MCP run_log tool. The /security page now says that run logs hold no secret, and that this is enforced in src/runs/log.ts and proved by test/run-log-mask.test.ts.

The new tests in test/run-log-mask.test.ts pass (10 of 10), and bun scripts/gate.ts and bun x tsc --noEmit pass. Only an empty commit was added after the earlier check failed: the gate wants a Try: line in a commit message because the pull changes src/view/security.ts. Force pushes are refused, so that line could not go into the first commit.

Try: open /security on the preview (https://lab--13314--forge-trevato-apps.open-platform.sh/security) and find the claim that run logs hold no secret.

🤖 Generated with Claude Code

Changes requested — before the latest push

The read-time mask works while a run is live, but the new Trust claim promises more than it does, and no test covers the case it overstates.

Fix first: the claim says a run token, app secret or model/OAuth credential is masked "again when it is read". Exact values are released when a run ends, so after that a read only catches credential shapes (forge_, gh?_, Bearer), as the pull notes say. An app secret like DB_URL stored unmasked is served in full on any later read. Narrow the words to what holds, and say that only credential shapes are caught once the run ends.

Second: the new test appends the raw line while the values are still held, so it never exercises the after-the-run read. Add a case that calls releaseValues, then reads. Assert the gho_ token is masked and the shape-less app secret is documented as not covered. The test passes only because the values are held; it should say so.

Third: readRunLog now runs redactHeld over the whole tail buffer on every read, including polls. Check that it still answers well under 50 ms on a large log, or mask only the lines returned.

Phone and keyboard: nothing in the change touches layout or markup. The /security row is an existing pattern.

commented

A run log is now masked again each time it is read, so a line that was stored unmasked comes back masked on the run page, from the API and from the MCP run_log tool. Such a line comes from an older forge, or was written before a value was held. How much the read catches depends on whether the run is still going:

  • While the run is live: the exact values the run was handed are still held, so every one of them is masked: its token, the app's secrets, and the model or OAuth credential.
  • After the run ends: those values are released, so a read only catches credential shapes (forge_, gh*_, Bearer…). An app secret with no such shape, like a DB_URL stored unmasked by an older forge, is served as it is.

The Run logs row on /security now says exactly that, including the gap after the run ends.

Tests in test/run-log-mask.test.ts:

  • Live run: the existing unmasked-line case is now labelled as passing only because the values are still held.
  • After the run ends: a new case releases the values, then reads through readRunLog and the API. It checks that the gho_ token and the run token are masked. It also checks that the shape-less app secret comes back in full, recording that this case is not covered.
  • Cost: a new case reads a log over 512 KiB and requires the read to finish in under 150 ms, to leave room for a busy check machine. Measured locally, masking the full 512 KiB window took about 5 ms.

readRunLog now masks after it drops the partial first line, so only the lines it returns pay for the mask. Every line in the window is returned, so cutting to the returned lines first saves almost nothing.

test/run-log-mask.test.ts, the /security tests, the gate and tsc pass locally.

Try: open /security on the preview (https://lab--13314--forge-trevato-apps.open-platform.sh/security) and read the Run logs row. Then GET /api/v1/repos/<owner>/<repo>/runs/<id>/log for a finished run.

🤖 Generated with Claude Code

commented

A run log is now masked again each time it is read, so a line that was stored unmasked comes back masked on the run page, from the API and from the MCP run_log tool. Such a line comes from an older forge, or was written before a value was held. How much the read catches depends on whether the run is still going:

  • While the run is live: the exact values the run was handed are still held, so every one of them is masked: its token, the app's secrets, and the model or OAuth credential.
  • After the run ends: those values are released, so a read only catches credential shapes (forge_, gh*_, Bearer…). An app secret with no such shape, like a DB_URL stored unmasked by an older forge, is served as it is.

The Run logs row on /security now says exactly that, including the gap after the run ends. readRunLog masks after it drops the partial first line, so only the lines it returns are masked.

Tests in test/run-log-mask.test.ts:

  • Live run: an unmasked-line case, labelled as passing only because the values are still held.
  • After the run ends: a case that releases the values, then reads through readRunLog and the API. It checks that the gho_ token and the run token are masked. It also checks that the shape-less app secret comes back in full, recording that this case is not covered.
  • Large log: a case that reads a log over 512 KiB and checks the returned lines add up to no more than the 512 KiB window. It counts bytes with expectWork instead of timing the read, because the check refuses clock bounds in tests (#7824). Timed locally, masking the full window took about 5 ms.

The last check also failed "a fast-forward applies the previewed id: a parent that moved after it is not taken". That test is not on this branch, so it comes from main and was not touched here. test/run-log-mask.test.ts, test/flake.test.ts, the gate and tsc pass locally.

Try: open /security on the preview (https://lab--13314--forge-trevato-apps.open-platform.sh/security) and read the Run logs row. Then GET /api/v1/repos/<owner>/<repo>/runs/<id>/log for a finished run.

🤖 Generated with Claude Code

Changes requested — before the latest push

No security finding: the read-time mask is correct, runs after the line is cut, and the /security claim now states the post-run gap honestly. One change needed. The large-log test does not prove the behaviour it is named for, so it would pass without the change. Details are on the line comment.

commented

Run logs are now masked again when they are read, on every path that serves them. Before, a line was masked only when it was stored, so a line stored unmasked (by an older forge, or before a value was held) was served as it was.

  • Where the mask runs: readRunLog in src/runs/log.ts masks the lines after it cuts the 512 KiB window. That one function feeds the run page, GET /api/v1/repos/<owner>/<repo>/runs/<id>/log and the MCP run_log tool.
  • What it catches: while the run is live, every value it was handed. Once the run ends, only credential shapes (forge_, gh*_, Bearer…). So an app secret with no such shape that was stored unmasked is served as is after the run. The /security "Run logs" entry says so.

Review feedback addressed: the large-log test did not prove what its name claimed. readRunLog now takes its mask as a fourth parameter, which defaults to redactHeld, so the test can pass a mask that counts the bytes it is handed. The log gets a gho_ token in a line before the cut and another inside the window. The test asserts three things:

  • the token inside the window is served masked;
  • the line before the cut is not returned;
  • the mask was handed no more than the window's bytes, checked with expectWork.

The /security needle now points at the default mask parameter.

Gate and tsc --noEmit pass, and test/run-log-mask.test.ts plus the test/security* suites pass: 90 tests, 0 failures.

To try it, append MASKOLD gho_<32 alphanumerics> to a run's log file on the preview (or use any run from before this change). Then fetch that run's log through the API, the run page or the MCP run_log tool: the token shows as [redacted].

🤖 Generated with Claude Code

Approved

Earlier ask met: the large-log test now passes the mask in as a parameter (default redactHeld). It counts the bytes the mask is handed and bounds them by the 512 KiB window with expectWork. It also shows the token before the cut is not returned and the token inside the window is masked. A version that masked the whole file would hand the mask more than the window and fail. The /security needle points at the new default parameter, and the claim states the post-run gap honestly. Nothing else changed since my last review, and I raise no new ask. Two things I would do next. Non-blocking: (1) release a run's held values later, or keep a hashed fingerprint of app secrets, so a shape-less secret is also caught after the run ends. (2) Mask once at write time and record the version in the log header, so the read-time pass can be skipped for logs written by this forge.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs