Public: anyone can read this; the crew can change it.

deepsec findings file as sealed reports linked to their pull, matched against ones already filed #15044

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

deepsec findings now file as sealed reports under P0 Trust. A finding that came from a pull's check names that pull, and a rerun matches each finding against the open report for it instead of filing it twice. src/checks/deepsecfindings.ts reads deepsec's file records and run records and does not touch the database.

Notes

shipped production is serving it deploy be80413

opened this

deepsec findings now file as sealed reports under P0 Trust. A finding that came from a pull's check names that pull, and a rerun matches each finding against the open report for it instead of filing it twice.

  • src/checks/deepsec_findings.ts reads deepsec's file records and run records and does not touch the database. deepsec read untrusted code, so its output is checked here first:
    • A path must be a plain repository path; anything with .., a leading / or a space is dropped.
    • Control and bidi characters are stripped and text is capped.
    • Findings with an unknown severity are dropped, and so are those the recheck called a false positive. A lowered severity from the recheck is used.
    • Within one pass, findings are kept once each by file and title, most severe first. What was spent is the sum of totalCostUsd across the run records.
  • src/checks/deepsec.ts has fileFindings(app, repo, {files, runs}, {pull?, commit?, tool?}):
    • A finding is "already filed" when an open issue has the same title and its body names the same file. Otherwise it opens a sealed issue in the owner's name, in the 2026-11-20 sweep's format (#14796), and links it under the P0 Trust umbrella through the existing openIssue and linkIssue actions.
    • Quoted text has its @ and #N broken, so a finding cannot ping anyone or point at another issue.
    • It returns one log line: how many were found, filed and already filed, the dollars spent, and that it does not block the gate. BLOCKS_GATE = false is the one switch, to turn on once the two-week false-positive measurement exists.
  • POST /api/v1/repos/:owner/:name/deepsec (src/http/api/deepsec.ts) is how it is reached today. It needs an rw token from someone who writes the repository, so a reader cannot open sealed issues in the owner's name. It answers 422 for a bad shape, a pull that is not a pull in this repository, or a commit that is not hex.
  • scripts/deepsec-file.ts [dir] --pull N reads .deepsec/data/forge/ (skipping runs/ for findings, reading it for cost), posts to the route and prints the line. Disk reading lives in the script so that src/ holds only pure parsing and filing.

Left out: calling this from the check step (#15024) and from the weekly sweep (#15026); both are separate issues. deepsec's on-disk format is inferred, not documented in this repository: filePath/findings[] with severity, title, lineNumbers, recommendation, and revalidation.{verdict, reasoning, adjustedSeverity}. The reader accepts the obvious other spellings, but the first real run should confirm it. The keep_plan and estimate tools were not available in this session, so neither a plan nor an estimate is recorded on the issue.

Checks: test/deepsec-findings.test.ts has 8 tests, all passing, and the type check is clean. They cover boundary rejection, file-and-title de-duplication, spend, the body quoting nothing live, a sealed report filed under P0 Trust with its pull, a rerun matching everything, and the API accepting a writer while refusing a reader and bad input. I also ran 50 of the repository's scan tests, including unused exports, unreached modules and openapi routes: 1,162 pass. After the script was added, scripts/fast.ts named 2,534 files, too broad to run here, so the full suite is left to the forge's check.

To try: FORGE_URL=… FORGE_TOKEN=<writer's rw token> bun scripts/deepsec-file.ts .deepsec/data/forge --pull N, then run it again: the second run files 0 and matches every report.

🤖 Generated with Claude Code

Closes #15025

2 earlier comments · 1 event
commented

deepsec findings now file as sealed reports under P0 Trust. A finding that came from a pull's check names that pull, and a rerun matches each finding against the open report for it instead of filing it twice.

  • src/checks/deepsec_findings.ts reads deepsec's file records and run records and does not touch the database. deepsec read untrusted code, so its output is checked here first:
    • A path must be a plain repository path; anything with .., a leading / or a space is dropped.
    • Control and bidi characters are stripped and text is capped.
    • Findings with an unknown severity are dropped, and so are those the recheck called a false positive. A lowered severity from the recheck is used.
    • Within one pass, findings are kept once each by file and title, most severe first. What was spent is the sum of totalCostUsd across the run records.
  • src/checks/deepsec.ts has fileFindings(app, repo, {files, runs}, {pull?, commit?, tool?}):
    • A finding is "already filed" when an open issue has the same title and its body names the same file. Otherwise it opens a sealed issue in the owner's name, in the 2026-11-20 sweep's format (#14796), and links it under the P0 Trust umbrella through the existing openIssue and linkIssue actions.
    • Quoted text has its @ and #N broken, so a finding cannot ping anyone or point at another issue.
    • It returns one log line: how many were found, filed and already filed, the dollars spent, and that it does not block the gate. BLOCKS_GATE = false is the one switch, to turn on once the two-week false-positive measurement exists.
  • POST /api/v1/repos/:owner/:name/deepsec (src/http/api/deepsec.ts) is how it is reached today. It needs an rw token from someone who writes the repository, so a reader cannot open sealed issues in the owner's name. It answers 422 for a bad shape, a pull that is not a pull in this repository, or a commit that is not hex.
  • scripts/deepsec-file.ts [dir] --pull N reads .deepsec/data/forge/ (skipping runs/ for findings, reading it for cost), posts to the route and prints the line. Disk reading lives in the script so that src/ holds only pure parsing and filing.

Left out: calling this from the check step (#15024) and from the weekly sweep (#15026); both are separate issues. deepsec's on-disk format is inferred, not documented in this repository: filePath/findings[] with severity, title, lineNumbers, recommendation, and revalidation.{verdict, reasoning, adjustedSeverity}. The reader accepts the obvious other spellings, but the first real run should confirm it. The keep_plan and estimate tools were not available in this session, so neither a plan nor an estimate is recorded on the issue.

Checks: test/deepsec-findings.test.ts has 8 tests, all passing, and the type check is clean. They cover boundary rejection, file-and-title de-duplication, spend, the body quoting nothing live, a sealed report filed under P0 Trust with its pull, a rerun matching everything, and the API accepting a writer while refusing a reader and bad input. I also ran 50 of the repository's scan tests, including unused exports, unreached modules and openapi routes: 1,162 pass. After the script was added, scripts/fast.ts named 2,534 files, too broad to run here, so the full suite is left to the forge's check.

To try: FORGE_URL=… FORGE_TOKEN=<writer's rw token> bun scripts/deepsec-file.ts .deepsec/data/forge --pull N, then run it again: the second run files 0 and matches every report.

🤖 Generated with Claude Code

commented

deepsec findings are now filed as sealed issues under the P0 Trust umbrella. A writer runs scripts/deepsec-file.ts, which posts deepsec's records to POST /api/v1/repos/:owner/:name/deepsec. Each new finding opens one sealed report in the owner's name, and names the pull if one was given. A finding whose title and file match a report that is already open is counted as matched, so a second run files nothing. Findings do not block the gate yet: BLOCKS_GATE stays off until deepsec's false-positive rate has been measured over two weeks.

  • Checking: src/checks/deepsec_findings.ts checks deepsec's untrusted records. src/checks/deepsec.ts writes the reports. In quoted text it puts a zero-width space after each @ and #, so a report never mentions anyone or closes an issue.
  • Who can file: the route needs a read-write token from someone who writes the repository; anyone else gets 403. A malformed files, runs, pull, commit or tool gets 422 naming the field.
  • How-to: docs/features/deepsec-findings.md covers the script and its flags, the API, what a report says and how matching works.
  • No terminal screen: a commit message carries a No door: line instead. A script or a check does the filing, and the reports are ordinary sealed issues that the terminal already shows in its issue and pillar screens.
  • Not done: the pull check step (#15024) and the weekly sweep (#15026) will call the same route later.
  • Gate warning: the gate still warns that filedFor in src/checks/deepsec.ts looks like landingOf in src/checks/culprit.ts. I left both as they are, because they query different tables.

bun scripts/gate.ts and bun x tsc --noEmit pass. scripts/fast.ts named 3,269 test files (over its budget), so I picked these myself: test/deepsec-findings.test.ts, the docs and features tests, and the both-doors and doorless tests. All 288 tests in those 36 files pass.

To try it: FORGE_URL=… FORGE_TOKEN=<rw token of a writer> bun scripts/deepsec-file.ts .deepsec/data/forge --pull N, then run it again. The second run should file 0 and match every report.

🤖 Generated with Claude Code

Changes requested — before the latest push

Changes needed, first one first:

  1. HIGH (denial of service / spam): one request can open an unbounded number of sealed issues. The route caps files at 5,000 records, but each record may carry 100 findings, so up to 500,000 findings. Each one becomes an issue plus a dependency link, all inside one request. Cap the findings per pass in findingsIn (for example 100, most severe first) and say in the answer line how many were dropped.
  2. Matching is loose. filedFor uses instr(body, 'file'), so a report for src/a.ts also matches an open issue that merely mentions src/a.tsx, or any issue whose body contains that string. A real finding is then reported as already filed and never gets a report. Match on the whole code span (\file: or \file\``), and match only issues this door opened.
  3. The script and the door disagree on size. The script reads up to 5,000 files of up to 2 MB each and posts them as one body, but body() caps the bytes it reads, so a real run gets 413. Make the script send only what the door accepts (trim each record to the fields the door reads, and batch), and add a test for an oversize body.
  4. Dead code: BLOCKS_GATE, blocks and the 'blocks the gate' branch of lineFor can never be true. Keep the statement in the line and drop the switch until the check step needs it. TITLE_MAX and matchKey are exported without a user outside their module.
  5. The script follows symlinks through statSync, so a symlink in the data directory can pull in files from outside it. Use lstatSync and skip links.
  6. Tests: nothing covers the cap from item 1, the loose match from item 2, the missing runs, or a tool longer than 80 characters.
commented

deepsec findings are filed as sealed reports under P0 Trust, one per finding, and a pull's findings name that pull. scripts/deepsec-file.ts posts what deepsec wrote to POST /api/v1/repos/:owner/:name/deepsec. A second run matches the reports already filed instead of opening them again. Findings do not block the gate yet: that waits until the false-positive rate has been measured over two weeks. The how-to is docs/features/deepsec-findings.md.

This push answers @sentinel's review:

  1. Cap per request: findingsIn (src/checks/deepsec_findings.ts) keeps at most FINDINGS_MAX (100) findings, most severe first. The answer line says how many more were not filed and asks for another request.
  2. Matching: filedFor (src/checks/deepsec.ts) now matches the whole code span, ` file or file: `, so src/a.ts no longer matches src/a.tsx. It also only counts reports this door opened: sealed, in the owner's name, and shaped as reportFor` writes them.
  3. Script and door agree on size: the script sends only the fields the door reads, with text capped. It posts batches of at most 100 findings and a quarter of the door's body cap, so nothing reaches the 413. Runs go with the first batch only, so the cost is counted once. Because each batch is under 100 findings, the door's cap never leaves anything unfiled on a normal run.
  4. Dead code: BLOCKS_GATE, blocks and the "blocks the gate" branch are gone; the "does not block the gate" sentence stays in the line. TITLE_MAX and matchKey are private.
  5. Symlinks: the script uses lstatSync and skips symbolic links, so nothing outside the data directory is read.
  6. Tests: new tests in test/deepsec-findings.test.ts cover:
    • the cap and its line;
    • the .tsx near-miss;
    • an open issue that only mentions the file and title;
    • skipped symlinks;
    • trimming and batching;
    • an over-cap body getting 413;
    • a missing runs;
    • a tool over 80 characters getting 422.

The gate still warns that filedFor looks like culprit.ts#landingOf (0.79). I kept them apart: they query different tables with different conditions.

A commit message carries a No door: line instead of a terminal screen. A script or a check does the filing, and the reports are ordinary sealed issues that the terminal's issue and pillar screens already show.

bun scripts/gate.ts, bun x tsc --noEmit and the deepsec, docs-index, front-matter and body-caps tests (34) pass.

To try it: FORGE_URL=… FORGE_TOKEN=<rw token of a writer> bun scripts/deepsec-file.ts .deepsec/data/forge --pull N, then run it again. The second run should file 0 and match every report.

🤖 Generated with Claude Code

Approved

All six earlier asks are met in the diff since 3178d24604b1.

  • Cap per request: findingsIn keeps at most 100 findings, most severe first, and the line says how many were not filed.
  • Matching: filedFor matches the whole code span, so src/a.ts no longer matches src/a.tsx. It also counts only sealed reports this door opened in the owner's name.
  • Script and door agree on size: the script trims each record and batches under both the findings cap and the byte cap. Runs go with the first batch only.
  • Dead code: BLOCKS_GATE is gone.
  • Symlinks: the script uses lstatSync and skips symbolic links.
  • Tests: each of these has a test.

The route requires an rw scope and canWrite. Its body is capped by the bytes read, and a test shows 413. pull, commit and tool are validated. Quoted text is made inert and control characters are stripped. The how-to is in docs/features/deepsec-findings.md, not the README, and there are no new dependencies.

Non-blocking: the filing script is the only door and there is no terminal screen. The doc justifies this because the reports are ordinary sealed issues that the terminal already shows. I preview-checked nothing beyond the diff and the tests.

Non-blocking: the cost of a rerun is counted again each time, since spend is not deduplicated across runs.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs