Public: anyone can read this; the crew can change it.
Weekly deepsec sweep as a durable forge job, bounded by security.sweep_max_usd #15223
Check · Review · Land · Live done — Landed and live in production.
torque/forge:torque/15026-weekly-deepsec-sweep-as-a-durable-forge → main 3 commits fork
12 files
changed +751
−93
AGENTS.md +2
| @@ -62,6 +62,8 @@ The check queue hands work to a runner through `src/checks/runner.ts` (#13652, R | ||
| 62 | 62 | |
| 63 | 63 | `src/actions/drop.ts`: `uploading`, per forge (a WeakMap on the app), each uploader's id and how many of their drop uploads are in flight, at most `UPLOADS_AT_ONCE` each (#14778); an id leaves when its count reaches zero, so it holds at most the people uploading now. Named as `drop uploads by person` (`heldOf`). |
| 64 | 64 | |
| 65 | `src/maintain/duties/deepsec_sweep.ts`: `running`, the one weekly deepsec sweep this process is running (#15026), so a tick never starts a second; one promise, gone on restart, when `<data>/deepsec-sweep/state.json` says the sweep was cut short and the next tick resumes it. | |
| 66 | ||
| 65 | 67 | `src/apps/timer_fire.ts`: `inFlight`, the ids of timers being fired in this process, so a slow app's timer is never fired twice at once; at most the timers due, gone when each settles. The timers themselves are rows (`timer`), read at every boot, never memory. |
| 66 | 68 | |
| 67 | 69 | `src/actions/jobs.ts`: `jobReach.override`, a test seam only (a test hands in its own way to reach an app); the real one is `runtimeReach`. A durable job's runs are the app's own rows (`src/jobs.ts`, carried in the app template bundle `templates/app-template.json`; the template repository `trevato/app-template` must take the same files before a forge that reads that repository hands them out); the template's `Jobs` keeps in process memory only the ids of runs being advanced now (`active`), so a restart forgets nothing it needs. |
docs/features/deepsec-findings.md +1 −1
| @@ -38,4 +38,4 @@ Filing is not a terminal screen: a script or a check runs it. The reports it ope | ||
| 38 | 38 | |
| 39 | 39 | ## Not yet |
| 40 | 40 | |
| 41 | A finding does not block the gate yet. It will block it once deepsec's false-positive rate has been measured over two weeks of runs. A pull's check step (#15024) and a weekly sweep (#15026) will call the same door. Today you run the script yourself. | |
| 41 | A finding does not block the gate yet. It will block it once deepsec's false-positive rate has been measured over two weeks of runs. The weekly sweep (#15026, `src/checks/deepsec_sweep.ts`) files through the same path. You can still run the script yourself on a run of your own. |
docs/features/deepsec.md +9
| @@ -33,6 +33,15 @@ The review never changes the check's verdict. It does not block the gate until i | ||
| 33 | 33 | |
| 34 | 34 | What is left: a prompt planted in the files reviewed could steer the agent to send the key to `api.anthropic.com`, the one host it can reach. The key cannot leave for anywhere else. |
| 35 | 35 | |
| 36 | ## Every week | |
| 37 | ||
| 38 | The forge sweeps its own repository (`FORGE_SENTINEL_REPO`, else `trevato/forge`) once a week. The maintenance duty `deepsec-sweep` (`src/maintain/duties/deepsec_sweep.ts`, #15026) ticks daily and starts `src/checks/deepsec_sweep.ts` a week after the last sweep began. It exports the default branch's tree, then runs `scan`, `process --limit n` and `revalidate --min-severity HIGH` in the same cell, with the same key and the same config as a pull's review. What it wrote is filed through the same path as a pull's findings: each new one a sealed report under P0 Trust, each one already open matched. | |
| 39 | ||
| 40 | - **Durable.** Its workspace is `<data>/deepsec-sweep/`. deepsec's record of what it processed is kept there, so the files past one week's ceiling are next week's. `state.json` says when the sweep began and on which commit. A sweep a restart cut short is resumed on the next tick, at most 3 tries, with only what is left of its ceiling. | |
| 41 | - **Only its own records.** Only the records written since the sweep began are filed, so a finding closed last month is never filed again from an old record. | |
| 42 | - **What it did.** Its end is a row on `/maintenance` with the line and what it spent, for example `deepsec sweep of main at 1a2b3c4d5 under a $25.00 ceiling: ran scan, process --limit 92, revalidate; 3 findings, 1 filed as sealed reports (#15301), 2 already filed (#14801, #14803); spent $24.30; …`. deepsec's own output, with the key masked, is in `<data>/deepsec-sweep/sweep.log`. | |
| 43 | - **Skipped.** A $0 ceiling, no key, no cell, or no scope on the default branch is said on that row, and so is a tree that could not be exported. A skip writes no state (a resumed sweep's failed export counts as one of its 3 tries), so the next day's tick tries again. | |
| 44 | ||
| 36 | 45 | ## Spend |
| 37 | 46 | |
| 38 | 47 | `process` has no spend cap of its own. The cap is a file count: floor(ceiling ÷ $0.27). $0.27 a file was measured on the 2026-11-20 sweep at thinking `high`; `xhigh` cost about six times that. |
scripts/deepsec-file.ts +5 −82
| @@ -19,14 +19,13 @@ | ||
| 19 | 19 | * link in the data directory is skipped: nothing outside it is read. |
| 20 | 20 | */ |
| 21 | 21 | |
| 22 | import { lstatSync, readdirSync, readFileSync } from "node:fs"; | |
| 23 | import { join } from "node:path"; | |
| 24 | import { clean, FINDINGS_MAX, spentIn, TEXT_MAX } from "../src/checks/deepsec_findings.ts"; | |
| 22 | import { FINDINGS_MAX } from "../src/checks/deepsec_findings.ts"; | |
| 23 | import { readWorkspace } from "../src/checks/deepsec_records.ts"; | |
| 25 | 24 | import { BODY_MAX } from "../src/http/limits.ts"; |
| 26 | 25 | |
| 27 | const FILES_MAX = 5_000; | |
| 28 | const BYTES_MAX = 2_000_000; | |
| 29 | const DEPTH_MAX = 12; | |
| 26 | // The reader is the forge's own, shared with the weekly sweep (`src/checks/deepsec_records.ts`). | |
| 27 | export { readWorkspace, trimRecord } from "../src/checks/deepsec_records.ts"; | |
| 28 | ||
| 30 | 29 | /** |
| 31 | 30 | * A quarter of the door's body cap: room for the JSON around the records and for multibyte text. |
| 32 | 31 | */ |
| @@ -37,82 +36,6 @@ function fail(message: string): never { | ||
| 37 | 36 | process.exit(1); |
| 38 | 37 | } |
| 39 | 38 | |
| 40 | /** Every `.json` under `dir`, bounded; `runs/` at the top is left to `runRecords`. */ | |
| 41 | function jsonFiles(dir: string, depth = 0, acc: string[] = []): string[] { | |
| 42 | if (depth > DEPTH_MAX || acc.length >= FILES_MAX) return acc; | |
| 43 | let names: string[]; | |
| 44 | try { | |
| 45 | names = readdirSync(dir).sort(); | |
| 46 | } catch { | |
| 47 | return acc; | |
| 48 | } | |
| 49 | for (const name of names) { | |
| 50 | if (acc.length >= FILES_MAX) break; | |
| 51 | if (depth === 0 && name === "runs") continue; | |
| 52 | const path = join(dir, name); | |
| 53 | const st = lstatSync(path, { throwIfNoEntry: false }); | |
| 54 | if (st === undefined || st.isSymbolicLink()) continue; | |
| 55 | if (st.isDirectory()) jsonFiles(path, depth + 1, acc); | |
| 56 | else if (st.isFile() && name.endsWith(".json") && st.size <= BYTES_MAX) acc.push(path); | |
| 57 | } | |
| 58 | return acc; | |
| 59 | } | |
| 60 | ||
| 61 | function parse(path: string): unknown { | |
| 62 | try { | |
| 63 | return JSON.parse(readFileSync(path, "utf8")); | |
| 64 | } catch { | |
| 65 | return undefined; | |
| 66 | } | |
| 67 | } | |
| 68 | ||
| 69 | const hasFindings = (v: unknown): boolean => | |
| 70 | typeof v === "object" && v !== null && Array.isArray((v as { findings?: unknown }).findings) && (v as { findings: unknown[] }).findings.length > 0; | |
| 71 | ||
| 72 | const obj = (v: unknown): Record<string, unknown> => | |
| 73 | typeof v === "object" && v !== null && !Array.isArray(v) ? (v as Record<string, unknown>) : {}; | |
| 74 | const text = (v: unknown): string | undefined => (typeof v === "string" ? clean(v, TEXT_MAX) : undefined); | |
| 75 | ||
| 76 | /** | |
| 77 | * A recheck as the door reads it: its verdict, reasoning and adjusted severity, or its bare word. | |
| 78 | */ | |
| 79 | function trimRecheck(v: unknown): unknown { | |
| 80 | if (typeof v === "string") return clean(v, 200); | |
| 81 | const r = obj(v); | |
| 82 | return { verdict: text(r["verdict"]), reasoning: text(r["reasoning"]), adjustedSeverity: text(r["adjustedSeverity"]) }; | |
| 83 | } | |
| 84 | ||
| 85 | /** One file record cut to the fields the door reads, at most `FINDINGS_MAX` findings. */ | |
| 86 | export function trimRecord(raw: unknown): { filePath: unknown; findings: unknown[] } { | |
| 87 | const r = obj(raw); | |
| 88 | const findings = Array.isArray(r["findings"]) ? r["findings"].slice(0, FINDINGS_MAX) : []; | |
| 89 | return { | |
| 90 | filePath: typeof r["filePath"] === "string" ? r["filePath"] : r["path"], | |
| 91 | findings: findings.map((item) => { | |
| 92 | const f = obj(item); | |
| 93 | const recheck = f["revalidation"] ?? f["revalidate"]; | |
| 94 | return { | |
| 95 | severity: text(f["severity"]), | |
| 96 | title: text(f["title"]), | |
| 97 | description: text(f["description"]), | |
| 98 | lineNumbers: f["lineNumbers"] ?? f["lines"], | |
| 99 | recommendation: text(f["recommendation"] ?? f["fix"]), | |
| 100 | ...(recheck === undefined ? {} : { revalidation: trimRecheck(recheck) }), | |
| 101 | }; | |
| 102 | }), | |
| 103 | }; | |
| 104 | } | |
| 105 | ||
| 106 | /** deepsec's records under `dir`: file records with findings, trimmed, and what each run spent. */ | |
| 107 | export function readWorkspace(dir: string): { files: ReturnType<typeof trimRecord>[]; runs: unknown[] } { | |
| 108 | const files = jsonFiles(dir).map(parse).filter(hasFindings).map(trimRecord); | |
| 109 | const runs = jsonFiles(join(dir, "runs")) | |
| 110 | .map(parse) | |
| 111 | .filter((r) => r !== undefined) | |
| 112 | .map((r) => ({ totalCostUsd: spentIn([r]) })); | |
| 113 | return { files, runs }; | |
| 114 | } | |
| 115 | ||
| 116 | 39 | /** |
| 117 | 40 | * The records as bodies the door accepts: each at most `FINDINGS_MAX` findings and `maxBytes` of |
| 118 | 41 | * records, the runs with the first. A record is never split; one trimmed record fits by itself. |
src/checks/deepsec.ts +4 −4
| @@ -11,10 +11,10 @@ | ||
| 11 | 11 | * findings, the most severe, and says how many it did not file. |
| 12 | 12 | * |
| 13 | 13 | * Whatever ran deepsec hands in the records it wrote and gets back one line |
| 14 | * for its log: today `scripts/deepsec-file.ts` through | |
| 15 | * `POST /api/v1/repos/:owner/:name/deepsec` (`http/api/deepsec.ts`); next | |
| 16 | * the check step on a pull's changed files (#15024) and the weekly sweep | |
| 17 | * (#15026). | |
| 14 | * for its log: `scripts/deepsec-file.ts` through | |
| 15 | * `POST /api/v1/repos/:owner/:name/deepsec` (`http/api/deepsec.ts`), and the | |
| 16 | * weekly sweep (`checks/deepsec_sweep.ts`, #15026), which files what its own | |
| 17 | * run wrote, with the commit it read. | |
| 18 | 18 | */ |
| 19 | 19 | |
| 20 | 20 | import type { App } from "../app.ts"; |
src/checks/deepsec_records.ts added +98
| @@ -0,0 +1,98 @@ | ||
| 1 | /** | |
| 2 | * deepsec's records as it wrote them under its project directory (`.deepsec/data/forge/`): every | |
| 3 | * file record with findings, cut to the fields `deepsec_findings.ts` reads, and what each run in | |
| 4 | * `runs/` spent (#15025). Read by `scripts/deepsec-file.ts` for a person's run and by the weekly | |
| 5 | * sweep (`deepsec_sweep.ts`, #15026), which reads only what its own run wrote (`since`). | |
| 6 | * | |
| 7 | * The walk is bounded (files, bytes, depth) and a symbolic link is skipped: nothing outside the | |
| 8 | * directory is read. What a record says is still untrusted; `deepsec_findings.ts` checks it. | |
| 9 | */ | |
| 10 | ||
| 11 | import { lstatSync, readdirSync, readFileSync } from "node:fs"; | |
| 12 | import { join } from "node:path"; | |
| 13 | import { clean, FILES_MAX, FINDINGS_MAX, spentIn, TEXT_MAX } from "./deepsec_findings.ts"; | |
| 14 | ||
| 15 | const BYTES_MAX = 2_000_000; | |
| 16 | const DEPTH_MAX = 12; | |
| 17 | ||
| 18 | /** | |
| 19 | * Every `.json` under `dir`, bounded; `runs/` at the top is left to the run records. With `since` | |
| 20 | * (ms), only files changed at or after it. | |
| 21 | */ | |
| 22 | function jsonFiles(dir: string, since = 0, depth = 0, acc: string[] = []): string[] { | |
| 23 | if (depth > DEPTH_MAX || acc.length >= FILES_MAX) return acc; | |
| 24 | let names: string[]; | |
| 25 | try { | |
| 26 | names = readdirSync(dir).sort(); | |
| 27 | } catch { | |
| 28 | return acc; | |
| 29 | } | |
| 30 | for (const name of names) { | |
| 31 | if (acc.length >= FILES_MAX) break; | |
| 32 | if (depth === 0 && name === "runs") continue; | |
| 33 | const path = join(dir, name); | |
| 34 | const st = lstatSync(path, { throwIfNoEntry: false }); | |
| 35 | if (st === undefined || st.isSymbolicLink()) continue; | |
| 36 | if (st.isDirectory()) jsonFiles(path, since, depth + 1, acc); | |
| 37 | else if (st.isFile() && name.endsWith(".json") && st.size <= BYTES_MAX && st.mtimeMs >= since) acc.push(path); | |
| 38 | } | |
| 39 | return acc; | |
| 40 | } | |
| 41 | ||
| 42 | function parse(path: string): unknown { | |
| 43 | try { | |
| 44 | return JSON.parse(readFileSync(path, "utf8")); | |
| 45 | } catch { | |
| 46 | return undefined; | |
| 47 | } | |
| 48 | } | |
| 49 | ||
| 50 | const hasFindings = (v: unknown): boolean => | |
| 51 | typeof v === "object" && v !== null && Array.isArray((v as { findings?: unknown }).findings) && (v as { findings: unknown[] }).findings.length > 0; | |
| 52 | ||
| 53 | const obj = (v: unknown): Record<string, unknown> => | |
| 54 | typeof v === "object" && v !== null && !Array.isArray(v) ? (v as Record<string, unknown>) : {}; | |
| 55 | const text = (v: unknown): string | undefined => (typeof v === "string" ? clean(v, TEXT_MAX) : undefined); | |
| 56 | ||
| 57 | /** | |
| 58 | * A recheck as the door reads it: its verdict, reasoning and adjusted severity, or its bare word. | |
| 59 | */ | |
| 60 | function trimRecheck(v: unknown): unknown { | |
| 61 | if (typeof v === "string") return clean(v, 200); | |
| 62 | const r = obj(v); | |
| 63 | return { verdict: text(r["verdict"]), reasoning: text(r["reasoning"]), adjustedSeverity: text(r["adjustedSeverity"]) }; | |
| 64 | } | |
| 65 | ||
| 66 | /** One file record cut to the fields the door reads, at most `FINDINGS_MAX` findings. */ | |
| 67 | export function trimRecord(raw: unknown): { filePath: unknown; findings: unknown[] } { | |
| 68 | const r = obj(raw); | |
| 69 | const findings = Array.isArray(r["findings"]) ? r["findings"].slice(0, FINDINGS_MAX) : []; | |
| 70 | return { | |
| 71 | filePath: typeof r["filePath"] === "string" ? r["filePath"] : r["path"], | |
| 72 | findings: findings.map((item) => { | |
| 73 | const f = obj(item); | |
| 74 | const recheck = f["revalidation"] ?? f["revalidate"]; | |
| 75 | return { | |
| 76 | severity: text(f["severity"]), | |
| 77 | title: text(f["title"]), | |
| 78 | description: text(f["description"]), | |
| 79 | lineNumbers: f["lineNumbers"] ?? f["lines"], | |
| 80 | recommendation: text(f["recommendation"] ?? f["fix"]), | |
| 81 | ...(recheck === undefined ? {} : { revalidation: trimRecheck(recheck) }), | |
| 82 | }; | |
| 83 | }), | |
| 84 | }; | |
| 85 | } | |
| 86 | ||
| 87 | /** | |
| 88 | * deepsec's records under `dir`: file records with findings, trimmed, and what each run spent. | |
| 89 | * With `since` (ms since the epoch), only the records written at or after it. | |
| 90 | */ | |
| 91 | export function readWorkspace(dir: string, since = 0): { files: ReturnType<typeof trimRecord>[]; runs: unknown[] } { | |
| 92 | const files = jsonFiles(dir, since).map(parse).filter(hasFindings).map(trimRecord); | |
| 93 | const runs = jsonFiles(join(dir, "runs"), since) | |
| 94 | .map(parse) | |
| 95 | .filter((r) => r !== undefined) | |
| 96 | .map((r) => ({ totalCostUsd: spentIn([r]) })); | |
| 97 | return { files, runs }; | |
| 98 | } |
src/checks/deepsec_step.ts +2 −2
| @@ -38,9 +38,9 @@ import { deepsecToolDir, ensureDeepsec, type Ensured } from "./deepsec_tool.ts"; | ||
| 38 | 38 | /** Where a repository opts in: its deepsec scope. Its absence means no review and no line. */ |
| 39 | 39 | export const SCOPE_FILE = ".deepsec/data/forge/config.json"; |
| 40 | 40 | /** The threat summary deepsec reads beside the scope. */ |
| 41 | const INFO_FILE = ".deepsec/data/forge/INFO.md"; | |
| 41 | export const INFO_FILE = ".deepsec/data/forge/INFO.md"; | |
| 42 | 42 | /** Neither file is anywhere near this big; a larger one is not read. */ |
| 43 | const FILE_MAX = 256 * 1024; | |
| 43 | export const FILE_MAX = 256 * 1024; | |
| 44 | 44 | /** A pull's review stops after this long; the check goes on without it. */ |
| 45 | 45 | export const REVIEW_TIMEOUT_MS = 20 * 60_000; |
| 46 | 46 |
src/checks/deepsec_sweep.ts added +252
| @@ -0,0 +1,252 @@ | ||
| 1 | /** | |
| 2 | * The weekly deepsec sweep (#15026): the forge reviews its own default branch, not a pull's diff. | |
| 3 | * `sweepArgvs` (`deepsec_plan.ts`) is the sweep: `scan`, then `process --limit n` with | |
| 4 | * n = floor(ceiling / $0.27), then `revalidate --min-severity HIGH`. The ceiling is the limit | |
| 5 | * `security.sweep_max_usd` (/admin/limits); its settings name no vendor, the key is the connected | |
| 6 | * model account's, handed in as the pull review's is (`deepsec_key.ts`). | |
| 7 | * | |
| 8 | * It runs in the pull review's cell (`deepsecCell`, #15024): the model host its only way out, | |
| 9 | * enforced; the exported tree and the tool revealed read-only; its workspace its one place to | |
| 10 | * write. Where no cell can be made, the sweep is skipped, never run bare. What it found is filed | |
| 11 | * through the pull review's path (`fileFindings`, #15025): each new finding a sealed report under | |
| 12 | * P0 Trust, each one already open matched. | |
| 13 | * | |
| 14 | * Durable: the workspace lives under `<data>/deepsec-sweep/`, so deepsec's record of what it has | |
| 15 | * processed carries from week to week (the files past one week's ceiling wait for the next), and | |
| 16 | * `state.json` there says when the sweep began and on which commit. A sweep the forge's restart | |
| 17 | * cut short is resumed on the next tick, at most `SWEEP_ATTEMPTS` times, with only what is left of | |
| 18 | * its ceiling; only the records written since it began are filed, so a finding closed last month | |
| 19 | * is never filed again from an old record. The duty `deepsec-sweep` | |
| 20 | * (`maintain/duties/deepsec_sweep.ts`) ticks it daily and records what it did. | |
| 21 | */ | |
| 22 | ||
| 23 | import { mkdirSync, readFileSync, realpathSync, writeFileSync } from "node:fs"; | |
| 24 | import { join } from "node:path"; | |
| 25 | import type { App } from "../app.ts"; | |
| 26 | import { cellsAvailable, resolveLimits, type CellSpec } from "../apps/contain.ts"; | |
| 27 | import { exportTree, readFileAt } from "../apps/checkout.ts"; | |
| 28 | import { childEnv, runStep } from "../apps/process.ts"; | |
| 29 | import { pickSubscription } from "../core/subscriptions.ts"; | |
| 30 | import type { Repo } from "../core/repos.ts"; | |
| 31 | import { forgeApiKey } from "../crew/runner_env.ts"; | |
| 32 | import { runGit } from "../git/env.ts"; | |
| 33 | import { repoDir } from "../git/repos.ts"; | |
| 34 | import { limitValue } from "../settings/limits.ts"; | |
| 35 | import { cellHides } from "../spawn/hosted-mark.ts"; | |
| 36 | import { fileFindings, type Filed } from "./deepsec.ts"; | |
| 37 | import { spentIn } from "./deepsec_findings.ts"; | |
| 38 | import { DEEPSEC_KEY_ENV, handKey, maskLine } from "./deepsec_key.ts"; | |
| 39 | import { DEEPSEC_VERSION, dollars, forgeConfig, parseScope, sweepArgvs } from "./deepsec_plan.ts"; | |
| 40 | import { readWorkspace } from "./deepsec_records.ts"; | |
| 41 | import { deepsecCell, FILE_MAX, INFO_FILE, SCOPE_FILE, shellWord } from "./deepsec_step.ts"; | |
| 42 | import { deepsecToolDir, ensureDeepsec, type Ensured } from "./deepsec_tool.ts"; | |
| 43 | ||
| 44 | /** A sweep starts at most this often: a week after the last one began. */ | |
| 45 | export const SWEEP_EVERY_MS = 7 * 86_400_000; | |
| 46 | /** A sweep cut short is resumed at most this many times in all, then left for next week. */ | |
| 47 | export const SWEEP_ATTEMPTS = 3; | |
| 48 | /** One step of the sweep (scan, process, revalidate) is cut off after this long. */ | |
| 49 | export const SWEEP_STEP_TIMEOUT_MS = 6 * 3_600_000; | |
| 50 | /** A file's time is read to the millisecond, but a filesystem may round it down to the second. */ | |
| 51 | const MTIME_SLACK_MS = 1_000; | |
| 52 | /** The lines of deepsec's own output kept in `sweep.log` beside the state, the last ones. */ | |
| 53 | export const LOG_LINES = 2_000; | |
| 54 | ||
| 55 | /** Where the sweep keeps its workspace and its state, under the forge's data. */ | |
| 56 | export const sweepRoot = (dataDir: string): string => join(dataDir, "deepsec-sweep"); | |
| 57 | ||
| 58 | /** What `state.json` holds: plain data, checked when read. */ | |
| 59 | export interface SweepState { | |
| 60 | /** ISO time the current or last sweep began. */ | |
| 61 | readonly startedAt?: string; | |
| 62 | /** The commit it reads. */ | |
| 63 | readonly commit?: string; | |
| 64 | /** Times it has been started or resumed. */ | |
| 65 | readonly attempts: number; | |
| 66 | /** ISO time it ended; absent, or before `startedAt`, while it has not. */ | |
| 67 | readonly finishedAt?: string; | |
| 68 | /** The line it ended with. */ | |
| 69 | readonly line?: string; | |
| 70 | } | |
| 71 | ||
| 72 | const ISO = (v: unknown): v is string => typeof v === "string" && v.length <= 40 && !Number.isNaN(Date.parse(v)); | |
| 73 | ||
| 74 | /** The state as written, or a fresh one when there is none or it does not parse. */ | |
| 75 | export function readState(root: string): SweepState { | |
| 76 | let raw: unknown; | |
| 77 | try { | |
| 78 | raw = JSON.parse(readFileSync(join(root, "state.json"), "utf8")); | |
| 79 | } catch { | |
| 80 | return { attempts: 0 }; | |
| 81 | } | |
| 82 | const r = typeof raw === "object" && raw !== null ? (raw as Record<string, unknown>) : {}; | |
| 83 | const attempts = Number.isSafeInteger(r["attempts"]) && (r["attempts"] as number) >= 0 ? (r["attempts"] as number) : 0; | |
| 84 | return { | |
| 85 | attempts, | |
| 86 | ...(ISO(r["startedAt"]) ? { startedAt: r["startedAt"] } : {}), | |
| 87 | ...(typeof r["commit"] === "string" && /^[0-9a-f]{40,64}$/.test(r["commit"]) ? { commit: r["commit"] } : {}), | |
| 88 | ...(ISO(r["finishedAt"]) ? { finishedAt: r["finishedAt"] } : {}), | |
| 89 | ...(typeof r["line"] === "string" ? { line: r["line"].slice(0, 2_000) } : {}), | |
| 90 | }; | |
| 91 | } | |
| 92 | ||
| 93 | export function writeState(root: string, state: SweepState): void { | |
| 94 | mkdirSync(root, { recursive: true }); | |
| 95 | writeFileSync(join(root, "state.json"), `${JSON.stringify(state, null, 2)}\n`); | |
| 96 | } | |
| 97 | ||
| 98 | export type Due = | |
| 99 | | { readonly kind: "start" } | |
| 100 | | { readonly kind: "resume"; readonly startedAt: string; readonly commit: string; readonly attempts: number } | |
| 101 | | { readonly kind: "wait"; readonly until: string }; | |
| 102 | ||
| 103 | /** | |
| 104 | * What the sweep does now, from its state. Pure. A sweep that began and never ended is resumed | |
| 105 | * while it has tries left; otherwise the next one starts a week after the last began. | |
| 106 | */ | |
| 107 | export function sweepDue(state: SweepState, now: Date): Due { | |
| 108 | if (state.startedAt === undefined) return { kind: "start" }; | |
| 109 | const began = Date.parse(state.startedAt); | |
| 110 | const ended = state.finishedAt !== undefined && Date.parse(state.finishedAt) >= began; | |
| 111 | if (!ended && state.commit !== undefined && state.attempts < SWEEP_ATTEMPTS) | |
| 112 | return { kind: "resume", startedAt: state.startedAt, commit: state.commit, attempts: state.attempts }; | |
| 113 | const next = began + SWEEP_EVERY_MS; | |
| 114 | return now.getTime() >= next ? { kind: "start" } : { kind: "wait", until: new Date(next).toISOString() }; | |
| 115 | } | |
| 116 | ||
| 117 | export interface SweepResult { | |
| 118 | /** True when deepsec ran; false when the sweep was skipped or not due. */ | |
| 119 | readonly ran: boolean; | |
| 120 | /** One line for the duty's record: what ran, what it found and filed, what it spent. */ | |
| 121 | readonly line: string; | |
| 122 | readonly spentUsd: number; | |
| 123 | readonly filed?: Filed; | |
| 124 | } | |
| 125 | ||
| 126 | export interface SweepDeps { | |
| 127 | readonly run?: typeof runStep; | |
| 128 | readonly ensure?: (dataDir: string, signal: AbortSignal) => Promise<Ensured>; | |
| 129 | readonly cells?: () => Promise<{ ok: true } | { ok: false; reason: string }>; | |
| 130 | readonly forgeKey?: string | undefined; | |
| 131 | readonly now?: () => Date; | |
| 132 | readonly signal?: AbortSignal; | |
| 133 | readonly exportTree?: typeof exportTree; | |
| 134 | } | |
| 135 | ||
| 136 | const skipped = (why: string): SweepResult => ({ ran: false, line: `deepsec sweep: skipped, ${why}`, spentUsd: 0 }); | |
| 137 | ||
| 138 | /** | |
| 139 | * Runs the sweep on `repo`'s default branch when it is due: scan, process up to the ceiling, | |
| 140 | * revalidate, then file what it wrote. A skip writes no state, so the next tick tries again. | |
| 141 | */ | |
| 142 | export async function runSweep(app: App, repo: Repo, deps: SweepDeps = {}): Promise<SweepResult> { | |
| 143 | const now = deps.now ?? (() => new Date()); | |
| 144 | const signal = deps.signal ?? new AbortController().signal; | |
| 145 | const root = sweepRoot(app.config.dataDir); | |
| 146 | const due = sweepDue(readState(root), now()); | |
| 147 | if (due.kind === "wait") return { ran: false, line: `deepsec sweep: not due until ${due.until}`, spentUsd: 0 }; | |
| 148 | ||
| 149 | const maxUsd = limitValue(app.db, "security.sweep_max_usd"); | |
| 150 | if (maxUsd <= 0) return skipped("its ceiling security.sweep_max_usd is $0.00 (/admin/limits)"); | |
| 151 | const bare = repoDir(app.reposRoot, repo.ownerName, repo.name); | |
| 152 | let commit: string; | |
| 153 | if (due.kind === "resume") commit = due.commit; | |
| 154 | else { | |
| 155 | const tip = await runGit(["rev-parse", "--verify", "--quiet", `refs/heads/${repo.defaultBranch}^{commit}`], { cwd: bare, signal }); | |
| 156 | if (!tip.ok || !/^[0-9a-f]{40,64}$/.test(tip.value.trim())) return skipped(`${repo.defaultBranch} has no commit to read`); | |
| 157 | commit = tip.value.trim(); | |
| 158 | } | |
| 159 | const scopeText = await readFileAt(bare, commit, SCOPE_FILE, FILE_MAX); | |
| 160 | if (scopeText === undefined) return skipped(`${repo.defaultBranch} has no ${SCOPE_FILE}`); | |
| 161 | const scope = parseScope(scopeText); | |
| 162 | if (typeof scope === "string") return skipped(scope); | |
| 163 | const picked = pickSubscription(app.db, app.config.dataDir, { repoId: repo.id, ownerId: repo.ownerId, agentId: "" }, now()); | |
| 164 | const handed = handKey(picked, "forgeKey" in deps ? deps.forgeKey : forgeApiKey()); | |
| 165 | if (handed.key === undefined) return skipped(`no key was handed in, ${handed.line.replace(/^deepsec key: (not handed in, )?/, "")}`); | |
| 166 | const cells = await (deps.cells ?? cellsAvailable)(); | |
| 167 | if (!cells.ok) return skipped(`no cell can be made here (${cells.reason}) and deepsec is never run bare`); | |
| 168 | const tool = await (deps.ensure ?? ensureDeepsec)(app.config.dataDir, signal); | |
| 169 | if (!tool.ok) return skipped(tool.reason); | |
| 170 | ||
| 171 | const startedAt = due.kind === "resume" ? due.startedAt : now().toISOString(); | |
| 172 | const attempts = due.kind === "resume" ? due.attempts + 1 : 1; | |
| 173 | const since = Date.parse(startedAt) - MTIME_SLACK_MS; | |
| 174 | ||
| 175 | // The tree is exported before the state says the sweep began: a failed export (a full disk, a | |
| 176 | // git error) is a skip, not an ended sweep, so the next day's tick tries again. A fresh sweep | |
| 177 | // leaves no state; a resumed one counts the try, so `SWEEP_ATTEMPTS` bounds a commit that can no | |
| 178 | // longer be exported. | |
| 179 | mkdirSync(root, { recursive: true }); | |
| 180 | const tree = join(root, "tree"); | |
| 181 | const exported = await (deps.exportTree ?? exportTree)(bare, commit, tree); | |
| 182 | if (!exported.ok) { | |
| 183 | if (due.kind === "resume") writeState(root, { startedAt, commit, attempts }); | |
| 184 | return skipped(`the tree of ${commit.slice(0, 12)} could not be exported: ${exported.error}`); | |
| 185 | } | |
| 186 | writeState(root, { startedAt, commit, attempts }); | |
| 187 | const work = join(root, "work"); | |
| 188 | const data = join(work, "data"); | |
| 189 | const project = join(data, "forge"); | |
| 190 | for (const dir of [project, join(work, "home"), join(work, "tmp")]) mkdirSync(dir, { recursive: true }); | |
| 191 | const checkout = realpathSync(tree); | |
| 192 | writeFileSync(join(project, "config.json"), scopeText); | |
| 193 | const info = await readFileAt(bare, commit, INFO_FILE, FILE_MAX); | |
| 194 | if (info !== undefined) writeFileSync(join(project, "INFO.md"), info); | |
| 195 | writeFileSync(join(work, "deepsec.config.mjs"), forgeConfig(checkout)); | |
| 196 | ||
| 197 | // A resumed sweep spends only what its earlier tries left of the ceiling. | |
| 198 | const spentBefore = spentIn(readWorkspace(project, since).runs); | |
| 199 | const leftUsd = Math.max(0, maxUsd - spentBefore); | |
| 200 | const base: CellSpec = { | |
| 201 | dir: work, | |
| 202 | writable: [], | |
| 203 | home: join(work, "home"), | |
| 204 | hide: cellHides(app), | |
| 205 | egress: [], | |
| 206 | reach: [], | |
| 207 | limits: { ...resolveLimits("step", undefined), timeMs: SWEEP_STEP_TIMEOUT_MS }, | |
| 208 | }; | |
| 209 | const cell = deepsecCell(base, { work: realpathSync(work), checkout, tool: deepsecToolDir(app.config.dataDir), runId: "sweep" }); | |
| 210 | const steps: string[] = []; | |
| 211 | const output: string[] = []; | |
| 212 | const keep = (l: string): void => { | |
| 213 | output.push(maskLine(l, handed.key)); | |
| 214 | if (output.length > LOG_LINES) output.shift(); | |
| 215 | }; | |
| 216 | for (const argv of sweepArgvs(leftUsd)) { | |
| 217 | if (signal.aborted) break; | |
| 218 | const command = [process.execPath, tool.cli, ...argv].map(shellWord).join(" "); | |
| 219 | const ran = await (deps.run ?? runStep)(`deepsec ${argv[0]}`, command, { | |
| 220 | cwd: work, | |
| 221 | env: childEnv({ | |
| 222 | HOME: join(work, "home"), | |
| 223 | TMPDIR: join(work, "tmp"), | |
| 224 | DEEPSEC_DATA_ROOT: data, | |
| 225 | JITI_FS_CACHE: "false", | |
| 226 | NO_COLOR: "1", | |
| 227 | [DEEPSEC_KEY_ENV]: handed.key, | |
| 228 | }), | |
| 229 | log: keep, | |
| 230 | signal, | |
| 231 | timeoutMs: SWEEP_STEP_TIMEOUT_MS, | |
| 232 | cell, | |
| 233 | }); | |
| 234 | // deepsec exits 1 on a finding, so a step that did not exit 0 does not stop the next. | |
| 235 | const said = argv[0] === "process" ? `process --limit ${argv[argv.indexOf("--limit") + 1]}` : argv[0]!; | |
| 236 | keep(`deepsec sweep: ${said} ${ran.ok ? "exited 0" : maskLine(ran.error, handed.key)}`); | |
| 237 | steps.push(ran.ok ? said : `${said} (${maskLine(ran.error, handed.key).slice(0, 120)})`); | |
| 238 | } | |
| 239 | ||
| 240 | writeFileSync(join(root, "sweep.log"), `${output.join("\n")}\n`); | |
| 241 | const filed = fileFindings(app, repo, readWorkspace(project, since), { commit, tool: `deepsec ${DEEPSEC_VERSION}`, now: now() }); | |
| 242 | const resumed = attempts > 1 ? `, try ${attempts} of ${SWEEP_ATTEMPTS}` : ""; | |
| 243 | const line = | |
| 244 | `deepsec sweep of ${repo.defaultBranch} at ${commit.slice(0, 9)} under a ${dollars(maxUsd)} ceiling${resumed}: ` + | |
| 245 | `ran ${steps.join(", ")}; ${filed.line.replace(/^deepsec: /, "")}`; | |
| 246 | return finish({ ran: true, line: maskLine(line, handed.key), spentUsd: filed.spentUsd, filed }); | |
| 247 | ||
| 248 | function finish(result: SweepResult): SweepResult { | |
| 249 | writeState(root, { startedAt, commit, attempts, finishedAt: now().toISOString(), line: result.line }); | |
| 250 | return result; | |
| 251 | } | |
| 252 | } |
src/maintain/duties/deepsec_sweep.ts added +77
| @@ -0,0 +1,77 @@ | ||
| 1 | /** | |
| 2 | * The weekly deepsec sweep as a duty (#15026): ticked daily, it starts the sweep | |
| 3 | * (`checks/deepsec_sweep.ts`) on the forge's own repository when a week has passed since the last | |
| 4 | * began, or resumes one a restart cut short. The sweep takes hours and a duty answers in seconds, | |
| 5 | * so the tick starts it and returns; when the sweep ends, its line, what it filed and what it | |
| 6 | * spent are a second `maintenance_run` row of this duty, read on /maintenance like any other. | |
| 7 | * | |
| 8 | * The spend ceiling is the limit `security.sweep_max_usd`, not this duty's dollars a day: those | |
| 9 | * only count what the sweep reports it spent, so they are set at the limit's own maximum. | |
| 10 | */ | |
| 11 | ||
| 12 | import type { App } from "../../app.ts"; | |
| 13 | import { getRepo, type Repo } from "../../core/repos.ts"; | |
| 14 | // Types only at the top: the sweep reaches the issue actions, which reach this roster (the | |
| 15 | // registry `require`s each duty), so the sweep itself is imported when the tick runs. | |
| 16 | import type { SweepDeps, SweepResult } from "../../checks/deepsec_sweep.ts"; | |
| 17 | import { ownRepoPath } from "../../kernel/own_repo.ts"; | |
| 18 | import { LIMITS } from "../../settings/limits.ts"; | |
| 19 | import type { Duty } from "../duty.ts"; | |
| 20 | import { record } from "../runs.ts"; | |
| 21 | ||
| 22 | export const NAME = "deepsec-sweep"; | |
| 23 | ||
| 24 | /** | |
| 25 | * The sweep this process is running, if any: one at a time. Gone on restart, when `state.json` | |
| 26 | * says the sweep was cut short and the next tick resumes it. | |
| 27 | */ | |
| 28 | let running: Promise<SweepResult> | undefined; | |
| 29 | ||
| 30 | /** Starts the sweep and records its end as this duty's row. */ | |
| 31 | export function startSweep(app: App, repo: Repo, started: Date, deps: SweepDeps = {}): Promise<SweepResult> { | |
| 32 | const sweep = import("../../checks/deepsec_sweep.ts") | |
| 33 | .then(({ runSweep }) => runSweep(app, repo, deps)) | |
| 34 | .catch((error: unknown): SweepResult => ({ ran: false, line: `deepsec sweep: failed, ${error instanceof Error ? error.message : String(error)}`, spentUsd: 0 })) | |
| 35 | .then((result) => { | |
| 36 | try { | |
| 37 | record(app.db, { duty: NAME, started, finished: new Date(), findings: [], actions: [result.line], outcome: "ok", dollars: result.spentUsd }); | |
| 38 | } catch (error) { | |
| 39 | app.log.error("deepsec sweep not recorded", { error: String(error) }); | |
| 40 | } | |
| 41 | return result; | |
| 42 | }) | |
| 43 | .finally(() => { | |
| 44 | running = undefined; | |
| 45 | }); | |
| 46 | running = sweep; | |
| 47 | return sweep; | |
| 48 | } | |
| 49 | ||
| 50 | /** The duty; `onStart` is handed each sweep it starts, so a caller can await its end. */ | |
| 51 | export function deepsecSweepDuty(deps: SweepDeps = {}, onStart?: (sweep: Promise<SweepResult>) => void): Duty { | |
| 52 | return { | |
| 53 | name: NAME, | |
| 54 | intervalMs: 86_400_000, | |
| 55 | budget: { | |
| 56 | runsPerDay: 4, | |
| 57 | dollarsPerDay: LIMITS["security.sweep_max_usd"].max, | |
| 58 | reason: "a daily tick and the sweep's own end are two rows a day; what it may spend is the limit security.sweep_max_usd, so the dollars here only count it", | |
| 59 | }, | |
| 60 | async run({ db, now, app }) { | |
| 61 | if (app === undefined) return { findings: [], actions: [] }; | |
| 62 | if (running !== undefined) return { findings: [], actions: ["the weekly deepsec sweep is still running"] }; | |
| 63 | const { readState, sweepDue, sweepRoot } = await import("../../checks/deepsec_sweep.ts"); | |
| 64 | if (running !== undefined) return { findings: [], actions: ["the weekly deepsec sweep is still running"] }; | |
| 65 | const due = sweepDue(readState(sweepRoot(app.config.dataDir)), now); | |
| 66 | if (due.kind === "wait") return { findings: [], actions: [] }; | |
| 67 | const [owner = "", name = ""] = ownRepoPath().split("/"); | |
| 68 | const repo = getRepo(db, owner, name); | |
| 69 | if (repo === undefined) return { findings: [], actions: [`no deepsec sweep: the forge's own repository ${ownRepoPath()} is not here`] }; | |
| 70 | const sweep = startSweep(app, repo, now, deps); | |
| 71 | onStart?.(sweep); | |
| 72 | return { findings: [], actions: [due.kind === "resume" ? "resumed the weekly deepsec sweep a restart cut short" : "started the weekly deepsec sweep"] }; | |
| 73 | }, | |
| 74 | }; | |
| 75 | } | |
| 76 | ||
| 77 | export const duty: Duty = deepsecSweepDuty(); |
src/perf/page_budgets/maintenance.ts +3 −2
| @@ -4,6 +4,7 @@ import type { PageBudget } from "../page_budget.ts"; | ||
| 4 | 4 | // own state and runs, so the count moves with the number of duties and background ticks; an N+1 |
| 5 | 5 | // over the duties would still show. One query reads every dismissal (#13967), one every quieted |
| 6 | 6 | // finding (#14557); one grouped read of the degraded findings (#14577); +3: the edge-kept duty |
| 7 | // (#14682); one grouped read of every duty's last run (#14674). 87 is the steady count measured | |
| 7 | // (#14682); one grouped read of every duty's last run (#14674); +3: the deepsec-sweep duty | |
| 8 | // (#15026). 90 is the steady count measured | |
| 8 | 9 | // warm on every run, not a load-sensitive figure. |
| 9 | export const budget: PageBudget = { page: "maintenance", pattern: "/maintenance", medianMs: 50, statements: 87 }; | |
| 10 | export const budget: PageBudget = { page: "maintenance", pattern: "/maintenance", medianMs: 50, statements: 90 }; |
test/deepsec-sweep.test.ts added +296
| @@ -0,0 +1,296 @@ | ||
| 1 | /** | |
| 2 | * The weekly deepsec sweep (#15026): scan, process up to the ceiling `security.sweep_max_usd` | |
| 3 | * (n = floor(ceiling / $0.27)), revalidate HIGH and up, in the deepsec cell, with the connected | |
| 4 | * account's key; then what it wrote is filed as sealed reports or matched. Durable: a week apart, | |
| 5 | * resumed with what is left of its ceiling after a restart, and never refiling an old record. | |
| 6 | */ | |
| 7 | import { afterAll, beforeAll, describe, expect, test } from "bun:test"; | |
| 8 | import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, utimesSync, writeFileSync } from "node:fs"; | |
| 9 | import { tmpdir } from "node:os"; | |
| 10 | import { join } from "node:path"; | |
| 11 | import { createApp, type App } from "../src/app.ts"; | |
| 12 | import { openIssue } from "../src/actions/issues.ts"; | |
| 13 | import { createRepo } from "../src/actions/repos.ts"; | |
| 14 | import type { CellSpec } from "../src/apps/contain.ts"; | |
| 15 | import type { runStep } from "../src/apps/process.ts"; | |
| 16 | import { | |
| 17 | readState, | |
| 18 | runSweep, | |
| 19 | SWEEP_ATTEMPTS, | |
| 20 | SWEEP_EVERY_MS, | |
| 21 | sweepDue, | |
| 22 | sweepRoot, | |
| 23 | writeState, | |
| 24 | type SweepDeps, | |
| 25 | type SweepResult, | |
| 26 | } from "../src/checks/deepsec_sweep.ts"; | |
| 27 | import { blockerNumbersFor } from "../src/core/dependencies.ts"; | |
| 28 | import { getIssue } from "../src/core/issues.ts"; | |
| 29 | import type { Repo } from "../src/core/repos.ts"; | |
| 30 | import { createUser, getUserByName, type User } from "../src/core/users.ts"; | |
| 31 | import { repoDir } from "../src/git/repos.ts"; | |
| 32 | import { commitFiles } from "../src/git/write.ts"; | |
| 33 | import { deepsecSweepDuty, NAME } from "../src/maintain/duties/deepsec_sweep.ts"; | |
| 34 | import { latest } from "../src/maintain/runs.ts"; | |
| 35 | import { setLimit } from "../src/settings/limits.ts"; | |
| 36 | ||
| 37 | const KEY = "sk-ant-api03-sweep-canary-never-printed-0123"; | |
| 38 | const SCOPE = JSON.stringify({ ignorePaths: ["test/**"], priorityPaths: ["src/http/"] }); | |
| 39 | const DAY = 86_400_000; | |
| 40 | ||
| 41 | describe("when the sweep is due", () => { | |
| 42 | const at = (iso: string) => new Date(iso); | |
| 43 | test("never run: start now", () => { | |
| 44 | expect(sweepDue({ attempts: 0 }, at("2026-11-21T00:00:00Z"))).toEqual({ kind: "start" }); | |
| 45 | }); | |
| 46 | test("finished: wait until a week after it began, then start", () => { | |
| 47 | const s = { startedAt: "2026-11-21T00:00:00.000Z", finishedAt: "2026-11-21T05:00:00.000Z", commit: "a".repeat(40), attempts: 1 }; | |
| 48 | expect(sweepDue(s, at("2026-11-27T23:59:00Z"))).toEqual({ kind: "wait", until: "2026-11-28T00:00:00.000Z" }); | |
| 49 | expect(sweepDue(s, at("2026-11-28T00:00:00Z"))).toEqual({ kind: "start" }); | |
| 50 | expect(SWEEP_EVERY_MS).toBe(7 * DAY); | |
| 51 | }); | |
| 52 | test("began and never ended: resume while it has tries, then wait for next week", () => { | |
| 53 | const s = { startedAt: "2026-11-21T00:00:00.000Z", commit: "b".repeat(40), attempts: 1 }; | |
| 54 | expect(sweepDue(s, at("2026-11-21T01:00:00Z"))).toEqual({ kind: "resume", startedAt: s.startedAt, commit: s.commit, attempts: 1 }); | |
| 55 | // An end older than the start is the last sweep's: this one is still unfinished. | |
| 56 | expect(sweepDue({ ...s, finishedAt: "2026-11-14T00:00:00.000Z" }, at("2026-11-21T01:00:00Z")).kind).toBe("resume"); | |
| 57 | expect(sweepDue({ ...s, attempts: SWEEP_ATTEMPTS }, at("2026-11-21T01:00:00Z")).kind).toBe("wait"); | |
| 58 | }); | |
| 59 | test("a state file that does not parse, or lies, is a fresh state", () => { | |
| 60 | const root = mkdtempSync(join(tmpdir(), "sweep-state-")); | |
| 61 | try { | |
| 62 | writeFileSync(join(root, "state.json"), "{nope"); | |
| 63 | expect(readState(root)).toEqual({ attempts: 0 }); | |
| 64 | writeFileSync(join(root, "state.json"), JSON.stringify({ startedAt: "never", commit: "; rm -rf /", attempts: -3 })); | |
| 65 | expect(readState(root)).toEqual({ attempts: 0 }); | |
| 66 | } finally { | |
| 67 | rmSync(root, { recursive: true, force: true }); | |
| 68 | } | |
| 69 | }); | |
| 70 | }); | |
| 71 | ||
| 72 | describe("the weekly sweep on the forge's own repository", () => { | |
| 73 | let dataDir = ""; | |
| 74 | let app: App; | |
| 75 | let alice: User; | |
| 76 | let repo: Repo; | |
| 77 | let umbrella = 0; | |
| 78 | let commit = ""; | |
| 79 | ||
| 80 | beforeAll(async () => { | |
| 81 | dataDir = mkdtempSync(join(tmpdir(), "forge-sweep-")); | |
| 82 | app = createApp({ dataDir, port: 0 }); | |
| 83 | const made = await createUser(app.db, { origin: "seed", trusted: true, username: "alice", email: "[email protected]", password: "a-good-passphrase", adminIfFirst: true }); | |
| 84 | if (!made.ok) throw new Error(made.error); | |
| 85 | alice = getUserByName(app.db, "alice")!; | |
| 86 | const created = await createRepo(app, alice, { name: "forge", visibility: "public" }); | |
| 87 | if (!created.ok) throw new Error(String(created.error)); | |
| 88 | repo = created.value; | |
| 89 | const c = await commitFiles(repoDir(app.reposRoot, repo.ownerName, repo.name), { | |
| 90 | branch: repo.defaultBranch, | |
| 91 | message: "deepsec scope", | |
| 92 | files: [ | |
| 93 | { path: ".deepsec/data/forge/config.json", content: SCOPE }, | |
| 94 | { path: ".deepsec/data/forge/INFO.md", content: "threats\n" }, | |
| 95 | { path: "src/http/a.ts", content: "export {};\n" }, | |
| 96 | ], | |
| 97 | author: { name: "T", email: "[email protected]" }, | |
| 98 | }); | |
| 99 | if (!c.ok) throw new Error(JSON.stringify(c.error)); | |
| 100 | commit = c.value.commitOid; | |
| 101 | const u = openIssue(app, alice, repo, { title: "P0 Trust: be trusted" }); | |
| 102 | if (!u.ok) throw new Error(String(u.error)); | |
| 103 | umbrella = u.value.number; | |
| 104 | }); | |
| 105 | afterAll(() => { | |
| 106 | app.db.close(); | |
| 107 | rmSync(dataDir, { recursive: true, force: true }); | |
| 108 | }); | |
| 109 | ||
| 110 | interface Seen { | |
| 111 | readonly argv: string; | |
| 112 | readonly env: Record<string, string>; | |
| 113 | readonly cell: CellSpec | undefined; | |
| 114 | readonly cwd: string; | |
| 115 | } | |
| 116 | ||
| 117 | function harness(over: Partial<SweepDeps> = {}, spent = 1.08) { | |
| 118 | const seen: Seen[] = []; | |
| 119 | const run: typeof runStep = async (_name, command, options) => { | |
| 120 | seen.push({ argv: command, env: options.env, cell: options.cell, cwd: options.cwd }); | |
| 121 | options.log(`deepsec sees ${options.env["DEEPSEC_ANTHROPIC_KEY"]}`); | |
| 122 | if (command.includes("'process'")) { | |
| 123 | const project = join(options.env["DEEPSEC_DATA_ROOT"]!, "forge"); | |
| 124 | mkdirSync(join(project, "files", "src", "http"), { recursive: true }); | |
| 125 | mkdirSync(join(project, "runs"), { recursive: true }); | |
| 126 | const finding = { severity: "HIGH", title: "Token compared with ==", description: "Timing leaks it.", lineNumbers: [3], recommendation: "Use timingSafeEqual." }; | |
| 127 | writeFileSync(join(project, "files", "src", "http", "a.ts.json"), JSON.stringify({ filePath: "src/http/a.ts", findings: [finding] })); | |
| 128 | writeFileSync(join(project, "runs", `run-${seen.length}-${Date.now()}.json`), JSON.stringify({ totalCostUsd: spent })); | |
| 129 | return { ok: false, error: `exited 1 near ${options.env["DEEPSEC_ANTHROPIC_KEY"]}` }; | |
| 130 | } | |
| 131 | return { ok: true, value: undefined }; | |
| 132 | }; | |
| 133 | const deps: SweepDeps = { | |
| 134 | run, | |
| 135 | ensure: async () => ({ ok: true, cli: "/tool/node_modules/deepsec/dist/cli.mjs", dir: "/tool" }), | |
| 136 | cells: async () => ({ ok: true }), | |
| 137 | forgeKey: KEY, | |
| 138 | ...over, | |
| 139 | }; | |
| 140 | return { deps, seen }; | |
| 141 | } | |
| 142 | const root = () => sweepRoot(app.config.dataDir); | |
| 143 | /** A fresh workspace: what earlier tests wrote within the clock's slack is not this sweep's. */ | |
| 144 | const fresh = () => rmSync(join(root(), "work"), { recursive: true, force: true }); | |
| 145 | const reports = () => | |
| 146 | app.db.query<{ number: number; title: string; sealed: number }, [string]>("SELECT number, title, sealed FROM issue WHERE repo_id = ?1 AND title = 'Token compared with ==' ORDER BY number").all(repo.id); | |
| 147 | ||
| 148 | test("scan, process up to floor($25 / $0.27) = 92 files, revalidate HIGH: in the model-host-only cell, the key never shown", async () => { | |
| 149 | const h = harness(); | |
| 150 | const result = await runSweep(app, repo, h.deps); | |
| 151 | expect(h.seen.map((s) => s.argv.split(" ").slice(2).join(" "))).toEqual([ | |
| 152 | "'scan' '--project-id' 'forge'", | |
| 153 | "'process' '--project-id' 'forge' '--limit' '92'", | |
| 154 | "'revalidate' '--project-id' 'forge' '--min-severity' 'HIGH'", | |
| 155 | ]); | |
| 156 | for (const s of h.seen) { | |
| 157 | expect(s.env["DEEPSEC_ANTHROPIC_KEY"]).toBe(KEY); | |
| 158 | expect(s.cell!.egress).toEqual(["api.anthropic.com"]); | |
| 159 | expect(s.cell!.egressMode).toBe("enforce"); | |
| 160 | expect(s.cell!.reach).toEqual([]); | |
| 161 | expect(s.cell!.hide).toContain(app.config.dataDir); | |
| 162 | expect(s.cell!.reveal).toContain(join(root(), "tree")); | |
| 163 | expect(s.cwd).toBe(join(root(), "work")); | |
| 164 | } | |
| 165 | // The forge's own config, rooted at the exported tree of the default branch. | |
| 166 | expect(readFileSync(join(root(), "work", "deepsec.config.mjs"), "utf8")).toContain(JSON.stringify(join(root(), "tree"))); | |
| 167 | expect(readFileSync(join(root(), "tree", "src/http/a.ts"), "utf8")).toBe("export {};\n"); | |
| 168 | expect(result.ran).toBe(true); | |
| 169 | expect(result.spentUsd).toBeCloseTo(1.08, 9); | |
| 170 | expect(result.line).toContain(`deepsec sweep of main at ${commit.slice(0, 9)} under a $25.00 ceiling`); | |
| 171 | expect(result.line).toContain("1 finding, 1 filed as sealed reports"); | |
| 172 | expect(result.line).toContain("spent $1.08"); | |
| 173 | const log = readFileSync(join(root(), "sweep.log"), "utf8"); | |
| 174 | expect(`${result.line}\n${log}`).not.toContain(KEY); | |
| 175 | expect(log).toContain("deepsec sees [DEEPSEC_ANTHROPIC_KEY]"); | |
| 176 | const [report] = reports(); | |
| 177 | expect(report!.sealed).toBe(1); | |
| 178 | expect(getIssue(app.db, repo.id, report!.number)!.body).toContain(`on ${commit.slice(0, 9)}`); | |
| 179 | const umbrellaId = getIssue(app.db, repo.id, umbrella)!.id; | |
| 180 | expect(blockerNumbersFor(app.db, [umbrellaId]).get(umbrellaId)).toEqual([report!.number]); | |
| 181 | const state = readState(root()); | |
| 182 | expect(state.commit).toBe(commit); | |
| 183 | expect(Date.parse(state.finishedAt!)).toBeGreaterThanOrEqual(Date.parse(state.startedAt!)); | |
| 184 | }); | |
| 185 | ||
| 186 | test("the same day it is not due, and nothing runs", async () => { | |
| 187 | const h = harness(); | |
| 188 | const result = await runSweep(app, repo, h.deps); | |
| 189 | expect(h.seen).toHaveLength(0); | |
| 190 | expect(result.line).toMatch(/^deepsec sweep: not due until /); | |
| 191 | }); | |
| 192 | ||
| 193 | test("a week on, the finding it writes again is matched, and an old record is never filed again", async () => { | |
| 194 | writeState(root(), { ...readState(root()), startedAt: new Date(Date.now() - 8 * DAY).toISOString(), finishedAt: new Date(Date.now() - 8 * DAY + 1000).toISOString() }); | |
| 195 | // A record from an earlier sweep whose report was since closed: not this sweep's to file. | |
| 196 | const old = join(root(), "work", "data", "forge", "files", "src", "old.ts.json"); | |
| 197 | writeFileSync(old, JSON.stringify({ filePath: "src/old.ts", findings: [{ severity: "CRITICAL", title: "Fixed last month", description: "d" }] })); | |
| 198 | const past = new Date(Date.now() - 30 * DAY); | |
| 199 | utimesSync(old, past, past); | |
| 200 | const h = harness(); | |
| 201 | const result = await runSweep(app, repo, h.deps); | |
| 202 | expect(h.seen).toHaveLength(3); | |
| 203 | expect(result.line).toContain("1 finding, 0 filed as sealed reports, 1 already filed"); | |
| 204 | expect(reports()).toHaveLength(1); | |
| 205 | expect(app.db.query("SELECT 1 FROM issue WHERE repo_id = ?1 AND title = 'Fixed last month'").get(repo.id)).toBeNull(); | |
| 206 | }); | |
| 207 | ||
| 208 | test("a sweep a restart cut short resumes with only what is left of its ceiling", async () => { | |
| 209 | const startedAt = new Date(Date.now() - 3_600_000).toISOString(); | |
| 210 | writeState(root(), { startedAt, commit, attempts: 1 }); | |
| 211 | fresh(); | |
| 212 | mkdirSync(join(root(), "work", "data", "forge", "runs"), { recursive: true }); | |
| 213 | // The cut-short try spent $20 of $25: $5 pays for floor(5 / 0.27) = 18 files. | |
| 214 | writeFileSync(join(root(), "work", "data", "forge", "runs", "cut-short.json"), JSON.stringify({ totalCostUsd: 20 })); | |
| 215 | const h = harness(); | |
| 216 | const result = await runSweep(app, repo, h.deps); | |
| 217 | expect(h.seen[1]!.argv).toEndWith("'--limit' '18'"); | |
| 218 | expect(result.line).toContain(`try 2 of ${SWEEP_ATTEMPTS}`); | |
| 219 | expect(result.spentUsd).toBeCloseTo(21.08, 9); | |
| 220 | expect(readState(root()).startedAt).toBe(startedAt); | |
| 221 | }); | |
| 222 | ||
| 223 | test("skips write no state and run nothing: a $0 ceiling, no key, no cell", async () => { | |
| 224 | const due = () => writeState(root(), { attempts: 0 }); | |
| 225 | const cases: [Partial<SweepDeps>, RegExp][] = [ | |
| 226 | [{ forgeKey: undefined }, /skipped, no key was handed in, the forge has no API key of its own/], | |
| 227 | [{ cells: async () => ({ ok: false, reason: "bwrap is not installed" }) }, /no cell can be made here \(bwrap is not installed\) and deepsec is never run bare/], | |
| 228 | ]; | |
| 229 | for (const [over, said] of cases) { | |
| 230 | due(); | |
| 231 | const h = harness(over); | |
| 232 | const result = await runSweep(app, repo, h.deps); | |
| 233 | expect(result.line).toMatch(said); | |
| 234 | expect(h.seen).toHaveLength(0); | |
| 235 | expect(readState(root()).startedAt).toBeUndefined(); | |
| 236 | } | |
| 237 | due(); | |
| 238 | expect(setLimit(app.db, "security.sweep_max_usd", 0, alice.id).ok).toBe(true); | |
| 239 | const h = harness(); | |
| 240 | const result = await runSweep(app, repo, h.deps); | |
| 241 | expect(result.line).toBe("deepsec sweep: skipped, its ceiling security.sweep_max_usd is $0.00 (/admin/limits)"); | |
| 242 | expect(h.seen).toHaveLength(0); | |
| 243 | expect(setLimit(app.db, "security.sweep_max_usd", null, alice.id).ok).toBe(true); | |
| 244 | }); | |
| 245 | ||
| 246 | test("a tree that cannot be exported is a skip, not an ended sweep: the next tick tries again", async () => { | |
| 247 | const broken: SweepDeps = { exportTree: async () => ({ ok: false, error: "No space left on device" }) }; | |
| 248 | // A fresh sweep: nothing runs and no state is left, so it is due again tomorrow. | |
| 249 | writeState(root(), { attempts: 0 }); | |
| 250 | const h = harness(broken); | |
| 251 | const result = await runSweep(app, repo, h.deps); | |
| 252 | expect(result.ran).toBe(false); | |
| 253 | expect(result.line).toBe(`deepsec sweep: skipped, the tree of ${commit.slice(0, 12)} could not be exported: No space left on device`); | |
| 254 | expect(h.seen).toHaveLength(0); | |
| 255 | expect(readState(root())).toEqual({ attempts: 0 }); | |
| 256 | expect(sweepDue(readState(root()), new Date()).kind).toBe("start"); | |
| 257 | // A resumed sweep: the try is counted and it is still unfinished, so SWEEP_ATTEMPTS bounds it. | |
| 258 | const startedAt = new Date(Date.now() - 3_600_000).toISOString(); | |
| 259 | writeState(root(), { startedAt, commit, attempts: 1 }); | |
| 260 | for (let tried = 2; tried <= SWEEP_ATTEMPTS; tried++) { | |
| 261 | const again = await runSweep(app, repo, harness(broken).deps); | |
| 262 | expect(again.line).toContain("could not be exported"); | |
| 263 | expect(readState(root())).toEqual({ startedAt, commit, attempts: tried }); | |
| 264 | } | |
| 265 | expect(sweepDue(readState(root()), new Date()).kind).toBe("wait"); | |
| 266 | writeState(root(), { attempts: 0 }); | |
| 267 | }); | |
| 268 | ||
| 269 | test("the duty starts it on its tick, and its end is a row on /maintenance with what it spent", async () => { | |
| 270 | writeState(root(), { attempts: 0 }); | |
| 271 | fresh(); | |
| 272 | const was = process.env["FORGE_SENTINEL_REPO"]; | |
| 273 | process.env["FORGE_SENTINEL_REPO"] = "alice/forge"; | |
| 274 | try { | |
| 275 | const h = harness(); | |
| 276 | const sweeps: Promise<SweepResult>[] = []; | |
| 277 | const duty = deepsecSweepDuty(h.deps, (sweep) => sweeps.push(sweep)); | |
| 278 | const started = await duty.run({ db: app.db, bus: app.bus, now: new Date(), app }); | |
| 279 | expect(started.actions).toEqual(["started the weekly deepsec sweep"]); | |
| 280 | const again = await duty.run({ db: app.db, bus: app.bus, now: new Date(), app }); | |
| 281 | expect(again.actions).toEqual(["the weekly deepsec sweep is still running"]); | |
| 282 | expect(sweeps).toHaveLength(1); | |
| 283 | const result = await sweeps[0]!; | |
| 284 | expect(result.ran).toBe(true); | |
| 285 | const row = latest(app.db, 10).find((r) => r.duty === NAME)!; | |
| 286 | expect(row.actions).toEqual([result.line]); | |
| 287 | expect(app.db.query<{ dollars: number }, []>(`SELECT dollars FROM maintenance_run WHERE duty = '${NAME}' ORDER BY id DESC LIMIT 1`).get()!.dollars).toBeCloseTo(1.08, 9); | |
| 288 | const quiet = await duty.run({ db: app.db, bus: app.bus, now: new Date(), app }); | |
| 289 | expect(quiet.actions).toEqual([]); | |
| 290 | } finally { | |
| 291 | if (was === undefined) delete process.env["FORGE_SENTINEL_REPO"]; | |
| 292 | else process.env["FORGE_SENTINEL_REPO"] = was; | |
| 293 | } | |
| 294 | expect(existsSync(join(root(), "state.json"))).toBe(true); | |
| 295 | }); | |
| 296 | }); |
test/pages-budget.test.ts +2 −2
| @@ -59,8 +59,8 @@ const HEAVIER: Record<string, { statements?: number; git?: number }> = { | ||
| 59 | 59 | "/api/v1/asks": { statements: 61 }, // +1: the stopped apps someone asked for, one read of the viewer's archived repositories (#12277) |
| 60 | 60 | "/board": { statements: 72 }, // +1: the forge-copy read for the tell box (#12037); +1: the vitals read the database's growth (#12025); +6: the money line (#510) — one settings read, then the month's and the 30 days' meter, storage and ledger reads; constant in the number of apps |
| 61 | 61 | "/settings/capacity": { statements: 70 }, |
| 62 | "/maintenance": { statements: 87 }, // +3: edge-kept (#14682), one more duty reading its own state; +1: every duty's last run in one query (#14674); +1: quieted findings (#14557); +1: the findings fold (#14577); +5: slow-tests (#13966); +3: untested-sources (#14125) and the dismissal query (#13967); +4: flaky-tests (#13937), red apps (#13938), chapter stall (#14057), dead-doc-paths (#13977); +3: stale-branches (#14141) — each reads its own state and runs | |
| 63 | "/api/v1/maintenance/duties": { statements: 69 }, // the same duties as JSON; slow-tests (#13966), stale-branches (#14141) and edge-kept (#14682) each add 3 | |
| 62 | "/maintenance": { statements: 90 }, // +3: deepsec-sweep (#15026); +3: edge-kept (#14682), one more duty reading its own state; +1: every duty's last run in one query (#14674); +1: quieted findings (#14557); +1: the findings fold (#14577); +5: slow-tests (#13966); +3: untested-sources (#14125) and the dismissal query (#13967); +4: flaky-tests (#13937), red apps (#13938), chapter stall (#14057), dead-doc-paths (#13977); +3: stale-branches (#14141) — each reads its own state and runs | |
| 63 | "/api/v1/maintenance/duties": { statements: 72 }, // the same duties as JSON; slow-tests (#13966), stale-branches (#14141), edge-kept (#14682) and deepsec-sweep (#15026) each add 3 | |
| 64 | 64 | }; |
| 65 | 65 | |
| 66 | 66 | /** Patterns with no fixture today: lower this as fixtures are added, never raise it silently. */ |
new above
offline