trevato/forge

Public: anyone can read this; the crew can change it.

More
Forks1g fMapg aRunsg rEventsFlows

Code scanning on every push and nightly: secrets, advisories and static analysis, and a new high finding holds a release until a person accepts it #9599

Merged scoutcrew opened this

Check · Review · Land · Live done — Landed and live in production.

ConversationFiles
Merged

scout/forge:scout/9593-code-scanning-on-every-push-and-nightly → main 8 commits fork

  • Merged. scout/9593-code-scanning-on-every-push-and-nightly was merged into main as 496fb11.

Review

Sign in to review this change.
19 files changed +1327 −2
docs/features/code-scanning.md added +52
@@ -0,0 +1,52 @@
1 # Code scanning
2
3 Every push to a repository is scanned (`src/checks/scan.ts`, run from the
4 post-receive hook, off git's answer):
5
6 - **Secrets** in the files the push changed: AWS, GitHub, Slack and Stripe
7 live keys, and private key blocks. High, except in tests and fixtures
8 (`test/`, `*.test.ts`, `fixtures/`), which quote fakes on purpose: medium there.
9 - **Advisories** in each changed `package.json`: a pin whose lowest admitted
10 version is below the fix of an advisory the forge keeps
11 (`ADVISORIES` in `src/checks/scan_rules.ts`, read without the network).
12
13 Each night (an hourly tick picks repositories whose last pass is a day old)
14 the default branch's whole tree gets both, plus **static analysis**: `eval`,
15 `new Function`, shell commands built from templates, and HTML written from a
16 value, in source files outside tests. Static findings are medium.
17
18 A finding is recorded once per fingerprint (rule, path and a hash of the
19 match). The matched text is never stored or shown: pages, logs and comments
20 name the rule, the file and the line.
21
22 A pass is bounded: at most 5 000 files, each at most 512 KiB, read in one
23 `git cat-file --batch` (a push of many refs costs one `diff-tree` and one
24 `cat-file`; a new branch is compared with the default branch's tip). A file
25 whose name holds a newline is never asked for, since it would shift every
26 later reply onto the wrong file. What a pass could not read is counted, and
27 the findings page says so: "partial: N files not read".
28
29 ## Where it shows
30
31 - `/:owner/:name/findings`: every finding, open high first.
32 - The flows view (`/:owner/:name/flows`): a "Code scanning" line with the
33 open count, linking to the findings.
34 - The terminal: the palette's **Findings in a repository** (`owner/name`,
35 `src/tui/screens/findings.ts`) lists the same findings from the same reads.
36 On an open finding, `a` asks why it may stand and accepts it through the
37 same action as the page's form (`src/actions/scan_accept.ts`): a reader,
38 an agent or a guest is refused, a guest sees nothing of a private
39 repository, and an acceptance made there shows on the web page.
40
41 ## Holding a release
42
43 When a landed tree of the forge's own repository (`FORGE_REPO`) would be
44 swapped in (`cut` in `src/checks/landing.ts`), an open high finding still in
45 that tree holds it: the landed pull is told why and where to accept. Each
46 open high finding's file is read at the landed commit and scanned again
47 (`presentAt`), so a secret on a branch that never landed holds nothing, and
48 one removed from the tree stops holding without an acceptance. If the tree
49 cannot be read, every open high finding holds. A person with write access (never an agent) accepts each
50 finding on the findings page or in the terminal with a reason of 3 to 500 characters; who, why
51 and when are recorded on the finding. Once none is open, the next landing
52 releases.
src/actions/scan_accept.ts added +30
@@ -0,0 +1,30 @@
1 /**
2 * Accept a code scanning finding (#9593): a person with write access says
3 * why it may stand, and the reason, who and when are recorded on the
4 * finding. An agent never accepts one: a finding that holds a release is a
5 * person's to lift, as a merge is. The page and the API answer from here.
6 */
7
8 import type { App } from "../app.ts";
9 import { getFinding, markAccepted } from "../checks/scan_store.ts";
10 import { canWrite, type Repo } from "../core/repos.ts";
11 import { isActive, type User } from "../core/users.ts";
12
13 export const REASON_MIN = 3;
14 export const REASON_MAX = 500;
15
16 export type Accepted = { ok: true } | { ok: false; status: 400 | 403 | 404 | 409; why: string };
17
18 export function acceptFinding(app: App, repo: Repo, user: User | undefined, id: number, rawReason: string): Accepted {
19 if (user === undefined || user.kind !== "user" || !isActive(user) || !canWrite(app.db, repo, user))
20 return { ok: false, status: 403, why: "Only a person with write access accepts a finding." };
21 if (!Number.isSafeInteger(id) || id <= 0) return { ok: false, status: 404, why: "No such finding." };
22 const reason = rawReason.replace(/\s+/g, " ").trim();
23 if (reason.length < REASON_MIN || reason.length > REASON_MAX)
24 return { ok: false, status: 400, why: `Say why it may stand, in ${REASON_MIN} to ${REASON_MAX} characters.` };
25 const finding = getFinding(app.db, repo.id, id);
26 if (finding === undefined) return { ok: false, status: 404, why: "No such finding." };
27 if (!markAccepted(app.db, repo.id, id, user.id, reason)) return { ok: false, status: 409, why: "That finding is already accepted." };
28 app.log.info("finding accepted", { repo: `${repo.ownerName}/${repo.name}`, finding: id, by: user.username });
29 return { ok: true };
30 }
src/checks/landing.ts +8 −1
@@ -22,6 +22,7 @@ import {
2222 import { beginSwap, endSwap, handover, startRelease } from "../release/swap.ts";
2323 import { running } from "../kernel/version.ts";
2424 import { recordRelease, type Release } from "../release/record.ts";
25 import { scanHold } from "../release/scan_hold.ts";
2526 import type { Pull } from "../core/pulls.ts";
2627 import type { Repo } from "../core/repos.ts";
2728 import { failuresOf } from "./failures.ts";
@@ -185,7 +186,13 @@ async function cut(
185186 `The landed tree ${sha.slice(0, 12)} failed its preflight, so it will not be released: ${verdict.line}`,
186187 );
187188 }
188 if (verdict.ok && out.ok) await swapIn(app, by, repo, pull, sha, out.value);
189 // A new high finding holds the release until a person accepts it (#9593).
190 const hold = verdict.ok && out.ok ? await scanHold(app, repo, sha) : { held: false as const };
191 if (hold.held) {
192 app.log.warn("release held by code scanning", { sha: sha.slice(0, 12), pull: pull.number, findings: hold.count });
193 addComment(app, by, repo, getIssueById(app.db, pull.id) ?? pull, `The landed tree ${sha.slice(0, 12)} is not released: ${hold.line}.`);
194 }
195 if (verdict.ok && out.ok && !hold.held) await swapIn(app, by, repo, pull, sha, out.value);
189196 return release;
190197 }
191198
src/checks/scan.ts added +222
@@ -0,0 +1,222 @@
1 /**
2 * Code scanning passes (#9593). On every push: the files each pushed branch
3 * or tag changed get the secret scan, and its package.json files the
4 * advisory check. Nightly: the default branch's whole tree gets both, and
5 * static analysis. Read from the git store with `cat-file --batch`; no
6 * network, no model, nothing run from the repository. What is found is
7 * recorded by fingerprint (`scan_store.ts`), never by its text.
8 */
9
10 import type { App } from "../app.ts";
11 import type { Repo } from "../core/repos.ts";
12 import { runGit, runGitRaw } from "../git/env.ts";
13 import { repoDir } from "../git/repos.ts";
14 import { advisoryFindings, type Finding, secretFindings, staticFindings } from "./scan_rules.ts";
15 import { type FindingRow, noteScanRun, recordFindings } from "./scan_store.ts";
16
17 /** A pass reads at most this many files, each at most this big: a scan is bounded work. */
18 export const MAX_FILES = 5_000;
19 export const MAX_BYTES = 512 * 1024;
20
21 const ZERO = /^0+$/;
22 const OID = /^[0-9a-f]{40}(?:[0-9a-f]{24})?$/;
23
24 /** What a read left out: past MAX_FILES, over MAX_BYTES, or a name `cat-file --batch` cannot ask for. */
25 interface Read {
26 readonly texts: Map<string, string>;
27 skipped: number;
28 }
29
30 /**
31 * A name holding a newline would split its `cat-file --batch` request in two
32 * and shift every later reply onto the wrong file: never asked for, counted
33 * as skipped instead.
34 */
35 const askable = (path: string): boolean => !path.includes("\n") && !path.includes("\r");
36
37 /**
38 * `oid:path → text` for each named blob, skipping the large and the binary:
39 * one `cat-file` for them all. Null when git could not be read at all, so a
40 * caller that must not pass on silence (the hold) can tell.
41 */
42 async function readFiles(dir: string, wanted: readonly { readonly oid: string; readonly path: string }[]): Promise<Read | null> {
43 const out: Read = { texts: new Map(), skipped: 0 };
44 const asked = wanted.filter((w) => askable(w.path));
45 out.skipped = wanted.length - asked.length;
46 if (asked.length > MAX_FILES) {
47 out.skipped += asked.length - MAX_FILES;
48 asked.length = MAX_FILES;
49 }
50 if (asked.length === 0) return out;
51 const raw = await runGitRaw(["cat-file", "--batch"], { cwd: dir, stdin: asked.map((w) => `${w.oid}:${w.path}\n`).join(""), timeoutMs: 60_000 });
52 if (!raw.ok) return null;
53 const bytes = raw.value;
54 const decoder = new TextDecoder();
55 let at = 0;
56 for (const w of asked) {
57 const nl = bytes.indexOf(10, at);
58 if (nl < 0) break;
59 const header = decoder.decode(bytes.subarray(at, nl)).split(" ");
60 at = nl + 1;
61 if (header[1] !== "blob") continue; // "missing", or a submodule's commit
62 const size = Number(header[2]);
63 const body = bytes.subarray(at, at + size);
64 at += size + 1;
65 if (size <= MAX_BYTES) out.texts.set(`${w.oid}:${w.path}`, decoder.decode(body));
66 else out.skipped++;
67 }
68 return out;
69 }
70
71 async function listTree(dir: string, oid: string): Promise<string[]> {
72 const r = await runGit(["ls-tree", "-r", "-z", "--name-only", oid], { cwd: dir });
73 return r.ok ? r.value.split("\0").filter((p) => p !== "") : [];
74 }
75
76 /**
77 * The files each `base → oid` pair changed, in one `diff-tree --stdin`: with
78 * `--always` every pair prints its new commit, then its paths, so the
79 * output splits back into pairs in order.
80 */
81 async function listChangedMany(dir: string, pairs: readonly { readonly base: string; readonly oid: string }[]): Promise<string[][]> {
82 const out: string[][] = pairs.map(() => []);
83 if (pairs.length === 0) return out;
84 const r = await runGit(["diff-tree", "--stdin", "--always", "-r", "-z", "--name-only", "--diff-filter=AMR"], {
85 cwd: dir,
86 stdin: pairs.map((p) => `${p.oid} ${p.base}\n`).join(""), // `commit parent`: diffs base → oid
87 timeoutMs: 60_000,
88 });
89 if (!r.ok) return out;
90 let k = -1;
91 for (const token of r.value.split("\0")) {
92 if (token === "") continue;
93 if (k + 1 < pairs.length && token === pairs[k + 1]!.oid) {
94 k++;
95 continue;
96 }
97 if (k >= 0) out[k]!.push(token);
98 }
99 return out;
100 }
101
102 function filesOf(read: Read | null, oid: string, paths: readonly string[]): Map<string, string> {
103 const files = new Map<string, string>();
104 for (const p of paths) {
105 const text = read?.texts.get(`${oid}:${p}`);
106 if (text !== undefined) files.set(p, text);
107 }
108 return files;
109 }
110
111 export function findingsIn(files: ReadonlyMap<string, string>, withStatic: boolean): Finding[] {
112 const out: Finding[] = [];
113 for (const [path, text] of files) {
114 out.push(...secretFindings(path, text), ...advisoryFindings(path, text));
115 if (withStatic) out.push(...staticFindings(path, text));
116 }
117 return out;
118 }
119
120 /**
121 * The push pass: returns how many findings were new. Never throws for a ref
122 * it cannot read. Bounded work however many refs a push names: one
123 * `diff-tree` and one `cat-file` for them all, at most MAX_FILES files. A new
124 * branch is compared with the default branch's tip; only a push with no
125 * default branch yet lists whole trees.
126 */
127 export async function scanPush(
128 app: App,
129 repo: Repo,
130 updates: readonly { readonly ref: string; readonly oldOid: string; readonly newOid: string }[],
131 ): Promise<number> {
132 const dir = repoDir(app.reposRoot, repo.ownerName, repo.name);
133 const live = updates.filter((u) => (u.ref.startsWith("refs/heads/") || u.ref.startsWith("refs/tags/")) && OID.test(u.newOid) && !ZERO.test(u.newOid));
134 if (live.length === 0) return 0;
135 let tip = live.find((u) => u.ref === `refs/heads/${repo.defaultBranch}`)?.newOid;
136 if (tip === undefined && live.some((u) => ZERO.test(u.oldOid))) {
137 const r = await runGit(["rev-parse", "--verify", "-q", `refs/heads/${repo.defaultBranch}^{commit}`], { cwd: dir });
138 if (r.ok && OID.test(r.value)) tip = r.value;
139 }
140 const paths: string[][] = live.map(() => []);
141 const pairs: { base: string; oid: string; at: number }[] = [];
142 for (const [i, u] of live.entries()) {
143 const base = ZERO.test(u.oldOid) ? tip : u.oldOid;
144 if (base === undefined) paths[i] = await listTree(dir, u.newOid);
145 else if (base !== u.newOid) pairs.push({ base, oid: u.newOid, at: i });
146 }
147 const changed = await listChangedMany(dir, pairs);
148 for (const [j, p] of pairs.entries()) paths[p.at] = changed[j]!;
149 const read = await readFiles(dir, live.flatMap((u, i) => paths[i]!.map((path) => ({ oid: u.newOid, path }))));
150 let fresh = 0;
151 for (const [i, u] of live.entries()) {
152 fresh += recordFindings(app.db, repo.id, u.newOid, findingsIn(filesOf(read, u.newOid, paths[i]!), false));
153 }
154 const total = live.reduce((n, _, i) => n + paths[i]!.length, 0);
155 noteScanRun(app.db, repo.id, "push", live.at(-1)!.newOid, read === null ? total : read.skipped);
156 return fresh;
157 }
158
159 /** The nightly pass over the default branch's whole tree, static analysis included. */
160 export async function scanNightly(app: App, repo: Repo): Promise<number> {
161 const dir = repoDir(app.reposRoot, repo.ownerName, repo.name);
162 const tip = await runGit(["rev-parse", "--verify", "-q", `refs/heads/${repo.defaultBranch}^{commit}`], { cwd: dir });
163 if (!tip.ok || !OID.test(tip.value)) {
164 noteScanRun(app.db, repo.id, "nightly", "");
165 return 0;
166 }
167 const paths = await listTree(dir, tip.value);
168 const read = await readFiles(dir, paths.map((path) => ({ oid: tip.value, path })));
169 const fresh = recordFindings(app.db, repo.id, tip.value, findingsIn(filesOf(read, tip.value, paths), true));
170 noteScanRun(app.db, repo.id, "nightly", tip.value, read === null ? paths.length : read.skipped);
171 return fresh;
172 }
173
174 /**
175 * Which of `findings` are still in the tree at `oid`: each one's file is read
176 * there and scanned again, and a finding stands only where its fingerprint
177 * (rule, path and the hashed match) comes back. So a secret on a branch that
178 * never reached `oid`, or one since removed, is not there. Null when git
179 * could not be read: the hold then believes every finding.
180 */
181 export async function presentAt(app: App, repo: Repo, oid: string, findings: readonly FindingRow[]): Promise<FindingRow[] | null> {
182 if (findings.length === 0) return [];
183 if (!OID.test(oid)) return null;
184 const dir = repoDir(app.reposRoot, repo.ownerName, repo.name);
185 const paths = [...new Set(findings.map((f) => f.path))];
186 const read = await readFiles(dir, paths.map((path) => ({ oid, path })));
187 if (read === null || read.skipped > 0) return null;
188 const there = new Set(findingsIn(filesOf(read, oid, paths), true).map((f) => f.fingerprint));
189 return findings.filter((f) => there.has(f.fingerprint));
190 }
191
192 /** Each app's push scans, chained per repository: one after another, and awaitable. */
193 const queues = new WeakMap<App, Map<string, Promise<unknown>>>();
194
195 /**
196 * Queues the push pass behind the repository's earlier ones and returns it.
197 * The hook does not wait for it; `scansSettled` does.
198 */
199 export function queueScanPush(
200 app: App,
201 repo: Repo,
202 updates: readonly { readonly ref: string; readonly oldOid: string; readonly newOid: string }[],
203 ): Promise<number> {
204 let byRepo = queues.get(app);
205 if (byRepo === undefined) queues.set(app, (byRepo = new Map()));
206 const prior = byRepo.get(repo.id) ?? Promise.resolve();
207 const next = prior.catch(() => undefined).then(() => scanPush(app, repo, updates));
208 const tail = next.catch(() => undefined);
209 byRepo.set(repo.id, tail);
210 void tail.then(() => {
211 if (byRepo.get(repo.id) === tail) byRepo.delete(repo.id);
212 });
213 return next;
214 }
215
216 /** Resolves once every push scan queued for the repository so far has finished. */
217 export async function scansSettled(app: App, repoId: string): Promise<void> {
218 for (let q = queues.get(app)?.get(repoId); q !== undefined; q = queues.get(app)?.get(repoId)) {
219 await q;
220 if (queues.get(app)?.get(repoId) === q) return;
221 }
222 }
src/checks/scan_rules.ts added +136
@@ -0,0 +1,136 @@
1 /**
2 * Code scanning's rules (#9593), pure: a file's path and text in, findings
3 * out. Three kinds: `secret` (a credential committed in plain text),
4 * `advisory` (a package.json pin a known advisory names) and `static` (a
5 * pattern that turns text into code). A finding never carries the matched
6 * text: the secret is hashed into the fingerprint and dropped, so a page,
7 * a log or a comment that prints a finding cannot print the secret.
8 */
9
10 import { createHash } from "node:crypto";
11
12 export type ScanKind = "secret" | "advisory" | "static";
13 export type Severity = "high" | "medium";
14
15 export interface Finding {
16 readonly kind: ScanKind;
17 readonly severity: Severity;
18 /** The rule's name, as a reader reads it: "AWS access key", "lodash < 4.17.21". */
19 readonly rule: string;
20 readonly path: string;
21 /** 1-based line, 0 when the finding is about the whole file. */
22 readonly line: number;
23 /** Stable across pushes for the same thing in the same file: rule, path and a hash of the match. */
24 readonly fingerprint: string;
25 }
26
27 const hash = (s: string): string => createHash("sha256").update(s).digest("hex").slice(0, 24);
28
29 const SECRETS: readonly { rule: string; re: RegExp }[] = [
30 { rule: "AWS access key", re: /\b(?:AKIA|ASIA)[0-9A-Z]{16}\b/g },
31 { rule: "GitHub token", re: /\bgh[pousr]_[A-Za-z0-9]{36,}\b/g },
32 { rule: "Slack token", re: /\bxox[abprs]-[A-Za-z0-9-]{10,}\b/g },
33 { rule: "Stripe live key", re: /\b[sr]k_live_[0-9A-Za-z]{24,}\b/g },
34 { rule: "Private key", re: /-----BEGIN (?:RSA |EC |DSA |OPENSSH |PGP )?PRIVATE KEY(?: BLOCK)?-----/g },
35 ];
36
37 /** Lines of `text` with their 1-based numbers; a file with a NUL is binary and has none. */
38 function lines(text: string): string[] {
39 return text.includes("\0") ? [] : text.split("\n");
40 }
41
42 /** Tests and fixtures quote fake credentials on purpose: what they hold is still shown, as medium, and holds nothing. */
43 export const TEST_PATH = /(?:^|\/)(?:test|tests|__tests__|fixtures?)\/|\.test\.[cm]?[jt]sx?$/;
44
45 export function secretFindings(path: string, text: string): Finding[] {
46 const severity: Severity = TEST_PATH.test(path) ? "medium" : "high";
47 const out: Finding[] = [];
48 lines(text).forEach((l, i) => {
49 for (const { rule, re } of SECRETS) {
50 for (const m of l.matchAll(re)) {
51 out.push({ kind: "secret", severity, rule, path, line: i + 1, fingerprint: hash(`secret\0${rule}\0${path}\0${m[0]}`) });
52 }
53 }
54 });
55 return out;
56 }
57
58 /** Known advisories, kept here and read without the network: a package, the first fixed version, and its severity. */
59 export const ADVISORIES: readonly { name: string; fixed: string; severity: Severity; id: string }[] = [
60 { name: "lodash", fixed: "4.17.21", severity: "high", id: "GHSA-35jh-r3h4-6jhm" },
61 { name: "minimist", fixed: "1.2.6", severity: "high", id: "GHSA-xvch-5gv4-984h" },
62 { name: "node-fetch", fixed: "2.6.7", severity: "high", id: "GHSA-r683-j2x4-v87g" },
63 { name: "jsonwebtoken", fixed: "9.0.0", severity: "high", id: "GHSA-27h2-hvpr-p74q" },
64 { name: "semver", fixed: "7.5.2", severity: "medium", id: "GHSA-c2qf-rxjj-qqgw" },
65 { name: "axios", fixed: "1.6.0", severity: "medium", id: "GHSA-wf5p-g6vw-rhxx" },
66 { name: "ws", fixed: "8.17.1", severity: "high", id: "GHSA-3h5v-q93c-6h6q" },
67 { name: "tar", fixed: "6.2.1", severity: "medium", id: "GHSA-f5x3-32g6-xq36" },
68 ];
69
70 /** `a < b` for dotted numeric versions; anything that is not one compares as not less. */
71 export function versionBelow(a: string, b: string): boolean {
72 const pa = a.split(".").map(Number);
73 const pb = b.split(".").map(Number);
74 if (pa.some((n) => !Number.isInteger(n)) || pb.some((n) => !Number.isInteger(n))) return false;
75 for (let i = 0; i < Math.max(pa.length, pb.length); i++) {
76 const x = pa[i] ?? 0;
77 const y = pb[i] ?? 0;
78 if (x !== y) return x < y;
79 }
80 return false;
81 }
82
83 /** The lowest version a range admits, when it says one plainly (`1.2.3`, `^1.2.3`, `~1.2.3`, `>=1.2.3`). */
84 function floorOf(range: string): string | undefined {
85 const m = /^\s*(?:\^|~|>=|=|v)?\s*(\d+(?:\.\d+){0,2})(?:[-+][0-9A-Za-z.-]*)?\s*$/.exec(range);
86 return m?.[1];
87 }
88
89 export function advisoryFindings(path: string, text: string): Finding[] {
90 if (path !== "package.json" && !path.endsWith("/package.json")) return [];
91 let pkg: unknown;
92 try {
93 pkg = JSON.parse(text);
94 } catch {
95 return [];
96 }
97 if (typeof pkg !== "object" || pkg === null) return [];
98 const out: Finding[] = [];
99 for (const field of ["dependencies", "devDependencies", "optionalDependencies"] as const) {
100 const deps = (pkg as Record<string, unknown>)[field];
101 if (typeof deps !== "object" || deps === null) continue;
102 for (const [name, range] of Object.entries(deps as Record<string, unknown>)) {
103 if (typeof range !== "string") continue;
104 const floor = floorOf(range);
105 if (floor === undefined) continue;
106 for (const a of ADVISORIES) {
107 if (a.name !== name || !versionBelow(floor, a.fixed)) continue;
108 const rule = `${name} < ${a.fixed} (${a.id})`;
109 out.push({ kind: "advisory", severity: a.severity, rule, path, line: 0, fingerprint: hash(`advisory\0${a.id}\0${path}`) });
110 }
111 }
112 }
113 return out;
114 }
115
116 const CODE = /\.(?:[cm]?[jt]sx?)$/;
117 const STATIC: readonly { rule: string; re: RegExp }[] = [
118 { rule: "eval of a value", re: /(?<![\w.$"'`])eval\s*\(/ },
119 { rule: "new Function from text", re: /\bnew\s+Function\s*\(/ },
120 { rule: "shell command built from a template", re: /\bexec(?:Sync)?\s*\(\s*`[^`]*\$\{/ },
121 { rule: "HTML written from a value", re: /\.(?:innerHTML|outerHTML)\s*=(?!=)/ },
122 ];
123
124 /** Static analysis for the nightly pass: source files only, tests left out (they quote these patterns on purpose). Medium: a reader judges it, no release waits. */
125 export function staticFindings(path: string, text: string): Finding[] {
126 if (!CODE.test(path) || /(?:^|\/)(?:test|tests|__tests__|vendor|node_modules)\//.test(path) || /\.test\.[jt]sx?$/.test(path)) return [];
127 const out: Finding[] = [];
128 lines(text).forEach((l, i) => {
129 const code = l.trimStart();
130 if (code.startsWith("//") || code.startsWith("*")) return;
131 for (const { rule, re } of STATIC) {
132 if (re.test(l)) out.push({ kind: "static", severity: "medium", rule, path, line: i + 1, fingerprint: hash(`static\0${rule}\0${path}\0${l.trim()}`) });
133 }
134 });
135 return out;
136 }
src/checks/scan_store.ts added +123
@@ -0,0 +1,123 @@
1 /**
2 * Code scanning's findings as rows (#9593): recorded once per fingerprint,
3 * listed for the repository's findings page and flows line, and the open
4 * high ones that hold a release until a person accepts each with a reason.
5 */
6
7 import type { Database } from "bun:sqlite";
8 import type { Finding, ScanKind, Severity } from "./scan_rules.ts";
9
10 export interface FindingRow {
11 readonly id: number;
12 readonly kind: ScanKind;
13 readonly severity: Severity;
14 readonly rule: string;
15 readonly path: string;
16 readonly line: number;
17 readonly oid: string;
18 /** The hash of rule, path and match: how a later read knows it again. Never the match. */
19 readonly fingerprint: string;
20 readonly foundAt: string;
21 /** Who accepted it, by username, with why and when; null while open. */
22 readonly accepted: { readonly by: string; readonly reason: string; readonly at: string } | null;
23 }
24
25 interface Raw {
26 id: number;
27 kind: ScanKind;
28 severity: Severity;
29 rule: string;
30 path: string;
31 line: number;
32 oid: string;
33 fingerprint: string;
34 found_at: string;
35 accepted_reason: string | null;
36 accepted_at: string | null;
37 username: string | null;
38 }
39
40 const SELECT = `SELECT f.id, f.kind, f.severity, f.rule, f.path, f.line, f.oid, f.fingerprint, f.found_at, f.accepted_reason, f.accepted_at, u.username
41 FROM scan_finding f LEFT JOIN user u ON u.id = f.accepted_by`;
42
43 const shape = (r: Raw): FindingRow => ({
44 id: r.id, kind: r.kind, severity: r.severity, rule: r.rule, path: r.path, line: r.line, oid: r.oid, fingerprint: r.fingerprint, foundAt: r.found_at,
45 accepted: r.accepted_at === null ? null : { by: r.username ?? "someone removed", reason: r.accepted_reason ?? "", at: r.accepted_at },
46 });
47
48 /** Record what a pass found at `oid`; a fingerprint already recorded is left as it stands. Returns how many were new. */
49 export function recordFindings(db: Database, repoId: string, oid: string, findings: readonly Finding[], at = new Date().toISOString()): number {
50 const insert = db.query(
51 `INSERT OR IGNORE INTO scan_finding (repo_id, fingerprint, kind, severity, rule, path, line, oid, found_at)
52 VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9)`,
53 );
54 let fresh = 0;
55 db.transaction(() => {
56 for (const f of findings) fresh += insert.run(repoId, f.fingerprint, f.kind, f.severity, f.rule, f.path, f.line, oid, at).changes;
57 })();
58 return fresh;
59 }
60
61 /** When a pass ran, at what, and how many files it could not read (past the limits, or unaskable): a partial pass says so. */
62 export function noteScanRun(db: Database, repoId: string, kind: "push" | "nightly", oid: string, skipped = 0, at = new Date().toISOString()): void {
63 db.query("INSERT INTO scan_run (repo_id, kind, oid, at, skipped) VALUES (?1, ?2, ?3, ?4, ?5) ON CONFLICT (repo_id, kind) DO UPDATE SET oid = ?3, at = ?4, skipped = ?5").run(repoId, kind, oid, at, skipped);
64 }
65
66 export function lastScan(db: Database, repoId: string, kind: "push" | "nightly"): { at: string; skipped: number } | null {
67 return db.query<{ at: string; skipped: number }, [string, string]>("SELECT at, skipped FROM scan_run WHERE repo_id = ?1 AND kind = ?2").get(repoId, kind) ?? null;
68 }
69
70 export function lastScanAt(db: Database, repoId: string, kind: "push" | "nightly"): string | null {
71 return lastScan(db, repoId, kind)?.at ?? null;
72 }
73
74 /** Newest first, open before accepted; bounded. */
75 export function listFindings(db: Database, repoId: string, limit = 200): FindingRow[] {
76 return db
77 .query<Raw, [string, number]>(`${SELECT} WHERE f.repo_id = ?1 ORDER BY f.accepted_at IS NOT NULL, f.id DESC LIMIT ?2`)
78 .all(repoId, limit)
79 .map(shape);
80 }
81
82 export function getFinding(db: Database, repoId: string, id: number): FindingRow | undefined {
83 const r = db.query<Raw, [string, number]>(`${SELECT} WHERE f.repo_id = ?1 AND f.id = ?2`).get(repoId, id);
84 return r ? shape(r) : undefined;
85 }
86
87 /** The open high findings: each holds a release of this repository. */
88 export function openHigh(db: Database, repoId: string): FindingRow[] {
89 return db
90 .query<Raw, [string]>(`${SELECT} WHERE f.repo_id = ?1 AND f.severity = 'high' AND f.accepted_at IS NULL ORDER BY f.id`)
91 .all(repoId)
92 .map(shape);
93 }
94
95 export function counts(db: Database, repoId: string): { openHigh: number; open: number; accepted: number } {
96 const r = db
97 .query<{ high: number | null; open: number | null; accepted: number | null }, [string]>(
98 `SELECT SUM(accepted_at IS NULL AND severity = 'high') AS high, SUM(accepted_at IS NULL) AS open, SUM(accepted_at IS NOT NULL) AS accepted
99 FROM scan_finding WHERE repo_id = ?1`,
100 )
101 .get(repoId);
102 return { openHigh: r?.high ?? 0, open: r?.open ?? 0, accepted: r?.accepted ?? 0 };
103 }
104
105 /** Mark an open finding accepted; false when it was not open. The caller has checked who and why. */
106 export function markAccepted(db: Database, repoId: string, id: number, userId: string, reason: string, at = new Date().toISOString()): boolean {
107 return (
108 db
109 .query("UPDATE scan_finding SET accepted_by = ?3, accepted_reason = ?4, accepted_at = ?5 WHERE repo_id = ?1 AND id = ?2 AND accepted_at IS NULL")
110 .run(repoId, id, userId, reason, at).changes === 1
111 );
112 }
113
114 /** Repositories whose nightly pass is due: never run, or last run before `before`. Bounded. */
115 export function nightlyDue(db: Database, before: string, limit: number): string[] {
116 return db
117 .query<{ id: string }, [string, number]>(
118 `SELECT r.id FROM repo r LEFT JOIN scan_run s ON s.repo_id = r.id AND s.kind = 'nightly'
119 WHERE s.at IS NULL OR s.at < ?1 ORDER BY s.at IS NOT NULL, s.at LIMIT ?2`,
120 )
121 .all(before, limit)
122 .map((r) => r.id);
123 }
src/core/migration-pins/20261108160000-scan-findings.pin added +1
@@ -0,0 +1,1 @@
1 96121abb61c647eca46f2ff63ece6b9a0db91e953ce6a174d3057c6d37f87066
src/core/migrations/20261108160000-scan-findings.ts added +32
@@ -0,0 +1,32 @@
1 // Code scanning (#9593): what a push's or the nightly pass's scan found in a
2 // repository, one row per finding fingerprint (never the matched text), and
3 // a person's acceptance with its reason. `scan_run` is when each kind of pass
4 // last ran on each repository, so the nightly tick picks the ones due, and
5 // how many files it could not read, so a partial pass says so.
6 // New tables only. Additive.
7 export default `
8 CREATE TABLE scan_finding (
9 id INTEGER PRIMARY KEY,
10 repo_id TEXT NOT NULL REFERENCES repo(id) ON DELETE CASCADE,
11 fingerprint TEXT NOT NULL,
12 kind TEXT NOT NULL CHECK (kind IN ('secret', 'advisory', 'static')),
13 severity TEXT NOT NULL CHECK (severity IN ('high', 'medium')),
14 rule TEXT NOT NULL,
15 path TEXT NOT NULL,
16 line INTEGER NOT NULL,
17 oid TEXT NOT NULL,
18 found_at TEXT NOT NULL,
19 accepted_by TEXT REFERENCES user(id) ON DELETE SET NULL,
20 accepted_reason TEXT,
21 accepted_at TEXT,
22 UNIQUE (repo_id, fingerprint)
23 );
24 CREATE TABLE scan_run (
25 repo_id TEXT NOT NULL REFERENCES repo(id) ON DELETE CASCADE,
26 kind TEXT NOT NULL CHECK (kind IN ('push', 'nightly')),
27 oid TEXT NOT NULL,
28 at TEXT NOT NULL,
29 skipped INTEGER NOT NULL DEFAULT 0,
30 PRIMARY KEY (repo_id, kind)
31 );
32 `;
src/flows/view.ts +20 −1
@@ -11,6 +11,7 @@ import { forkFollowMode } from "../core/fork_follow.ts";
1111 import { canRead, canWrite, getRepoById, repoDatesFor, visibleForks, type Repo } from "../core/repos.ts";
1212 import type { User } from "../core/users.ts";
1313 import { aheadWords, parseAhead } from "../origins/ahead.ts";
14 import { counts, lastScanAt } from "../checks/scan_store.ts";
1415
1516 /** A followed link is stale when its last head comparison is older than this (three follow periods). */
1617 export const STALE_MS = 3 * 30 * 60_000;
@@ -26,7 +27,7 @@ export interface FlowAction {
2627 }
2728
2829 export interface FlowLine {
29 readonly kind: "origin" | "source" | "upstream" | "fork" | "mirror" | "forge";
30 readonly kind: "origin" | "source" | "upstream" | "fork" | "mirror" | "forge" | "scan";
3031 readonly label: string;
3132 /** In-forge link, when the line is a repository here. */
3233 readonly href: string;
@@ -146,6 +147,7 @@ export function readFlows(db: Database, repo: Repo, viewer: User | undefined, no
146147 health: mirror.last_error !== "" ? "unread" : "ok",
147148 });
148149 }
150 inbound.push(scanLine(db, repo));
149151 // Forges that follow this one: a child forge holds a proposal token for it. Their names are for those who write here.
150152 if (writer) {
151153 for (const c of db
@@ -163,3 +165,20 @@ export function readFlows(db: Database, repo: Repo, viewer: User | undefined, no
163165 }
164166 return { inbound, outbound };
165167 }
168
169 /** Code scanning as one line (#9593): what the push and nightly passes found, open high first. */
170 export function scanLine(db: Database, repo: Repo): FlowLine {
171 const c = counts(db, repo.id);
172 const push = lastScanAt(db, repo.id, "push");
173 const nightly = lastScanAt(db, repo.id, "nightly");
174 const at = push === null ? nightly : nightly === null || push > nightly ? push : nightly;
175 const state =
176 at === null ? "not scanned yet"
177 : c.openHigh > 0 ? `${c.openHigh} high ${c.openHigh === 1 ? "finding holds" : "findings hold"} releases`
178 : c.open > 0 ? `${c.open} open ${c.open === 1 ? "finding" : "findings"}`
179 : "no open findings";
180 return {
181 kind: "scan", label: "Code scanning", href: `${base(repo)}/findings`, state, at, problem: c.openHigh > 0,
182 health: at === null ? "none" : c.openHigh > 0 ? "unread" : c.open > 0 ? "behind" : "ok",
183 };
184 }
src/http/server.ts +12
@@ -20,6 +20,7 @@ import { signupOptions, signupPasskey } from "../web/pages/signup-passkey.ts";
2020 import { toolbarComment, toolbarPage } from "../web/studio/toolbar.ts";
2121 import { TOOLBAR_NAME, toolbarResponse } from "../apps/toolbar.ts";
2222 import { unsealReleased } from "../actions/sealed.ts";
23 import { queueScanPush } from "../checks/scan.ts";
2324 import { parentCallback, parentStart } from "./rp.ts";
2425 import { prerenderComments } from "../web/prerender.ts";
2526 import { systemPage, systemStream } from "../web/pages/system.ts";
@@ -152,6 +153,7 @@ import { activityAtom, changelogAtom } from "../web/feeds.ts";
152153 import { auditCsv, auditPage, auditProbeSubmit } from "../web/pages/audit.ts";
153154 import { apiPage, appApiPage } from "../web/pages/api.ts";
154155 import { flowsCheck, flowsPage, flowsProbe } from "../web/pages/flows.ts";
156 import { findingAccept, findingsPage } from "../web/pages/findings.ts";
155157 import { betAct, betPage, betPropose, betsPage } from "../web/pages/bets.ts";
156158 import { changelogJson, changelogPage } from "../web/pages/changelog.ts";
157159 import { releaseHoldSubmit, liftSubmit, releasesPage, rollbackSubmit } from "../web/pages/releases.ts";
@@ -1152,6 +1154,8 @@ export function buildRouter(): Router<Handler> {
11521154 router.add("GET", "/:owner/:name/flows", flowsPage);
11531155 router.add("POST", "/:owner/:name/flows/check", flowsCheck);
11541156 router.add("POST", "/:owner/:name/flows/probe", flowsProbe);
1157 router.add("GET", "/:owner/:name/findings", findingsPage);
1158 router.add("POST", "/:owner/:name/findings/:id/accept", findingAccept);
11551159 router.add("GET", "/:owner/:name/bets", betsPage);
11561160 router.add("POST", "/:owner/:name/bets/propose", betPropose);
11571161 router.add("GET", "/:owner/:name/bets/:id", betPage);
@@ -2035,6 +2039,14 @@ async function handleHook(app: App, request: Request): Promise<Response> {
20352039 error: String(error),
20362040 }),
20372041 );
2042 // Every push is scanned for secrets and advisories (`checks/scan.ts`,
2043 // #9593). Off the hook's answer too: a finding is recorded, not refused.
2044 void queueScanPush(app, repo, input.updates).catch((error) =>
2045 app.log.warn("could not scan the push", {
2046 repo: `${repo.ownerName}/${repo.name}`,
2047 error: String(error),
2048 }),
2049 );
20382050 return new Response("");
20392051 }
20402052
src/main.ts +12
@@ -4,6 +4,8 @@
44 * environment-only — a handful of variables, all with sane defaults.
55 */
66
7 import { scanNightly } from "./checks/scan.ts";
8 import { nightlyDue } from "./checks/scan_store.ts";
79 import { runModelReview } from "./actions/model_review.ts";
810 import { getRepoById } from "./core/repos.ts";
911 import { plannedRepoIds, tickWeeks } from "./flow/retro.ts";
@@ -697,6 +699,15 @@ const weekTimer = setInterval(() => {
697699 app.log.warn("week retrospective failed", { error: String(error) });
698700 }
699701 }, 3_600_000);
702 // Code scanning's nightly pass (#9593): each hour, the repositories whose last pass is a day old get secrets, advisories and static analysis over their default branch.
703 const nightlyScanTimer = setInterval(() => {
704 void (async () => {
705 for (const id of nightlyDue(app.db, new Date(Date.now() - 24 * 3_600_000).toISOString(), 25)) {
706 const repo = getRepoById(app.db, id);
707 if (repo !== undefined) await scanNightly(app, repo);
708 }
709 })().catch((error) => app.log.warn("nightly scan failed", { error: String(error) }));
710 }, 3_600_000);
700711 const processTimer = setInterval(processTick, 60_000);
701712 processTick();
702713 // A new version proves itself against the public URL; a failed one again, 5 min after and backing off to hourly (release/prove.ts).
@@ -837,6 +848,7 @@ function drain(signal: NodeJS.Signals): Promise<unknown> {
837848 // process that replaced this one, not to this one while it finishes.
838849 drainStreams();
839850 clearInterval(deliveryTimer);
851 clearInterval(nightlyScanTimer);
840852 clearInterval(heartbeatTimer);
841853 clearInterval(upstreamTimer);
842854 clearInterval(reachTimer);
src/release/scan_hold.ts added +31
@@ -0,0 +1,31 @@
1 /**
2 * A new high finding holds a release (#9593). Before a landed tree of the
3 * forge's own repository is swapped in, its open high findings are asked —
4 * only those still in that tree (`presentAt`): a secret on a branch that
5 * never landed, or one since removed, holds nothing. Any one left and the
6 * release waits, saying how many and where to accept them. A person with
7 * write access lifts the hold by accepting each with a reason
8 * (`actions/scan_accept.ts`); nothing here prints the match. When the tree
9 * cannot be read, every open high finding holds: silence never releases.
10 */
11
12 import type { App } from "../app.ts";
13 import { presentAt } from "../checks/scan.ts";
14 import { openHigh } from "../checks/scan_store.ts";
15 import type { Repo } from "../core/repos.ts";
16
17 export type ScanHold = { held: false } | { held: true; count: number; line: string };
18
19 export async function scanHold(app: App, repo: Repo, sha: string): Promise<ScanHold> {
20 const all = openHigh(app.db, repo.id);
21 const open = (await presentAt(app, repo, sha, all)) ?? all;
22 if (open.length === 0) return { held: false };
23 const first = open[0]!;
24 const where = `${first.rule} in ${first.path}${first.line > 0 ? `:${first.line}` : ""}`;
25 const more = open.length === 1 ? "" : ` and ${open.length - 1} more`;
26 return {
27 held: true,
28 count: open.length,
29 line: `held by ${open.length} open high ${open.length === 1 ? "finding" : "findings"} (${where}${more}); a person with write access accepts each with a reason at /${repo.ownerName}/${repo.name}/findings`,
30 };
31 }
src/tui/forge.ts +7
@@ -81,6 +81,7 @@ import { stalledScreen } from "./screens/stalled.ts";
8181 import { issuesScreen, newIssueScreen } from "./screens/issues.ts";
8282 import { limbsScreen } from "./screens/limbs.ts";
8383 import { nextScreen } from "./screens/next.ts";
84 import { findingsScreen } from "./screens/findings.ts";
8485 import { flowsScreen } from "./screens/flows.ts";
8586 import { weekScreen } from "./screens/week.ts";
8687 import { machinesScreen } from "./screens/machines.ts";
@@ -152,6 +153,12 @@ const REPO_VERBS: readonly {
152153 title: "Flows of a repository",
153154 open: (f, ctx, repo) => flowsScreen(f, ctx, repo),
154155 },
156 {
157 // What code scanning found, and acceptance with a reason for a writer (#9593).
158 id: "findings",
159 title: "Findings in a repository",
160 open: (f, ctx, repo) => findingsScreen(f, ctx, repo),
161 },
155162 {
156163 // Settings → Machines' other door: the machines and last shell sessions, writers only (#9471).
157164 id: "machines",
src/tui/screens/findings.ts added +89
@@ -0,0 +1,89 @@
1 /**
2 * Findings in the terminal (#9593): the findings page's list from the same
3 * reads — `listFindings`, `lastScan` — newest open first, never the matched
4 * text. `a` on an open finding asks why it may stand and accepts it through
5 * `acceptFinding`, the action the page's form calls, so the same people are
6 * refused (agents, readers, guests) and the acceptance shows on the page.
7 * Access is read at every draw: a person who cannot read sees nothing.
8 */
9
10 import { acceptFinding, REASON_MAX, REASON_MIN } from "../../actions/scan_accept.ts";
11 import { lastScan, listFindings, type FindingRow } from "../../checks/scan_store.ts";
12 import { canRead, canWrite, type Repo } from "../../core/repos.ts";
13 import type { Ctx, Screen } from "../app/app.ts";
14 import { binding } from "../app/keys.ts";
15 import type { Session } from "../session.ts";
16 import { style } from "../term/style.ts";
17 import { emptyState } from "../ui/widgets.ts";
18 import { ListState, drawList, promptScreen, type ListRow } from "./common.ts";
19
20 const pass = (run: { at: string; skipped: number } | null): string =>
21 run === null ? "never" : `${run.at.slice(0, 16).replace("T", " ")}${run.skipped === 0 ? "" : ` (partial: ${run.skipped} not read)`}`;
22
23 export function findingRow(f: FindingRow): ListRow {
24 const where = `${f.path}${f.line > 0 ? `:${f.line}` : ""}`;
25 return {
26 title: f.rule,
27 secondary: f.accepted !== null ? "accepted" : f.severity,
28 ...(f.accepted === null ? { tone: f.severity === "high" ? ("danger" as const) : ("attn" as const) } : {}),
29 meta: where,
30 note: f.accepted !== null ? `accepted by ${f.accepted.by}: ${f.accepted.reason}` : `${f.kind} · found ${f.foundAt.slice(0, 10)} at ${f.oid.slice(0, 12)}`,
31 };
32 }
33
34 export function findingsScreen(f: Session, ctx: Ctx, repo: Repo): Screen {
35 const state = new ListState();
36 const visible = (): boolean => canRead(f.db, repo, f.user);
37 const writer = (): boolean => f.user !== undefined && f.user.kind === "user" && canWrite(f.db, repo, f.user);
38 const rows = (): FindingRow[] => (visible() ? listFindings(f.db, repo.id) : []);
39 const screen: Screen = {
40 title: "Findings",
41 keys: [binding(["j", "k"], "move cursor"), ...(writer() ? [binding(["a"], "accept with a reason")] : []), binding(["esc"], "back")],
42 render(area, s) {
43 const c = ctx.theme.c;
44 if (!visible()) {
45 emptyState(s, area, c, { title: "Not yours to see", body: "Findings are for the repository's readers." });
46 return;
47 }
48 const list = rows();
49 const head = `Last push scan ${pass(lastScan(f.db, repo.id, "push"))} · last nightly scan ${pass(lastScan(f.db, repo.id, "nightly"))}`;
50 if (list.length === 0) {
51 emptyState(s, area, c, { title: "Nothing found", body: head });
52 return;
53 }
54 s.text(area.x, area.y, head.slice(0, area.w), style(c.mutedFg, c.bg));
55 drawList(s, { ...area, y: area.y + 2, h: Math.max(0, area.h - 2) }, c, state, list.map(findingRow));
56 },
57 key(e) {
58 if (e.ctrl || e.alt) return state.key(e, rows().length);
59 if (e.key === "a") {
60 const finding = rows()[state.index];
61 if (finding === undefined) return false;
62 if (finding.accepted !== null) {
63 ctx.toast("That finding is already accepted.");
64 return true;
65 }
66 if (!writer()) {
67 ctx.toast("Only a person with write access accepts a finding.", "danger");
68 return true;
69 }
70 ctx.push(
71 promptScreen(ctx, screen, {
72 title: `Accept ${finding.rule}`,
73 label: `why it may stand (${REASON_MIN} to ${REASON_MAX} characters)`,
74 onSubmit: (reason) => {
75 const done = acceptFinding(f.app, repo, f.user, finding.id, reason);
76 if (!done.ok) return done.why;
77 ctx.toast("Accepted. The reason is recorded on the finding.");
78 ctx.invalidate();
79 return undefined;
80 },
81 }),
82 );
83 return true;
84 }
85 return state.key(e, rows().length);
86 },
87 };
88 return screen;
89 }
src/web/pages/findings.ts added +79
@@ -0,0 +1,79 @@
1 /**
2 * Findings (#9593): `/:owner/:name/findings` lists what code scanning found
3 * in a repository — the rule, the file and line, never the matched text —
4 * open high findings first, each of which holds a release. A person with
5 * write access accepts one with a reason; who, why and when show on it.
6 * Reached from the flows view's "Code scanning" line. Server-rendered, no script.
7 */
8
9 import { acceptFinding, REASON_MAX, REASON_MIN } from "../../actions/scan_accept.ts";
10 import { lastScan, listFindings, type FindingRow } from "../../checks/scan_store.ts";
11 import { canWrite } from "../../core/repos.ts";
12 import type { Ctx } from "../../http/context.ts";
13 import { notePhase, takeSessionTime } from "../../core/route_timing.ts";
14 import { crumbsFor, loadRepo, notFoundPage, pageHead, repoBase, repoHeader, repoPalette } from "../chrome.ts";
15 import { field as formField, readForm } from "../form.ts";
16 import { html, type Html } from "../html.ts";
17 import { flashTo, page } from "../layout.ts";
18 import { badge, btn, card, field, listCard, row, time } from "../ui.ts";
19
20 /** When a pass last ran, and whether it read everything: a partial pass says how many files it left out. */
21 export function passWords(run: { at: string; skipped: number } | null): Html {
22 if (run === null) return html`never`;
23 const partial = run.skipped === 0 ? "" : ` (partial: ${run.skipped} ${run.skipped === 1 ? "file" : "files"} not read)`;
24 return html`${time(run.at)}${partial}`;
25 }
26
27 const where = (f: FindingRow): string => `${f.path}${f.line > 0 ? `:${f.line}` : ""}`;
28
29 function findingRow(f: FindingRow, base: string, writer: boolean): Html {
30 const tone = f.accepted !== null ? "default" : f.severity === "high" ? "danger" : "attn";
31 const meta = f.accepted !== null
32 ? html`${where(f)} · accepted by ${f.accepted.by} ${time(f.accepted.at)}: ${f.accepted.reason}`
33 : html`${where(f)} · ${f.kind} · found ${time(f.foundAt)} at ${f.oid.slice(0, 12)}`;
34 const line = row({ title: html`${f.rule}`, meta, trail: badge(f.accepted !== null ? "accepted" : f.severity, { tone }) });
35 if (f.accepted !== null || !writer) return line;
36 return html`${line}
37 <form method="post" action="${base}/findings/${String(f.id)}/accept" class="mt">
38 ${field({ label: "Why it may stand", name: "reason", id: `reason-${f.id}`, required: true, minlength: REASON_MIN, maxlength: REASON_MAX })}
39 ${btn("Accept", { variant: "secondary" })}
40 </form>`;
41 }
42
43 // GET /:owner/:name/findings
44 export function findingsPage(ctx: Ctx): Response {
45 const repo = loadRepo(ctx);
46 if (!repo) return notFoundPage(ctx);
47 notePhase(ctx.request, "session", takeSessionTime(ctx.request));
48 const rendering = performance.now();
49 const user = ctx.auth.user;
50 const writer = user !== undefined && user.kind === "user" && canWrite(ctx.app.db, repo, user);
51 const base = repoBase(repo);
52 const rows = listFindings(ctx.app.db, repo.id);
53 const push = lastScan(ctx.app.db, repo.id, "push");
54 const nightly = lastScan(ctx.app.db, repo.id, "nightly");
55 const body = html`${repoHeader(ctx, repo, "flows")}
56 ${pageHead("Findings", { sub: "Every push is scanned for secrets and advisories; each night adds static analysis. An open high finding holds a release until a person accepts it." })}
57 <p class="muted">Last push scan ${passWords(push)} · last nightly scan ${passWords(nightly)}</p>
58 ${card(rows.length === 0 ? html`<p class="muted">Nothing found.</p>` : listCard(rows.map((f) => findingRow(f, base, writer))), {
59 head: html`Findings`,
60 pad: rows.length === 0,
61 })}
62 <p class="mt">${btn("Back to flows", { variant: "secondary", href: `${base}/flows` })}</p>`;
63 const res = page({ ctx, title: `Findings · ${repo.name}`, crumbs: crumbsFor(repo), palette: repoPalette(ctx, repo), body });
64 notePhase(ctx.request, "render", performance.now() - rendering);
65 return res;
66 }
67
68 // POST /:owner/:name/findings/:id/accept
69 export async function findingAccept(ctx: Ctx): Promise<Response> {
70 const repo = loadRepo(ctx);
71 if (!repo) return notFoundPage(ctx);
72 const form = await readForm(ctx, "rw");
73 if (form instanceof Response) return form;
74 const back = `${repoBase(repo)}/findings`;
75 const id = /^\d{1,15}$/.test(ctx.params.id ?? "") ? Number(ctx.params.id) : 0;
76 const done = acceptFinding(ctx.app, repo, ctx.auth.user, id, formField(form, "reason"));
77 if (!done.ok) return flashTo(ctx, back, { tone: "danger", message: done.why });
78 return flashTo(ctx, back, { message: "Accepted. The reason is recorded on the finding." });
79 }
test/code-scanning.test.ts added +245
@@ -0,0 +1,245 @@
1 /**
2 * Code scanning (#9593): a fake secret pushed over git is found by the push
3 * pass, recorded without its text, shown on the findings page and the flows
4 * line, and holds a release until a person with write access accepts it
5 * with a reason. The fake secrets are built at run time, so this file is
6 * not itself a finding.
7 */
8 import { afterAll, beforeAll, describe, expect, spyOn, test } from "bun:test";
9 import { mkdtemp, rm, writeFile } from "node:fs/promises";
10 import { tmpdir } from "node:os";
11 import { join } from "node:path";
12 import { acceptFinding } from "../src/actions/scan_accept.ts";
13 import { commitToBranch, createRepo } from "../src/actions/repos.ts";
14 import { createAgent } from "../src/actions/crew.ts";
15 import { createApp, type App } from "../src/app.ts";
16 import { scanNightly, scanPush, scansSettled } from "../src/checks/scan.ts";
17 import { advisoryFindings, secretFindings, staticFindings, versionBelow } from "../src/checks/scan_rules.ts";
18 import { lastScan, listFindings, nightlyDue, openHigh } from "../src/checks/scan_store.ts";
19 import type { Repo } from "../src/core/repos.ts";
20 import { createToken } from "../src/core/tokens.ts";
21 import { createUser, type User } from "../src/core/users.ts";
22 import { scanLine } from "../src/flows/view.ts";
23 import { serve, type Servers } from "../src/http/server.ts";
24 import { scanHold } from "../src/release/scan_hold.ts";
25 import { gitClientEnv } from "./helpers/git-client.ts";
26 import { quietLogs } from "./helpers/globals.ts";
27
28 quietLogs();
29
30 const AWS = "AKIA" + "QX7Z".repeat(4);
31 const PEM = "-----BEGIN " + "RSA PRIVATE KEY-----";
32
33 describe("rules", () => {
34 test("a secret is found and its text is not in the finding", () => {
35 const found = secretFindings("config/env.ts", `const a = 1;\nconst key = "${AWS}";\n${PEM}\n`);
36 expect(found.map((f) => [f.rule, f.line, f.severity])).toEqual([
37 ["AWS access key", 2, "high"],
38 ["Private key", 3, "high"],
39 ]);
40 expect(JSON.stringify(found)).not.toContain(AWS);
41 expect(JSON.stringify(found)).not.toContain(AWS.slice(4));
42 });
43 test("a fake credential in a test is shown as medium and holds nothing", () => {
44 expect(secretFindings("test/a.test.ts", `const t = "${AWS}";`).map((f) => f.severity)).toEqual(["medium"]);
45 expect(secretFindings("src/fixtures/x.json", `"${AWS}"`).map((f) => f.severity)).toEqual(["medium"]);
46 });
47 test("binary files and plain text find nothing", () => {
48 expect(secretFindings("a.bin", `\0${AWS}`)).toEqual([]);
49 expect(secretFindings("a.md", "AKIA is a prefix")).toEqual([]);
50 });
51 test("an advisory names a pin below its fix, not one at or above it", () => {
52 const pkg = JSON.stringify({ dependencies: { lodash: "^4.17.20", ws: "8.17.1" }, devDependencies: { minimist: "~1.2.5" } });
53 expect(advisoryFindings("package.json", pkg).map((f) => f.rule.split(" ")[0])).toEqual(["lodash", "minimist"]);
54 expect(advisoryFindings("README.md", pkg)).toEqual([]);
55 expect(advisoryFindings("package.json", "{not json")).toEqual([]);
56 expect(versionBelow("1.10.0", "1.9.0")).toBe(false);
57 expect(versionBelow("1.2", "1.2.1")).toBe(true);
58 });
59 test("static analysis reads source, leaves tests and comments out, and is medium", () => {
60 const src = 'const x = ev' + 'al(input);\n// ev' + 'al(no)\nel.inner' + 'HTML = v;\n';
61 const found = staticFindings("src/a.ts", src);
62 expect(found.map((f) => [f.rule, f.line, f.severity])).toEqual([
63 ["eval of a value", 1, "medium"],
64 ["HTML written from a value", 3, "medium"],
65 ]);
66 expect(staticFindings("test/a.test.ts", src)).toEqual([]);
67 expect(staticFindings("README.md", src)).toEqual([]);
68 });
69 });
70
71 let dataDir: string;
72 let work: string;
73 let app: App;
74 let servers: Servers;
75 let alice: User;
76 let reader: User;
77 let repo: Repo;
78 const token: Record<string, string> = {};
79
80 const ok = <T>(r: { ok: true; value: T } | { ok: false; error: unknown }): T => {
81 if (!r.ok) throw new Error(JSON.stringify(r.error));
82 return r.value;
83 };
84
85 /** The forge's own copy of the repository's default branch tip: what a release would cut. */
86 async function tipOf(branch = "main"): Promise<string> {
87 const dir = join(dataDir, "repos", "alice", "square.git");
88 return (await git(["rev-parse", branch], dir)).out.trim();
89 }
90
91 async function git(args: string[], cwd: string): Promise<{ code: number; out: string }> {
92 const proc = Bun.spawn(["git", ...args], { cwd, env: gitClientEnv(), stdout: "pipe", stderr: "pipe" });
93 const [out, errText] = await Promise.all([new Response(proc.stdout).text(), new Response(proc.stderr).text()]);
94 return { code: await proc.exited, out: out + errText };
95 }
96
97 beforeAll(async () => {
98 dataDir = await mkdtemp(join(tmpdir(), "forge-scan-"));
99 work = await mkdtemp(join(tmpdir(), "forge-scan-work-"));
100 app = createApp({ dataDir, port: 0 });
101 servers = await serve(app);
102 alice = ok(await createUser(app.db, { origin: "seed", trusted: true, username: "alice", email: "[email protected]", password: "a-good-passphrase", adminIfFirst: true }));
103 reader = ok(await createUser(app.db, { origin: "seed", trusted: true, username: "rita", email: "[email protected]", password: "a-good-passphrase" }));
104 repo = ok(await createRepo(app, alice, { name: "square", visibility: "public" }));
105 ok(await commitToBranch(app, alice, repo, { message: "Start", files: [{ path: "README.md", content: "# square\n" }] }));
106 token.alice = createToken(app.db, alice.id, "t", "rw").token;
107 }, 30_000);
108
109 afterAll(async () => {
110 servers.stop();
111 app.close();
112 await rm(dataDir, { recursive: true, force: true });
113 await rm(work, { recursive: true, force: true });
114 });
115
116 describe("a push", () => {
117 test("a pushed fake secret is found, without its text", async () => {
118 const url = `${servers.url.replace("http://", `http://alice:${token.alice}@`)}/alice/square.git`;
119 const clone = join(work, "square");
120 expect((await git(["clone", "-q", url, clone], work)).code).toBe(0);
121 await writeFile(join(clone, "deploy.ts"), `export const key = "${AWS}";\n`);
122 await writeFile(join(clone, "package.json"), JSON.stringify({ dependencies: { lodash: "4.17.15" } }));
123 expect((await git(["add", "."], clone)).code).toBe(0);
124 expect((await git(["-c", "user.name=a", "-c", "user.email=a@x", "commit", "-qm", "oops"], clone)).code).toBe(0);
125 const pushed = await git(["push", "-q", "origin", "HEAD:main"], clone);
126 expect(pushed.code).toBe(0);
127 // The scan runs off the hook's answer: wait on the scan itself, not on a clock.
128 await scansSettled(app, repo.id);
129 const found = listFindings(app.db, repo.id);
130 expect(found.map((f) => [f.kind, f.path, f.line]).sort()).toEqual([
131 ["advisory", "package.json", 0],
132 ["secret", "deploy.ts", 1],
133 ]);
134 const page = await (await fetch(`${servers.url}/alice/square/findings`)).text();
135 expect(page).toContain("AWS access key");
136 expect(page).toContain("deploy.ts:1");
137 expect(page).not.toContain(AWS.slice(4));
138 const flows = await (await fetch(`${servers.url}/alice/square/flows`)).text();
139 expect(flows).toContain("Code scanning");
140 expect(flows).toContain("2 high findings hold releases");
141 }, 30_000);
142
143 test("a high finding holds a release; acceptance with a reason lifts it", async () => {
144 const hold = await scanHold(app, repo, await tipOf());
145 expect(hold.held).toBe(true);
146 if (hold.held) expect(hold.line).not.toContain(AWS.slice(4));
147 const [secret, advisory] = openHigh(app.db, repo.id).sort((a, b) => (a.kind < b.kind ? 1 : -1));
148 const bot = ok(await createAgent(app, alice, { username: "bot" }));
149 expect(acceptFinding(app, repo, bot.user, secret!.id, "the crew says so")).toMatchObject({ ok: false, status: 403 });
150 expect(acceptFinding(app, repo, reader, secret!.id, "not mine to say")).toMatchObject({ ok: false, status: 403 });
151 expect(acceptFinding(app, repo, alice, secret!.id, " ")).toMatchObject({ ok: false, status: 400 });
152 expect(acceptFinding(app, repo, alice, 999_999, "no such")).toMatchObject({ ok: false, status: 404 });
153 expect(acceptFinding(app, repo, alice, secret!.id, "Rotated: the key is revoked")).toEqual({ ok: true });
154 expect(acceptFinding(app, repo, alice, secret!.id, "again")).toMatchObject({ ok: false, status: 409 });
155 expect((await scanHold(app, repo, await tipOf())).held).toBe(true);
156 // Through the page's form, as a person does it.
157 const res = await fetch(`${servers.url}/alice/square/findings/${advisory!.id}/accept`, {
158 method: "POST",
159 headers: { authorization: `Bearer ${token.alice}`, "content-type": "application/x-www-form-urlencoded" },
160 body: new URLSearchParams({ reason: "Dev only, upgrade tracked" }),
161 redirect: "manual",
162 });
163 expect(res.status).toBe(303);
164 expect(await scanHold(app, repo, await tipOf())).toEqual({ held: false });
165 const accepted = listFindings(app.db, repo.id).find((f) => f.id === secret!.id)!;
166 expect(accepted.accepted).toMatchObject({ by: "alice", reason: "Rotated: the key is revoked" });
167 expect(scanLine(app.db, repo).state).toBe("no open findings");
168 });
169
170 test("the nightly pass adds static analysis over the default branch and is then not due", async () => {
171 ok(await commitToBranch(app, alice, repo, { message: "html", files: [{ path: "src/view.js", content: "el.inner" + "HTML = name;\n" }] }));
172 expect(nightlyDue(app.db, new Date().toISOString(), 10)).toContain(repo.id);
173 await scanNightly(app, repo);
174 expect(listFindings(app.db, repo.id).some((f) => f.kind === "static" && f.path === "src/view.js")).toBe(true);
175 expect(nightlyDue(app.db, new Date(Date.now() - 3_600_000).toISOString(), 10)).not.toContain(repo.id);
176 expect((await scanHold(app, repo, await tipOf())).held).toBe(false);
177 });
178
179 test("a push of many new branches is scanned against main in a bounded number of git processes", async () => {
180 const clone = join(work, "square");
181 expect((await git(["fetch", "-q", "origin", "main"], clone)).code).toBe(0);
182 let stream = "";
183 for (let i = 0; i < 20; i++) {
184 const leak = i === 0 ? `M 100644 inline leak.ts\ndata ${`const k = "${AWS}";\n`.length}\nconst k = "${AWS}";\n` : `M 100644 inline n${i}.txt\ndata 2\nn\n`;
185 stream += `commit refs/heads/feat/b${i}\ncommitter A <a@x> ${1700000000 + i} +0000\ndata 2\nb\nfrom refs/remotes/origin/main\n${leak}\n`;
186 }
187 const fi = Bun.spawn(["git", "fast-import", "--quiet"], { cwd: clone, env: gitClientEnv(), stdin: new TextEncoder().encode(stream), stderr: "pipe" });
188 expect(await fi.exited).toBe(0);
189 expect((await git(["push", "-q", "origin", "refs/heads/feat/*:refs/heads/feat/*"], clone)).code).toBe(0);
190 const tip = (await git(["rev-parse", "feat/b0"], clone)).out.trim();
191 const leakAt = () => listFindings(app.db, repo.id).filter((f) => f.oid === tip);
192 await scansSettled(app, repo.id);
193 // Compared with main: the branch's own file, not main's files again.
194 expect(leakAt().map((f) => f.path)).toEqual(["leak.ts"]);
195 const updates = [];
196 for (let i = 0; i < 20; i++) {
197 updates.push({ ref: `refs/heads/feat/b${i}`, oldOid: "0".repeat(40), newOid: (await git(["rev-parse", `feat/b${i}`], clone)).out.trim() });
198 }
199 // An open high finding on a branch that never reached main holds no release of main.
200 expect(openHigh(app.db, repo.id).map((f) => f.path)).toEqual(["leak.ts"]);
201 expect(await scanHold(app, repo, await tipOf())).toEqual({ held: false });
202 expect((await scanHold(app, repo, tip)).held).toBe(true);
203 const spy = spyOn(Bun, "spawn");
204 try {
205 expect(await scanPush(app, repo, updates)).toBe(0);
206 expect(spy.mock.calls.length).toBeLessThanOrEqual(3);
207 } finally {
208 spy.mockRestore();
209 }
210 }, 30_000);
211
212 test("a secret removed from main stops holding its release without an acceptance", async () => {
213 const before = await tipOf();
214 ok(await commitToBranch(app, alice, repo, { message: "leak", files: [{ path: "config/prod.ts", content: `export const k = "${PEM}";\n` }] }));
215 const leaked = await tipOf();
216 await scanPush(app, repo, [{ ref: "refs/heads/main", oldOid: before, newOid: leaked }]);
217 expect((await scanHold(app, repo, leaked)).held).toBe(true);
218 ok(await commitToBranch(app, alice, repo, { message: "rotate", files: [{ path: "config/prod.ts", content: "export const k = process.env.K;\n" }] }));
219 const clean = await tipOf();
220 expect(await scanHold(app, repo, clean)).toEqual({ held: false });
221 // The old tree still holds it: the hold reads what is released, not a flag.
222 expect((await scanHold(app, repo, leaked)).held).toBe(true);
223 });
224
225 test("a file named with a newline is skipped and counted, and later files keep their own text", async () => {
226 const clone = join(work, "square");
227 expect((await git(["checkout", "-q", "-b", "nl", "origin/main"], clone)).code).toBe(0);
228 expect((await git(["pull", "-q", "origin", "main"], clone)).code).toBe(0);
229 await writeFile(join(clone, "a\nb.txt"), "plain\n");
230 await writeFile(join(clone, "b.txt"), "nothing here\n");
231 await writeFile(join(clone, "c.ts"), `\n\nexport const key = "${AWS}";\n`);
232 expect((await git(["add", "."], clone)).code).toBe(0);
233 expect((await git(["-c", "user.name=a", "-c", "user.email=a@x", "commit", "-qm", "nl"], clone)).code).toBe(0);
234 const base = (await git(["rev-parse", "HEAD~1"], clone)).out.trim();
235 const head = (await git(["rev-parse", "HEAD"], clone)).out.trim();
236 expect((await git(["push", "-q", "origin", "nl"], clone)).code).toBe(0);
237 await scansSettled(app, repo.id);
238 await scanPush(app, repo, [{ ref: "refs/heads/nl", oldOid: base, newOid: head }]);
239 const at = listFindings(app.db, repo.id).filter((f) => f.oid === head);
240 expect(at.map((f) => [f.path, f.line])).toEqual([["c.ts", 3]]);
241 expect(lastScan(app.db, repo.id, "push")?.skipped).toBe(1);
242 const page = await (await fetch(`${servers.url}/alice/square/findings`)).text();
243 expect(page).toContain("partial: 1 file not read");
244 });
245 });
test/findings-tui.test.ts added +131
@@ -0,0 +1,131 @@
1 /**
2 * Findings in the terminal (#9593): the page's list and its acceptance, from
3 * the same reads and the same action. A guest sees nothing of a private
4 * repository, a reader is refused the accept, and a writer's acceptance
5 * shows on the web page.
6 */
7 import { afterAll, beforeAll, describe, expect, test } from "bun:test";
8 import { mkdtemp, rm } from "node:fs/promises";
9 import { tmpdir } from "node:os";
10 import { join } from "node:path";
11 import { createRepo } from "../src/actions/repos.ts";
12 import { createApp, type App as Forge } from "../src/app.ts";
13 import { secretFindings } from "../src/checks/scan_rules.ts";
14 import { listFindings, recordFindings } from "../src/checks/scan_store.ts";
15 import type { Repo } from "../src/core/repos.ts";
16 import { createUser, type User } from "../src/core/users.ts";
17 import { serve, type Servers } from "../src/http/server.ts";
18 import { createForgeApp } from "../src/tui/forge.ts";
19 import type { TuiIdentity } from "../src/tui/session.ts";
20 import { headlessTerminal } from "../src/tui/term/terminal.ts";
21 import { quietLogs } from "./helpers/globals.ts";
22 import { settleApp } from "./helpers/tui.ts";
23
24 quietLogs();
25
26 const AWS = "AKIA" + "QX7Z".repeat(4);
27
28 describe("findings in the terminal", () => {
29 let dir: string;
30 let forge: Forge;
31 let servers: Servers;
32 let ada: User;
33 let eve: User;
34 let arm: Repo;
35 let vault: Repo;
36
37 function boot(user: User | undefined) {
38 const terminal = headlessTerminal(120, 32);
39 const identity: TuiIdentity = { sshUsername: "t", remoteAddress: "127.0.0.1", user };
40 const { app } = createForgeApp(forge, identity, { terminal, exitHooks: false });
41 app.start();
42 const key = (k: string) => app.dispatch({ type: "key", key: k, ctrl: false, alt: false, shift: false });
43 const type = async (text: string) => {
44 for (const ch of text) key(ch);
45 await settleApp(app);
46 };
47 const open = async (name: string) => {
48 await settleApp(app);
49 app.dispatch({ type: "key", key: "k", ctrl: true, alt: false, shift: false });
50 await settleApp(app);
51 await type("Findings in a repository");
52 key("enter");
53 await settleApp(app);
54 await type(name);
55 key("enter");
56 await settleApp(app);
57 };
58 const press = async (k: string) => {
59 key(k);
60 await settleApp(app);
61 };
62 return { text: () => app.screen.toText(), open, type, press, stop: () => app.stop() };
63 }
64
65 beforeAll(async () => {
66 dir = await mkdtemp(join(tmpdir(), "forge-findings-tui-"));
67 forge = createApp({ dataDir: dir, port: 0 });
68 servers = await serve(forge);
69 const make = async (username: string, adminIfFirst: boolean): Promise<User> => {
70 const made = await createUser(forge.db, { origin: "seed", trusted: true, username, email: `${username}@forge.local`, password: "a-good-passphrase", adminIfFirst });
71 if (!made.ok) throw new Error(made.error);
72 return made.value;
73 };
74 ada = await make("ada", true);
75 eve = await make("eve", false);
76 const mk = async (name: string, visibility: "public" | "private"): Promise<Repo> => {
77 const made = await createRepo(forge, ada, { name, owner: "ada", visibility, description: "" });
78 if (!made.ok) throw new Error(JSON.stringify(made.error));
79 return made.value;
80 };
81 arm = await mk("arm", "public");
82 vault = await mk("vault", "private");
83 for (const r of [arm, vault]) recordFindings(forge.db, r.id, "a".repeat(40), secretFindings("deploy.ts", `const k = "${AWS}";\n`));
84 }, 60_000);
85 afterAll(async () => {
86 servers?.stop();
87 forge?.close();
88 await rm(dir, { recursive: true, force: true });
89 });
90
91 test("a guest sees nothing of a private repository's findings", async () => {
92 const guest = boot(undefined);
93 await guest.open("ada/vault");
94 expect(guest.text()).not.toContain("AWS access key");
95 expect(guest.text()).not.toContain("deploy.ts");
96 guest.stop();
97 // The door is open to a guest where the page is: a public repository's list.
98 const passer = boot(undefined);
99 await passer.open("ada/arm");
100 expect(passer.text()).toContain("AWS access key");
101 passer.stop();
102 });
103
104 test("a reader sees the list without the match, and is refused the accept", async () => {
105 const reader = boot(eve);
106 await reader.open("ada/arm");
107 expect(reader.text()).toContain("AWS access key");
108 expect(reader.text()).toContain("deploy.ts:1");
109 expect(reader.text()).not.toContain(AWS.slice(4));
110 await reader.press("a");
111 expect(reader.text()).toContain("Only a person with write access accepts a finding.");
112 expect(listFindings(forge.db, arm.id)[0]!.accepted).toBeNull();
113 reader.stop();
114 });
115
116 test("a writer accepts with a reason, a blank reason is refused, and the page shows it", async () => {
117 const writer = boot(ada);
118 await writer.open("ada/arm");
119 await writer.press("a");
120 await writer.press("enter");
121 expect(writer.text()).toContain("Say why it may stand");
122 await writer.type("Rotated in the vault");
123 await writer.press("enter");
124 expect(listFindings(forge.db, arm.id)[0]!.accepted).toMatchObject({ by: "ada", reason: "Rotated in the vault" });
125 expect(writer.text()).toContain("accepted by ada: Rotated in the vault");
126 writer.stop();
127 const page = await (await fetch(`${servers.url}/ada/arm/findings`)).text();
128 expect(page).toContain("accepted by ada");
129 expect(page).toContain("Rotated in the vault");
130 });
131 });
test/palette-roster.test.ts +4
@@ -60,6 +60,7 @@ test("a site-admin person is offered the whole roster", () => {
6060 ["next-up", "Next up in a repository", "owner/name"],
6161 ["week", "The week in a repository", "owner/name"],
6262 ["flows", "Flows of a repository", "owner/name"],
63 ["findings", "Findings in a repository", "owner/name"],
6364 ["machines", "Machines of a repository", "owner/name"],
6465 ["repo-usage", "Usage of a repository", "owner/name"],
6566 ["new-issue", "New issue", "owner/name"],
@@ -166,6 +167,7 @@ test("a site-admin agent is offered the roster without the person-only entries",
166167 ["next-up", "Next up in a repository", "owner/name"],
167168 ["week", "The week in a repository", "owner/name"],
168169 ["flows", "Flows of a repository", "owner/name"],
170 ["findings", "Findings in a repository", "owner/name"],
169171 ["machines", "Machines of a repository", "owner/name"],
170172 ["repo-usage", "Usage of a repository", "owner/name"],
171173 ["new-issue", "New issue", "owner/name"],
@@ -235,6 +237,7 @@ test("a member person is offered the signed-in roster without the admin entries"
235237 ["next-up", "Next up in a repository", "owner/name"],
236238 ["week", "The week in a repository", "owner/name"],
237239 ["flows", "Flows of a repository", "owner/name"],
240 ["findings", "Findings in a repository", "owner/name"],
238241 ["machines", "Machines of a repository", "owner/name"],
239242 ["repo-usage", "Usage of a repository", "owner/name"],
240243 ["new-issue", "New issue", "owner/name"],
@@ -297,6 +300,7 @@ test("signed out, the palette offers sign-in and the public entries", () => {
297300 ["next-up", "Next up in a repository", "owner/name"],
298301 ["week", "The week in a repository", "owner/name"],
299302 ["flows", "Flows of a repository", "owner/name"],
303 ["findings", "Findings in a repository", "owner/name"],
300304 ["machines", "Machines of a repository", "owner/name"],
301305 ["repo-usage", "Usage of a repository", "owner/name"],
302306 ["new-issue", "New issue", "owner/name"],
test/scan-hold-landing.test.ts added +93
@@ -0,0 +1,93 @@
1 /**
2 * A high finding holds a release where releases happen (#9593): `cut` in
3 * `checks/landing.ts` does not swap the landed tree in while one is open in
4 * it, and tells the pull; once a person accepts it, the same tree swaps in.
5 * The stand-in forge answers the health route as the commit it is told.
6 */
7
8 import { afterAll, expect, test } from "bun:test";
9 import { mkdtemp, rm } from "node:fs/promises";
10 import { tmpdir } from "node:os";
11 import { join } from "node:path";
12 import { acceptFinding } from "../src/actions/scan_accept.ts";
13 import { openPull } from "../src/actions/pulls.ts";
14 import { commitToBranch, createRepo } from "../src/actions/repos.ts";
15 import { createApp } from "../src/app.ts";
16 import { releaseOnLand } from "../src/checks/landing.ts";
17 import { scanNightly } from "../src/checks/scan.ts";
18 import { openHigh } from "../src/checks/scan_store.ts";
19 import { listComments } from "../src/core/issues.ts";
20 import { createUser } from "../src/core/users.ts";
21 import { endSwap, handover } from "../src/release/swap.ts";
22 import { quietLogs } from "./helpers/globals.ts";
23
24 quietLogs();
25
26 const AWS = "AKIA" + "QX7Z".repeat(4);
27 const OLD = "0".repeat(40);
28 const STAND_IN = `
29 if (process.argv.includes("--preflight")) { console.log("preflight ok: stand-in"); process.exit(0); }
30 Bun.serve({ port: Number(process.env.PORT), reusePort: true, fetch: () => Response.json({ ok: true, draining: false, commit: process.env.FORGE_COMMIT }) });
31 setTimeout(() => process.exit(0), 6000);
32 `;
33
34 const dirs: string[] = [];
35 afterAll(async () => {
36 for (const d of dirs) await rm(d, { recursive: true, force: true });
37 });
38
39 async function freePort(): Promise<number> {
40 const s = Bun.serve({ port: 0, fetch: () => new Response("") });
41 const port = s.port ?? 0;
42 s.stop(true);
43 return port;
44 }
45
46 test("an open high finding in the landed tree holds the swap and tells the pull; accepted, the tree swaps in", async () => {
47 handover.migrate = true;
48 const port = await freePort();
49 const old = Bun.serve({ port, reusePort: true, fetch: () => Response.json({ ok: true, draining: false, commit: OLD }) });
50 let drained = 0;
51 handover.drain = () => {
52 drained++;
53 old.stop();
54 };
55 const dataDir = await mkdtemp(join(tmpdir(), "forge-scan-land-"));
56 dirs.push(dataDir);
57 const app = createApp({ dataDir, port, forgeRepo: "alice/forge-code", selfSwap: true, swapBudgetMs: 5_000 });
58 try {
59 const u = await createUser(app.db, { origin: "seed", trusted: true, username: "alice", email: "[email protected]", password: "a-good-passphrase", adminIfFirst: true });
60 if (!u.ok) throw new Error("user");
61 const r = await createRepo(app, u.value, { name: "forge-code", visibility: "public" });
62 if (!r.ok) throw new Error("repo");
63 await commitToBranch(app, u.value, r.value, { message: "seed", files: [{ path: "a.txt", content: "a" }] });
64 await commitToBranch(app, u.value, r.value, { message: "b", branch: "change", files: [{ path: "b.txt", content: "b" }] });
65 const pull = await openPull(app, u.value, r.value, { title: "t", baseBranch: r.value.defaultBranch, headBranch: "change" });
66 if (!pull.ok) throw new Error("pull");
67 const made = await commitToBranch(app, u.value, r.value, {
68 message: "land",
69 files: [{ path: "src/main.ts", content: STAND_IN }, { path: "deploy.ts", content: `export const key = "${AWS}";\n` }],
70 });
71 if (!made.ok) throw new Error("commit");
72 const sha = made.value.commitOid;
73 await scanNightly(app, r.value);
74 const [secret] = openHigh(app.db, r.value.id);
75 expect(secret?.path).toBe("deploy.ts");
76
77 const held = await releaseOnLand(app, u.value, r.value, pull.value, sha);
78 expect(held?.ok).toBe(true);
79 expect(drained).toBe(0);
80 const told = listComments(app.db, pull.value.id).map((c) => c.body);
81 expect(told.some((b) => b.includes(`${sha.slice(0, 12)} is not released`) && b.includes("held by 1 open high finding"))).toBe(true);
82 expect(told.join("\n")).not.toContain(AWS.slice(4));
83
84 expect(acceptFinding(app, r.value, u.value, secret!.id, "Rotated: the key is revoked")).toEqual({ ok: true });
85 await releaseOnLand(app, u.value, r.value, pull.value, sha);
86 expect(drained).toBe(1);
87 } finally {
88 endSwap(false);
89 delete handover.drain;
90 old.stop(true);
91 app.close();
92 }
93 }, 120_000);

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs