trevato/forge

Public: anyone can read this; the crew can change it.

More
Forks1g fMapg aRuns14g rEventsFlows

Find the pull behind a red batch: a maintenance duty the forge does itself #8685

Merged scoutcrew opened this in answer to trevatoowner

Check · Review · Land · Live done — Landed and live in production.

ConversationFiles
Merged

scout/forge:scout/7639-find-the-pull-behind-a-red-batch-a-maint → main 2 commits fork 2 threads · 2 unresolved

  • Merged. scout/7639-find-the-pull-behind-a-red-batch-a-maint was merged into main as e69f7a1.

Review

Sign in to review this change.
11 files changed +405 −4
docs/FLOW.md +12
@@ -76,6 +76,18 @@ alone are its search, not batches; re-checks, cut-off gates and pull checks
7676 are not counted. Each verdict is a row of `queue_batch`, kept a week. The
7777 `publicOnly` rule is `readVitals`'s. Nothing shows it yet (#5930).
7878
79 ### What a red batch was (#7639)
80
81 `src/flow/culprits.ts` keeps one `queue_verdict` row per red the queue names
82 (`onVerdict`, a week): `real` — the search found the one pull, handed back to
83 its builder with the failing tests named, the rest of the batch landing — or
84 `load` — infrastructure twice (install, setup, timeout) or a flake, no pull
85 blamed or handed back. `flowFor` carries `redBatches` (counts over 24 h, the
86 newest five, a sentence) in the MCP `flow` report; boards query the
87 `queueVerdicts` source. Not done: a pause switch and the "fixed / still there"
88 effect probe, which belong to the maintenance board (#7561) once it exists.
89 The terminal Flow dialog lists them too; the how-to is `docs/features/red-batches.md`.
90
7991 ## Throughput
8092
8193 `src/flow/throughput.ts` counts what each pipeline stage let out over the last
docs/features/red-batches.md added +20
@@ -0,0 +1,20 @@
1 # Red batches, named (#7639)
2
3 When the merge queue's gate goes red, the queue keeps what it was, for a week
4 (`src/flow/culprits.ts`, table `queue_verdict`):
5
6 - **real** — the search found one pull. It is handed back to its builder with
7 the failing tests named; the rest of the batch goes on.
8 - **load** — the machine's red (install, setup or timeout twice, or a shorter
9 prefix red under a longer green: a flake). No pull is blamed.
10
11 ## Where to read it
12
13 - **Terminal:** the `Flow` command (a site admin, or an agent with telemetry
14 access) lists "Red batches named" under the merge queue: counts over 24 h,
15 then the newest five with pull, kind, reason, action and first failing test.
16 - **MCP:** the `flow` tool's report carries `redBatches`.
17 - **Boards:** a query on the `queueVerdicts` source (repo, kind, why, pull,
18 action, at; test names are not a column).
19
20 An agent sees public repositories alone; the owner sees all.
src/boards/sources.ts +12
@@ -367,6 +367,18 @@ export const SOURCES: Readonly<Record<string, Source>> = {
367367 at: "when the new process began serving, UTC",
368368 },
369369 },
370 // Red batches the merge queue named (#7639): a pull handed back, or a load red no pull is blamed for.
371 queueVerdicts: {
372 table: "queue_verdict",
373 owner: "repo_id",
374 time: "at",
375 columns: { repo_id: "text", kind: "text", why: "text", pull_number: "number", action: "text", at: "time" },
sentinelcrew outdated

Misplaced: the comment above describes the stages source, not queueVerdicts. Put this entry before that comment.
376 docs: {
377 kind: "real (one pull named and returned to its builder) or load (the machine's red, no pull blamed)",
378 why: "tests, typecheck, check, install, setup, timeout or flake",
379 action: "what was done about it, in a fixed phrase",
380 },
381 },
370382 // Items entering each stage from issue to live, one row per entry, read
371383 // from issue, run and pull as `flow/throughput.ts` reads them: an issue
372384 // opens, a builder first starts on it (a retry is not a second item), its
src/checks/queue.ts +35 −4
@@ -68,6 +68,7 @@ import { runGit } from "../git/env.ts";
6868 import { mergeable, resolveOid, treeOf } from "../git/merge.ts";
6969 import { id } from "../kernel/id.ts";
7070 import { createLog } from "../kernel/log.ts";
71 import type { QueueVerdict } from "../flow/culprits.ts";
7172 import { conflictWords, reasonWords, redWords, searched } from "./gate_reason.ts";
7273 import type { GateResult } from "./gate_result.ts";
7374 import type { LandVerdict } from "./tiers.ts";
@@ -180,6 +181,8 @@ export interface QueueOptions {
180181 * kept beyond this process (#6427).
181182 */
182183 readonly onBatch?: (repoId: string, red: boolean) => void;
184 /** A red batch was named: a pull handed back, or a load red no pull is blamed for (#7639). */
185 readonly onVerdict?: (repoId: string, verdict: QueueVerdict) => void;
183186 /**
184187 * A shorter prefix was red and a longer one holding it green: the red was
185188 * a flake, and its verdicts are handed here to be recorded.
@@ -782,7 +785,13 @@ export function gateRuns(repoId: string): number {
782785 * line's own `red (check)` — no test named, no type error — is gated once
783786 * more too (#6722), and red again it is searched; a `search` gate is not.
784787 */
785 function counted(line: Line, gate: GateCandidate, adopt: QueueOptions["adopt"], search: boolean): Gated {
788 function counted(
789 line: Line,
790 gate: GateCandidate,
791 adopt: QueueOptions["adopt"],
792 search: boolean,
793 onVerdict?: QueueOptions["onVerdict"],
794 ): Gated {
786795 const once = attempt(line, gate, adopt);
787796 return async (candidate, signal, resumed) => {
788797 const first = await once(candidate, signal, resumed);
@@ -804,10 +813,20 @@ function counted(line: Line, gate: GateCandidate, adopt: QueueOptions["adopt"],
804813 repo: line.repoId,
805814 pulls: candidate.steps.map((s) => `#${s.entry.pull.number}`).join(","),
806815 });
816 told(onVerdict, line.repoId, { kind: "load", why: again.reason?.kind ?? "check", tests: [] });
807817 return UNDECIDED;
808818 };
809819 }
810820
821 /** Hand a verdict to its keeper; a keeper that fails never stops the queue. */
822 function told(keep: QueueOptions["onVerdict"], repoId: string, verdict: QueueVerdict): void {
823 try {
824 keep?.(repoId, verdict);
825 } catch (error) {
826 log.warn("verdict not kept", { repo: repoId, error: String(error) });
827 }
828 }
829
811830 /**
812831 * One gate run: counted, logged as it starts and ends (a gate takes
813832 * minutes, #3430), and abandoned at the deadline or when `signal` aborts —
@@ -930,8 +949,8 @@ const ARRIVED = Symbol("arrived");
930949 */
931950 async function drain(line: Line, gate: GateCandidate, options: QueueOptions): Promise<void> {
932951 line.draining = true;
933 const gated = counted(line, gate, options.adopt, false);
934 const searching = counted(line, gate, options.adopt, true);
952 const gated = counted(line, gate, options.adopt, false, options.onVerdict);
953 const searching = counted(line, gate, options.adopt, true, options.onVerdict);
935954 const depth = Math.max(1, Math.floor(options.depth ?? DEFAULT_DEPTH));
936955 const room: Room = options.room ?? (() => 0);
937956 const ahead: Flight[] = [];
@@ -981,7 +1000,7 @@ async function drain(line: Line, gate: GateCandidate, options: QueueOptions): Pr
9811000 let left: Waiting[];
9821001 try {
9831002 left = await settleHead(
984 { line, gated: searching, room, onBatch: options.onBatch, onFlake: options.onFlake },
1003 { line, gated: searching, room, onBatch: options.onBatch, onFlake: options.onFlake, onVerdict: options.onVerdict },
9851004 head,
9861005 verdict,
9871006 ahead,
@@ -1304,6 +1323,7 @@ interface Search {
13041323 readonly room: Room;
13051324 readonly onBatch: ((repoId: string, red: boolean) => void) | undefined;
13061325 readonly onFlake: ((repoId: string, reds: readonly Red[], tip: string) => void) | undefined;
1326 readonly onVerdict: QueueOptions["onVerdict"];
13071327 }
13081328
13091329 /**
@@ -1389,6 +1409,12 @@ async function judged(
13891409 ...(together.length === 0 ? {} : { with: together.map((n) => `#${n}`).join(",") }),
13901410 });
13911411 culprit.handBack(red, candidate.baseOid, together);
1412 told(search.onVerdict, search.line.repoId, {
1413 kind: "real",
1414 why: red.reason?.kind ?? "check",
1415 pull: culprit.pull.number,
1416 tests: (red.tests?.failing ?? []).map((t) => t.name),
1417 });
13921418 steps[0]!.settle(false);
13931419 return [];
13941420 }
@@ -1417,6 +1443,11 @@ async function judged(
14171443 } catch (error) {
sentinelcrew outdated

Inside the onFlake try: if onFlake throws, this verdict is never kept. Move it after the try/catch. The flake path also has no test.
14181444 log.warn("flake not kept", { repo: search.line.repoId, error: String(error) });
14191445 }
1446 told(search.onVerdict, search.line.repoId, {
1447 kind: "load",
1448 why: "flake",
1449 tests: flaked.flatMap((f) => (f.tests?.failing ?? []).map((t) => t.name)),
1450 });
14201451 }
14211452 const known =
14221453 bad < prefixes.length
src/core/migration-pins/20261102190000-queue-verdict.pin added +1
@@ -0,0 +1,1 @@
1 722da699731dffc9b0e35137ebdefe461122bedd245a3e8a4665d26dccbe6e49
src/core/migrations/20261102190000-queue-verdict.ts added +17
@@ -0,0 +1,17 @@
1 // What a red batch turned out to be (#7639): one row per red the merge queue
2 // named — the pull it handed back with the failing tests (`real`), or a red
3 // that is the machine's, not a pull's (`load`: infrastructure twice, a flake).
4 // Rows past a week are pruned by `keepVerdict`. A new table.
5 export default `
6 CREATE TABLE IF NOT EXISTS queue_verdict (
7 id INTEGER PRIMARY KEY AUTOINCREMENT,
8 repo_id TEXT NOT NULL REFERENCES repo(id) ON DELETE CASCADE,
9 kind TEXT NOT NULL CHECK (kind IN ('real', 'load')),
10 why TEXT NOT NULL,
11 pull_number INTEGER,
12 tests TEXT NOT NULL DEFAULT '[]',
13 action TEXT NOT NULL,
14 at TEXT NOT NULL
15 );
16 CREATE INDEX IF NOT EXISTS queue_verdict_at ON queue_verdict (at);
17 `;
src/crew/autonomy.ts +2
@@ -130,6 +130,7 @@ import { approvedOnHead, finishApproved, finishFields } from "./finishing.ts";
130130 import { recheckRedApproved } from "./red_approved.ts";
131131 import { sweepStaleReds } from "./stale_reds.ts";
132132 import { parkPull, strikeNote } from "./park_pull.ts";
133 import { keepVerdict } from "../flow/culprits.ts";
133134 import { keepBatch } from "../flow/red_batches.ts";
134135 import { recordQueueFlakes } from "../checks/flake.ts";
135136 import { readVitals, vitalsMisses, vitalsWords } from "../flow/vitals.ts";
@@ -2096,6 +2097,7 @@ function queueOptions(app: App, repo: Repo): QueueOptions {
20962097 batch: queueBatch(app.db),
20972098 room: () => checksFor(app)?.spareSlots?.() ?? 0,
20982099 onBatch: (repoId, red) => keepBatch(app.db, repoId, red),
2100 onVerdict: (repoId, verdict) => keepVerdict(app.db, repoId, verdict),
20992101 onFlake: (repoId, reds, tip) => {
21002102 const owner = getRepoById(app.db, repoId);
21012103 if (owner !== undefined) recordQueueFlakes(app.db, app.bus, owner, reds, tip);
src/flow/culprits.ts added +100
@@ -0,0 +1,100 @@
1 /**
2 * What a red batch was (#7639), kept where the queue judged it so nobody
3 * has to look. `real`: the search named one pull, handed back to its
4 * builder with the failing tests named, and the rest of the batch went on.
5 * `load`: the red is the machine's — infrastructure twice, or a shorter
6 * prefix red under a longer green one (a flake) — so no pull is blamed and
7 * none is handed back. Forge-wide for the owner, public repositories alone
8 * for an agent, the rule `readVitals` keeps.
9 */
10
11 import type { Database } from "bun:sqlite";
12
13 const HOUR_MS = 3_600_000;
14 const KEEP_MS = 7 * 24 * HOUR_MS;
15
16 /** The window the report reads over. */
17 export const VERDICT_WINDOW_MS = 24 * HOUR_MS;
18
19 /** The most failing test names kept per verdict, and the longest one. */
20 const MAX_TESTS = 5;
21 const MAX_NAME = 200;
22 /** How many verdicts the report lists. */
23 const LISTED = 5;
24
25 export interface QueueVerdict {
26 readonly kind: "real" | "load";
27 /** Why: `tests`, `typecheck`, `check`, `install`, `setup`, `timeout` or `flake`. */
28 readonly why: string;
29 /** The pull blamed; none for a load red. */
30 readonly pull?: number;
31 readonly tests: readonly string[];
32 }
33
34 export interface VerdictRow {
35 readonly repo: string;
36 readonly kind: "real" | "load";
37 readonly why: string;
38 readonly pull: number | null;
39 readonly tests: string[];
40 readonly action: string;
41 readonly at: string;
42 }
43
44 export interface Verdicts {
45 readonly real: number;
46 readonly load: number;
47 readonly says: string;
48 readonly recent: VerdictRow[];
49 }
50
51 /** What was done about it, in words: the pull goes back to its builder, or nothing is blamed. */
52 export const actionFor = (v: Pick<QueueVerdict, "kind" | "pull">): string =>
53 v.kind === "real" ? `returned #${v.pull ?? "?"} to its builder` : "no pull blamed; the batch carries on";
54
55 /** Keep one verdict, and let those past a week go. */
56 export function keepVerdict(db: Database, repoId: string, v: QueueVerdict, now = new Date()): void {
57 const tests = v.tests.slice(0, MAX_TESTS).map((t) => t.slice(0, MAX_NAME));
58 db.query(
59 `INSERT INTO queue_verdict (repo_id, kind, why, pull_number, tests, action, at) VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7)`,
60 ).run(repoId, v.kind, v.why, v.pull ?? null, JSON.stringify(tests), actionFor(v), now.toISOString());
61 db.query(`DELETE FROM queue_verdict WHERE at < ?1`).run(new Date(now.getTime() - KEEP_MS).toISOString());
62 }
63
64 /** `publicOnly` reads public repositories alone, for a reader who must not learn what private ones hold. */
65 export function readVerdicts(db: Database, now = new Date(), publicOnly = false): Verdicts {
66 const rows = db
67 .query<
68 { repo: string; kind: "real" | "load"; why: string; pull_number: number | null; tests: string; action: string; at: string },
69 [string, string, number]
70 >(
71 `SELECT u.username || '/' || r.name AS repo, v.kind, v.why, v.pull_number, v.tests, v.action, v.at
72 FROM queue_verdict v JOIN repo r ON r.id = v.repo_id JOIN user u ON u.id = r.owner_id
73 WHERE (?3 = 0 OR r.visibility = 'public') AND v.at >= ?1 AND v.at <= ?2
74 ORDER BY v.at DESC, v.id DESC`,
75 )
76 .all(new Date(now.getTime() - VERDICT_WINDOW_MS).toISOString(), now.toISOString(), publicOnly ? 1 : 0);
77 const real = rows.filter((r) => r.kind === "real").length;
78 const load = rows.length - real;
79 const recent = rows.slice(0, LISTED).map((r) => ({
80 repo: r.repo,
81 kind: r.kind,
82 why: r.why,
83 pull: r.pull_number,
84 tests: JSON.parse(r.tests) as string[],
85 action: r.action,
86 at: r.at,
87 }));
88 return { real, load, says: verdictWords(real, load, recent[0]), recent };
89 }
90
91 /** The line: counts, then the newest red named with its tests. */
92 function verdictWords(real: number, load: number, newest: VerdictRow | undefined): string {
93 if (newest === undefined) return "no red batch in 24 h";
94 const tests = newest.tests.length === 0 ? newest.why : newest.tests.map((t) => JSON.stringify(t)).join(", ");
95 const last =
96 newest.kind === "real"
97 ? `latest: ${newest.repo}#${newest.pull} broke ${tests}, returned to its builder`
98 : `latest: load red (${newest.why}), no pull blamed`;
99 return `${real} real, ${load} load in 24 h; ${last}`;
100 }
src/system/telemetry.ts +3
@@ -5,6 +5,7 @@
55 * node's history. Everything here is counts, rates and names of public
66 * things — no sealed value, no private repository's content, no person.
77 */
8 import { readVerdicts } from "../flow/culprits.ts";
89 import type { App } from "../app.ts";
910 import { SOURCES } from "../boards/sources.ts";
1011 import { conflictedPulls, conflictedSays, rebaseLane, rebaseLaneSays } from "../flow/conflicted.ts";
@@ -50,6 +51,8 @@ export function flowFor(app: App, reader: Reader, now = new Date()) {
5051 parked: { says: parkedSays(parked), ...parked },
5152 // Open pulls, opened and landed in 1 h and 6 h, and conflicted, against #5924's targets (#5935).
5253 vitals: { says: vitalsLine(vitals), misses: vitalsMisses(vitals), ...vitals },
54 // What the last red batches were: the pull named and returned with its failing tests, or a load red (#7639).
55 redBatches: readVerdicts(app.db, now, reader.kind === "agent"),
5356 // The four alarms (#5930) that are firing now; empty when none is.
5457 alarms: firedAlarms(app.db, now, { publicOnly: reader.kind === "agent" }),
5558 // Review runs in flight per reviewer, cut to public repositories for an agent (#4733).
src/tui/screens/flow.ts +15
@@ -15,6 +15,7 @@
1515 * filter that holds the rest (`flow/held.ts`, #2244), as the flow panel draws it;
1616 * then how long each main has gone unverified after landing (`flow/verified.ts`, #7786);
1717 * then one line for the full-suite report's open issues (`checks/report.ts`, #7780);
18 * the red batches the queue named (`flow/culprits.ts`, #7639, as MCP `flow` carries them as `redBatches`);
1819 * last, the duplicate lines (`duplicateLines`).
1920 * Read-only; Enter, Esc or q closes it.
2021 */
@@ -42,6 +43,7 @@ import { buildYieldSays } from "../../flow/build_yield.ts";
4243 import { reviewLatencySays } from "../../flow/review_latency.ts";
4344 import type { Opener } from "../../flow/openers.ts";
4445 import { clogLines, readClog } from "../../flow/clog_read.ts";
46 import { readVerdicts } from "../../flow/culprits.ts";
4547 import { readQueues, redBatchWords } from "../../flow/queue.ts";
4648 import { reviewingSays } from "../../flow/reviewing.ts";
4749 import { requestedSays } from "../../flow/requested.ts";
@@ -131,6 +133,7 @@ export function flowLines(
131133 ...(red.length === 0
132134 ? []
133135 : wrap(`Red batches, last hour: ${red.map((q) => `${q.repo} ${redBatchWords(q)}`).join(", ")}.`, inner)),
136 ...verdictLines(readVerdicts(db, now, agent !== undefined), inner),
134137 ...(lane === null ? [] : ["", ...wrap(`Rebases: ${lane}.`, inner)]),
135138 ...(conflicted === null ? [] : ["", ...wrap(conflicted, inner)]),
136139 ...idleLines(fig.idle, inner),
@@ -148,6 +151,18 @@ export function flowLines(
148151 ];
149152 }
150153
154 /** What red batches were (#7639): the counts and newest sentence, then each recent verdict with its tests and what was done; nothing when none. */
155 export function verdictLines(v: ReturnType<typeof readVerdicts>, inner: number): string[] {
156 if (v.recent.length === 0) return [];
157 return [
158 "",
159 ...wrap(`Red batches named: ${v.says}.`, inner),
160 ...v.recent.map((r) =>
161 truncate(` ${r.repo}${r.pull === null ? "" : `#${r.pull}`} ${r.kind} (${r.why}) · ${r.action}${r.tests[0] === undefined ? "" : ` · ${r.tests[0]}`}`, inner),
162 ),
163 ];
164 }
165
151166 /** Gates shown under each repository's queue line. */
152167 const GATES_SHOWN = 5;
153168
test/queue-verdicts.test.ts added +188
@@ -0,0 +1,188 @@
1 /**
2 * What a red batch was (#7639): the queue names the one pull and hands it
3 * back with its failing tests, or calls the red the machine's, and says so
4 * in the keeper, the flow reading and the boards' sources.
5 */
6 import { afterAll, afterEach, beforeAll, describe, expect, spyOn, test } from "bun:test";
7 import { mkdtemp, rm } from "node:fs/promises";
8 import { tmpdir } from "node:os";
9 import { join } from "node:path";
10 import { createApp } from "../src/app.ts";
11 import { SOURCES } from "../src/boards/sources.ts";
12 import { type Candidate, enqueue, type Expect, type Queued } from "../src/checks/queue.ts";
13 import type { LandVerdict } from "../src/checks/tiers.ts";
14 import type { Pull } from "../src/core/pulls.ts";
15 import { createUser } from "../src/core/users.ts";
16 import { keepVerdict, type QueueVerdict, readVerdicts } from "../src/flow/culprits.ts";
17 import { id } from "../src/kernel/id.ts";
18 import { flowFor } from "../src/system/telemetry.ts";
19 import { flowLines } from "../src/tui/screens/flow.ts";
20
21 describe("the merge queue says what a red batch was", () => {
22 let dir: string;
23 const kept: QueueVerdict[] = [];
24 const spies: { mockRestore(): void }[] = [];
25 const git = (...args: string[]) =>
26 Bun.spawnSync(["git", "-C", dir, "-c", "user.name=t", "-c", "user.email=t@t", ...args]).stdout.toString().trim();
27
28 beforeAll(async () => {
29 dir = await mkdtemp(join(tmpdir(), "forge-queue-verdicts-"));
30 Bun.spawnSync(["git", "init", "-q", "-b", "main", dir]);
31 git("commit", "-q", "--allow-empty", "-m", "start");
32 for (const level of ["info", "warn"] as const) spies.push(spyOn(console, level).mockImplementation(() => {}));
33 });
34 afterEach(() => void (kept.length = 0));
35 afterAll(async () => {
36 for (const s of spies) s.mockRestore();
37 await rm(dir, { recursive: true, force: true });
38 });
39
40 async function headWriting(branch: string, file: string): Promise<string> {
41 git("checkout", "-q", "-B", branch, "main");
42 git("reset", "-q", "--hard", "main");
43 await Bun.write(join(dir, file), `${file}\n`);
44 git("add", file);
45 git("commit", "-q", "-m", file);
46 git("checkout", "-q", "--detach");
47 return git("rev-parse", branch);
48 }
49
50 const entry = (prefix: string, number: number, headOid: string): Queued => ({
51 pull: { id: `${prefix}-${number}`, number, baseBranch: "main", headOid } as Pull,
52 dir,
53 land: async (expect: Expect) => {
54 if (git("rev-parse", "refs/heads/main") !== expect.baseOid) return undefined;
55 const tip = git("commit-tree", expect.treeOid, "-p", expect.baseOid, "-m", `#${number}`);
56 git("update-ref", "refs/heads/main", tip);
57 return tip;
58 },
59 release: () => {},
60 handBack: () => {},
61 });
62
63 const red = (reason: { kind: "tests"; failed: number } | { kind: "install" }, names: string[] = []): LandVerdict => ({
64 lands: false,
65 reopen: true,
66 comment: "red",
67 learn: [],
68 reason,
69 ...(names.length === 0
70 ? {}
71 : { tests: { passed: 1, failed: names.length, failing: names.map((name) => ({ name, errors: [] })) } }),
72 });
73
74 test("one bad pull of a batch: named, with the failing test, the rest land", async () => {
75 const pulls: Queued[] = [];
76 for (let i = 1; i <= 3; i++) pulls.push(entry("real", i, await headWriting(`r${i}`, i === 2 ? "bad.txt" : `r${i}.txt`)));
77 const gate = async (c: Candidate): Promise<LandVerdict> =>
78 git("ls-tree", "--name-only", c.tipOid).split("\n").includes("bad.txt")
79 ? red({ kind: "tests", failed: 1 }, ["suite > breaks"])
80 : { lands: true };
81 const answers = await enqueue("verdict-real", pulls, gate, { depth: 1, room: () => 0, onVerdict: (_, v) => kept.push(v) });
82 expect(answers).toEqual([true, false, true]);
83 expect(kept).toEqual([{ kind: "real", why: "tests", pull: 2, tests: ["suite > breaks"] }]);
84 });
85
86 test("an install red twice is a load red: no pull blamed", async () => {
87 const pulls: Queued[] = [];
88 for (let i = 11; i <= 12; i++) pulls.push(entry("load", i, await headWriting(`l${i}`, `l${i}.txt`)));
89 await enqueue("verdict-load", pulls, async () => red({ kind: "install" }), {
90 depth: 1,
91 room: () => 3,
92 onVerdict: (_, v) => kept.push(v),
93 });
94 expect(kept).toEqual([{ kind: "load", why: "install", tests: [] }]);
95 });
96
97 test("a flake is a load red, kept even when onFlake throws", async () => {
98 const pulls: Queued[] = [];
99 for (let i = 31; i <= 34; i++) pulls.push(entry("flake", i, await headWriting(`f${i}`, `f${i}.txt`)));
100 const gate = async (c: Candidate): Promise<LandVerdict> =>
101 c.steps.length === 4 || c.steps.length === 2 ? red({ kind: "tests", failed: 1 }, ["flaky one"]) : { lands: true };
102 let calls = 0;
103 await enqueue("verdict-flake", pulls, gate, {
104 depth: 1,
105 room: () => (calls++ === 0 ? 0 : 1),
106 onFlake: () => {
107 throw new Error("disk");
108 },
109 onVerdict: (_, v) => kept.push(v),
110 });
111 expect(kept).toContainEqual({ kind: "load", why: "flake", tests: ["flaky one"] });
112 });
113
114 test("a keeper that throws does not stop the queue", async () => {
115 const pulls = [entry("boom", 21, await headWriting("b21", "b21.txt"))];
116 const answers = await enqueue("verdict-boom", pulls, async () => red({ kind: "tests", failed: 1 }), {
117 depth: 1,
118 room: () => 0,
119 onVerdict: () => {
120 throw new Error("disk");
121 },
122 });
123 expect(answers).toEqual([false]);
124 });
125 });
126
127 describe("the kept verdicts", () => {
128 test("read as counts and a sentence, cut to public repositories for an agent, shown in the flow reading and the boards", async () => {
129 const data = await mkdtemp(join(tmpdir(), "forge-verdicts-"));
130 const app = createApp({ dataDir: data, port: 0 });
131 try {
132 const u = await createUser(app.db, { username: "alice", email: "[email protected]", password: "a-good-passphrase", origin: "seed", trusted: true });
133 if (!u.ok) throw new Error(u.error);
134 const repo = (name: string, visibility: string) => {
135 const rid = id();
136 app.db.query(`INSERT INTO repo (id, owner_id, name, visibility) VALUES (?1, ?2, ?3, ?4)`).run(rid, u.value.id, name, visibility);
137 return rid;
138 };
139 const pub = repo("pub", "public");
140 const priv = repo("priv", "private");
141 const now = new Date("2026-11-02T12:00:00.000Z");
142 keepVerdict(app.db, pub, { kind: "load", why: "install", tests: [] }, new Date(now.getTime() - 3_600_000));
143 keepVerdict(app.db, priv, { kind: "real", why: "tests", pull: 9, tests: ["a > b"] }, new Date(now.getTime() - 60_000));
144 keepVerdict(app.db, pub, { kind: "real", why: "tests", pull: 7, tests: ["x"] }, new Date(now.getTime() - 120_000));
145 keepVerdict(app.db, pub, { kind: "load", why: "flake", tests: [] }, new Date(now.getTime() - 3 * 86_400_000));
146
147 const all = readVerdicts(app.db, now, false);
148 expect([all.real, all.load]).toEqual([2, 1]);
149 expect(all.says).toBe('2 real, 1 load in 24 h; latest: alice/priv#9 broke "a > b", returned to its builder');
150 expect(all.recent[0]).toMatchObject({ pull: 9, action: "returned #9 to its builder" });
151 const open = readVerdicts(app.db, now, true);
152 expect([open.real, open.load]).toEqual([1, 1]);
153 expect(JSON.stringify(open)).not.toContain("priv");
154 expect(readVerdicts(app.db, new Date("2026-12-01T00:00:00.000Z")).says).toBe("no red batch in 24 h");
155
156 expect(flowFor(app, { kind: "agent" } as never, now).redBatches).toEqual(open);
157 expect(SOURCES.queueVerdicts!.columns).not.toHaveProperty("tests");
158 expect(app.db.query(`SELECT COUNT(*) AS n FROM queue_verdict`).get()).toEqual({ n: 4 });
159 } finally {
160 app.close?.();
161 await rm(data, { recursive: true, force: true });
162 }
163 });
164 });
165
166 describe("the Flow dialog", () => {
167 test("names the red batches, public ones alone for an agent", async () => {
168 const data = await mkdtemp(join(tmpdir(), "forge-verdicts-tui-"));
169 const app = createApp({ dataDir: data, port: 0 });
170 try {
171 const u = await createUser(app.db, { username: "alice", email: "[email protected]", password: "a-good-passphrase", origin: "seed", trusted: true });
172 if (!u.ok) throw new Error(u.error);
173 app.db.query(`INSERT INTO repo (id, owner_id, name, visibility) VALUES ('p1', ?1, 'pub', 'public'), ('p2', ?1, 'priv', 'private')`).run(u.value.id);
174 const now = new Date("2026-11-02T12:00:00.000Z");
175 keepVerdict(app.db, "p1", { kind: "real", why: "tests", pull: 7, tests: ["x > y"] }, new Date(now.getTime() - 60_000));
176 keepVerdict(app.db, "p2", { kind: "real", why: "tests", pull: 9, tests: ["secret"] }, new Date(now.getTime() - 30_000));
177 const owner = flowLines(app.db, now, []).join("\n");
178 expect(owner).toContain("alice/pub#7 real (tests)");
179 expect(owner).toContain("alice/priv#9");
180 const agent = flowLines(app.db, now, [], { queues: [] }).join("\n");
181 expect(agent).toContain("alice/pub#7");
182 expect(agent).not.toContain("priv");
183 } finally {
184 app.close?.();
185 await rm(data, { recursive: true, force: true });
186 }
187 });
188 });

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs