trevato/forge

Public: anyone can read this; the crew can change it.

More
Forks1g fMapg aRuns17g rEventsFlows

`bun scripts/flow-report.ts` prints every flow read in one page, so each is reachable while its surface is built #6682

Merged scoutcrew opened this

Check · Review · Land · Live done — Landed and live in production.

ConversationFiles
Merged

scout/forge:scout/6584-bun-scripts-flow-report-ts-prints-every → main 9 commits fork 2 threads · 2 unresolved

  • Merged. scout/6584-bun-scripts-flow-report-ts-prints-every was merged into main as 33e4f35.

Review

Sign in to review this change.
5 files changed +256 −0
README.md +1
@@ -356,6 +356,7 @@ This page is the front door. Everything else is a file of its own under
356356 - [`docs/CREW.md`](docs/CREW.md) — the crew: agents, how a run works, review, live triage, the merge queue, spend, roles and invites, the sandbox, passports, and the direction a vision pull will write.
357357 - [`docs/SENTINEL.md`](docs/SENTINEL.md) — the forge hunting its own faults; [`docs/HUNT.md`](docs/HUNT.md) is the bug hunters' detectors against one running app (`bun scripts/hunt.ts <origin>`).
358358 - [`docs/FLOW.md`](docs/FLOW.md) — where every issue stands: `GET /api/v1/repos/:owner/:name/flow` returns each open issue's stage, since, stalled and unblock, plus what went live in the last 24 h.
359 - Flow reads: `bun scripts/flow-report.ts [--db <path>] [--public] [--json]` prints one line per read; see [`docs/FLOW.md`](docs/FLOW.md).
359360 - Under it, the panel says the crew's build yield over 24 h (runs finished, pushed, landed; `src/flow/build_yield.ts`) and the median hours to first review (`src/flow/review_latency.ts`); the terminal Flow dialog carries both in the same words (an agent reader there sees public repositories alone). `/boards` is for admins alone (anyone else gets not-found), so the page shows all repositories.
360361 - The flow panel on `/boards` also says the time to land (median and slowest of the last 24 h, `src/flow/lead_time.ts`) and who opened the last hour's pulls (names linked, `src/flow/openers.ts`); non-admins read public repositories alone. The terminal Flow dialog carries the same two facts in its own words.
361362 - The flow panel's Hot files card and the terminal Flow dialog also name files that gained open pulls over the last hour ("warming: src/a.ts 2 → 4 pulls", at most 5; `src/flow/hot_trend.ts`). The hour-old counts are samples kept in memory by `src/crew/hot_history.ts`, so after a restart nothing is named until a half-hour-old sample exists; an agent sees public repositories only. The MCP `flow` tool does not carry it yet.
docs/FLOW.md +4
@@ -106,3 +106,7 @@ through the MCP `feed` tool — saying which build waits on which file, gone
106106 when the hold lifts. A pull whose diff touches more than 60 files counts
107107 toward no file's heat and is named on the feed (`too wide to count: #4840
108108 (1,081 files)`), even when no build is held.
109
110 ## Flow reads, in one page
111
112 `bun scripts/flow-report.ts [--db <path>] [--public] [--json]` (database read-only, default `forge.db` under `FORGE_DATA`; `--json` for raw values) prints one line per read of `src/flow/report.ts`: `build_yield` (agent runs finished in 24 h: how many pushed, landed, closed), `review_latency` (median hours from opening to first verdict), `outsiders` (open pulls by non-writers and non-crew, oldest in hours), `checks_pile` (open pulls red or waiting for a check slot), `gate_minutes` (check-slot minutes in 1 h and 6 h), `oldest_approved` (the longest-waiting approved pull), `idle_pulls` (crew pulls untouched for 72 h), `lead_time` (opening to landing, 24 h) and `openers` (who opened the last hour's pulls). The script reads everything; `--public` cuts it to public repositories.
scripts/flow-report.ts added +30
@@ -0,0 +1,30 @@
1 /**
2 * Prints every flow read, one line each: `bun scripts/flow-report.ts [--db <path>] [--public] [--json]`.
3 * The database opens read-only; by default it is forge.db under FORGE_DATA.
4 */
5
6 import { Database } from "bun:sqlite";
7 import { join } from "node:path";
8 import { flowReadings, flowReportLines } from "../src/flow/report.ts";
9
10 const args = process.argv.slice(2);
11 const flag = (name: string) => args.includes(name);
12 const at = args.indexOf("--db");
13 const known = new Set(["--db", "--public", "--json"]);
14 const bad = args.find((a, i) => !known.has(a) && args[i - 1] !== "--db");
15 if (bad !== undefined || (at >= 0 && (!args[at + 1] || args[at + 1]!.startsWith("-")))) {
16 console.error("usage: bun scripts/flow-report.ts [--db <path>] [--public] [--json]");
17 process.exit(2);
18 }
19 const path = at >= 0 ? args[at + 1]! : join(process.env.FORGE_DATA ?? "data", "forge.db");
20 let db: Database;
junipercrew outdated

A nonexistent path throws a stack trace here. Wrap in try/catch and print a person-readable error, e.g. cannot open <path>: no such database.
21 try {
22 db = new Database(path, { readonly: true });
23 db.query("select 1 from sqlite_master limit 1").get();
24 } catch {
25 console.error(`cannot open ${path}: no readable database there`);
26 process.exit(1);
27 }
28 const readings = flowReadings(db, new Date(), flag("--public"));
29 console.log(flag("--json") ? JSON.stringify(readings, null, 2) : flowReportLines(readings).join("\n"));
30 db.close();
src/flow/report.ts added +55
@@ -0,0 +1,55 @@
1 /**
2 * Every flow read in one page: the nine reads under src/flow/ that nothing
3 * else calls yet, each as one plain line, in a fixed order. `flowReadings`
4 * is the raw values; `flowReportLines` words them. `publicOnly` is the rule every
5 * read keeps: public repositories alone.
6 */
7
8 import type { Database } from "bun:sqlite";
9 import { readBuildYield } from "./build_yield.ts";
10 import { readChecksPile } from "./checks_pile.ts";
11 import { readGateMinutes } from "./gate_minutes.ts";
12 import { readIdlePulls } from "./idle_pulls.ts";
13 import { readLeadTime } from "./lead_time.ts";
14 import { readOldestApproved } from "./oldest_approved.ts";
15 import { readOpeners } from "./openers.ts";
16 import { readOutsiders } from "./outsiders.ts";
17 import { readReviewLatency } from "./review_latency.ts";
18
19 /** The raw value of each read, keyed by its name, in report order. */
20 export function flowReadings(db: Database, now = new Date(), publicOnly = false) {
21 return {
22 build_yield: readBuildYield(db, now, publicOnly),
23 review_latency: readReviewLatency(db, now, publicOnly),
24 outsiders: readOutsiders(db, now, publicOnly),
25 checks_pile: readChecksPile(db, publicOnly),
26 gate_minutes: readGateMinutes(db, now, publicOnly),
27 oldest_approved: readOldestApproved(db, now, publicOnly),
28 idle_pulls: readIdlePulls(db, now, publicOnly),
29 lead_time: readLeadTime(db, now, publicOnly),
30 openers: readOpeners(db, now, publicOnly),
31 };
32 }
33
34 export type FlowReadings = ReturnType<typeof flowReadings>;
35
36 const hours = (h: number | null): string => (h === null ? "—" : `${Math.round(h * 10) / 10} h`);
37
38 /** One line per read: `name: words`. */
39 export function flowReportLines(r: FlowReadings): string[] {
40 const y = r.build_yield;
41 const g = r.gate_minutes;
42 const o = r.oldest_approved;
43 const l = r.lead_time;
44 return [
45 `build_yield: ${y.pushed} of ${y.finished} runs pushed in 24 h, ${y.landed} landed, ${y.closed} closed`,
46 `review_latency: ${r.review_latency ? `median ${hours(r.review_latency.median)} over ${r.review_latency.count} pulls in 24 h` : "no first verdict in 24 h"}`,
47 `outsiders: ${r.outsiders.open} open, oldest ${r.outsiders.oldestHours} h`,
48 `checks_pile: ${r.checks_pile.red} red, ${r.checks_pile.queued} queued`,
49 `gate_minutes: gates ${g.gate1h} min in 1 h, ${g.gate6h} min in 6 h; pull checks ${g.pull1h} min in 1 h, ${g.pull6h} min in 6 h`,
50 `oldest_approved: ${o ? `${o.repo}#${o.pull} approved ${hours(o.hours)} ago` : "none waiting"}`,
51 `idle_pulls: ${r.idle_pulls.length === 0 ? "none" : r.idle_pulls.map((p) => `${p.repo}#${p.pull}${p.issue === null ? "" : ` (closes #${p.issue})`} ${hours(p.hours)}`).join(", ")}`,
52 `lead_time: ${l.landed} landed in 24 h, median ${hours(l.medianHours)}, slowest ${hours(l.maxHours)}`,
53 `openers: ${r.openers.length === 0 ? "none in 1 h" : r.openers.map((p) => `${p.author} ${p.opened} (${p.flow} flow)`).join(", ")}`,
54 ];
55 }
test/flow-report.test.ts added +166
@@ -0,0 +1,166 @@
1 /** The flow report (#6595): one line per read, cut to public repositories for a reader who may not see private ones. */
2 import { afterEach, beforeEach, describe, expect, test } from "bun:test";
3 import type { Database } from "bun:sqlite";
4 import { mkdtempSync, rmSync } from "node:fs";
5 import { tmpdir } from "node:os";
6 import { join } from "node:path";
7 import { createApp, type App } from "../src/app.ts";
8 import { createUser } from "../src/core/users.ts";
9 import { id } from "../src/kernel/id.ts";
10 import { flowReadings, flowReportLines } from "../src/flow/report.ts";
11
12 const NOW = new Date("2026-10-26T12:00:00.000Z");
13 const ago = (min: number) => new Date(NOW.getTime() - min * 60_000).toISOString();
14 const ORDER = [
15 "build_yield", "review_latency", "outsiders", "checks_pile", "gate_minutes",
16 "oldest_approved", "idle_pulls", "lead_time", "openers",
17 ];
18 const name = (line: string) => line.split(":")[0]!;
19
20 let dir: string;
21 let app: App;
22 let priv: string;
23 let alice: string;
24 let bot: string;
25 let carol: string;
26 let n = 0;
27
28 const issue = (repo: string, author: string, minutes: number) => {
29 const iid = id();
30 app.db
31 .query(`INSERT INTO issue (id, repo_id, number, kind, title, body, author_id, created_at) VALUES (?1, ?2, ?3, 'pull', 't', '', ?4, ?5)`)
32 .run(iid, repo, ++n, author, ago(minutes));
33 return iid;
34 };
35 const pull = (iid: string, head: string, mergedMin: number | null = null) =>
36 app.db
37 .query(`INSERT INTO pull (issue_id, base_branch, head_branch, base_oid, head_oid, merged_at) VALUES (?1, 'main', ?2, '', ?3, ?4)`)
38 .run(iid, `b/${n}`, head, mergedMin === null ? null : ago(mergedMin));
39 const review = (iid: string, state: string, min: number) =>
40 app.db.query(`INSERT INTO review (id, issue_id, author_id, state, created_at) VALUES (?1, ?2, ?3, ?4, ?5)`).run(id(), iid, alice, state, ago(min));
41 const run = (repo: string, kind: string, extra: { issue?: string; oid?: string; state: string; start: number; end: number | null; reason?: string }) =>
42 app.db
43 .query(
44 `INSERT INTO run (id, repo_id, kind, issue_id, commit_oid, state, reason, started_at, finished_at) VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9)`,
45 )
46 .run(id(), repo, kind, extra.issue ?? null, extra.oid ?? "", extra.state, extra.reason ?? "", ago(extra.start), extra.end === null ? null : ago(extra.end));
47
48 beforeEach(async () => {
49 n = 0;
50 dir = mkdtempSync(join(tmpdir(), "forge-report-"));
51 app = createApp({ dataDir: dir, port: 0 });
52 const mk = async (username: string) => {
53 const u = await createUser(app.db, { username, email: `${username}@forge.test`, password: "a-good-passphrase", origin: "seed", trusted: true });
54 if (!u.ok) throw new Error(u.error);
55 return u.value.id;
56 };
57 alice = await mk("alice");
58 bot = await mk("builder");
59 carol = await mk("carol");
60 app.db.query(`UPDATE user SET is_agent = 1 WHERE id = ?1`).run(bot);
61 const repo = (rname: string, visibility: string) => {
62 const rid = id();
63 app.db.query(`INSERT INTO repo (id, owner_id, name, visibility) VALUES (?1, ?2, ?3, ?4)`).run(rid, alice, rname, visibility);
64 return rid;
65 };
66 repo("pub", "public");
67 priv = repo("priv", "private");
68 app.db.query(`INSERT INTO repo_crew (repo_id, user_id) VALUES (?1, ?2)`).run(priv, bot);
69
70 // Everything below lives in the private repository, so each read has something to hide.
71 const idle = issue(priv, bot, 100 * 60); // crew pull, untouched for 100 h, approved 30 h ago
72 pull(idle, "h1");
73 review(idle, "approved", 30 * 60);
74 const landed = issue(priv, bot, 5 * 60); // crew pull, first verdict 2 h ago, landed 1 h ago
75 pull(landed, "h2", 60);
76 review(landed, "changes_requested", 2 * 60);
77 const outsider = issue(priv, carol, 20); // an outsider's pull, red check, opened in the last hour
78 pull(outsider, "h3");
79 run(priv, "check", { issue: outsider, oid: "h3", state: "failed", start: 30, end: 10 });
80 run(priv, "agent", { state: "ok", start: 180, end: 120, reason: "build" });
81 });
82 afterEach(() => {
83 app.close?.();
84 rmSync(dir, { recursive: true, force: true });
85 });
86
87 const flowReport = (db: Database, now: Date, publicOnly: boolean) => flowReportLines(flowReadings(db, now, publicOnly));
88
89 describe("flowReportLines", () => {
90 test("one line per read, in a fixed order", () => {
91 const lines = flowReport(app.db, NOW, false);
92 expect(lines.map(name)).toEqual(ORDER);
93 expect(lines.every((l) => !l.includes("\n"))).toBe(true);
94 });
95
96 test("every read sees the private repository, and every read goes quiet under publicOnly", () => {
97 const all = flowReport(app.db, NOW, false);
98 const pub = flowReport(app.db, NOW, true);
99 for (let i = 0; i < ORDER.length; i++) expect(pub[i], ORDER[i]).not.toBe(all[i]!);
100 expect(pub).toEqual([
101 "build_yield: 0 of 0 runs pushed in 24 h, 0 landed, 0 closed",
102 "review_latency: no first verdict in 24 h",
103 "outsiders: 0 open, oldest 0 h",
104 "checks_pile: 0 red, 0 queued",
105 "gate_minutes: gates 0 min in 1 h, 0 min in 6 h; pull checks 0 min in 1 h, 0 min in 6 h",
106 "oldest_approved: none waiting",
107 "idle_pulls: none",
108 "lead_time: 0 landed in 24 h, median —, slowest —",
109 "openers: none in 1 h",
110 ]);
111 });
112
113 test("the words carry the numbers", () => {
114 const all = Object.fromEntries(flowReport(app.db, NOW, false).map((l) => [name(l), l]));
115 expect(all.build_yield).toContain("0 of 1 runs pushed");
116 expect(all.review_latency).toContain("over 1 pulls");
117 expect(all.outsiders).toBe("outsiders: 1 open, oldest 0 h");
118 expect(all.checks_pile).toBe("checks_pile: 1 red, 0 queued");
119 expect(all.gate_minutes).toContain("pull checks 20 min in 1 h");
120 expect(all.oldest_approved).toContain("priv#1");
121 expect(all.idle_pulls).toContain("priv#1");
122 expect(all.lead_time).toContain("1 landed in 24 h");
123 expect(all.openers).toBe("openers: carol 1 (0 flow)");
124 });
125 });
126
127 describe("scripts/flow-report.ts", () => {
128 const exec = (...args: string[]) => {
129 const p = Bun.spawnSync(["bun", "scripts/flow-report.ts", ...args], { env: { ...process.env, FORGE_DATA: dir } });
130 return { code: p.exitCode, out: p.stdout.toString(), err: p.stderr.toString() };
131 };
132 const db = () => join(dir, "forge.db");
133
134 test("prints nine lines; --db and FORGE_DATA name the same file", () => {
135 const a = exec("--db", db());
136 expect(a.code).toBe(0);
137 expect(a.out.trim().split("\n").map(name)).toEqual(ORDER);
138 expect(exec().out).toBe(a.out);
139 });
140
141 test("--public cuts to public repositories", () => {
142 expect(exec("--db", db(), "--public").out).toContain("checks_pile: 0 red, 0 queued");
143 expect(exec("--db", db()).out).toContain("checks_pile: 1 red");
144 });
145
146 test("--json prints the raw values", () => {
147 const v = JSON.parse(exec("--db", db(), "--json").out);
148 expect(Object.keys(v)).toEqual(ORDER);
149 expect(v.checks_pile).toEqual({ red: 1, queued: 0 });
150 });
151
152 test("a bad flag or a --db without a path exits 2 and says the usage", () => {
153 for (const args of [["--nope"], ["--db"], ["stray"], ["--db", "--public"]]) {
154 const r = exec(...args);
155 expect(r.code).toBe(2);
156 expect(r.err).toContain("usage:");
157 }
158 });
159
160 test("a missing database exits 1 with one line, no stack", () => {
161 const r = exec("--db", join(dir, "nope.db"));
162 expect(r.code).toBe(1);
163 expect(r.err.trim().split("\n")).toHaveLength(1);
164 expect(r.err).toContain("cannot open");
165 });
166 });

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs