Public: anyone can read this; the crew can change it.
An admin sees flows across all repositories in one view #9620
Check · Review · Land · Live done — Landed and live in production.
mason/forge:mason/9594-an-admin-sees-flows-across-all-repositor → main 5 commits fork 3 threads · 3 unresolved
12 files
changed +390
−1
docs/design/audit.md +1
| @@ -100,6 +100,7 @@ Machine routes are not pages and have no entry: `/audit.csv`, | ||
| 100 | 100 | - `/system` — `pages/system.ts` · the machine as it is now: cores, load, memory, processes · `/flow`, `/` · none seen; updates in place over `/system/stream`, no reload script. |
| 101 | 101 | - `/resilience` — `pages/resilience.ts` · each protection in `docs/THREATS.md` against the last gate's tests · `/settings/resilience` (similar name, different content) · none seen. |
| 102 | 102 | - `/admin/limits` — `pages/admin-limits.ts` · every limit with value, default and reason · `/settings/capacity` (also a numbers-you-can-change page) · none seen. |
| 103 | - `/admin/flows` — `pages/flows-all.ts` · every repository's flow lines with state and last moved, by kind and stale · `/:owner/:name/flows` · none seen. | |
| 103 | 104 | - `/settings/capacity` — `pages/capacity.ts` · how many runs and checks go at once · `/admin/limits`, `/settings/usage` · none seen. |
| 104 | 105 | - `/settings/usage` — `pages/usage.ts` · route patterns by use, unused first · `/settings/telemetry`, `/board/returns` · none seen. Not about cost, despite the name. |
| 105 | 106 | - `/settings/telemetry` — `pages/telemetry.ts` · which agents may read telemetry · `/settings/crew` (agent grants) · none seen. |
docs/features/admin-flows.md added +9
| @@ -0,0 +1,9 @@ | ||
| 1 | # Flows across the forge | |
| 2 | ||
| 3 | `/admin/flows` (#9594) is one list of every repository's flow lines, for a site admin: each followed upstream, followed source, fork family, mirror and code scan, with its state and when it last moved. `docs/features/flows.md` is the per-repository picture; this is the forge-wide one. | |
| 4 | ||
| 5 | - **Who.** Site admins who are people. Everyone else, signed in or not, and every agent, gets the not-found page; the terminal offers them nothing. | |
| 6 | - **Filters.** `?kind=upstream|source|fork|mirror|scan` and `?stale=1` (a followed link not read in 90 minutes, or never; a scan whose last run is over three days old, `SCAN_STALE_MS`). A scan with an open high finding reads as holding releases instead. | |
| 7 | - **Terminal.** *Flows across the forge* in the palette (`src/tui/screens/flowsall.ts`), same gate and same rows; `f` cycles the kind, `s` flips stale only. | |
| 8 | - **Fast.** `readAllFlows` (`src/flows/all.ts`) runs seven set-based reads whatever the number of repositories, no network, no model. At most 500 lines are drawn; the count says how many matched. Followed URLs are shown without userinfo, query or fragment. | |
| 9 | - **Test.** `test/flows-all.test.ts`: exact rows, the stale threshold, the gate on both doors, and statement counts for the reader and the page at 300 repositories. |
docs/features/flows.md +1 −1
| @@ -18,7 +18,7 @@ | ||
| 18 | 18 | - **Forks.** Each fork out reads *fork, takes updates* or *fork, told of updates* (its `fork_follow` mode), and its last-moved time is its newest push (its creation until it has one). A fork following its parent is the only "follower" the forge records; there is no table of people following a repository. |
| 19 | 19 | - **Forges that follow.** A child forge holding a proposal token for the repository is a *forge follows* line (its origin without userinfo or query; last use of the token, else when it was minted; *token ended* once expired). Only people who can write to the repository see these lines, page and terminal alike. |
| 20 | 20 | |
| 21 | Left out: the CHANGELOG sections between pin and newest (dropped until a lookup can read one within a byte bound); the nightly radar is below; code scanning (and the release block on a new high finding); per-line buttons (actions are listed under the columns); counting a fork's commits ahead of its parent; GitHub release tags; the all-repositories admin view. Each is its own pull on top of `dep_latest` and `readFlows`. | |
| 21 | Left out: the CHANGELOG sections between pin and newest (dropped until a lookup can read one within a byte bound); the nightly radar is below; code scanning (and the release block on a new high finding); per-line buttons (actions are listed under the columns); counting a fork's commits ahead of its parent; GitHub release tags; the all-repositories admin view is `admin-flows.md`. Each is its own pull on top of `dep_latest` and `readFlows`. | |
| 22 | 22 | |
| 23 | 23 | ## The nightly radar (#9592) |
| 24 | 24 |
src/flows/all.ts added +96
| @@ -0,0 +1,96 @@ | ||
| 1 | /** | |
| 2 | * Every repository's flows in one list (#9594), for a site admin: one line | |
| 3 | * per followed upstream, followed source, fork family, mirror and scan, with | |
| 4 | * its state and when it last moved. A handful of set-based reads of state the | |
| 5 | * forge already keeps (no per-repository query, no network, no model), so it | |
| 6 | * answers in milliseconds for hundreds of repositories. Doors: the page | |
| 7 | * `web/pages/flows-all.ts` and the terminal screen `tui/screens/flowsall.ts`; | |
| 8 | * how to use it: docs/features/admin-flows.md. | |
| 9 | */ | |
| 10 | ||
| 11 | import type { Database } from "bun:sqlite"; | |
| 12 | import { followHealth, plainUrl, type Health } from "./view.ts"; | |
| 13 | ||
| 14 | /** A scan is stale when its last run is older than this: three days, since scans run on pushes and nightly, not every follow period. */ | |
| 15 | export const SCAN_STALE_MS = 3 * 24 * 60 * 60_000; | |
| 16 | ||
| 17 | export const ALL_KINDS = ["upstream", "source", "fork", "mirror", "scan"] as const; | |
| 18 | export type AllKind = (typeof ALL_KINDS)[number]; | |
| 19 | ||
| 20 | export interface AllLine { | |
| 21 | readonly repo: string; | |
| 22 | readonly href: string; | |
| 23 | readonly kind: AllKind; | |
| 24 | readonly label: string; | |
| 25 | readonly state: string; | |
| 26 | readonly at: string | null; | |
| 27 | readonly health: Health; | |
| 28 | /** Not read lately: a followed link past STALE_MS, or one never read. */ | |
| 29 | readonly stale: boolean; | |
| 30 | } | |
| 31 | ||
| 32 | export interface AllFilter { | |
| 33 | readonly kind?: AllKind; | |
| 34 | readonly stale?: boolean; | |
| 35 | } | |
| 36 | ||
| 37 | /** A query value as a kind, or undefined when it is not one. */ | |
| 38 | export const parseKind = (raw: string | null | undefined): AllKind | undefined => ALL_KINDS.find((k) => k === raw); | |
| 39 | ||
| 40 | const plural = (n: number, one: string, many: string) => `${n} ${n === 1 ? one : many}`; | |
| 41 | ||
| 42 | interface Followed { repo_id: string; url: string; branch: string; enabled: number; checked_at: string | null; behind: number; error: string } | |
| 43 | ||
| 44 | export function readAllFlows(db: Database, filter: AllFilter = {}, now = Date.now()): AllLine[] { | |
| 45 | const repos = new Map<string, { name: string; href: string }>(); | |
| 46 | for (const r of db | |
| 47 | .query<{ id: string; name: string; owner: string }, []>("SELECT repo.id, repo.name, user.username AS owner FROM repo JOIN user ON user.id = repo.owner_id") | |
| 48 | .all()) | |
| 49 | repos.set(r.id, { name: `${r.owner}/${r.name}`, href: `/${r.owner}/${r.name}` }); | |
| 50 | const lines: AllLine[] = []; | |
| 51 | const add = (repoId: string, l: Omit<AllLine, "repo" | "href" | "stale">) => { | |
| 52 | const r = repos.get(repoId); | |
| 53 | if (!r) return; | |
| 54 | lines.push({ repo: r.name, href: `${r.href}/flows`, ...l, stale: l.health === "stale" }); | |
| 55 | }; | |
| 56 | const followed = (kind: "upstream" | "source", table: string) => { | |
| 57 | for (const s of db.query<Followed, []>(`SELECT repo_id, url, branch, enabled, checked_at, behind, error FROM ${table}`).all()) { | |
| 58 | const health = followHealth({ enabled: s.enabled !== 0, checkedAt: s.checked_at, behind: s.behind, error: s.error }, now); | |
| 59 | add(s.repo_id, { | |
| 60 | kind, label: `${plainUrl(s.url)} (${s.branch})`, at: s.checked_at, health, | |
| 61 | state: s.enabled === 0 ? "not following" : s.error !== "" ? "could not be read" : s.behind === 0 ? "up to date" : plural(s.behind, "change", "changes") + " behind", | |
| 62 | }); | |
| 63 | } | |
| 64 | }; | |
| 65 | followed("upstream", "repo_upstream"); | |
| 66 | followed("source", "repo_source"); | |
| 67 | for (const f of db | |
| 68 | .query<{ parent: string; n: number; at: string | null }, []>( | |
| 69 | "SELECT fork_parent_id AS parent, count(*) AS n, max(created_at) AS at FROM repo WHERE fork_parent_id IS NOT NULL GROUP BY fork_parent_id", | |
| 70 | ) | |
| 71 | .all()) | |
| 72 | add(f.parent, { kind: "fork", label: "forks", state: plural(f.n, "fork", "forks"), at: f.at, health: "none" }); | |
| 73 | for (const m of db | |
| 74 | .query<{ repo_id: string; full_name: string; pushed_at: string | null; last_error: string }, []>("SELECT repo_id, full_name, pushed_at, last_error FROM migrate_mirror") | |
| 75 | .all()) | |
| 76 | add(m.repo_id, { | |
| 77 | kind: "mirror", label: m.full_name, at: m.pushed_at, | |
| 78 | state: m.last_error !== "" ? "last push failed" : "mirrored", health: m.last_error !== "" ? "unread" : "ok", | |
| 79 | }); | |
| 80 | const high = new Map<string, number>(); | |
| 81 | for (const h of db | |
| 82 | .query<{ repo_id: string; n: number }, []>("SELECT repo_id, count(*) AS n FROM scan_finding WHERE accepted_at IS NULL AND severity = 'high' GROUP BY repo_id") | |
| 83 | .all()) | |
| 84 | high.set(h.repo_id, h.n); | |
| 85 | for (const s of db.query<{ repo_id: string; at: string }, []>("SELECT repo_id, max(at) AS at FROM scan_run GROUP BY repo_id").all()) { | |
outdated
Magic number STALE_MS * 48 for scan staleness: name it as a constant with its reason, and test it. |
||
| 86 | const n = high.get(s.repo_id) ?? 0; | |
| 87 | add(s.repo_id, { | |
| 88 | kind: "scan", label: "Code scanning", at: s.at, | |
| 89 | state: n > 0 ? `${plural(n, "high finding holds", "high findings hold")} releases` : "no open high findings", | |
| 90 | health: n > 0 ? "unread" : Date.parse(s.at) < now - SCAN_STALE_MS ? "stale" : "ok", | |
| 91 | }); | |
| 92 | } | |
| 93 | const kept = lines.filter((l) => (filter.kind === undefined || l.kind === filter.kind) && (filter.stale !== true || l.stale)); | |
| 94 | // Newest movement first; never-moved last. | |
| 95 | return kept.sort((a, b) => (b.at ?? "").localeCompare(a.at ?? "") || a.repo.localeCompare(b.repo)); | |
| 96 | } |
src/http/admin_gate.ts +2
| @@ -19,6 +19,8 @@ const under = (path: string, root: string): boolean => path === root || path.sta | ||
| 19 | 19 | |
| 20 | 20 | /** Whether this path is one the `administer` ability guards. */ |
| 21 | 21 | export function isAdminPath(path: string): boolean { |
| 22 | // `/admin/flows` answers anyone but a site admin with the not-found page (#9594), so it keeps its own refusal. | |
| 23 | if (path === "/admin/flows") return false; | |
| 22 | 24 | return under(path, "/admin") || under(path, "/api/v1/admin") || under(path, "/releases"); |
| 23 | 25 | } |
| 24 | 26 |
src/http/server.ts +2
| @@ -28,6 +28,7 @@ import { contributePage } from "../web/pages/contribute.ts"; | ||
| 28 | 28 | import { pipelinePage } from "../web/pages/pipeline.ts"; |
| 29 | 29 | import { nightlyOutputPage, resiliencePage } from "../web/pages/resilience.ts"; |
| 30 | 30 | import { limitsPage, limitsSet, quotasSet, signupSet, inviteMint, inviteRevoke } from "../web/pages/admin-limits.ts"; |
| 31 | import { allFlowsPage } from "../web/pages/flows-all.ts"; | |
| 31 | 32 | import { measureSubmit } from "./measure.ts"; |
| 32 | 33 | import { settingsRoutes } from "./settings_routes.ts"; |
| 33 | 34 | import { pinnedRoutes } from "./pinned_routes.ts"; |
| @@ -960,6 +961,7 @@ export function buildRouter(): Router<Handler> { | ||
| 960 | 961 | router.add("GET", "/boards/:slug/live", boardLive); |
| 961 | 962 | router.add("POST", "/boards/:slug/:op", boardPost); |
| 962 | 963 | router.add("GET", "/admin/limits", limitsPage); |
| 964 | router.add("GET", "/admin/flows", allFlowsPage); | |
| 963 | 965 | router.add("POST", "/admin/limits", limitsSet); |
| 964 | 966 | router.add("POST", "/admin/quotas", quotasSet); |
| 965 | 967 | router.add("POST", "/admin/signup", signupSet); |
src/tui/screens/entries.ts +2
| @@ -27,6 +27,7 @@ import { entry as edgeOnly } from "./edgeonly.ts"; | ||
| 27 | 27 | import { entry as traffic } from "./trafficview.ts"; |
| 28 | 28 | import { entry as feed } from "./feed.ts"; |
| 29 | 29 | import { entry as forgeView } from "./forgeview.ts"; |
| 30 | import { entry as flowsAll } from "./flowsall.ts"; | |
| 30 | 31 | import { entry as front } from "./front.ts"; |
| 31 | 32 | import { entry as forges } from "./forges.ts"; |
| 32 | 33 | import { entry as homePins } from "./home_pins.ts"; |
| @@ -96,6 +97,7 @@ export const ENTRIES: readonly PaletteEntry[] = [ | ||
| 96 | 97 | people, |
| 97 | 98 | quotas, |
| 98 | 99 | limits, |
| 100 | flowsAll, | |
| 99 | 101 | scrapes, |
| 100 | 102 | teams, |
| 101 | 103 | orgTeams, |
src/tui/screens/flowsall.ts added +76
| @@ -0,0 +1,76 @@ | ||
| 1 | /** | |
| 2 | * Every repository's flows in the terminal (#9594): the rows of the web page | |
| 3 | * `/admin/flows` (`flows/all.ts`), gated the same way. A site admin who is a | |
| 4 | * person is offered it in the palette and sees every line; anyone else is | |
| 5 | * offered nothing and, were the screen reached, sees no rows. | |
| 6 | * | |
| 7 | * `k` cycles the kind filter, `s` flips stale only. Read-only. | |
| 8 | */ | |
| 9 | ||
| 10 | import { ALL_KINDS, readAllFlows, type AllKind } from "../../flows/all.ts"; | |
| 11 | import type { Ctx, Screen } from "../app/app.ts"; | |
| 12 | import { binding } from "../app/keys.ts"; | |
| 13 | import type { Session } from "../session.ts"; | |
| 14 | import { style } from "../term/style.ts"; | |
| 15 | import { pad } from "../term/width.ts"; | |
| 16 | import { Scroll, drawLines, hitScroll } from "./common.ts"; | |
| 17 | import type { PaletteEntry } from "./entries.ts"; | |
| 18 | ||
| 19 | const mayRead = (session: Session): boolean => session.user?.isAdmin === true && session.user.kind !== "agent"; | |
| 20 | ||
| 21 | /** Lines drawn at once, as on the page. */ | |
| 22 | const SHOWN = 500; | |
| 23 | ||
| 24 | /** The header, then a line a row; empty for anyone the gate refuses. */ | |
| 25 | export function flowLines(session: Session, kind?: AllKind, stale = false, now = Date.now()): string[] { | |
| 26 | if (!mayRead(session)) return []; | |
| 27 | const rows = readAllFlows(session.db, { ...(kind ? { kind } : {}), stale }, now).slice(0, SHOWN); | |
| 28 | return [ | |
| 29 | `${pad("Repository", 28)}${pad("Kind", 10)}${pad("State", 34)}Last moved`, | |
| 30 | ...rows.map((l) => `${pad(l.repo, 28)}${pad(l.kind, 10)}${pad(l.state, 34)}${l.at ?? "never moved"}`), | |
| 31 | ]; | |
| 32 | } | |
| 33 | ||
| 34 | export function flowsAllScreen(session: Session, ctx: Ctx): Screen { | |
| 35 | const scroll = new Scroll(); | |
| 36 | let kind: AllKind | undefined; | |
| 37 | let stale = false; | |
| 38 | return { | |
| 39 | title: "Flows across the forge", | |
| 40 | keys: [binding(["j", "k"], "scroll"), binding(["f"], "kind"), binding(["s"], "stale only"), binding(["esc", "q"], "back")], | |
| 41 | render(area, s) { | |
| 42 | const c = ctx.theme.c; | |
| 43 | const lines = flowLines(session, kind, stale); | |
| 44 | const head = `Kind: ${kind ?? "all"} · ${stale ? "stale only" : "all states"}`; | |
| 45 | const spans = [head, ...lines].map((text, i) => [{ text, style: style(i <= 1 ? c.fg : c.mutedFg, c.bg) }]); | |
| 46 | drawLines(s, area, c, scroll, spans); | |
| 47 | hitScroll(ctx, area, scroll, spans.length); | |
| 48 | }, | |
| 49 | key(e) { | |
| 50 | if (scroll.key(e, flowLines(session, kind, stale).length + 1)) return true; | |
| 51 | if (e.key === "f") { | |
| 52 | const i = kind === undefined ? -1 : ALL_KINDS.indexOf(kind); | |
| 53 | kind = ALL_KINDS[i + 1]; | |
| 54 | return true; | |
| 55 | } | |
| 56 | if (e.key === "s") { | |
| 57 | stale = !stale; | |
| 58 | return true; | |
| 59 | } | |
| 60 | if (e.key === "esc" || e.key === "q") { | |
| 61 | ctx.pop(); | |
| 62 | return true; | |
| 63 | } | |
| 64 | return false; | |
| 65 | }, | |
| 66 | }; | |
| 67 | } | |
| 68 | ||
| 69 | export const entry: PaletteEntry = { | |
| 70 | id: "flows-all", | |
| 71 | title: "Flows across the forge", | |
| 72 | hint: "every repository's flow lines, by kind and stale", | |
| 73 | after: ["limits"], | |
| 74 | offered: mayRead, | |
| 75 | run: (session, ctx) => ctx.push(flowsAllScreen(session, ctx)), | |
| 76 | }; |
src/web/pages/flows-all.ts added +52
| @@ -0,0 +1,52 @@ | ||
| 1 | /** | |
| 2 | * `/admin/flows` (#9594): every repository's flow lines in one list, with | |
| 3 | * state and when each last moved, filterable by kind and by stale. Site | |
| 4 | * admins only; anyone else gets the not-found page. Reads stored state only. | |
| 5 | */ | |
| 6 | ||
| 7 | import { ALL_KINDS, parseKind, readAllFlows } from "../../flows/all.ts"; | |
| 8 | import type { Health } from "../../flows/view.ts"; | |
| 9 | import type { Ctx } from "../../http/context.ts"; | |
| 10 | import { notePhase, takeSessionTime, timePhase } from "../../core/route_timing.ts"; | |
| 11 | import { notFoundPage, pageHead, settingsLayout } from "../chrome.ts"; | |
| 12 | import { html } from "../html.ts"; | |
| 13 | import { page } from "../layout.ts"; | |
| 14 | import { accountSections } from "./origins.ts"; | |
| 15 | import { badge, btn, listCard, row, time, type Tone } from "../ui.ts"; | |
| 16 | ||
| 17 | /** Lines drawn at once; the count above says how many matched. */ | |
| 18 | const SHOWN = 500; | |
| 19 | ||
| 20 | const TONE: Record<Health, Tone> = { ok: "default", behind: "attn", diverged: "danger", unread: "danger", stale: "attn", off: "default", none: "default" }; | |
| 21 | ||
| 22 | // GET /admin/flows?kind=&stale=1 | |
| 23 | export function allFlowsPage(ctx: Ctx): Response { | |
| 24 | notePhase(ctx.request, "session", takeSessionTime(ctx.request)); | |
| 25 | const user = ctx.auth.user; | |
| 26 | if (user?.isAdmin !== true || user.kind === "agent") return notFoundPage(ctx); | |
| 27 | const q = new URL(ctx.request.url).searchParams; | |
| 28 | const kind = parseKind(q.get("kind")); | |
| 29 | const stale = q.get("stale") === "1"; | |
| 30 | const lines = timePhase(ctx.request, "flows", () => readAllFlows(ctx.app.db, { ...(kind ? { kind } : {}), stale })); | |
| 31 | const rendering = performance.now(); | |
| 32 | const shown = lines.slice(0, SHOWN); | |
| 33 | const filters = html`<form method="get" action="/admin/flows" class="row-fields"> | |
| 34 | <label>Kind <select name="kind"><option value="">All</option>${ALL_KINDS.map((k) => html`<option value="${k}" ${k === kind ? html`selected` : null}>${k}</option>`)}</select></label> | |
| 35 | <label><input type="checkbox" name="stale" value="1" ${stale ? html`checked` : null}> Stale only</label> | |
| 36 | ${btn("Filter", { variant: "secondary" })} | |
| 37 | </form>`; | |
| 38 | const rows = shown.map((l) => | |
| 39 | row({ | |
| 40 | href: l.href, | |
| 41 | title: html`${l.repo} · ${l.kind}`, | |
| 42 | meta: html`${l.label} · ${l.at === null ? "never moved" : time(l.at)}`, | |
| 43 | trail: badge(l.state, { tone: TONE[l.health] }), | |
| 44 | }), | |
| 45 | ); | |
| 46 | const body = html`${pageHead("Flows across the forge", { sub: `${lines.length} ${lines.length === 1 ? "line" : "lines"}${lines.length > SHOWN ? `, the newest ${SHOWN} shown` : ""}.` })} | |
outdated
Principles 6 and 7: the filter form sits inside a card, a box around a thing. Render the form bare above the rows. Also the empty state is a card around a sentence; use plain muted text. |
||
| 47 | ${filters} | |
| 48 | ${rows.length === 0 ? html`<p class="muted">No flow lines match.</p>` : listCard(rows)}`; | |
| 49 | const response = page({ ctx, title: "Flows · admin", body: settingsLayout(accountSections("adminflows", { admin: true }), body, "Account settings") }); | |
| 50 | notePhase(ctx.request, "render", performance.now() - rendering); | |
| 51 | return response; | |
| 52 | } |
src/web/pages/origins.ts +2
| @@ -89,6 +89,7 @@ import { | ||
| 89 | 89 | } from "../ui.ts"; |
| 90 | 90 | |
| 91 | 91 | export type AccountSection = |
| 92 | | "adminflows" | |
| 92 | 93 | | "security" |
| 93 | 94 | | "profile" |
| 94 | 95 | | "confirms" |
| @@ -157,6 +158,7 @@ const SECTION_ROWS: readonly (readonly [ | ||
| 157 | 158 | ["This forge", "Accounts", "/settings/accounts", "users", (c) => c === "accounts", true], |
| 158 | 159 | ["This forge", "People", "/settings/people", "users", (c) => c === "people", true], |
| 159 | 160 | ["This forge", "Limits", "/admin/limits", "activity", (c) => c === "limits", true], |
| 161 | ["This forge", "Flows", "/admin/flows", "activity", (c) => c === "adminflows", true], | |
| 160 | 162 | ["This forge", "Flags", "/settings/flags", "activity", (c) => c === "flags", true], |
| 161 | 163 | ["This forge", "Features", "/settings/features", "rocket", (c) => c === "features", true], |
| 162 | 164 | ["This forge", "Backups", "/settings/backups", "download", (c) => c === "backups", true], |
test/flows-all.test.ts added +146
| @@ -0,0 +1,146 @@ | ||
| 1 | /** `/admin/flows` (#9594): every repository's lines, filterable, admin only, from stored state, fast. */ | |
| 2 | import { afterAll, beforeAll, expect, test } from "bun:test"; | |
| 3 | import { mkdtempSync, rmSync } from "node:fs"; | |
| 4 | import { tmpdir } from "node:os"; | |
| 5 | import { join } from "node:path"; | |
| 6 | import { createApp, type App } from "../src/app.ts"; | |
| 7 | import { createUser } from "../src/core/users.ts"; | |
| 8 | import { readAllFlows, SCAN_STALE_MS } from "../src/flows/all.ts"; | |
| 9 | import { serve, type Servers } from "../src/http/server.ts"; | |
| 10 | import { createForgeApp, Session } from "../src/tui/forge.ts"; | |
| 11 | import { flowLines } from "../src/tui/screens/flowsall.ts"; | |
| 12 | import { headlessTerminal } from "../src/tui/term/terminal.ts"; | |
| 13 | import type { User } from "../src/core/users.ts"; | |
| 14 | import { requestStatements, expectWork } from "./helpers/work.ts"; | |
| 15 | import { statements } from "./helpers/statements.ts"; | |
| 16 | import { proveAll } from "./helpers/verified.ts"; | |
| 17 | import { quietLogs } from "./helpers/globals.ts"; | |
| 18 | ||
| 19 | quietLogs(); | |
| 20 | let root = ""; | |
| 21 | let app: App; | |
| 22 | let servers: Servers; | |
| 23 | const NOW = Date.parse("2026-11-08T12:00:00Z"); | |
| 24 | ||
| 25 | beforeAll(async () => { | |
| 26 | root = mkdtempSync(join(tmpdir(), "forge-flows-all-")); | |
| 27 | app = createApp({ dataDir: root, port: 0, forgeRepo: "alice/forge" }); | |
| 28 | for (const [username, adminIfFirst] of [["alice", true], ["bob", false]] as const) { | |
| 29 | const u = await createUser(app.db, { origin: "seed", trusted: true, username, email: `${username}@forge.local`, password: "a-good-passphrase", adminIfFirst }); | |
| 30 | if (!u.ok) throw new Error(u.error); | |
| 31 | } | |
| 32 | proveAll(app); | |
| 33 | const alice = app.db.query<{ id: string }, []>("SELECT id FROM user WHERE username = 'alice'").get()!.id; | |
| 34 | for (let i = 0; i < 300; i++) app.db.query("INSERT INTO repo (id, owner_id, name) VALUES (?1, ?2, ?3)").run(`r${i}`, alice, `repo${i}`); | |
| 35 | const up = app.db.query("INSERT INTO repo_upstream (repo_id, url, branch, enabled, checked_at, behind, error, by_id, updated_at) VALUES (?1, ?2, 'main', 1, ?3, ?4, '', ?5, '2026-11-08T00:00:00Z')"); | |
| 36 | up.run("r0", "https://u:[email protected]/a.git?x=1", new Date(NOW - 60_000).toISOString(), 2, alice); | |
| 37 | up.run("r1", "https://example.com/b.git", new Date(NOW - 86_400_000).toISOString(), 0, alice); | |
| 38 | for (let i = 2; i < 300; i++) up.run(`r${i}`, `https://example.com/${i}.git`, new Date(NOW - 1000).toISOString(), 0, alice); | |
| 39 | app.db.query("UPDATE repo SET fork_parent_id = 'r0' WHERE id = 'r5'").run(); | |
| 40 | app.db.run("PRAGMA foreign_keys = OFF"); | |
| 41 | app.db.query("INSERT INTO repo_source (id, repo_id, url, branch, by_id, created_at, checked_at, error) VALUES ('s1', 'r2', 'https://example.com/s.git', 'dev', ?1, '2026-11-01T00:00:00Z', ?2, 'refused')").run(alice, new Date(NOW - 1000).toISOString()); | |
| 42 | const mirror = app.db.query("INSERT INTO migrate_mirror (repo_id, origin_id, full_name, clone_url, pushed_at, last_error) VALUES (?1, 'o', ?2, 'https://x', ?3, ?4)"); | |
| 43 | mirror.run("r3", "acme/ok", "2026-11-07T00:00:00Z", ""); | |
| 44 | mirror.run("r4", "acme/bad", "2026-11-06T00:00:00Z", "denied"); | |
| 45 | const scan = app.db.query("INSERT INTO scan_run (repo_id, kind, oid, at) VALUES (?1, 'push', 'abc', ?2)"); | |
| 46 | scan.run("r6", new Date(NOW - SCAN_STALE_MS + 60_000).toISOString()); | |
| 47 | scan.run("r7", new Date(NOW - SCAN_STALE_MS - 60_000).toISOString()); | |
| 48 | scan.run("r8", new Date(NOW - 1000).toISOString()); | |
| 49 | app.db.query("INSERT INTO scan_finding (repo_id, fingerprint, kind, severity, rule, path, line, oid, found_at) VALUES ('r8', 'f', 'secret', 'high', 'k', 'a', 1, 'abc', '2026-11-08T00:00:00Z')").run(); | |
| 50 | servers = await serve(app); | |
| 51 | }); | |
| 52 | afterAll(() => { | |
| 53 | servers.stop(); | |
| 54 | app.close(); | |
| 55 | rmSync(root, { recursive: true, force: true }); | |
| 56 | }); | |
| 57 | ||
| 58 | const signedIn = async (username: string): Promise<string> => { | |
| 59 | const r = await fetch(`${servers.url}/signin`, { | |
| 60 | method: "POST", headers: { origin: servers.url }, redirect: "manual", | |
| 61 | body: new URLSearchParams({ username, password: "a-good-passphrase" }), | |
| 62 | }); | |
| 63 | return (r.headers.get("set-cookie") ?? "").split(";")[0]!; | |
| 64 | }; | |
| 65 | ||
| 66 | test("lists lines with state, filtered by kind and by stale", () => { | |
| 67 | const all = readAllFlows(app.db, {}, NOW); | |
| 68 | expect(all.filter((l) => l.kind === "upstream")).toHaveLength(300); | |
| 69 | const first = all.find((l) => l.repo === "alice/repo0" && l.kind === "upstream")!; | |
| 70 | expect(first.state).toBe("2 changes behind"); | |
| 71 | expect(first.label).toBe("https://example.com/a.git (main)"); | |
| 72 | expect(readAllFlows(app.db, { kind: "fork" }, NOW).map((l) => [l.repo, l.state])).toEqual([["alice/repo0", "1 fork"]]); | |
| 73 | expect(readAllFlows(app.db, { stale: true }, NOW).map((l) => l.repo)).toEqual(["alice/repo1", "alice/repo7"]); | |
| 74 | }); | |
| 75 | ||
| 76 | const rows = (kind: "source" | "mirror" | "scan") => readAllFlows(app.db, { kind }, NOW).map((l) => [l.repo, l.label, l.state, l.health]); | |
| 77 | ||
| 78 | test("source, mirror and scan lines say their state", () => { | |
| 79 | expect(rows("source")).toEqual([["alice/repo2", "https://example.com/s.git (dev)", "could not be read", "unread"]]); | |
| 80 | expect(rows("mirror")).toEqual([ | |
| 81 | ["alice/repo3", "acme/ok", "mirrored", "ok"], | |
| 82 | ["alice/repo4", "acme/bad", "last push failed", "unread"], | |
| 83 | ]); | |
| 84 | }); | |
| 85 | ||
| 86 | test("a scan is stale after SCAN_STALE_MS, and a high finding outranks it", () => { | |
| 87 | const scans = Object.fromEntries(readAllFlows(app.db, { kind: "scan" }, NOW).map((l) => [l.repo, [l.state, l.health, l.stale]])); | |
| 88 | expect(scans).toEqual({ | |
| 89 | "alice/repo6": ["no open high findings", "ok", false], | |
| 90 | "alice/repo7": ["no open high findings", "stale", true], | |
| 91 | "alice/repo8": ["1 high finding holds releases", "unread", false], | |
| 92 | }); | |
| 93 | expect(readAllFlows(app.db, { kind: "scan", stale: true }, NOW).map((l) => l.repo)).toEqual(["alice/repo7"]); | |
| 94 | }); | |
| 95 | ||
| 96 | test("a site admin reads it; everyone else gets 404", async () => { | |
| 97 | const ok = await fetch(`${servers.url}/admin/flows?kind=upstream&stale=1`, { headers: { cookie: await signedIn("alice") } }); | |
| 98 | const body = await ok.text(); | |
| 99 | expect(ok.status).toBe(200); | |
| 100 | expect([...body.matchAll(/alice\/repo\d+ · (\w+)/g)].map((m) => m[0])).toEqual(["alice/repo1 · upstream"]); | |
| 101 | expect(body).not.toContain("u:secret"); | |
| 102 | expect(body).not.toMatch(/<div class="card[^"]*">\s*<form/); | |
| 103 | expect((await fetch(`${servers.url}/admin/flows`, { headers: { cookie: await signedIn("bob") } })).status).toBe(404); | |
| 104 | expect((await fetch(`${servers.url}/admin/flows`)).status).toBe(404); | |
| 105 | }); | |
| 106 | ||
| 107 | test("the nav highlights Flows on the page", async () => { | |
| 108 | const body = await (await fetch(`${servers.url}/admin/flows`, { headers: { cookie: await signedIn("alice") } })).text(); | |
| 109 | expect(body).toMatch(/href="\/admin\/flows"[^>]*aria-current/); | |
| 110 | }); | |
| 111 | ||
| 112 | test("the page at 300 repositories costs a fixed handful of statements", async () => { | |
| 113 | const cookie = await signedIn("alice"); | |
| 114 | const small = await requestStatements(app.db, `${servers.url}/admin/flows?kind=fork`, { headers: { cookie } }); | |
| 115 | const full = await requestStatements(app.db, `${servers.url}/admin/flows`, { headers: { cookie } }); | |
| 116 | expect(full).toBe(small); | |
| 117 | expectWork(full, { max: 30 }); | |
| 118 | }); | |
| 119 | ||
| 120 | const identity = (user?: User) => ({ sshUsername: user?.username ?? "git", remoteAddress: "127.0.0.1", fingerprint: "SHA256:test", user }); | |
| 121 | const userOf = (name: string) => app.db.query<{ id: string }, [string]>("SELECT id FROM user WHERE username = ?1").get(name)!.id; | |
| 122 | ||
| 123 | test("the terminal gives an admin person every line and nobody else anything", async () => { | |
| 124 | const get = async (name: string): Promise<User> => { | |
| 125 | const { getUserById } = await import("../src/core/users.ts"); | |
| 126 | return getUserById(app.db, userOf(name))!; | |
| 127 | }; | |
| 128 | const admin = new Session(app, identity(await get("alice"))); | |
| 129 | const lines = flowLines(admin, "mirror", false, NOW); | |
| 130 | expect(lines[0]).toContain("Repository"); | |
| 131 | expect(lines.slice(1).map((l) => l.split(/\s{2,}/).slice(0, 3))).toEqual([ | |
| 132 | ["alice/repo3", "mirror", "mirrored"], | |
| 133 | ["alice/repo4", "mirror", "last push failed"], | |
| 134 | ]); | |
| 135 | expect(flowLines(new Session(app, identity(await get("bob"))), undefined, false, NOW)).toEqual([]); | |
| 136 | expect(flowLines(new Session(app, identity(undefined)), undefined, false, NOW)).toEqual([]); | |
| 137 | const { app: tui, session } = createForgeApp(app, identity(await get("alice")), { terminal: headlessTerminal(80, 24), exitHooks: false }); | |
| 138 | const { forgeCommands } = await import("../src/tui/forge.ts"); | |
| 139 | expect(forgeCommands(session)(tui.ctx).map((c) => c.id)).toContain("flows-all"); | |
| 140 | tui.stop?.(); | |
| 141 | }); | |
| 142 | ||
| 143 | test("reads 300 repositories in a fixed handful of statements, never one per repository", async () => { | |
| 144 | // Seven set-based reads, whatever the number of repositories (#7824: work, not a clock). | |
| 145 | expectWork(await statements(app.db, () => void readAllFlows(app.db, {}, NOW)), { max: 7 }); | |
| 146 | }); |
test/palette-roster.test.ts +1
| @@ -124,6 +124,7 @@ test("a site-admin person is offered the whole roster", () => { | ||
| 124 | 124 | ["people", "People", "who is a guest, a member or an admin; promote or demote"], |
| 125 | 125 | ["quotas", "Account quotas", "what one account may own, and who changed it"], |
| 126 | 126 | ["limits", "Limits", "what the forge caps, with default and bounds; change one"], |
| 127 | ["flows-all", "Flows across the forge", "every repository's flow lines, by kind and stale"], | |
| 127 | 128 | ["teams", "Crew teams", "each team's charter, dial and members"], |
| 128 | 129 | ["agents", "Crew agents", "each agent's model, budget and brief"], |
| 129 | 130 | ["invites", "Invite links", "make a single-use link, see which are open, revoke one"], |
Both doors: this list has no terminal screen in src/tui/ and no docs/features/admin-flows.md. Add both; the TUI must gate reads to site admins as the page does.