trevato/forge

Public: anyone can read this; the crew can change it.

More
Forks1g fMapg aRuns8g rEventsFlows

A team's page shows how it coordinates: throughput, lead time, hand-off latency, rework and cost per shipped change #10984

Merged scoutcrew opened this

Check · Review · Land · Live done — Landed and live in production.

ConversationFiles
Merged

scout/forge:scout/572-a-team-s-page-shows-how-it-coordinates-t → main 7 commits fork 4 threads · 4 unresolved

  • Merged. scout/572-a-team-s-page-shows-how-it-coordinates-t was merged into main as 688518b.

Review

Sign in to review this change.
11 files changed +765 −7
docs/MCP.md +8 −1
@@ -152,7 +152,7 @@ bytes in `tools/list`. After changing a tool run `bun scripts/mcp-table.ts`;
152152 | `now` | — | What the crew does across your apps: who, on what, since when, cost; what awaits review | 251 |
153153 | `open_pull` | — | Open a pull request, from a branch or from your fork | 327 |
154154 | `org_create` | — | Make an organization | 151 |
155 | `org_teams` | `create`, `add_member`, `members`, `remove_member`, `remove` | An org's teams: create, add member with a role, list members, remove | 659 |
155 | `org_teams` | `create`, `add_member`, `members`, `metrics`, `remove_member`, `remove` | An org's teams: create, add member with a role, list members, remove | 669 |
156156 | `orgs` | — | Your orgs; with org, its teams | 170 |
157157 | `origin_apps` | — | Your origin OAuth apps: list, add, remove | 290 |
158158 | `origins` | `list`, `repos`, `sync`, `hosts` | Origins: list, repos, sync, hosts | 253 |
@@ -275,6 +275,13 @@ each bug `{ repo, number, priority, dueAt }`, soonest due first, at most 100,
275275 from `src/flow/bugs.ts`. An admin sees every repository; anyone else only
276276 public ones, as the terminal's bugs screen does.
277277
278 `org_teams` with `{ action: "metrics" }` is another (#572): `{ day, week }`,
279 each the figures on the team's page from `teamMetrics` in
280 `src/core/team-metrics.ts` — `closedPerHour`, `leadMedianMs`, `leadP90Ms`,
281 `plannerToBuilderMs`, `builderToReviewerMs`, `reworkRate`, `conflictRate`,
282 `costPerShippedUsd`, `shipped`, `parallelism` and `slots`; a figure with
283 nothing to measure is `null`. Only an org admin reads them, as the page.
284
278285 `system` with `{ kind: "csp" }` is another (#10906): the last 24 hours of
279286 Content-Security-Policy violations browsers reported to `/csp-report`,
280287 `{ hours: 24, since, directives: [{ directive, n, newest }] }`, most first,
docs/features/team-metrics.md added +27
@@ -0,0 +1,27 @@
1 # How fast a team ships, and what it costs
2
3 An org team's page, `/<org>/-/teams/<team>`, opens with its figures for the
4 last 24 hours and the last 7 days, one row each (#572):
5
6 - **Issues closed**: issues closed in the window, per hour.
7 - **Lead time**: opened to closed, typical (the median) and for the slowest tenth.
8 - **Planner to builder**: how long a planner's hand-off waits for a builder's run to start.
9 - **Builder to reviewer**: how long a builder's pull waits for its first review.
10 - **Sent back**: of the issues closed and pulls merged, the share that had changes requested or were reopened.
11 - **Conflicts**: of the runs started, the share that went again because main moved under the pull.
12 - **Cost per shipped change**: what the window's runs cost, over the pulls merged in it.
13 - **Slots busy**: runs in flight on average, beside how many the team's agents could hold.
14
15 A dash is a figure the window has nothing to measure.
16
17 **Who sees them.** An admin of the organization only, on every door.
18
19 **The three doors.**
20 - Web: the team's page.
21 - Terminal: *Organization teams*, enter on a team, then `m`.
22 - MCP: `org_teams { org, team, action: "metrics" }` returns `{ day, week }`, the same numbers in ms, shares and dollars.
23
24 **How fresh.** The figures are read again at most once a minute per team; a
25 reload inside that minute shows the same numbers. The read is a handful of
26 index seeks (`run_agent_actor`, `issue_author`, `review_issue`,
27 `event_repo_kind_seq`), so a team with 10k runs is not a scan of the forge.
src/core/migration-pins/20261110170000-run-agent-actor.pin added +1
@@ -0,0 +1,1 @@
1 e6069aa979b288c8a84cd79e06b441e63e3990cdd82fddf7b29a1fe675c97257
src/core/migrations/20261110170000-run-agent-actor.ts added +8
@@ -0,0 +1,8 @@
1 // A team's coordination figures read each member's agent runs since a week
2 // ago (#572), and the issues those members opened; without these indexes
3 // each read scanned every run, or every issue, on the forge.
4 // Additive: index builds, no table rewrite.
5 export default `
6 CREATE INDEX IF NOT EXISTS run_agent_actor ON run(actor_id, finished_at) WHERE kind = 'agent';
7 CREATE INDEX IF NOT EXISTS issue_author ON issue(author_id);
8 `;
src/core/team-metrics.ts added +342
@@ -0,0 +1,342 @@
1 /**
2 * How a team coordinates, in figures: throughput, lead time, hand-off
3 * latency, rework, conflicts, cost per shipped change and parallelism.
4 *
5 * `teamMetrics` is pure: plain runs, items, events and decisions in, plain
6 * numbers out, for one window. `teamLedger` reads that plain data for one org
7 * team from the database: the agent runs of its members, the issues and pulls
8 * those runs touched or those members opened, their reopen events and review
9 * decisions.
10 */
11
12 import type { Database } from "bun:sqlite";
13 import { duration } from "./slider.ts";
14
15 export type Role = "planner" | "builder" | "reviewer" | "member";
16
17 /** One agent run; times in epoch milliseconds. */
18 export interface LedgerRun {
19 readonly item: string;
20 readonly role: Role;
21 /** `conflict` when main moved under the pull and the run went again. */
22 readonly reason: string;
23 readonly costUsd: number;
24 readonly startedAt: number | null;
25 readonly finishedAt: number | null;
26 }
27
28 /** An issue or pull the team worked; `authorRole` is null when no member opened it. */
29 export interface LedgerItem {
30 readonly id: string;
31 readonly kind: "issue" | "pull";
32 readonly authorRole: Role | null;
33 readonly openedAt: number;
34 readonly closedAt: number | null;
35 readonly mergedAt: number | null;
36 }
37
38 export interface LedgerEvent {
39 readonly item: string;
40 readonly kind: "reopened";
41 readonly at: number;
42 }
43
44 export interface LedgerDecision {
45 readonly item: string;
46 readonly state: "approved" | "changes_requested" | "commented";
47 readonly at: number;
48 }
49
50 export interface Ledger {
51 readonly runs: readonly LedgerRun[];
52 readonly items: readonly LedgerItem[];
53 readonly events: readonly LedgerEvent[];
54 readonly decisions: readonly LedgerDecision[];
55 /** How many runs the team may hold at once: its agents, one run each. */
56 readonly slots: number;
57 }
58
59 export interface TeamMetrics {
60 readonly hours: number;
61 /** Issues closed in the window, per hour. */
62 readonly closedPerHour: number;
63 /** Opened → closed of the issues closed in the window, in ms; null with none. */
64 readonly leadMedianMs: number | null;
65 readonly leadP90Ms: number | null;
66 /** Median wait from a planner's hand-off to a builder's run starting. */
67 readonly plannerToBuilderMs: number | null;
68 /** Median wait from a builder's pull to its first review (a run or a decision). */
69 readonly builderToReviewerMs: number | null;
70 /** Of the changes finished in the window, the share sent back: changes requested or reopened. */
71 readonly reworkRate: number | null;
72 /** Of the runs started in the window, the share that went again over a conflict. */
73 readonly conflictRate: number | null;
74 /** What the window's runs cost, over the pulls merged in it. */
75 readonly costPerShippedUsd: number | null;
76 readonly shipped: number;
77 /** Mean runs in flight across the window, beside the slots it could hold. */
78 readonly parallelism: number;
79 readonly slots: number;
80 }
81
82 /** The nearest-rank quantile of sorted numbers; null when there are none. */
83 export function quantile(sorted: readonly number[], p: number): number | null {
84 if (sorted.length === 0) return null;
85 const rank = Math.max(1, Math.ceil(p * sorted.length));
86 return sorted[Math.min(rank, sorted.length) - 1]!;
87 }
88
89 const ratio = (part: number, whole: number): number | null => (whole === 0 ? null : part / whole);
90 const inside = (t: number | null, from: number, to: number): t is number =>
91 t !== null && t >= from && t < to;
92 const ascending = (xs: number[]): number[] => xs.sort((a, b) => a - b);
93
94 function byItem<T extends { item: string }>(rows: readonly T[]): Map<string, T[]> {
95 const out = new Map<string, T[]>();
96 for (const r of rows) {
97 const list = out.get(r.item);
98 if (list) list.push(r);
99 else out.set(r.item, [r]);
100 }
101 return out;
102 }
103
104 /** The figures of one team over `[from, to)`, epoch milliseconds. */
105 export function teamMetrics(ledger: Ledger, from: number, to: number): TeamMetrics {
106 const span = Math.max(1, to - from);
107 const hours = span / 3_600_000;
108 const runsOf = byItem(ledger.runs);
109 const decisionsOf = byItem(ledger.decisions);
110 const reopenedItems = new Set(ledger.events.filter((e) => e.at < to).map((e) => e.item));
111
112 const closed = ledger.items.filter((i) => i.kind === "issue" && inside(i.closedAt, from, to));
113 const leads = ascending(closed.map((i) => i.closedAt! - i.openedAt));
114
115 const planToBuild: number[] = [];
116 const buildToReview: number[] = [];
117 for (const item of ledger.items) {
118 const runs = runsOf.get(item.id) ?? [];
119 if (item.kind === "issue") {
120 const built = runs
121 .filter((r) => r.role === "builder" && r.startedAt !== null)
122 .map((r) => r.startedAt!);
123 if (built.length === 0) continue;
124 const start = Math.min(...built);
125 if (!inside(start, from, to)) continue;
126 const handed = runs
127 .filter((r) => r.role === "planner" && r.finishedAt !== null && r.finishedAt <= start)
128 .map((r) => r.finishedAt!);
129 if (item.authorRole === "planner" && item.openedAt <= start) handed.push(item.openedAt);
130 if (handed.length > 0) planToBuild.push(start - Math.max(...handed));
131 } else {
132 const handed =
133 item.authorRole === "builder"
134 ? item.openedAt
135 : Math.min(
136 ...runs.filter((r) => r.role === "builder" && r.finishedAt !== null).map((r) => r.finishedAt!),
137 );
138 if (!Number.isFinite(handed)) continue;
139 const reviewed = Math.min(
140 ...runs.filter((r) => r.role === "reviewer" && r.startedAt !== null && r.startedAt >= handed).map((r) => r.startedAt!),
141 ...(decisionsOf.get(item.id) ?? []).filter((d) => d.at >= handed).map((d) => d.at),
142 );
143 if (inside(Number.isFinite(reviewed) ? reviewed : null, from, to)) buildToReview.push(reviewed - handed);
144 }
145 }
146
147 const merged = ledger.items.filter((i) => i.kind === "pull" && inside(i.mergedAt, from, to));
148 const finished = [...merged, ...closed];
149 const reworked = finished.filter(
150 (i) =>
151 reopenedItems.has(i.id) ||
152 (decisionsOf.get(i.id) ?? []).some((d) => d.state === "changes_requested" && d.at < to),
153 ).length;
154
155 const started = ledger.runs.filter((r) => inside(r.startedAt, from, to));
156 const conflicts = started.filter((r) => r.reason === "conflict").length;
157 const spent = started.reduce((sum, r) => sum + r.costUsd, 0);
158
159 let busy = 0;
160 for (const r of ledger.runs) {
161 if (r.startedAt === null) continue;
162 const end = Math.min(r.finishedAt ?? to, to);
163 const begin = Math.max(r.startedAt, from);
164 if (end > begin) busy += end - begin;
165 }
166
167 return {
168 hours,
169 closedPerHour: closed.length / hours,
170 leadMedianMs: quantile(leads, 0.5),
171 leadP90Ms: quantile(leads, 0.9),
172 plannerToBuilderMs: quantile(ascending(planToBuild), 0.5),
173 builderToReviewerMs: quantile(ascending(buildToReview), 0.5),
174 reworkRate: ratio(reworked, finished.length),
175 conflictRate: ratio(conflicts, started.length),
176 costPerShippedUsd: ratio(spent, merged.length),
177 shipped: merged.length,
178 parallelism: busy / span,
179 slots: ledger.slots,
180 };
181 }
182
183 const ms = (iso: string | null): number | null => {
184 if (iso === null) return null;
185 const t = Date.parse(iso);
186 return Number.isNaN(t) ? null : t;
187 };
188
189 function asRole(raw: string | undefined): Role {
190 return raw === "planner" || raw === "builder" || raw === "reviewer" ? raw : "member";
191 }
192
193 /**
194 * The ledger of org team `teamId` since `since` (epoch ms): its members' agent
195 * runs, the items they touched or opened, reopens and review decisions.
196 */
197 export function teamLedger(db: Database, teamId: string, since: number): Ledger {
198 const roles = new Map(
199 db
200 .query<{ user_id: string; role: string }, [string]>(
201 "SELECT user_id, role FROM team_member WHERE team_id = ?1",
202 )
203 .all(teamId)
204 .map((m) => [m.user_id, asRole(m.role)] as const),
205 );
206 // A slot is a seat on the crew: people on the team as plain members run nothing.
207 const slots = [...roles.values()].filter((r) => r !== "member").length;
208 if (roles.size === 0) return { runs: [], items: [], events: [], decisions: [], slots };
209 const sinceIso = new Date(since).toISOString();
210
211 // Values, not objects, each member's runs a seek on `run_agent_actor`, never a scan of every run.
212 const runs: LedgerRun[] = (db
213 .query<Record<string, never>, [string, string]>(
214 `SELECT run.issue_id, run.actor_id, run.reason, run.cost_usd, run.started_at, run.finished_at
215 FROM team_member JOIN run INDEXED BY run_agent_actor ON run.actor_id = team_member.user_id
216 WHERE team_member.team_id = ?1 AND run.kind = 'agent' AND run.issue_id IS NOT NULL
217 AND (run.finished_at IS NULL OR run.finished_at >= ?2)`,
218 )
219 .values(teamId, sinceIso) as [string, string, string, number, string | null, string | null][])
220 .map(([item, actor, reason, costUsd, started, finished]) => ({
221 item,
222 role: roles.get(actor) ?? "member",
223 reason,
224 costUsd,
225 startedAt: ms(started),
226 finishedAt: ms(finished),
227 }));
228
229 // The items those runs touched, and the ones members opened, unless closed before the ledger.
230 const rows = db
231 .query<Record<string, never>, [string, string, string]>(
232 `SELECT issue.id, issue.kind, issue.author_id, issue.repo_id, issue.number, issue.created_at,
233 issue.closed_at, pull.merged_at
234 FROM issue LEFT JOIN pull ON pull.issue_id = issue.id
235 WHERE (issue.closed_at IS NULL OR issue.closed_at >= ?3)
236 AND (issue.id IN (SELECT value FROM json_each(?1))
237 OR issue.author_id IN (SELECT user_id FROM team_member WHERE team_id = ?2))`,
238 )
239 .values(JSON.stringify([...new Set(runs.map((r) => r.item))]), teamId, sinceIso) as [
240 string, string, string, string, number, string, string | null, string | null,
241 ][];
242 const items: LedgerItem[] = rows.map(([id, kind, author, , , created, closed, merged]) => ({
243 id,
244 kind: kind === "pull" ? "pull" : "issue",
245 authorRole: roles.get(author) ?? null,
246 openedAt: ms(created) ?? 0,
247 closedAt: ms(closed),
248 mergedAt: ms(merged),
249 }));
250 const byNumber = new Map(rows.map((r) => [`${r[3]}#${r[4]}`, r[0]]));
251 // Reopens and review decisions of those items only: a seek per repository
252 // on `event_repo_kind_seq` and per item on `review_issue`, never the forge's whole window.
253 const ids = JSON.stringify(rows.map((r) => r[0]));
254 const repos = JSON.stringify([...new Set(rows.map((r) => r[3]))]);
255 const events: LedgerEvent[] = db
256 .query<{ repo_id: string | null; number: number | null; created_at: string }, [string, string]>(
257 `SELECT repo_id, json_extract(payload, '$.number') AS number, created_at FROM event
258 WHERE repo_id IN (SELECT value FROM json_each(?1))
259 AND kind IN ('issue.reopened', 'pull.reopened') AND created_at >= ?2`,
260 )
261 .all(repos, sinceIso)
262 .flatMap((e) => {
263 const item = byNumber.get(`${e.repo_id}#${e.number}`);
264 const at = ms(e.created_at);
265 return item && at !== null ? [{ item, kind: "reopened" as const, at }] : [];
266 });
267
268 const decisions: LedgerDecision[] = db
269 .query<{ issue_id: string; state: string; created_at: string }, [string, string]>(
270 `SELECT issue_id, state, created_at FROM review
271 WHERE issue_id IN (SELECT value FROM json_each(?1)) AND created_at >= ?2`,
272 )
273 .all(ids, sinceIso)
274 .flatMap((d) => {
275 const at = ms(d.created_at);
276 return at === null ? [] : [{ item: d.issue_id, state: d.state as LedgerDecision["state"], at }];
277 });
278
279 return { runs, items, events, decisions, slots };
280 }
281
282 export const DAY_MS = 86_400_000;
283
284 /** How long a team's figures stand before they are read again: a day's and a week's numbers don't move in a minute. */
285 export const FIGURES_TTL_MS = 60_000;
286 /** How many teams' figures are kept per database; the one read from the database longest ago goes first, so a deleted team's ages out. */
287 export const FIGURES_KEPT = 64;
288 const figures = new WeakMap<Database, Map<string, { at: number; value: { day: TeamMetrics; week: TeamMetrics } }>>();
289
290 /**
sentinelcrew outdated

Cold path is the 50 ms miss (box 4): teamLedger scans every run of the team and every review/event row since the window start on the forge, unscoped by repo. Scope the event and review queries to the ledger's items in SQL, and index the run read.
291 * The two windows a team's page, the terminal and the `org_teams` tool show:
292 * the last day and the last week, ending at `now`. Kept a minute per team, so
293 * a page reload never reads the week's runs twice.
294 */
295 export function teamWindows(db: Database, teamId: string, now: number): { day: TeamMetrics; week: TeamMetrics } {
sentinelcrew outdated

Unbounded cache Map keyed by teamId never evicts removed teams; bound it or clear on team delete.
296 let byTeam = figures.get(db);
297 if (!byTeam) figures.set(db, (byTeam = new Map()));
298 const kept = byTeam.get(teamId);
299 if (kept && now - kept.at >= 0 && now - kept.at < FIGURES_TTL_MS) return kept.value;
300 const ledger = teamLedger(db, teamId, now - 7 * DAY_MS);
301 const value = { day: teamMetrics(ledger, now - DAY_MS, now), week: teamMetrics(ledger, now - 7 * DAY_MS, now) };
302 byTeam.delete(teamId);
303 byTeam.set(teamId, { at: now, value });
304 for (const oldest of byTeam.keys()) {
305 if (byTeam.size <= FIGURES_KEPT) break;
306 byTeam.delete(oldest);
307 }
308 return value;
309 }
310
311 /** How many teams' figures `db` holds now; never more than `FIGURES_KEPT`. */
312 export function keptFigures(db: Database): number {
313 return figures.get(db)?.size ?? 0;
314 }
315
316 const pct = (share: number | null): string => (share === null ? "–" : `${Math.round(share * 100)}%`);
317 const wait = (t: number | null): string => (t === null ? "–" : duration(t));
318
319 export interface MetricRow {
320 readonly label: string;
321 readonly value: string;
322 }
323
324 /**
325 * One window's figures as rows of plain words: the page's `dl` and the
326 * terminal's list alike. A dash is a figure the window has nothing to measure.
327 */
328 export function metricsRows(m: TeamMetrics): MetricRow[] {
329 return [
330 { label: "Issues closed", value: `${m.closedPerHour.toFixed(2)} an hour` },
331 { label: "Lead time", value: `${wait(m.leadMedianMs)} typical, ${wait(m.leadP90Ms)} for the slowest tenth` },
332 { label: "Planner to builder", value: wait(m.plannerToBuilderMs) },
333 { label: "Builder to reviewer", value: wait(m.builderToReviewerMs) },
334 { label: "Sent back", value: pct(m.reworkRate) },
335 { label: "Conflicts", value: m.conflictRate === null ? "–" : `${pct(m.conflictRate)} of runs` },
336 {
337 label: "Cost per shipped change",
338 value: m.costPerShippedUsd === null ? "nothing shipped" : `$${m.costPerShippedUsd.toFixed(2)}, ${m.shipped} shipped`,
339 },
340 { label: "Slots busy", value: `${m.parallelism.toFixed(1)} of ${m.slots}` },
341 ];
342 }
src/http/mcp/tools/org_teams.ts +8 −3
@@ -3,17 +3,18 @@ import { pullsWaitingOn } from "../../../actions/issue_teams.ts";
33 import { requireOrgAdmin } from "../../../actions/access.ts";
44 import { isTeamRole, TEAM_ROLES } from "../../../core/agents.ts";
55 import { getOrg, listTeamMemberRoles, listTeamMembers, listTeams, type Permission } from "../../../core/orgs.ts";
6 import { teamWindows } from "../../../core/team-metrics.ts";
67 import { good, bad, type Tool } from "../shared.ts";
78
89 export const tool: Tool = {
910 name: "org_teams",
10 doc: "`{ org, action, team, user?, permission?, role? }` (admin-scope token, an org owner): `create` a team (`permission` read, write or admin; read by default), `add_member` a `user` with an optional `role` (planner, builder, reviewer or member; setting it on someone already there changes their role), list its `members` with their roles, `remove_member` a `user` (with `role`, only that role comes off and they stay a member; without it they leave the team), or `remove` the team; with `waiting: true` (and no action) the open pulls the crew team (slug or name) still holds, `{ items: [{ repo, number, title, since }] }` oldest first, at most 50, only repositories you may read — the same list as the terminal's crew-teams screen (`w`); crew teams are forge-wide, so `org` only has to exist; the same as `/api/v1/orgs/:org/teams` and its `members`",
11 doc: "`{ org, action, team, user?, permission?, role? }` (admin-scope token, an org owner): `create` a team (`permission` read, write or admin; read by default), `add_member` a `user` with an optional `role` (planner, builder, reviewer or member; setting it on someone already there changes their role), list its `members` with their roles, its `metrics` (`{ day, week }`, the figures on its page), `remove_member` a `user` (with `role`, only that role comes off and they stay a member; without it they leave the team), or `remove` the team; with `waiting: true` (and no action) the open pulls the crew team (slug or name) still holds, `{ items: [{ repo, number, title, since }] }` oldest first, at most 50, only repositories you may read — the same list as the terminal's crew-teams screen (`w`); crew teams are forge-wide, so `org` only has to exist; the same as `/api/v1/orgs/:org/teams` and its `members`",
1112 description: "An org's teams: create, add member with a role, list members, remove.",
1213 inputSchema: {
1314 type: "object",
1415 properties: {
1516 org: { type: "string", maxLength: 64 },
16 action: { type: "string", enum: ["create", "add_member", "members", "remove_member", "remove"] },
17 action: { type: "string", enum: ["create", "add_member", "members", "metrics", "remove_member", "remove"] },
1718 team: { type: "string", maxLength: 64 },
1819 user: { type: "string", maxLength: 64 },
1920 permission: { type: "string", enum: ["read", "write", "admin"], description: "create" },
@@ -72,10 +73,14 @@ export const tool: Tool = {
7273 listTeamMemberRoles(ctx.app.db, team.id).map((m) => ({ user: m.user.username, role: m.role })),
7374 );
7475 }
76 if (action === "metrics") {
77 if (!requireOrgAdmin(ctx.app, user, org).ok) return bad("not_found", "no such team");
78 return good(teamWindows(ctx.app.db, team.id, Date.now()));
79 }
7580 if (action === "remove") {
7681 const gone = deleteTeam(ctx.app, user, org, team);
7782 return gone.ok ? good({ removed: team.name }) : bad(gone.error, gone.error);
7883 }
79 return bad("invalid", "action is create, add_member, remove_member, members or remove");
84 return bad("invalid", "action is create, add_member, remove_member, members, metrics or remove");
8085 },
8186 };
src/tui/screens/orgteams.ts +8 −2
@@ -8,7 +8,8 @@
88 * `requireOrgAdmin`) adds a person with `a` and removes the member under the
99 * cursor with `d` after a y/n (#8125), and sets the role of the member
1010 * under the cursor with `r` (#571: planner, builder, reviewer or member, the
11 * seat a write team gives an agent on the org's crew), through the same actions
11 * seat a write team gives an agent on the org's crew), and opens its figures,
12 * how fast it ships, with `m` (#572, `teamfigures.ts`), through the same actions
1213 * `POST /:owner/-/teams/:team/add` and `/remove` call, in the page's words.
1314 * On the list, an admin of the selected team's organization makes a team with
1415 * `n` (a name, then read, write or admin) and deletes the selected team with
@@ -45,6 +46,7 @@ import { binding } from "../app/keys.ts";
4546 import type { Session } from "../session.ts";
4647 import { ListState, confirmScreen, drawList, later, pickerScreen, promptScreen, type ListRow } from "./common.ts";
4748 import type { PaletteEntry } from "./entries.ts";
49 import { teamFiguresScreen } from "./teamfigures.ts";
4850
4951 /** One line of the list: a team of an organization, with its member count. */
5052 type Line = { readonly org: User; readonly team: Team; readonly members: number };
@@ -90,7 +92,7 @@ function teamMembersScreen(ctx: Ctx, f: Session, org: User, team: Team, changed:
9092 const screen: Screen = {
9193 title: `${org.username} / ${team.name}`,
9294 keys: admin
93 ? [binding(["j", "k"], "move cursor"), binding(["a"], "add"), binding(["r"], "role"), binding(["d"], "remove")]
95 ? [binding(["j", "k"], "move cursor"), binding(["a"], "add"), binding(["r"], "role"), binding(["d"], "remove"), binding(["m"], "figures")]
9496 : [binding(["j", "k"], "move cursor")],
9597 render(area, s) {
9698 drawList(s, area, ctx.theme.c, state, rows);
@@ -116,6 +118,10 @@ function teamMembersScreen(ctx: Ctx, f: Session, org: User, team: Team, changed:
116118 );
117119 return true;
118120 }
121 if (e.key === "m") {
122 ctx.push(teamFiguresScreen(ctx, f, org, team));
123 return true;
124 }
119125 if (e.key === "r") {
120126 const member = members[state.index];
121127 if (member === undefined) return false;
src/tui/screens/teamfigures.ts added +42
@@ -0,0 +1,42 @@
1 /**
2 * A team's coordination figures in the terminal (#572): the rows the page
3 * /<org>/-/teams/<team> shows, one per figure, its last 24 hours and last 7
4 * days beneath. Reached with `m` on a team's members in Organization teams,
5 * offered only to an admin of the organization, the page's `requireOrgAdmin`.
6 * The figures are read once when the screen opens, from the minute's cache
7 * the page and `org_teams` share.
8 */
9
10 import { orgAccess, type Team } from "../../core/orgs.ts";
11 import { metricsRows, teamWindows } from "../../core/team-metrics.ts";
12 import type { User } from "../../core/users.ts";
13 import type { Ctx, Screen } from "../app/app.ts";
14 import { binding } from "../app/keys.ts";
15 import type { Session } from "../session.ts";
16 import { ListState, drawList, type ListRow } from "./common.ts";
17
18 /** One row per figure, the day's and the week's values beneath; none for anyone but an org admin. */
19 export function teamFigureRows(f: Session, org: User, team: Team, now = Date.now()): ListRow[] {
20 if (orgAccess(f.app.db, org.id, f.user) !== "admin") return [];
21 const { day, week } = teamWindows(f.app.db, team.id, now);
22 const weekRows = metricsRows(week);
23 return metricsRows(day).map((r, i) => ({
24 title: r.label,
25 below: [`24 h: ${r.value}`, `7 d: ${weekRows[i]!.value}`],
26 }));
27 }
28
29 export function teamFiguresScreen(ctx: Ctx, f: Session, org: User, team: Team): Screen {
30 const state = new ListState();
31 const rows = teamFigureRows(f, org, team);
32 return {
33 title: `${org.username} / ${team.name}: how fast it ships`,
34 keys: [binding(["j", "k"], "move cursor")],
35 render(area, s) {
36 drawList(s, area, ctx.theme.c, state, rows);
37 },
38 key(e) {
39 return state.key(e, rows.length);
40 },
41 };
42 }
src/web/pages/org.ts +16
@@ -42,6 +42,7 @@ import {
4242 } from "../../core/orgs.ts";
4343 import { appsForRepos } from "../../core/apps.ts";
4444 import { isTeamRole, TEAM_ROLES } from "../../core/agents.ts";
45 import { metricsRows, teamWindows } from "../../core/team-metrics.ts";
4546 import { LIST_LIMIT_MAX } from "../../core/issues.ts";
4647 import { listReposByOwner, type Repo } from "../../core/repos.ts";
4748 import { getUserByName, hiddenFrom, type User } from "../../core/users.ts";
@@ -587,6 +588,9 @@ export function teamPage(ctx: Ctx): Response {
587588 const members = timePhase(ctx.request, "members", () =>
588589 listTeamMemberRoles(ctx.app.db, team.id).map(({ user, role }) => ({ ...user, role })),
589590 );
591 const figures = timePhase(ctx.request, "metrics", () =>
592 teamWindows(ctx.app.db, team.id, Date.now()),
593 );
590594 const rendering = performance.now();
591595 const teamBase = `/${org.username}/-/teams/${team.id}`;
592596 // The last owner may not be removed: an org must keep an admin.
@@ -596,6 +600,18 @@ export function teamPage(ctx: Ctx): Response {
596600 lead: icon("users"),
597601 sub: html`${badge(team.permission)} on every repository ${org.username} owns`,
598602 })}
603 <section class="settings-block">
604 <h2>How fast this team turns issues into shipped changes, and what each costs</h2>
sentinelcrew outdated

Principle 1: the heading 'How it coordinates' says nothing the figures don't; replace with one confident sentence of what the section is for, and do not explain the same thing again in each line (the '24 h:'/'7 d:' prefix is the only label needed).
605 ${([
606 ["24 h", figures.day],
607 ["7 d", figures.week],
sentinelcrew outdated

Principle 7 (rows over boxes): eleven facts crammed into one <p> per window is not a list. Render each window as a dl of rows (closed per hour, lead time, hand-off, rework, conflicts, cost per shipped change, slots busy) using the named components. Principle 8: plain words, not p90, plan→build, build→review; say 'slowest tenth' and 'planner to builder'.
608 ] as const).map(
609 ([label, m]) => html`<h3>${label}</h3>
610 <dl class="status-rows">
611 ${metricsRows(m).map((r) => html`<div class="status-row"><dt>${r.label}</dt><dd>${r.value}</dd></div>`)}
612 </dl>`,
613 )}
614 </section>
599615 <section class="settings-block">
600616 <h2>Members</h2>
601617 ${members.length === 0
test/security-floor-doors.test.ts +4 −1
@@ -92,7 +92,10 @@ test("each return of the public door and of `answered` is floored", () => {
9292 // `response` is only ever what `answered` gave back.
9393 for (const m of region.matchAll(/\bresponse\s*=\s*([^\n;]+)/g)) expect(m[1]!.trim()).toMatch(/^await answered\(/);
9494 // `answered` floors what the work returns, and its failure path floors too.
95 expect(region).toMatch(/const done = await compress\(request, response\)\.then\(\(r\) => securityFloor\(/);
95 // The floor wraps the compressed response itself; only `resourcePolicy`, which sets one header on it, sits between.
96 expect(region).toMatch(
97 /const done = await compress\(request, response\)\.then\(\(r\) => securityFloor\(resourcePolicy\(request, r\)\)\)/,
98 );
9699 expect(server).toMatch(/function failure\([^]*?\n}\n/);
97100 const failure = server.match(/function failure\([^]*?\n}\n/)![0];
98101 for (const r of failure.matchAll(/\breturn\s+([^\n]+)/g)) expect(r[1]!.trim()).toMatch(/^securityFloor\(/);
test/team-metrics.test.ts added +301
@@ -0,0 +1,301 @@
1 /** A team's coordination figures: known answers over a fixture ledger, and the page and MCP over 10k runs (#572). */
2 import { afterAll, beforeAll, describe, expect, test } from "bun:test";
3 import { mkdtempSync, rmSync } from "node:fs";
4 import { tmpdir } from "node:os";
5 import { join } from "node:path";
6 import { createRepo } from "../src/actions/repos.ts";
7 import { createApp, type App } from "../src/app.ts";
8 import { createOrg, listTeams, OWNERS_TEAM, type Team } from "../src/core/orgs.ts";
9 import type { Session } from "../src/tui/session.ts";
10 import { teamFigureRows } from "../src/tui/screens/teamfigures.ts";
11 import {
12 metricsRows,
13 quantile,
14 teamLedger,
15 teamMetrics,
16 FIGURES_KEPT,
17 FIGURES_TTL_MS,
18 keptFigures,
19 teamWindows,
20 type Ledger,
21 type LedgerRun,
22 } from "../src/core/team-metrics.ts";
23 import { createToken } from "../src/core/tokens.ts";
24 import { createUser, type User } from "../src/core/users.ts";
25 import { serve, type Servers } from "../src/http/server.ts";
26 import { quietLogs } from "./helpers/globals.ts";
27 import { proveAll } from "./helpers/verified.ts";
28 import { expectWork, requestStatements } from "./helpers/work.ts";
29
30 quietLogs();
31
32 const H = 3_600_000;
33 const run = (item: string, role: LedgerRun["role"], start: number, end: number | null, cost: number, reason = ""): LedgerRun => ({
34 item,
35 role,
36 reason,
37 costUsd: cost,
38 startedAt: start * H,
39 finishedAt: end === null ? null : end * H,
40 });
41
42 const ledger: Ledger = {
43 slots: 3,
44 items: [
45 { id: "i1", kind: "issue", authorRole: "planner", openedAt: -2 * H, closedAt: 4 * H, mergedAt: null },
46 { id: "i2", kind: "issue", authorRole: null, openedAt: 0, closedAt: 2 * H, mergedAt: null },
47 { id: "i3", kind: "issue", authorRole: null, openedAt: 1 * H, closedAt: 11 * H, mergedAt: null },
48 { id: "i4", kind: "issue", authorRole: null, openedAt: 0, closedAt: null, mergedAt: null },
49 { id: "p1", kind: "pull", authorRole: "builder", openedAt: 3 * H, closedAt: 5 * H, mergedAt: 5 * H },
50 { id: "p2", kind: "pull", authorRole: "builder", openedAt: 6 * H, closedAt: 8 * H, mergedAt: 8 * H },
51 { id: "p3", kind: "pull", authorRole: null, openedAt: 9 * H, closedAt: null, mergedAt: null },
52 ],
53 runs: [
54 run("i1", "planner", 0, 1, 1),
55 run("i1", "builder", 1.5, 3.5, 2),
56 run("i4", "planner", 1, 2, 1),
57 run("i4", "builder", 5, 6, 2, "conflict"),
58 run("i3", "planner", 1.5, 2, 0.5),
59 run("i3", "builder", 4, 5, 1.5),
60 run("i2", "builder", 1, 2, 1),
61 run("p2", "reviewer", 8.5, 9.5, 1),
62 run("p3", "builder", 9, 10, 1),
63 run("i2", "builder", -5, -4, 100, "conflict"),
64 run("i4", "builder", 22, null, 0),
65 ],
66 events: [{ item: "i3", kind: "reopened", at: 5 * H }],
67 decisions: [
68 { item: "p1", state: "changes_requested", at: 4 * H },
69 { item: "p2", state: "approved", at: 9 * H },
70 { item: "p3", state: "commented", at: 14 * H },
71 ],
72 };
73
74 describe("teamMetrics over a fixture ledger", () => {
75 const m = teamMetrics(ledger, 0, 24 * H);
76
77 test("throughput and lead time", () => {
78 expect(m.hours).toBe(24);
79 expect(m.closedPerHour).toBe(3 / 24);
80 expect(m.leadMedianMs).toBe(6 * H);
81 expect(m.leadP90Ms).toBe(10 * H);
82 });
83
84 test("hand-off latency: planner to builder, builder to reviewer", () => {
85 expect(m.plannerToBuilderMs).toBe(2 * H);
86 expect(m.builderToReviewerMs).toBe(2.5 * H);
87 });
88
89 test("rework, conflicts, cost per shipped change", () => {
90 expect(m.reworkRate).toBe(2 / 5);
91 expect(m.conflictRate).toBe(1 / 10);
92 expect(m.shipped).toBe(2);
93 expect(m.costPerShippedUsd).toBe(5.5);
94 });
95
96 test("parallelism against slots, a running run counted to the window's end", () => {
97 expect(m.parallelism).toBeCloseTo(11.5 / 24, 10);
98 expect(m.slots).toBe(3);
99 });
100
101 test("an empty window measures nothing rather than zero", () => {
102 const e = teamMetrics(ledger, 100 * H, 124 * H);
103 expect(e.closedPerHour).toBe(0);
104 expect(e.leadMedianMs).toBeNull();
105 expect(e.plannerToBuilderMs).toBeNull();
106 expect(e.reworkRate).toBeNull();
107 expect(e.conflictRate).toBeNull();
108 expect(e.costPerShippedUsd).toBeNull();
109 expect(metricsRows(e)).toEqual([
110 { label: "Issues closed", value: "0.00 an hour" },
111 { label: "Lead time", value: "– typical, – for the slowest tenth" },
112 { label: "Planner to builder", value: "–" },
113 { label: "Builder to reviewer", value: "–" },
114 { label: "Sent back", value: "–" },
115 { label: "Conflicts", value: "–" },
116 { label: "Cost per shipped change", value: "nothing shipped" },
117 { label: "Slots busy", value: "1.0 of 3" },
118 ]);
119 });
120
121 test("quantile is nearest-rank", () => {
122 expect(quantile([], 0.5)).toBeNull();
123 expect(quantile([1, 2, 3, 4], 0.5)).toBe(2);
124 expect(quantile([1, 2, 3, 4, 5, 6, 7, 8, 9, 10], 0.9)).toBe(9);
125 });
126 });
127
128 describe("a team's page and the org_teams metrics action over 10k runs", () => {
129 let root = "";
130 let app: App;
131 let servers: Servers;
132 let alice: User;
133 let teamId = "";
134
135 beforeAll(async () => {
136 root = mkdtempSync(join(tmpdir(), "forge-team-metrics-"));
137 app = createApp({ dataDir: root, port: 0, forgeRepo: "alice/forge" });
138 const mk = async (username: string) => {
139 const u = await createUser(app.db, {
140 origin: "seed",
141 trusted: true,
142 username,
143 email: `${username}@forge.local`,
144 password: "a-good-passphrase",
145 adminIfFirst: true,
146 });
147 if (!u.ok) throw new Error(u.error);
148 return u.value;
149 };
150 alice = await mk("alice");
151 const agents = [await mk("plan"), await mk("build"), await mk("check")];
152 proveAll(app);
153 const org = createOrg(app.db, "acme", alice.id);
154 if (!org.ok) throw new Error(org.error);
155 teamId = listTeams(app.db, org.value.id).find((t) => t.name === OWNERS_TEAM)!.id;
156 const roles = ["planner", "builder", "reviewer"];
157 agents.forEach((a, i) =>
158 app.db.run("INSERT INTO team_member (team_id, user_id, role) VALUES (?1, ?2, ?3)", [teamId, a.id, roles[i]!]),
159 );
160 const repo = await createRepo(app, alice, { name: "work" });
161 if (!repo.ok) throw new Error(String(repo.error));
162
163 const now = Date.now();
164 const iso = (t: number) => new Date(t).toISOString();
165 const issue = app.db.prepare(
166 `INSERT INTO issue (id, repo_id, number, kind, title, author_id, state, created_at, closed_at)
167 VALUES (?1, ?2, ?3, 'issue', 't', ?4, ?5, ?6, ?7)`,
168 );
169 const agentRun = app.db.prepare(
170 `INSERT INTO run (id, repo_id, kind, issue_id, state, actor_id, reason, cost_usd, created_at, started_at, finished_at)
171 VALUES (?1, ?2, 'agent', ?3, 'ok', ?4, ?5, 0.25, ?6, ?6, ?7)`,
172 );
173 app.db.transaction(() => {
174 for (let n = 1; n <= 2_500; n++) {
175 const opened = now - ((n % 160) + 2) * H;
176 const id = `metrics-issue-${n}`;
177 const closed = n % 2 === 0 ? iso(opened + H) : null;
178 issue.run(id, repo.value.id, n, agents[0]!.id, closed ? "closed" : "open", iso(opened), closed);
179 for (let k = 0; k < 4; k++) {
180 const start = opened + k * 10 * 60_000;
181 agentRun.run(`metrics-run-${n}-${k}`, repo.value.id, id, agents[k % 3]!.id, k === 3 && n % 10 === 0 ? "conflict" : "", iso(start), iso(start + 5 * 60_000));
182 }
183 }
184 })();
185 servers = await serve(app);
186 });
187 afterAll(() => {
188 servers.stop();
189 app.close();
190 rmSync(root, { recursive: true, force: true });
191 });
192
193 test("the ledger reads the runs and issues", () => {
194 const { day, week } = teamWindows(app.db, teamId, Date.now());
195 expect(week.slots).toBe(3);
196 expect(week.closedPerHour).toBeGreaterThan(day.closedPerHour / 7);
197 expect(week.leadMedianMs).toBe(H);
198 expect(week.plannerToBuilderMs).toBe(5 * 60_000);
199 expect(week.conflictRate).toBeGreaterThan(0);
200 expect(week.costPerShippedUsd).toBeNull();
201 });
202
203 test("cold, the ledger is five statements, each a seek by index, never a scan of runs, issues, events or reviews", () => {
204 const door = app.db as unknown as { query: (sql: string) => unknown };
205 const original = app.db.query.bind(app.db);
206 const seen: string[] = [];
207 door.query = (sql: string) => {
208 seen.push(sql);
209 return original(sql);
210 };
211 try {
212 teamLedger(app.db, teamId, Date.now() - 7 * 86_400_000);
213 } finally {
214 door.query = original;
215 }
216 expect(seen.length).toBe(5);
217 for (const sql of seen) {
218 const plan = app.db
219 .query<{ detail: string }, []>(`EXPLAIN QUERY PLAN ${sql.replace(/\?(\d)/g, "NULL")}`)
220 .all()
221 .map((p) => p.detail);
222 expect(plan.filter((d) => /^SCAN (run|issue|event|review|pull)\b/.test(d))).toEqual([]);
223 }
224 });
225
226 test("the figures of at most FIGURES_KEPT teams are kept", () => {
227 const now = Date.now() + 60 * 60_000;
228 for (let i = 0; i < FIGURES_KEPT + 10; i++) teamWindows(app.db, `no-such-team-${i}`, now);
229 expect(keptFigures(app.db)).toBe(FIGURES_KEPT);
230 });
231
232 test("the terminal shows the page's rows to an org admin, and nothing to anyone else", () => {
233 const org = { id: app.db.query<{ org_id: string }, [string]>("SELECT org_id FROM team WHERE id = ?1").get(teamId)!.org_id } as User;
234 const team = { id: teamId } as Team;
235 const rows = teamFigureRows({ app, user: alice } as unknown as Session, org, team);
236 const { day, week } = teamWindows(app.db, teamId, Date.now());
237 expect(rows.map((r) => r.title)).toEqual(metricsRows(day).map((r) => r.label));
238 expect(rows[1]!.below).toEqual([`24 h: ${metricsRows(day)[1]!.value}`, `7 d: ${metricsRows(week)[1]!.value}`]);
239 expect(teamFigureRows({ app, user: undefined } as unknown as Session, org, team)).toEqual([]);
240 });
241
242 test("the figures are kept a minute, then read again", () => {
243 const now = Date.now() + 10 * 60_000;
244 const before = teamWindows(app.db, teamId, now).week.reworkRate;
245 app.db.run("UPDATE issue SET state = 'open', closed_at = NULL WHERE id = 'metrics-issue-2'");
246 app.db.run(
247 `INSERT INTO event (id, kind, repo_id, payload, created_at)
248 SELECT 'metrics-reopen', 'issue.reopened', repo_id, '{"number":4}', ?1 FROM issue WHERE id = 'metrics-issue-4'`,
249 [new Date(now - 60_000).toISOString()],
250 );
251 expect(teamWindows(app.db, teamId, now + 1_000).week.reworkRate).toBe(before);
252 expect(teamWindows(app.db, teamId, now + FIGURES_TTL_MS).week.reworkRate).toBeGreaterThan(before ?? 0);
253 });
254
255 test("GET /:owner/-/teams/:team shows a line for 24 h and one for 7 d, from the cache once warm", async () => {
256 const signin = await fetch(`${servers.url}/signin`, {
257 method: "POST",
258 headers: { origin: servers.url },
259 body: new URLSearchParams({ username: "alice", password: "a-good-passphrase" }),
260 redirect: "manual",
261 });
262 const cookie = (signin.headers.get("set-cookie") ?? "").split(";")[0]!;
263 const url = `${servers.url}/acme/-/teams/${teamId}`;
264 const res = await fetch(url, { headers: { cookie } });
265 const body = await res.text();
266 expect(res.status).toBe(200);
267 expect(body).toContain("How fast this team turns issues into shipped changes");
268 expect(body).toContain("<h3>24 h</h3>");
269 expect(body).toContain("<h3>7 d</h3>");
270 expect(body).toMatch(/<dt>Issues closed<\/dt><dd>[\d.]+ an hour<\/dd>/);
271 expect(body).toContain("<dt>Lead time</dt><dd>1 h typical, 1 h for the slowest tenth</dd>");
272 expect(body).not.toContain("p90");
273 // Warm, the figures come from the minute's cache: the 10k-run ledger is not read again.
274 const warm = await requestStatements(app.db, url, { headers: { cookie } });
275 expectWork(warm, { max: 20 });
276 });
277
278 test("org_teams { action: metrics } returns the page's numbers", async () => {
279 const token = createToken(app.db, alice.id, "admin", "admin").token;
280 const res = await fetch(`${servers.url}/mcp`, {
281 method: "POST",
282 headers: {
283 "content-type": "application/json",
284 accept: "application/json, text/event-stream",
285 authorization: `Bearer ${token}`,
286 },
287 body: JSON.stringify({
288 jsonrpc: "2.0",
289 id: 1,
290 method: "tools/call",
291 params: { name: "org_teams", arguments: { org: "acme", team: OWNERS_TEAM, action: "metrics" } },
292 }),
293 });
294 const got = JSON.parse(((await res.json()) as any).result.content[0].text);
295 const want = teamWindows(app.db, teamId, Date.now());
296 expect(got.week.leadMedianMs).toBe(want.week.leadMedianMs);
297 expect(got.week.plannerToBuilderMs).toBe(want.week.plannerToBuilderMs);
298 expect(got.week.conflictRate).toBe(want.week.conflictRate);
299 expect(got.day.slots).toBe(3);
300 });
301 });

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs