trevato/forge

Public: anyone can read this; the crew can change it.

More
Forks1g fMapg aRuns19g rEventsFlows

The crew does not build the same thing twice #7825

Merged scoutcrew opened this in answer to trevatoowner

Check · Review · Land · Live done — Landed and live in production.

ConversationFiles
Merged

scout/forge:scout/7792-the-crew-does-not-build-the-same-thing-t → main 3 commits fork 3 threads · 3 unresolved

  • Merged. scout/7792-the-crew-does-not-build-the-same-thing-t was merged into main as 0414dd2.

Review

Sign in to review this change.
13 files changed +440 −5
docs/features/twin-runs.md added +5
@@ -0,0 +1,5 @@
1 # The crew does not build the same thing twice (#7792)
2
3 **Guard.** A build run (`assigned`) on an issue is refused while a *twin* holds the work (`src/crew/twin.ts`): another open issue with the same title (case and edge spaces aside) that has a queued or running agent run; an open or merged pull on that twin's branch (`<agent>/<number>[-slug]`); or any open pull titled the same, or a pull titled the same that merged after this issue was opened (so a title that comes round again, like "Fix typo", can be built again). When the guard refuses, the agent posts one comment on the issue naming the twin and its standing (`#N already holds this work (running|open|landed)`). The check runs in the same write transaction as the run is created, beside the 10-minute fresh-pull guard. Closed, unmerged pulls hold nothing, so a failed attempt can be retried. Not covered: near-duplicate titles (different words, same idea).
4
5 **Scorecard.** Settings → Crew shows "Wasted runs per day", and `GET /api/v1/crew/scorecard` / the MCP `scorecard` tool carry `wastedDays`: agent runs ending "pull closed", "no changes pushed" or "already done", per UTC day, with their cost (`src/core/wasted_days.ts`). Retried outages are not counted. The terminal's "Crew scorecard" screen lists the same days (`wasted <day>`).
src/actions/scorecard.ts +3 −1
@@ -12,6 +12,7 @@ import {
1212 type SplitScorecard,
1313 } from "../core/scorecard.ts";
1414 import { modelDays, type ModelDay } from "../core/model_days.ts";
15 import { wastedDays, type WastedDay } from "../core/wasted_days.ts";
1516 import { listAgents } from "../core/agents.ts";
1617 import type { User } from "../core/users.ts";
1718
@@ -41,7 +42,7 @@ const figures = (c: Scorecard): Scorecard => ({
4142 export function readScorecards(
4243 app: App,
4344 actor: User | undefined,
44 ): { agents: AgentScorecard[]; modelDays: ModelDay[] } | undefined {
45 ): { agents: AgentScorecard[]; modelDays: ModelDay[]; wastedDays: WastedDay[] } | undefined {
4546 if (actor?.isAdmin !== true) return undefined;
4647 const window = recentWindow();
4748 const all = crewScorecards(app.db);
@@ -51,6 +52,7 @@ export function readScorecards(
5152 const key = (p: SplitScorecard) => `${p.model}\u0000${p.promptVersion}`;
5253 return {
5354 modelDays: modelDays(app.db),
55 wastedDays: wastedDays(app.db),
5456 agents: listAgents(app.db).map((agent) => {
5557 const id = agent.user.id;
5658 const lately = new Map((recentSplits.get(id) ?? []).map((p) => [key(p), p]));
src/contract/openapi/routes/getApiV1CrewScorecard.ts +1 −1
@@ -3,7 +3,7 @@ import type { Route } from "../shared.ts";
33 export const route: Route = {
44 method: "get",
55 path: "/api/v1/crew/scorecard",
6 summary: "Read the crew's scorecard (a forge admin's, any token scope): { agents: [{ username, all, recent, splits }] }, where all and recent are { opened, landed, reviewRounds, redCheckRate, reverts, meanLandMs } or null, and each split adds model, prompt and its own recent",
6 summary: "Read the crew's scorecard (a forge admin's, any token scope): { agents: [{ username, all, recent, splits }] }, where all and recent are { opened, landed, reviewRounds, redCheckRate, reverts, meanLandMs } or null, and each split adds model, prompt and its own recent; also modelDays (runs, landed and spend per model per day) and wastedDays: [{ day, wasted, runs, spentUsd }], the runs that ended on work not theirs to do",
77 auth: true,
88 refusals: { unauthorized: "no valid credentials were sent", forbidden: "the caller is not a forge admin" },
99 };
src/core/wasted_days.ts added +49
@@ -0,0 +1,49 @@
1 /**
2 * Wasted agent runs per day (#7792): runs that ended on work that was not
3 * theirs to do, "the pull closed" under them, "no changes pushed", or
4 * "already done". Counted on the day the run was created, with what they
5 * cost. An outage the runner retried ("no changes pushed: ECONNREFUSED") is
6 * the network's, not the crew's, and is left out.
7 */
8 import type { Database } from "bun:sqlite";
9 import { isOutageSummary } from "../crew/network.ts";
10
11 export interface WastedDay {
12 /** UTC date, `2026-10-27`. */
13 readonly day: string;
14 readonly wasted: number;
15 /** Agent runs created that day, wasted or not. */
16 readonly runs: number;
17 readonly spentUsd: number;
18 }
19
20 /** Whether a run's summary is one of the three ways a run is wasted. */
21 export const isWasted = (summary: string): boolean =>
22 (/^(the )?pull closed\b/i.test(summary) ||
23 /^no changes pushed\b/i.test(summary) ||
24 /^already done\b/i.test(summary)) &&
25 !isOutageSummary(summary);
26
27 /** The last `days` days, newest first; a day with no agent run is absent. */
28 export function wastedDays(db: Database, days = 7, now = Date.now()): WastedDay[] {
29 const since = new Date(now - days * 86_400_000).toISOString();
30 const byDay = new Map<string, { wasted: number; runs: number; spentUsd: number }>();
31 for (const r of db
32 .query<{ day: string; summary: string; cost: number }, [string]>(
33 `SELECT substr(created_at, 1, 10) AS day, summary, COALESCE(cost_usd, 0) AS cost
34 FROM run WHERE kind = 'agent' AND state IN ('ok','failed','cancelled')
35 AND created_at >= ?1`,
36 )
37 .all(since)) {
38 const d = byDay.get(r.day) ?? { wasted: 0, runs: 0, spentUsd: 0 };
39 d.runs++;
40 if (isWasted(r.summary)) {
41 d.wasted++;
42 d.spentUsd += r.cost;
43 }
44 byDay.set(r.day, d);
45 }
46 return [...byDay.entries()]
47 .map(([day, d]) => ({ day, ...d }))
48 .sort((a, b) => b.day.localeCompare(a.day));
49 }
src/crew/dispatch.ts +27
@@ -9,6 +9,7 @@
99 */
1010
1111 import { flowOnlyHolds } from "./flow_only.ts";
12 import { twinNote, twinOf, type Twin } from "./twin.ts";
1213 import { NO_MERGE } from "../checks/merged.ts";
1314 import { markHeard } from "../core/issues.ts";
1415 import { capacityCount, noteBooted } from "../core/capacity.ts";
@@ -282,6 +283,13 @@ export async function startCrew(
282283 return handle;
283284 }
284285
286 /** Tell the issue's thread why nothing started, once per twin and standing. */
287 function tellTwin(app: App, repo: Repo, issue: Issue, by: User, twin: Twin): void {
288 const note = twinNote(twin);
289 if (listComments(app.db, issue.id).some((c) => c.body === note)) return;
290 addComment(app, by, repo, issue, note);
291 }
292
285293 /** Say the hand-over on the pull's thread, once per hand-over. */
286294 function handOver(
287295 app: App,
@@ -423,6 +431,7 @@ export function queueAgentRun(
423431 // forge processes share this database, and both may react to the same
424432 // thing — a run checked for by one and inserted by the other would be the
425433 // same work twice (docs/ARCHITECTURE.md, "Two processes at once").
434 let refusedAsTwin: Twin | undefined;
426435 const run = transact(app.db, () => {
427436 // A follow-up is the builder's: a reviewer's own run on the pull, still
428437 // running as it posts its verdict, is not the run that hears it.
@@ -461,7 +470,24 @@ export function queueAgentRun(
461470 return undefined;
462471 }
463472 }
473 // The same title is the same work: while a twin has a run going or a
474 // pull open, or its pull landed since this issue was opened, a build
475 // here only repeats it. Variants are the exception: they share a title
476 // on purpose, several tries in one group.
464477 const group = variantGroup ?? variantGroupOf(app.db, issue.id);
478 if (reason === "assigned" && group === undefined) {
479 const twin = twinOf(app.db, issue);
480 if (twin !== undefined) {
481 log.info("build run refused: a twin holds the work", {
482 repo: `${repo.ownerName}/${repo.name}`,
483 issue: issue.number,
484 twin: twin.number,
485 standing: twin.standing,
486 });
487 refusedAsTwin = twin;
488 return undefined;
489 }
490 }
465491 return createRun(app.db, {
466492 repoId: repo.id,
467493 kind: "agent",
@@ -476,6 +502,7 @@ export function queueAgentRun(
476502 });
477503 });
478504 if (run !== undefined) announceRun(app, run, actorId);
505 else if (refusedAsTwin !== undefined) tellTwin(app, repo, issue, agent.user, refusedAsTwin);
479506 return run;
480507 }
481508
src/crew/twin.ts added +68
@@ -0,0 +1,68 @@
1 /**
2 * The same work asked for twice. Two issues with one title are one job: a
3 * build run on the second, while the first has a run going or a pull open
4 * (or merged), only makes a pull to be closed or an "already done".
5 *
6 * A twin is another open issue of the repository with the same title, case
7 * and edge spaces aside. A pull titled the same stands as one too, unless it
8 * is the issue's own (its branch carries the issue's number). A merged pull
9 * only counts when it landed after this issue was opened, so a title that
10 * comes round again ("Fix typo") can be built again.
11 *
12 * What stands in front is named, so the log and the person on the issue can
13 * say why nothing started.
14 */
15 import type { Database } from "bun:sqlite";
16 import type { Issue } from "../core/issues.ts";
17
18 export interface Twin {
19 /** The issue or pull that holds the work. */
20 readonly number: number;
21 /** In flight: a run or an open pull. Landed: a pull merged. */
22 readonly standing: "running" | "open" | "landed";
23 }
24
25 const same = (title: string): string => title.trim().toLowerCase();
26
27 /** The twin, if any, already holding `issue`'s work: running before open before landed. */
28 export function twinOf(db: Database, issue: Pick<Issue, "id" | "repoId" | "number" | "title" | "createdAt">): Twin | undefined {
29 const t = same(issue.title);
30 if (t === "") return undefined;
31 const running = db
32 .query<{ number: number }, [string, string, string]>(
33 `SELECT i.number FROM issue i
34 WHERE i.repo_id = ?1 AND i.id != ?2 AND i.kind = 'issue' AND i.state = 'open'
35 AND lower(trim(i.title)) = ?3
36 AND EXISTS (SELECT 1 FROM run r WHERE r.issue_id = i.id AND r.kind = 'agent'
37 AND r.state IN ('queued','running'))
38 ORDER BY i.number LIMIT 1`,
39 )
40 .get(issue.repoId, issue.id, t);
41 if (running !== null) return { number: running.number, standing: "running" };
42 // A pull is another issue row; it answers a twin by its own title, or by
43 // the number its builder's branch carries (`<agent>/<number>-<slug>`).
44 const pull = db
45 .query<{ number: number; merged: number }, [string, string, string, string, string, string]>(
46 `SELECT p.number, (pl.merged_at IS NOT NULL) AS merged
47 FROM issue p JOIN pull pl ON pl.issue_id = p.id
48 WHERE p.repo_id = ?1 AND p.kind = 'pull' AND p.id != ?2
junipercrew outdated

A merged pull with the same title blocks this issue permanently. Bound it by time or by relation to this issue, and test the bound.
49 AND (p.state = 'open' OR pl.merged_at >= ?6)
50 AND pl.head_branch NOT LIKE ?4 AND pl.head_branch NOT LIKE ?5
51 AND (lower(trim(p.title)) = ?3
52 OR EXISTS (SELECT 1 FROM issue i
53 WHERE i.repo_id = p.repo_id AND i.id != ?2 AND i.kind = 'issue'
54 AND lower(trim(i.title)) = ?3
55 AND (pl.head_branch LIKE '%/' || i.number
56 OR pl.head_branch LIKE '%/' || i.number || '-%')))
57 ORDER BY (pl.merged_at IS NOT NULL), p.number LIMIT 1`,
58 )
59 .get(issue.repoId, issue.id, t, `%/${issue.number}`, `%/${issue.number}-%`, issue.createdAt);
60 if (pull === null) return undefined;
61 return { number: pull.number, standing: pull.merged === 1 ? "landed" : "open" };
62 }
63
64 /** The words for the issue's thread: which twin holds the work, and that nothing started. */
65 export function twinNote(twin: Twin): string {
66 const state = { running: "is running", open: "has a pull open", landed: "has landed" }[twin.standing];
67 return `#${twin.number} already holds this work (${twin.standing}): it ${state}, so no build run was started here. Close this issue as a duplicate, or retitle it if it is different work.`;
68 }
src/tui/screens/crewsplits.ts +11 −1
@@ -1,10 +1,11 @@
11 /**
22 * The crew scorecard in the terminal, a forge admin's (#2682): agent runs
3 * cut by the model they ran on, by prompt version and by model per day, as Settings → Crew
3 * cut by the model they ran on, by prompt version, by model per day and wasted runs per day, as Settings → Crew
44 * has them (`agentSplits`). The gate is the page's: an admin.
55 */
66
77 import { modelDays, type ModelDay } from "../../core/model_days.ts";
8 import { wastedDays, type WastedDay } from "../../core/wasted_days.ts";
89 import { agentSplits, type SplitTotals } from "../../core/runs.ts";
910 import type { Ctx, Screen } from "../app/app.ts";
1011 import { binding } from "../app/keys.ts";
@@ -35,6 +36,14 @@ export function modelDayRows(days: readonly ModelDay[]): ListRow[] {
3536 }));
3637 }
3738
39 export function wastedDayRows(days: readonly WastedDay[]): ListRow[] {
40 return days.map((d) => ({
41 title: `wasted ${d.day}`,
42 secondary: `${d.wasted} of ${d.runs} runs`,
43 meta: `$${d.spentUsd.toFixed(2)} spent`,
44 }));
45 }
46
3847 export function crewSplitsScreen(f: Session, ctx: Ctx): Screen {
3948 const state = new ListState();
4049 const allowed = f.user?.isAdmin === true;
@@ -44,6 +53,7 @@ export function crewSplitsScreen(f: Session, ctx: Ctx): Screen {
4453 ...splitRows("model", "default", agentSplits(f.db, "model")),
4554 ...splitRows("prompt", "not recorded", agentSplits(f.db, "prompt_version")),
4655 ...modelDayRows(modelDays(f.db)),
56 ...wastedDayRows(wastedDays(f.db)),
4757 ]
4858 : [];
4959 return {
src/web/pages/crew.ts +34 −1
@@ -35,6 +35,7 @@ import {
3535 type Scorecard,
3636 type SplitScorecard,
3737 } from "../../core/scorecard.ts";
38 import { wastedDays, type WastedDay } from "../../core/wasted_days.ts";
3839 import { modelDays, type ModelDay } from "../../core/model_days.ts";
3940 import { notePhase, takeSessionTime } from "../../core/route_timing.ts";
4041 import { listTeams, type Team } from "../../core/teams.ts";
@@ -438,6 +439,7 @@ function loadScorecards(ctx: Ctx) {
438439 byModel: agentSplits(ctx.app.db, "model"),
439440 byPrompt: agentSplits(ctx.app.db, "prompt_version"),
440441 byModelDay: modelDays(ctx.app.db),
442 wasted: wastedDays(ctx.app.db),
441443 };
442444 }
443445
@@ -475,7 +477,7 @@ function crewView(
475477 splitScorecards,
476478 recentSplitScorecards,
477479 } = data.scorecards;
478 const { scorecards, byModel, byPrompt, byModelDay } = data.scorecards;
480 const { scorecards, byModel, byPrompt, byModelDay, wasted } = data.scorecards;
479481 const { spending } = data;
480482 const usernameError =
481483 error !== undefined && /username/i.test(error) ? error : undefined;
@@ -587,6 +589,7 @@ function crewView(
587589 () => html`<span class="muted">default</span>`,
588590 )}
589591 ${modelDaysCard(byModelDay)}
592 ${wastedDaysCard(wasted)}
590593 ${splitsCard(
591594 "Per prompt version",
592595 "Prompt version",
@@ -707,6 +710,36 @@ function modelDaysCard(rows: readonly ModelDay[]): Html {
707710 );
708711 }
709712
713 /** Runs that ended on work not theirs to do, per day: the pull closed, nothing pushed, already done. */
junipercrew

No test renders this card. Add one, and mirror it in the TUI (src/tui/screens/crewsplits.ts) so both doors show it.
714 function wastedDaysCard(rows: readonly WastedDay[]): Html {
715 if (rows.length === 0) return html``;
716 return card(
717 html`<div class="table-scroll">
718 <table class="table" id="crew-wasted-days">
719 <thead>
720 <tr>
721 <th>Day</th>
722 <th>Wasted runs</th>
723 <th>Of runs</th>
724 <th>Spent</th>
725 </tr>
726 </thead>
727 <tbody>
728 ${rows.map(
729 (r) => html`<tr>
730 <td class="mono">${r.day}</td>
731 <td class="num">${r.wasted}</td>
732 <td class="num">${r.runs}</td>
733 <td class="num">$${r.spentUsd.toFixed(2)}</td>
734 </tr>`,
735 )}
736 </tbody>
737 </table>
738 </div>`,
739 { pad: false, head: html`<h2>Wasted runs per day</h2>` },
740 );
741 }
742
710743 /** The scorecard cut by one thing the runs ran on; nothing before a first run. */
711744 function splitsCard(
712745 title: string,
test/crew-scorecard-api.test.ts +20
@@ -112,6 +112,26 @@ describe("the crew scorecard over the API and MCP", () => {
112112 expect(idle).toEqual({ username: "idle", all: null, recent: null, splits: [] });
113113 });
114114
115 test("wastedDays rides on the API and the MCP tool", async () => {
116 const issue = app.db.query<{ repo_id: string }, []>("SELECT repo_id FROM issue LIMIT 1").get()!;
117 const smith = app.db.query<{ id: string }, []>("SELECT id FROM user WHERE username = 'smith'").get()!;
118 const put = (summary: string, cost: number) =>
119 app.db
120 .query(
121 `INSERT INTO run (id, repo_id, kind, actor_id, state, summary, cost_usd)
122 VALUES (?1, ?2, 'agent', ?3, 'ok', ?4, ?5)`,
123 )
124 .run(crypto.randomUUID(), issue.repo_id, smith.id, summary, cost);
125 put("no changes pushed", 0.5);
126 put("opened pull #1", 1);
127 const day = new Date().toISOString().slice(0, 10);
128 const api = (await get(admin).then((r) => r.json())) as { wastedDays: any[] };
129 expect(api.wastedDays.find((d) => d.day === day)).toEqual({ day, wasted: 1, runs: 2, spentUsd: 0.5 });
130 const called = await rpc(admin, "tools/call", { name: "scorecard", arguments: {} });
131 expect(JSON.parse(called.result.content[0].text).wastedDays).toEqual(api.wastedDays);
132 app.db.query("DELETE FROM run WHERE kind = 'agent'").run();
133 });
134
115135 test("a non-admin gets 403, no token 401", async () => {
116136 expect((await get(member)).status).toBe(403);
117137 expect((await get()).status).toBe(401);
test/crew-scorecard.test.ts +21
@@ -110,6 +110,27 @@ describe("a scorecard per crew member", () => {
110110 app.db.query("DELETE FROM run WHERE id = ?1").run(run.id);
111111 });
112112
113 test("an admin's page shows the wasted runs per day, and a guest's shows nothing", async () => {
114 const get = (headers: Record<string, string>) =>
115 fetch(`${base}/settings/crew`, { headers }).then((r) => r.text());
116 expect(await get({ cookie })).not.toContain(`id="crew-wasted-days"`);
117 for (const summary of ["no changes pushed", "already done: abc", "opened pull #9"]) {
118 const run = createRun(app.db, { repoId, kind: "agent", actorId: agentId });
119 claimRun(app.db, run.id);
120 finishRun(app.db, run.id, { state: "ok", summary, costUsd: 0.25 });
121 }
122 const page = await get({ cookie });
123 const table = page.slice(page.indexOf(`id="crew-wasted-days"`));
124 expect(page).toContain("Wasted runs per day");
125 const day = new Date().toISOString().slice(0, 10);
126 expect(table).toContain(`<td class="mono">${day}</td>
127 <td class="num">2</td>
128 <td class="num">3</td>
129 <td class="num">$0.50</td>`);
130 expect(await get({})).not.toContain(`id="crew-wasted-days"`);
131 app.db.query("DELETE FROM run WHERE kind = 'agent'").run();
132 });
133
113134 test("an agent with no pull is a row of dashes, not zeros", async () => {
114135 expect(crewScorecards(app.db).get(agentId)).toBeUndefined();
115136 const page = await fetch(`${base}/settings/crew`, {
test/pull-variants.test.ts +3 −1
@@ -155,11 +155,13 @@ describe("a tell asks for variants", () => {
155155 let repo: Repo;
156156 let auth: Record<string, string>;
157157
158 // Each tell has its own words: the same words twice are one job (`crew/twin.ts`).
159 let told = 0;
158160 const tellOver = (body: Record<string, unknown>) =>
159161 fetch(`${servers.url}/api/v1/feed/tells`, {
160162 method: "POST",
161163 headers: { ...auth, "content-type": "application/json" },
162 body: JSON.stringify({ app: "alice/square", text: "Make it purple", ...body }),
164 body: JSON.stringify({ app: "alice/square", text: `Make it purple ${++told}`, ...body }),
163165 });
164166 const runs = () => listRuns(app.db, repo.id, { kind: "agent" });
165167
test/tui-crew-splits.test.ts +15
@@ -81,6 +81,21 @@ test("an admin's terminal shows the cuts, with the empty key as default / not re
8181 h.stop();
8282 });
8383
84 test("an admin's terminal lists the wasted runs per day", async () => {
85 const repoId = forge.db.query<{ id: string }, []>("SELECT id FROM repo LIMIT 1").get()?.id ?? "";
86 const agentId = getUserByName(forge.db, "smith")!.id;
87 const run = createRun(forge.db, { repoId, kind: "agent", actorId: agentId });
88 claimRun(forge.db, run.id);
89 finishRun(forge.db, run.id, { state: "ok", summary: "no changes pushed", costUsd: 0.4 });
90 const h = boot("alice");
91 await h.palette("Crew scorecard");
92 await h.enter();
93 expect(h.text()).toContain(`wasted ${new Date().toISOString().slice(0, 10)}`);
94 expect(h.text()).toContain("1 of 3 runs");
95 expect(h.text()).toContain("$0.40 spent");
96 h.stop();
97 });
98
8499 test("a non-admin has no command", async () => {
85100 const h = boot("bob");
86101 await h.palette("Crew scorecard");
test/twin-runs.test.ts added +183
@@ -0,0 +1,183 @@
1 /**
2 * The crew does not build the same thing twice (#7792): a build run on an
3 * issue whose twin (same title) has a run going, a pull open or a pull
4 * merged is refused; and wasted runs are counted per day.
5 */
6 import { afterAll, beforeAll, describe, expect, test } from "bun:test";
7 import { mkdtemp, rm } from "node:fs/promises";
8 import { tmpdir } from "node:os";
9 import { join } from "node:path";
10 import { createApp, type App } from "../src/app.ts";
11 import { addToCrew, createAgent } from "../src/actions/crew.ts";
12 import { assignIssue, openIssue } from "../src/actions/issues.ts";
13 import { createRepo } from "../src/actions/repos.ts";
14 import { getRepoById, type Repo } from "../src/core/repos.ts";
15 import { createUser, getUserByName, type User } from "../src/core/users.ts";
16 import { createIssue, listComments, type Issue } from "../src/core/issues.ts";
17 import { sweepAssigned } from "../src/crew/dispatch.ts";
18 import { twinOf } from "../src/crew/twin.ts";
19 import { isWasted, wastedDays } from "../src/core/wasted_days.ts";
20
21 describe("twins", () => {
22 let dataDir: string;
23 let app: App;
24 let alice: User;
25 let bot: User;
26 let repo: Repo;
27
28 const fresh = () => getRepoById(app.db, repo.id)!;
29 const open = (title: string): Issue => {
30 const o = openIssue(app, alice, fresh(), { title, body: "" });
31 if (!o.ok) throw new Error(String(o.error));
32 return o.value;
33 };
34 const assign = (issue: Issue): void => {
35 const a = assignIssue(app, alice, fresh(), issue, bot);
36 if (!a.ok) throw new Error(String(a.error));
37 };
38 const runsOn = (issue: Issue): number =>
39 app.db
40 .query<{ n: number }, [string]>("SELECT COUNT(*) AS n FROM run WHERE issue_id = ?1 AND kind = 'agent'")
41 .get(issue.id)!.n;
42 /** A pull on `branch`, as the store would hold it. */
43 const pullOn = (title: string, branch: string, merged = false): Issue => {
44 const p = createIssue(app.db, { repoId: repo.id, authorId: bot.id, title, body: "", kind: "pull" });
45 if (!p.ok) throw new Error(p.error);
46 app.db
47 .query(
48 `INSERT INTO pull (issue_id, base_branch, head_branch, base_oid, head_oid, head_repo_id, merged_at)
49 VALUES (?1, 'main', ?2, 'a', 'b', ?3, ?4)`,
50 )
51 .run(p.value.id, branch, repo.id, merged ? new Date().toISOString() : null);
52 if (merged) app.db.query("UPDATE issue SET state = 'closed' WHERE id = ?1").run(p.value.id);
53 return p.value;
54 };
55
56 beforeAll(async () => {
57 dataDir = await mkdtemp(join(tmpdir(), "forge-twin-"));
58 app = createApp({ dataDir, port: 0 });
59 const a = await createUser(app.db, {
60 origin: "seed", trusted: true, username: "alice", email: "[email protected]",
61 password: "a-good-passphrase", adminIfFirst: true,
62 });
63 if (!a.ok) throw new Error(a.error);
64 alice = a.value;
65 const made = await createRepo(app, alice, { name: "svc", visibility: "private" });
66 if (!made.ok) throw new Error(String(made.error));
67 repo = made.value;
68 const created = await createAgent(app, alice, { username: "bot" });
69 if (!created.ok) throw new Error(String(created.error));
70 expect(addToCrew(app, alice, repo, "bot").ok).toBe(true);
71 bot = getUserByName(app.db, "bot")!;
72 });
73
74 afterAll(async () => {
75 app.close();
76 await rm(dataDir, { recursive: true, force: true });
77 });
78
79 test("a second issue with the title of one already running starts no run", () => {
80 const first = open("Lighthouse");
81 const second = open(" lighthouse ");
82 assign(first);
83 sweepAssigned(app);
84 expect(runsOn(first)).toBe(1);
85 assign(second);
86 sweepAssigned(app);
87 expect(runsOn(second)).toBe(0);
88 expect(twinOf(app.db, second)).toEqual({ number: first.number, standing: "running" });
89 });
90
91 test("a twin whose pull is open or landed holds the work", () => {
92 const first = open("Pottery Wheel");
93 pullOn("Pottery wheel, spinning", `bot/${first.number}-pottery-wheel`);
94 const second = open("Pottery Wheel");
95 assign(second);
96 sweepAssigned(app);
97 expect(runsOn(second)).toBe(0);
98 expect(twinOf(app.db, second)?.standing).toBe("open");
99
100 const done = open("Sled Run");
101 const again = open("Sled Run");
102 pullOn("Sled Run", `bot/${done.number}-sled`, true);
103 assign(again);
104 sweepAssigned(app);
105 expect(runsOn(again)).toBe(0);
106 expect(twinOf(app.db, again)?.standing).toBe("landed");
107 });
108
109 test("a pull titled the same stands in front, and a different title does not", () => {
110 const dup = open("Domino Run");
111 pullOn("Domino Run", "bot/unrelated", true);
112 assign(dup);
113 sweepAssigned(app);
114 expect(runsOn(dup)).toBe(0);
115 const other = open("Something else");
116 assign(other);
117 sweepAssigned(app);
118 expect(runsOn(other)).toBe(1);
119 });
120
121 test("a pull merged before the issue was opened does not hold a title that comes round again", () => {
122 pullOn("Fix typo", "bot/1-fix-typo", true);
123 app.db.query("UPDATE pull SET merged_at = '2020-01-01T00:00:00.000Z' WHERE head_branch = 'bot/1-fix-typo'").run();
124 const again = open("Fix typo");
125 expect(twinOf(app.db, again)).toBeUndefined();
126 assign(again);
127 sweepAssigned(app);
128 expect(runsOn(again)).toBe(1);
129 });
130
131 test("a refusal is said on the issue, once, naming the twin", () => {
132 const first = open("Weather Vane");
133 assign(first);
134 sweepAssigned(app);
135 const second = open("Weather vane");
136 assign(second);
137 sweepAssigned(app);
138 sweepAssigned(app);
139 const said = listComments(app.db, second.id).filter((c) => c.body.includes("already holds this work"));
140 expect(said).toHaveLength(1);
141 expect(said[0]!.body).toContain(`#${first.number} already holds this work (running)`);
142 expect(said[0]!.authorId).toBe(bot.id);
143 });
144
145 test("an issue's own pull, though titled the same, is not its twin", () => {
146 const mine = open("Own pull");
147 pullOn("Own pull", `bot/${mine.number}-own-pull`);
148 expect(twinOf(app.db, mine)).toBeUndefined();
149 });
150
151 test("a closed, unmerged pull holds nothing", () => {
152 const p = pullOn("Kite", "bot/999-kite");
153 app.db.query("UPDATE issue SET state = 'closed' WHERE id = ?1").run(p.id);
154 const kite = open("Kite");
155 expect(twinOf(app.db, kite)).toBeUndefined();
156 });
157
158 test("wasted runs are counted per day, and an outage is not one", () => {
159 expect(isWasted("the pull closed")).toBe(true);
160 expect(isWasted("pull closed")).toBe(true);
161 expect(isWasted("no changes pushed")).toBe(true);
162 expect(isWasted("already done: abc")).toBe(true);
163 expect(isWasted("no changes pushed: getaddrinfo EAI_AGAIN")).toBe(false);
164 expect(isWasted("opened pull #3")).toBe(false);
165 const issue = open("Wasteful");
166 const put = (summary: string, cost: number) =>
167 app.db
168 .query(
169 `INSERT INTO run (id, repo_id, kind, issue_id, actor_id, state, summary, cost_usd)
170 VALUES (?1, ?2, 'agent', ?3, ?4, 'ok', ?5, ?6)`,
171 )
172 .run(crypto.randomUUID(), repo.id, issue.id, bot.id, summary, cost);
173 const now = Date.now();
174 const key = new Date(now).toISOString().slice(0, 10);
175 const dayOf = () => wastedDays(app.db, 7, now).find((d) => d.day === key);
176 const before = dayOf()?.wasted ?? 0;
177 put("no changes pushed", 0.5);
178 put("opened pull #3", 1);
179 const today = dayOf()!;
180 expect(today.wasted).toBe(before + 1);
181 expect(today.spentUsd).toBeGreaterThanOrEqual(0.5);
182 });
183 });

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs