Public: anyone can read this; the crew can change it.

Red main: a header line and a "red on main, not on this pull" mark when the size ceilings fail on the base #13841

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

mason/forge:mason/13840-red-main-a-header-line-and-a-red-on-main → main 7 commits fork 7 threads · 7 unresolved

  • Merged. mason/13840-red-main-a-header-line-and-a-red-on-main was merged into main as dd33527.

Review

Sign in to review this change.
7 files changed +345 −2
docs/features/pull-verdict.md +2
@@ -27,3 +27,5 @@ To see how far behind a green is, compare `checkedAgainst` with the base branch'
2727 ## Try it
2828
2929 Open any pull at `/<owner>/<repo>/pulls/<number>`; its first line is the verdict. Over `ssh`, open the pull from the repository's list.
30
31 When a pull's check failed while its base branch is red, one more sentence names the base's failing tests, and ends "red on main, not on this pull" if the pull fails only those. The page and the terminal screen say it; a pull whose check has not failed hears nothing.
src/checks/red_main_line.ts added +51
@@ -0,0 +1,51 @@
1 /**
2 * What a pull's page says when its base is red (#13840, item 2 of #13805):
3 * when the pull's check failed, one sentence naming the base as red and the tests it
4 * fails, and, when the pull's check names only those tests, that the red is main's.
5 * Plain data in, plain words out; the page escapes them.
6 */
7
8 import type { Run } from "../core/runs.ts";
9 import { manifestError, sharedWithMain, type RedMain } from "./red_main.ts";
10
11 /** The most failing tests the line names. */
12 const NAMED = 3;
13
14 /**
15 * Check output with its control characters (C0, DEL, C1) made spaces: the terminal prints the
16 * line as it is, so a test's escape sequence must not reach a viewer's terminal.
17 */
18 export function plainText(text: string): string {
19 return text.replace(/[\u0000-\u001f\u007f-\u009f]/g, " ");
20 }
21
22 /**
outdated

Raw check output goes into the terminal screen here. Strip control characters (e.g. replace /[\u0000-\u001f\u007f-\u009f]/g with a space) before slicing, and add a test with an ESC byte in the summary.

23 * The tests a red main fails, in words: `"a", "b" and 2 more`, or its summary when it named none.
24 */
25 function whatFails(red: RedMain): string {
26 const names = red.failing.slice(0, NAMED).map((n) => JSON.stringify(plainText(n)));
27 if (names.length === 0) return plainText(red.summary.replace(/^check failed: /, "")).slice(0, 200);
28 const more = red.failing.length - names.length;
29 return "failing: " + names.join(", ") + (more > 0 ? ` and ${more} more` : "");
30 }
31
32 /** Whether a failed check is red on main and not on the pull: it fails only what main fails. */
33 export function redOnMain(red: RedMain | undefined, run: Pick<Run, "state" | "checkDetail"> | null): boolean {
34 return run !== null && run.state === "failed" && !red?.limit && sharedWithMain(red, run);
35 }
36
37 /**
38 * The line for a pull whose check failed while its base branch `baseBranch` is red: one sentence,
39 * ending "red on main, not on this pull" when the pull fails only what main fails. "" for any
40 * other pull (check not failed, main green, stopped by a limit, another base).
41 */
42 export function redMainLine(
43 red: RedMain | undefined,
outdated

Principles 1 and 8: this branch tells a reader of a pull with a green or running check that 'a failure of these tests is main's, not this pull's', which is an explanation of a thing that has not happened. And the marked branch says 'red on main, not on this pull' and then 'the check runs again when main is green', two sentences for one fact, with an unverified promise (nothing in this diff proves the recheck). Fix: when the pull is not failed, return '' (no header on healthy pulls); when marked, one sentence: 'main is red at abcdef0 ("sizes hold" fails): red on main, not on this pull.' Drop the recheck claim unless a test shows waitingOnMain does it.

44 run: Pick<Run, "state" | "checkDetail"> | null,
outdated

Wording bug: when red.failing is empty, whatFails returns the summary ('1 failed') and the line reads '1 failed fail;'. Plural is chosen from failing.length, which is 0 here. Name the summary case separately ('main is red at abcdef0: 1 failed.') and test it. The manifestError branch and the 'and N more' branch also have no test.

45 baseBranch: string,
46 ): string {
47 if (red === undefined || red.limit || baseBranch !== red.branch || run === null || run.state !== "failed") return "";
48 const head = `${baseBranch} is red at ${red.commitOid.slice(0, 7)}`;
49 if (manifestError(red) !== undefined) return `${head}: its forge.json is invalid, so no check can run.`;
50 return `${head} (${whatFails(red)})${redOnMain(red, run) ? ": red on main, not on this pull." : "."}`;
51 }
src/tui/screens/pull.ts +8 −1
@@ -28,6 +28,8 @@
2828
2929 import { teamHoldRefusal, waitingOnTeams } from "../../core/teams_words.ts";
3030 import { crewFilters, guardedRefusal, isStrangerPull, strangerGuard, strangerNext } from "../../crew/stranger.ts";
31 import { redMain } from "../../checks/red_main.ts";
32 import { redMainLine } from "../../checks/red_main_line.ts";
3133 import { RED_ALONE_WAIT, redAloneAt, redMark } from "../../checks/queue.ts";
3234 import {
3335 addCommentThrough,
@@ -459,7 +461,12 @@ export function pullScreen(
459461 repo === undefined || row === undefined
460462 ? undefined
461463 : redMark(repo.id, row.id, row.headOid || (facts?.headOid ?? ""));
462 redWords = mark === undefined ? "" : `red alone at ${redAloneAt(mark)}, ${RED_ALONE_WAIT}`;
464 const mainRed =
465 repo === undefined || row === undefined || check.state !== "failed"
466 ? ""
467 : redMainLine(redMain(f.db, repo.id), check.run, row.baseBranch);
468 const alone = mark === undefined ? "" : `red alone at ${redAloneAt(mark)}, ${RED_ALONE_WAIT}`;
469 redWords = [mainRed, alone].filter((w) => w !== "").join(" · ");

This new terminal behaviour has no test. Add one that renders the pull screen with a red main and a failed pull check and asserts the line, with and without the mark.

463470 checkSteps =
464471 settings === undefined
465472 ? ""
src/web/pages/pull.ts +7 −1
@@ -52,6 +52,8 @@ import {
5252 } from "../../core/pulls.ts";
5353 import { isProposed, pullMayRun } from "../../core/pull_trust.ts";
5454 import { checkWords } from "../../checks/summary.ts";
55 import { redMain } from "../../checks/red_main.ts";
56 import { redMainLine } from "../../checks/red_main_line.ts";
5557 import { runningProgress } from "../../checks/progress.ts";
5658 import { CHECK_STALE_REFUSAL, checkStale, checksHold } from "../../checks/stale.ts";
5759 import { checkSummary, type CheckSummary } from "../../core/runs.ts";
@@ -509,7 +511,11 @@ function pullChecksLine(
509511 );
510512 const found = !isApp ? null : pullPreviewFindings(ctx.app.db, repo.id, previewEnv(pull.number));
511513 const red = redAloneLine(redMark(repo.id, pull.id, status.headOid));
512 return html`${originCheckLine(ctx, repo, pull)}${line}${red}${found === null ? "" : specFindings(found)}`;
514 // Read main only for a pull whose check failed: a healthy pull pays nothing.
outdated

Every pull page now pays one extra SQL read and shows a red line on every open pull while main is red, even a pull whose check passed. Gate the read behind check.run?.state === 'failed' (or waiting on main) so the budget bump in pages-budget is not paid by healthy pulls, and so the line stays a signal. Also no test renders the pull page with a red main: deleting these two lines would leave bun test green. Add a page test that fails without them.

515 const mainRed =
outdated

Design: the line is hand-rolled markup (<p class="muted small"> plus an icon) where docs/DESIGN.md names components for a status line; reuse the same component redAloneLine uses so the two red marks on one page look and read as one. As a failure mark it should not be a bare muted paragraph a screen reader reads as prose; check that the icon has no accessible name noise (decorative) and the text carries the state.

516 check.state === "failed" ? redMainLine(redMain(ctx.app.db, repo.id), check.run, pull.baseBranch) : "";
517 const header = mainRed === "" ? html`` : html`<p class="muted small">${icon("x")}${mainRed}</p>`;
518 return html`${originCheckLine(ctx, repo, pull)}${header}${line}${red}${found === null ? "" : specFindings(found)}`;
513519 }
514520
515521 /**
test/pull-page-red-main.test.ts added +98
@@ -0,0 +1,98 @@
1 /**
2 * The pull page's red-main line (#13840): shown for a pull whose check failed while main is
3 * red, with the red-on-main mark when it fails only what main fails; absent when main is green
4 * or the pull's check did not fail.
5 */
6 import { afterAll, beforeAll, expect, test } from "bun:test";
7 import { mkdtemp, rm } from "node:fs/promises";
8 import { tmpdir } from "node:os";
9 import { join } from "node:path";
10 import { createApp, type App } from "../src/app.ts";
11 import { openPull } from "../src/actions/pulls.ts";
12 import { commitToBranch, createRepo } from "../src/actions/repos.ts";
13 import { claimRun, createRun, finishRun } from "../src/core/runs.ts";
14 import { createSession } from "../src/core/sessions.ts";
15 import { createUser } from "../src/core/users.ts";
16 import { serve, type Servers } from "../src/http/server.ts";
17 import { quietLogs } from "./helpers/globals.ts";
18
19 quietLogs();
20
21 let dataDir: string;
22 let app: App;
23 let servers: Servers;
24 let cookie = "";
25 let repoId = "";
26 let pullId = "";
27 let headOid = "";
28 let number = 0;
29
30 function check(env: string, commitOid: string, state: "ok" | "failed", names: string[], issueId?: string) {
31 const run = createRun(app.db, { repoId, kind: "check", env, commitOid, reason: "push", ...(issueId ? { issueId } : {}) });
32 claimRun(app.db, run.id);
33 finishRun(app.db, run.id, {
34 state,
35 summary: state === "ok" ? "check passed" : "check failed: tests failed",
36 ...(state === "failed" ? { checkDetail: { passed: 1, failed: names.length, names } } : {}),
37 });
38 }
39
40 beforeAll(async () => {
41 dataDir = await mkdtemp(join(tmpdir(), "forge-pull-red-main-"));
42 app = createApp({ dataDir, port: 0 });
43 servers = await serve(app);
44 const made = await createUser(app.db, {
45 origin: "seed", trusted: true, username: "alice", email: "[email protected]",
46 password: "a-good-passphrase", adminIfFirst: true,
47 });
48 if (!made.ok) throw new Error(String(made.error));
49 cookie = `forge_session=${createSession(app.db, made.value.id).token}`;
50 const repo = await createRepo(app, made.value, { name: "r", visibility: "public" });
51 if (!repo.ok) throw new Error(String(repo.error));
52 repoId = repo.value.id;
53 for (const [branch, file] of [[undefined, "a.txt"], ["topic", "t.txt"]] as const) {
54 const done = await commitToBranch(app, made.value, repo.value, {
55 ...(branch === undefined ? {} : { branch }), message: "change", files: [{ path: file, content: "x" }],
56 });
57 if (!done.ok) throw new Error(JSON.stringify(done.error));
58 }
59 const pull = await openPull(app, made.value, repo.value, { title: "t", body: "", baseBranch: "main", headBranch: "topic" });
60 if (!pull.ok) throw new Error(JSON.stringify(pull.error));
61 number = pull.value.number;
62 pullId = pull.value.id;
63 headOid = pull.value.headOid;
64 });
65
66 afterAll(async () => {
67 await servers.stop();
68 app.close();
69 await rm(dataDir, { recursive: true, force: true });
70 });
71
72 const page = async () => (await fetch(`${servers.url}/alice/r/pulls/${number}`, { headers: { cookie } })).text();
73
74 test("red main and a pull failing the same test: one sentence marks it red on main", async () => {
75 check("main", "a".repeat(40), "failed", ["sizes hold"]);
76 check(`pull-${number}`, headOid, "failed", ["sizes hold"], pullId);
77 const body = await page();
78 expect(body).toContain("main is red at aaaaaaa");
79 expect(body).toContain("red on main, not on this pull.");
80 });
81
82 test("a pull with its own failure still hears that main is red, without the mark", async () => {
83 check(`pull-${number}`, headOid, "failed", ["sizes hold", "mine"], pullId);
84 const body = await page();
85 expect(body).toContain("main is red at aaaaaaa");
86 expect(body).not.toContain("red on main, not on this pull");
87 });
88
89 test("a pull whose check passed shows nothing while main is red", async () => {
90 check(`pull-${number}`, headOid, "ok", [], pullId);
91 expect(await page()).not.toContain("main is red");
92 });
93
94 test("green main again: no line on a failed pull", async () => {
95 check("main", "b".repeat(40), "ok", []);
96 check(`pull-${number}`, headOid, "failed", ["sizes hold"], pullId);
97 expect(await page()).not.toContain("main is red");
98 });
test/red-main-line.test.ts added +56
@@ -0,0 +1,56 @@
1 import { expect, test } from "bun:test";
2 import type { RedMain } from "../src/checks/red_main.ts";
3 import { redMainLine, redOnMain } from "../src/checks/red_main_line.ts";
4
5 const red = (over: Partial<RedMain> = {}): RedMain => ({
6 repoId: "r", repo: "a/forge", runId: "x", commitOid: "abcdef0123456789", failing: ["sizes hold"],
7 summary: "check failed: 1 failed", branch: "main", limit: false, since: "2026-11-16T00:00:00Z", ...over,
8 });
9 const failed = (names: string[]) => ({ state: "failed" as const, checkDetail: { passed: 1, failed: names.length, names } });
10
11 test("a pull failing only what main fails is marked red on main, not on this pull", () => {
12 const line = redMainLine(red(), failed(["sizes hold"]), "main");
13 expect(line).toBe('main is red at abcdef0 (failing: "sizes hold"): red on main, not on this pull.');
14 });
15
16 test("a pull with its own failure is not marked, but the header still names the base", () => {
17 const run = failed(["sizes hold", "mine"]);
18 expect(redOnMain(red(), run)).toBe(false);
19 const line = redMainLine(red(), run, "main");
20 expect(line).toBe('main is red at abcdef0 (failing: "sizes hold").');
21 expect(line).not.toContain("red on main, not on this pull");
22 });
23
24 test("a pull whose check did not fail gets no line, red main or not", () => {
25 expect(redMainLine(red(), null, "main")).toBe("");
26 expect(redMainLine(red(), { state: "ok", checkDetail: null }, "main")).toBe("");
27 });
28
29 test("a red main that named no test says its summary, once", () => {
outdated

Tests cover only the pure function. Add: the empty-failing summary case, the 'and N more' case, the invalid-manifest case, and a pull-page test showing the line appears with red main and is absent with green main.

30 const line = redMainLine(red({ failing: [] }), failed(["x"]), "main");
31 expect(line).toBe("main is red at abcdef0 (1 failed).");
32 });
33
34 test("more than three failing tests: the first three and a count", () => {
35 const line = redMainLine(red({ failing: ["a", "b", "c", "d", "e"] }), failed(["z"]), "main");
36 expect(line).toBe('main is red at abcdef0 (failing: "a", "b", "c" and 2 more).');
37 });
38
39 test("a main whose forge.json is invalid says no check can run", () => {
40 const bad = red({ failing: [], summary: "check failed: forge.json: run is missing" });
41 expect(redMainLine(bad, failed(["x"]), "main")).toBe("main is red at abcdef0: its forge.json is invalid, so no check can run.");
42 });
43
44 test("no line when main is green, stopped by a limit, or the pull targets another branch", () => {
45 expect(redMainLine(undefined, null, "main")).toBe("");
46 expect(redMainLine(red({ limit: true }), null, "main")).toBe("");
47 expect(redMainLine(red(), failed(["x"]), "dev")).toBe("");
48 });
49
50 test("control characters in a summary or a test name reach the line as spaces", () => {
51 const summary = redMainLine(red({ failing: [], summary: "check failed: \u001b[2J1 failed\u009b" }), failed(["x"]), "main");
52 expect(summary).toBe("main is red at abcdef0 ( [2J1 failed ).");
53 const named = redMainLine(red({ failing: ["a\u001b]0;pwned\u0007b"] }), failed(["x"]), "main");
54 expect(named).toBe('main is red at abcdef0 (failing: "a ]0;pwned b").');
55 expect(/[\u0000-\u001f\u007f-\u009f]/.test(summary + named)).toBe(false);
56 });
test/tui-pull-red-main.test.ts added +123
@@ -0,0 +1,123 @@
1 /**
2 * The terminal's pull screen says the red-main line (#13840) as the page does: for a pull whose
3 * check failed while main is red, with "red on main, not on this pull" when it fails only what
4 * main fails, joined by " · " to the red-alone words when the merge queue holds the pull red.
5 */
6 import { afterAll, beforeAll, expect, test } from "bun:test";
7 import { mkdtemp, rm } from "node:fs/promises";
8 import { tmpdir } from "node:os";
9 import { join } from "node:path";
10 import { openPull } from "../src/actions/pulls.ts";
11 import { commitToBranch, createRepo } from "../src/actions/repos.ts";
12 import type { Repo } from "../src/core/repos.ts";
13 import { createApp, type App } from "../src/app.ts";
14 import { enqueue, RED_ALONE_WAIT } from "../src/checks/queue.ts";
15 import { claimRun, createRun, finishRun } from "../src/core/runs.ts";
16 import { getPull, pullDir, type Pull } from "../src/core/pulls.ts";
17 import { createUser, type User } from "../src/core/users.ts";
18 import { createForgeApp } from "../src/tui/forge.ts";
19 import { pullScreen } from "../src/tui/screens/pull.ts";
20 import { local } from "../src/tui/screens/where.ts";
21 import { headlessTerminal } from "../src/tui/term/terminal.ts";
22 import { quietLogs } from "./helpers/globals.ts";
23 import { settleApp } from "./helpers/tui.ts";
24
25 quietLogs();
26
27 const cells = process.env["FORGE_CELLS"];
28 let dataDir: string;
29 let app: App;
30 let alice: User;
31 let repo: Repo;
32 let pull: Pull;
33
34 function check(env: string, commitOid: string, state: "ok" | "failed", names: string[], issueId?: string) {
35 const run = createRun(app.db, { repoId: repo.id, kind: "check", env, commitOid, reason: "push", ...(issueId ? { issueId } : {}) });
36 claimRun(app.db, run.id);
37 finishRun(app.db, run.id, {
38 state,
39 summary: state === "ok" ? "check passed" : "check failed: tests failed",
40 ...(state === "failed" ? { checkDetail: { passed: 1, failed: names.length, names } } : {}),
41 });
42 }
43
44 beforeAll(async () => {
45 process.env["FORGE_CELLS"] = "off";
46 dataDir = await mkdtemp(join(tmpdir(), "forge-tui-red-main-"));
47 app = createApp({ dataDir, port: 0, host: "forge.test" });
48 const made = await createUser(app.db, {
49 origin: "seed", trusted: true, username: "alice", email: "[email protected]",
50 password: "a-good-passphrase", adminIfFirst: true,
51 });
52 if (!made.ok) throw new Error(String(made.error));
53 alice = made.value;
54 const r = await createRepo(app, alice, { name: "lib", visibility: "public" });
55 if (!r.ok) throw new Error(String(r.error));
56 repo = r.value;
57 for (const branch of ["main", "feat"]) {
58 const c = await commitToBranch(app, alice, repo, { branch, message: branch, files: [{ path: `${branch}.md`, content: "x\n" }] });
59 if (!c.ok) throw new Error(JSON.stringify(c.error));
60 }
61 const o = await openPull(app, alice, repo, { title: "Pull feat", body: "", baseBranch: "main", headBranch: "feat" });
62 if (!o.ok) throw new Error(String(o.error));
63 pull = getPull(app.db, repo.id, o.value.number)!;
64 });
65
66 afterAll(async () => {
67 app.close();
68 if (cells === undefined) delete process.env["FORGE_CELLS"];
69 else process.env["FORGE_CELLS"] = cells;
70 await rm(dataDir, { recursive: true, force: true });
71 });
72
73 /** The pull screen's text, once it has drawn the pull. */
74 async function screen(): Promise<string> {
75 const { app: tui, session } = createForgeApp(
76 app,
77 { sshUsername: "alice", remoteAddress: "127.0.0.1", fingerprint: "SHA256:alice", user: alice },
78 { terminal: headlessTerminal(300, 44), exitHooks: false },
79 );
80 tui.start();
81 try {
82 tui.ctx.push(pullScreen(session, tui.ctx, local(session, repo), pull.number));
83 await settleApp(tui);
84 const text = tui.screen.toText();
85 if (!text.includes(pull.title)) throw new Error(`the pull screen settled without the pull drawn:\n${text}`);
86 return text;
87 } finally {
88 tui.stop();
89 }
90 }
91
92 const RED = 'main is red at aaaaaaa (failing: "sizes hold")';
93
94 test("red main and a pull failing the same test: the screen marks it red on main", async () => {
95 check("main", "a".repeat(40), "failed", ["sizes hold"]);
96 check(`pull-${pull.number}`, pull.headOid, "failed", ["sizes hold"], pull.id);
97 expect(await screen()).toContain(`${RED}: red on main, not on this pull.`);
98 }, 30_000);
99
100 test("a pull with its own failure hears that main is red, without the mark", async () => {
101 check(`pull-${pull.number}`, pull.headOid, "failed", ["sizes hold", "mine"], pull.id);
102 const text = await screen();
103 expect(text).toContain(`${RED}.`);
104 expect(text).not.toContain("red on main, not on this pull");
105 }, 30_000);
106
107 test("held red alone too: the two are one row, joined by ' · '", async () => {
108 const dir = await pullDir(app.db, app.reposRoot, repo, pull);
109 const verdicts = await enqueue(
110 repo.id,
111 [{ pull, dir, land: async () => undefined, release: () => {}, handBack: () => {} }],
112 async () => ({ lands: false, reopen: true, comment: "red", learn: [] }),
113 );
114 expect(verdicts).toEqual([false]);
115 const text = await screen();
116 expect(text).toMatch(new RegExp(`${RED.replace(/[()]/g, "\\$&")}\\. · red alone at \\d{4}-\\d\\d-\\d\\d \\d\\d:\\d\\d UTC, ${RED_ALONE_WAIT}`));
117 }, 30_000);
118
119 test("main green again: no red-main words on the screen", async () => {
120 check("main", "b".repeat(40), "ok", []);
121 check(`pull-${pull.number}`, pull.headOid, "failed", ["sizes hold"], pull.id);
122 expect(await screen()).not.toContain("main is red");
123 }, 30_000);

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs