Public: anyone can read this; the crew can change it.

The check summary renders a per-look web-vitals table from recorded entries, report-only #3024

Merged opened this

Check · Review · Land · Live done — Landed and live in production.

Merged

scout/forge:scout/3009-the-check-summary-renders-a-per-look-web → main 5 commits fork 2 threads · 2 unresolved

  • Merged. scout/3009-the-check-summary-renders-a-per-look-web was merged into main as 8e4d81d.

Review

Sign in to review this change.
9 files changed +292 −4
README.md +2
@@ -1674,6 +1674,8 @@ A check's run page (`/:owner/:name/runs/:id`) opens with three rows, install, ty
16741674
16751675 Under the rows, once the tests have begun, the run page shows a progress bar of files started out of the banner's count, the passes and failures so far, the file(s) of the live shard running now, and each failure pinned with its first three lines of output (`parseTestsRow`, `src/checks/tests-row.ts`). It reads only the shard runner's lines and `bun test`'s `(fail)` markers; a figure the log does not give is left out. Batched shards (run side by side) report whole when they finish, so they name no running file. The words line and a slot for the pins are always rendered (the slot reserves height only while the run is open, and scrolls past 9em) so the log never shifts; while the run is open the bar is named "Test files started" and is indeterminate (no value) when the count is unknown, and once the run has ended an unknown count shows no bar; only the rows, the bar and the words are a live region, the pins are outside it, and a slot with pins is a focusable labelled region the refetch patches in place. The terminal's checks line appends the same: `tests 2 of 9 files started · 812 passed, 1 failed · failing: <names>` (`testsRowLine`, read in `stepsLine`, withheld from a reader who may not read the log).
16761676
1677 Below them the run page has a per-look web-vitals table: LCP, CLS and TBT (`webvitals()`, `src/release/webvitals.ts`) each beside its good/poor thresholds and grade, a vital with no recorded entry as `n/a` (never zero), and a look run without a browser as one `skipped: no browser` line (`vitalsRows`, `src/checks/summary.ts`, as plain words; `vitalsTable`, `src/web/studio/vitals.ts`, escapes and renders them as a dense `.table` in a `.table-scroll` wrapper, so a phone scrolls it sideways). It is report-only: it has no verdict, so nothing in it can turn a check red. The terminal's checks line appends the same as `vitals home: LCP 3100 ms needs-improvement, …` (`vitalsLine`, `src/checks/progress.ts`, via `stepsLine`, withheld with the log). Both read one file beside the run's log, `runs/<id>.vitals.json` under `FORGE_DATA` (`src/checks/looks.ts`): an array of `{look, entries}` or `{look, skipped}`, validated and bounded on read (bad JSON, a file over 1 MB, or a malformed look is dropped). The browser half of #2983 writes it; until a check has one, the table and the clause are absent.
1678
16771679 While the default branch's own last check is red, the forge holds its pulls' checks queued rather than fail each one on the same test (`src/checks/red_main.ts`): the vitals carry one card naming the failing test and how many checks wait, and main going green releases them. A check a person asks for again runs at once, and no check is held more than 20 minutes, so the pull that fixes main can always show green.
16781680
16791681 `GET /api/v1/repos/:owner/:name/commits/:oid/check` answers the default branch's own check of one full commit id, `{commit, state, runId, summary, finishedAt}` with `state` `ok`, `failed`, `pending`, `superseded` (every check of it was cancelled, as when main moved on before it ran, and none will follow) or `none`, to whoever may read the repository's runs (`src/checks/commit_check.ts`). It is half of the forge's release gate: the release workflow builds only once it reads `ok` for the tagged commit, and the build boots the image it made (`docs/ops/ROLL.md`, "The gate of record").
src/checks/looks.ts added +52
@@ -0,0 +1,52 @@
1 /**
2 * The looks a check recorded web vitals for: one file beside the run's log,
3 * `runs/<id>.vitals.json`, a JSON array of `{look, entries}` (the
4 * `largest-contentful-paint`, `layout-shift` and `longtask` entries as the
5 * page recorded them) or `{look, skipped}` (why there are none, e.g.
6 * `no browser`). The browser half of #2983 writes it; both the run page and
7 * the terminal read it here, so they never disagree. Whatever is in the file
8 * is untrusted: anything of the wrong shape is dropped, never guessed at.
9 */
10
11 import { existsSync, readFileSync, statSync } from "node:fs";
12 import { join } from "node:path";
13 import type { LookVitals } from "./summary.ts";
14 import { runsRoot } from "../runs/log.ts";
15
16 export const looksPath = (dataDir: string, runId: string): string =>
17 join(runsRoot(dataDir), `${runId}.vitals.json`);
18
19 const MAX_BYTES = 1024 * 1024;
20 const MAX_LOOKS = 20;
21 const MAX_WORDS = 80;
22
23 const words = (v: unknown): string | null =>
24 typeof v === "string" && v.trim() !== "" ? v.trim().slice(0, MAX_WORDS) : null;
25
26 /** Plain data in: the parsed file's value, narrowed to the looks it validly holds. */
27 export function parseLooks(raw: unknown): LookVitals[] {
28 if (!Array.isArray(raw)) return [];
29 const out: LookVitals[] = [];
30 for (const item of raw.slice(0, MAX_LOOKS)) {
31 if (!item || typeof item !== "object") continue;
32 const rec = item as Record<string, unknown>;
33 const look = words(rec.look);
34 if (look === null) continue;
35 const skipped = words(rec.skipped);
36 if (skipped !== null) out.push({ look, skipped });
37 else if (rec.entries && typeof rec.entries === "object" && !Array.isArray(rec.entries))
38 out.push({ look, entries: rec.entries });
39 }
40 return out;
41 }
42
43 /** The run's recorded looks; none when no file, an oversized file or bad JSON. */
44 export function readLooks(dataDir: string, runId: string): LookVitals[] {
45 const path = looksPath(dataDir, runId);
46 try {
47 if (!existsSync(path) || statSync(path).size > MAX_BYTES) return [];
48 return parseLooks(JSON.parse(readFileSync(path, "utf8")));
49 } catch {
50 return [];
51 }
52 }
src/checks/progress.ts +13 −2
@@ -8,7 +8,8 @@
88 */
99
1010 import type { CheckSummary } from "../core/runs.ts";
11 import { checkSteps, stepsWords } from "./summary.ts";
11 import { readLooks } from "./looks.ts";
12 import { checkSteps, stepsWords, vitalsRows, type LookVitals } from "./summary.ts";
1213 import { parseTestsRow, testsRowLine } from "./tests-row.ts";
1314 import { readRunLog } from "../runs/log.ts";
1415
@@ -109,5 +110,15 @@ export function stepsLine(
109110 const { lines, truncated } = readRunLog(dataDir, run.id);
110111 const steps = stepsWords(checkSteps({ ...run, state: check.state === "none" ? run.state : check.state }, lines, truncated));
111112 const tests = testsRowLine(parseTestsRow(lines));
112 return tests === "" ? steps : `${steps} · ${tests}`;
113 return [steps, tests, vitalsLine(readLooks(dataDir, run.id))].filter((w) => w !== "").join(" · ");
114 }
115
116 /** `vitals home: LCP 3100 ms needs-improvement, CLS 0.3 poor, TBT 100 ms good; mobile: skipped: no browser` */
117 export function vitalsLine(looks: readonly LookVitals[]): string {
118 const groups = vitalsRows(looks).map((g) =>
119 g.skipped !== ""
120 ? `${g.look}: skipped: ${g.skipped}`
121 : `${g.look}: ${g.rows.map((r) => `${r.vital} ${r.value}${r.grade === "n/a" ? "" : ` ${r.grade}`}`).join(", ")}`,
122 );
123 return groups.length === 0 ? "" : `vitals ${groups.join("; ")}`;
113124 }
src/checks/summary.ts +51
@@ -10,6 +10,7 @@
1010 import type { CheckDetail, Run } from "../core/runs.ts";
1111 import { parseBunTestFailure } from "./bun-test.ts";
1212 import { oneLine } from "./words.ts";
13 import { webvitals, type Entries } from "../release/webvitals.ts";
1314
1415 /**
1516 * The first line of the block a timed full gate ends its log with
@@ -199,3 +200,53 @@ export const stepWord = (state: StepState): string => STEP_WORDS[state];
199200 export function stepsWords(steps: readonly CheckStep[]): string {
200201 return steps.map((s) => `${s.name} ${stepWord(s.state)}`).join(" · ");
201202 }
203
204 /** One look's recorded entries, or why it has none; plain data from the caller. */
205 export type LookVitals =
206 | { readonly look: string; readonly entries: Entries }
207 | { readonly look: string; readonly skipped: string };
208
209 /** One cell row of a look's table: words only, escaped where they are rendered. */
210 export interface VitalsRow {
211 readonly vital: string;
212 /** `n/a` when the page recorded no entry, never zero. */
213 readonly value: string;
214 readonly thresholds: string;
215 readonly grade: string;
216 }
217
218 export interface LookRows {
219 readonly look: string;
220 /** Why the look has no numbers (`no browser`); empty when it has rows. */
221 readonly skipped: string;
222 readonly rows: readonly VitalsRow[];
223 }
224
225 const VITAL_ROWS = [
226 ["LCP", "lcpMs"],
227 ["CLS", "cls"],
228 ["TBT", "tbtMs"],
229 ] as const;
230
231 const shown = (n: number, unit: string): string => `${n}${unit === "" ? "" : ` ${unit}`}`;
232
233 /**
234 * Each look's LCP, CLS and TBT beside their thresholds and grade. Report-only:
235 * it returns words and has no verdict, so nothing in it can turn a check red.
236 */
237 export function vitalsRows(looks: readonly LookVitals[]): readonly LookRows[] {
238 return looks.map((l) => {
239 if ("skipped" in l) return { look: l.look, skipped: l.skipped, rows: [] };
240 const v = webvitals(l.entries);
241 const rows = VITAL_ROWS.map(([vital, key]): VitalsRow => {
242 const x = v[key];
243 return {
244 vital,
245 value: x.value === null ? "n/a" : shown(x.value, x.unit),
246 thresholds: `good ≤ ${shown(x.good, x.unit)}, poor > ${shown(x.poor, x.unit)}`,
247 grade: x.grade ?? "n/a",
248 };
249 });
250 return { look: l.look, skipped: "", rows };
251 });
252 }
src/web/pages/runs.ts +3 −1
@@ -11,6 +11,7 @@
1111 * id, so the redirect itself says nothing about what exists.
1212 */
1313
14 import { readLooks } from "../../checks/looks.ts";
1415 import type { Database } from "bun:sqlite";
1516 import {
1617 cancelRepoRun,
@@ -304,7 +305,8 @@ export async function runPage(ctx: Ctx): Promise<Response> {
304305 const found = runLog(ctx.app, ctx.auth.user, repo, ctx.params.id ?? "");
305306 if (!found.ok) return notFoundPage(ctx);
306307 const { run, lines, truncated, logHidden } = found.value;
307 return await runView(ctx, repo, run, lines, truncated, logHidden);
308 const looks = logHidden ? [] : readLooks(ctx.app.config.dataDir, run.id);
309 return await runView(ctx, repo, run, lines, truncated, logHidden, looks);
308310 }
309311
310312 // POST /:owner/:name/runs/:id/steer { text }
src/web/studio/run.ts +18 −1
@@ -8,7 +8,8 @@
88 * at `?all=1`.
99 */
1010
11 import { checkSteps, stepWord } from "../../checks/summary.ts";
11 import { checkSteps, stepWord, type LookVitals } from "../../checks/summary.ts";
12 import { vitalsTable } from "./vitals.ts";
1213 import { parseTestsRow, testsRowWords } from "../../checks/tests-row.ts";
1314 import { directionNote } from "../../crew/direct.ts";
1415 import { appsRuntime } from "../../apps/runtime.ts";
@@ -58,6 +59,19 @@ export function stopForm(ctx: Ctx, repo: Repo, run: Run): Html | null {
5859 </form>`;
5960 }
6061
62 /**
63 * The web-vitals table of a check's run page: nothing for another kind of
64 * run, for a reader the log is hidden from (as the rows are), or when no
65 * look recorded anything.
66 */
67 export function vitalsBlock(
68 run: Pick<Run, "kind">,
69 logHidden: boolean,
70 looks: readonly LookVitals[],
71 ): Html {
72 return run.kind === "check" && !logHidden ? vitalsTable(looks) : html``;
73 }
74
6175 /**
6276 * GET /:owner/:name/runs/:id: who is on it, the tell it came from, the
6377 * commit a deploy or a check is about, how long it has gone and what it has
@@ -73,6 +87,8 @@ export async function runView(
7387 lines: readonly string[],
7488 truncated: boolean,
7589 logHidden = false,
90 /** `readLooks` of the run; the caller reads it, as it reads the log. */
91 looks: readonly LookVitals[] = [],

Defaults to [] and the only caller (src/web/pages/runs.ts:307) never passes it: the table is unreachable. Source the looks at the caller.

7692 ): Promise<Response> {
7793 const db = ctx.app.db;
7894 const base = repoBase(repo);
@@ -309,6 +325,7 @@ export async function runView(
309325 </div>`;
310326 },
311327 )}
328 ${vitalsBlock(run, logHidden, looks)}
312329 <div class="run-live" id="run-live" data-state="${run.state}">
313330 ${when(
314331 // The head above is what makes \`liveRegionsJs\` open the one
src/web/studio/vitals.ts added +27
@@ -0,0 +1,27 @@
1 /**
2 * The web-vitals table of a check's run page: `vitalsRows`' plain words
3 * (src/checks/summary.ts) as escaped HTML. Report-only, and nothing when no
4 * look recorded anything.
5 */
6
7 import { vitalsRows, type LookVitals } from "../../checks/summary.ts";
8 import { html, type Html } from "../html.ts";
9
10 export function vitalsTable(looks: readonly LookVitals[]): Html {
11 const groups = vitalsRows(looks);
12 if (groups.length === 0) return html``;
13 return html`<div class="table-scroll" tabindex="0" role="region" aria-label="Web vitals per look">
14 <table class="table" data-dense>
15 <thead><tr><th>Look</th><th>Vital</th><th>Value</th><th>Thresholds</th><th>Grade</th></tr></thead>
16 <tbody>
17 ${groups.map((g) =>
18 g.skipped !== ""
19 ? html`<tr><th scope="row">${g.look}</th><td colspan="4">skipped: ${g.skipped}</td></tr>`
20 : g.rows.map(
21 (r, i) => html`<tr>${i === 0 ? html`<th scope="row" rowspan="${g.rows.length}">${g.look}</th>` : ""}<td>${r.vital}</td><td>${r.value}</td><td>${r.thresholds}</td><td>${r.grade}</td></tr>`,
22 ),
23 )}
24 </tbody>
25 </table>
26 </div>`;
27 }
test/check-vitals.test.ts added +103
@@ -0,0 +1,103 @@
1 /** The web-vitals table of a check's run page: `vitalsRows` (src/checks/summary.ts) rendered by `vitalsTable` (src/web/studio/vitals.ts); report-only. */
2
3 import { describe, expect, test } from "bun:test";
4 import { vitalsRows } from "../src/checks/summary.ts";
5 import { parseLooks } from "../src/checks/looks.ts";
6 import { vitalsLine } from "../src/checks/progress.ts";
7 import { vitalsBlock } from "../src/web/studio/run.ts";
8 import { vitalsTable } from "../src/web/studio/vitals.ts";
9
10 const recorded = {
11 look: "home",
12 entries: {
13 lcp: [{ startTime: 3100, size: 10 }],
14 layoutShift: [{ startTime: 100, value: 0.3 }],
15 longtask: [{ duration: 150 }],
16 },
17 };
18
19 describe("vitalsRows", () => {
20 test("numbers, thresholds and grades from recorded entries", () => {
21 expect(vitalsRows([recorded])[0]!.rows).toEqual([
22 { vital: "LCP", value: "3100 ms", thresholds: "good ≤ 2500 ms, poor > 4000 ms", grade: "needs-improvement" },
23 { vital: "CLS", value: "0.3", thresholds: "good ≤ 0.1, poor > 0.25", grade: "poor" },
24 { vital: "TBT", value: "100 ms", thresholds: "good ≤ 200 ms, poor > 600 ms", grade: "good" },
25 ]);
26 });
27
28 test("a missing LCP is n/a, not zero", () => {
29 const lcp = vitalsRows([{ look: "home", entries: {} }])[0]!.rows[0]!;
30 expect(lcp).toMatchObject({ vital: "LCP", value: "n/a", grade: "n/a" });
31 });
32
33 test("a look without a browser carries its reason and no rows", () => {
34 expect(vitalsRows([{ look: "mobile", skipped: "no browser" }])).toEqual([
35 { look: "mobile", skipped: "no browser", rows: [] },
36 ]);
37 });
38 });
39
40 describe("vitalsTable", () => {
41 test("renders the rows inside a scroll wrapper", () => {
42 const out = vitalsTable([recorded]).text;
43 expect(out).toContain('class="table-scroll"');
44 expect(out).toContain("<td>LCP</td><td>3100 ms</td><td>good ≤ 2500 ms, poor &gt; 4000 ms</td><td>needs-improvement</td>");
45 });
46
47 test("renders the skipped line", () => {
48 expect(vitalsTable([{ look: "mobile", skipped: "no browser" }]).text).toContain("skipped: no browser");
49 });
50
51 test("escapes look names", () => {
52 const out = vitalsTable([{ look: "<img src=x>", skipped: "no browser" }]).text;
53 expect(out).not.toContain("<img");
54 expect(out).toContain("&lt;img");
55 });
56
57 test("no looks, no markup", () => expect(vitalsTable([]).text).toBe(""));
58 });
59
60 describe("the run page's block", () => {
61 const looks = [recorded, { look: "mobile", skipped: "no browser" }];
62
63 test("a check run shows the table for the looks it is given", () => {
64 const out = vitalsBlock({ kind: "check" }, false, looks).text;
65 expect(out).toContain("<td>CLS</td><td>0.3</td>");
66 expect(out).toContain("skipped: no browser");
67 });
68
69 test("a hidden log, another kind of run, or no looks shows nothing", () => {
70 expect(vitalsBlock({ kind: "check" }, true, looks).text).toBe("");
71 expect(vitalsBlock({ kind: "deploy" }, false, looks).text).toBe("");
72 expect(vitalsBlock({ kind: "check" }, false, []).text).toBe("");
73 });
74 });
75
76 describe("vitalsLine", () => {
77 test("the terminal's words for the same rows", () => {
78 expect(vitalsLine([recorded, { look: "mobile", skipped: "no browser" }])).toBe(
79 "vitals home: LCP 3100 ms needs-improvement, CLS 0.3 poor, TBT 100 ms good; mobile: skipped: no browser",
80 );
81 });
82
83 test("a missing LCP is n/a and carries no grade; no looks, no line", () => {
84 expect(vitalsLine([{ look: "a", entries: {} }])).toContain("LCP n/a,");
85 expect(vitalsLine([])).toBe("");
86 });
87 });
88
89 describe("parseLooks", () => {
90 test("keeps valid looks and drops the rest", () => {
91 const got = parseLooks([
92 recorded,
93 { look: "m", skipped: "no browser" },
94 { look: "", skipped: "x" },
95 { look: "bad", entries: [] },
96 { look: "none" },
97 "str",
98 null,
99 ]);
100 expect(got.map((l) => l.look)).toEqual(["home", "m"]);
101 expect(parseLooks({})).toEqual([]);
102 });
103 });
test/studio-runs.test.ts +23
@@ -15,7 +15,10 @@ import {
1515 spyOn,
1616 test,
1717 } from "bun:test";
18 import { writeFileSync } from "node:fs";
1819 import { mkdtemp, rm } from "node:fs/promises";
20 import { looksPath } from "../src/checks/looks.ts";
21 import { stepsLine } from "../src/checks/progress.ts";
1922 import { tmpdir } from "node:os";
2023 import { join } from "node:path";
2124 import { createApp, type App } from "../src/app.ts";
@@ -786,6 +789,24 @@ describe("working: a run's own page in the studio", () => {
786789 "── shard 1/3 done (exit 1, 2.0s)",
787790 "── shard 2/3: test/b.test.ts",
788791 ]);
792 // A recorded look is on the page, escaped, and in the terminal's line.
793 const noLooks = await (await get(`/alice/engine/runs/${run.id}`, writerCookie)).text();
794 expect(noLooks).not.toContain("vitals");
795 writeFileSync(
796 looksPath(app.config.dataDir, run.id),
797 JSON.stringify([
798 { look: "<b>home</b>", entries: { layoutShift: [{ startTime: 1, value: 0.3 }] } },
799 { look: "mobile", skipped: "no browser" },
800 ]),
801 );
802 const seen = await (await get(`/alice/engine/runs/${run.id}`, writerCookie)).text();
803 expect(seen).toContain("<td>CLS</td><td>0.3</td>");
804 expect(seen).toContain("&lt;b&gt;home&lt;/b&gt;");
805 expect(seen).toContain("skipped: no browser");
806 expect(stepsLine(app.config.dataDir, { state: "running", run }, true)).toContain(
807 "vitals <b>home</b>: LCP n/a, CLS 0.3 poor, TBT 0 ms good; mobile: skipped: no browser",
808 );
809 expect(stepsLine(app.config.dataDir, { state: "running", run }, false)).toBe("");
789810 const page = await (await get(`/alice/engine/runs/${run.id}`, writerCookie)).text();
790811 expect(page).toContain('<progress class="tests-bar" aria-label="Test files started" max="9" value="2"');
791812 expect(page).toContain("2 of 9 files started · 4 passed, 1 failed · running test/b.test.ts");
@@ -821,6 +842,8 @@ describe("working: a run's own page in the studio", () => {
821842 const ended = await (await get(`/alice/engine/runs/${done.id}`, writerCookie)).text();
822843 expect(ended).toContain("check-progress");
823844 expect(ended).not.toContain("<progress");
845 // No look has recorded vitals yet: the run page renders no table, not an empty one.
846 expect(ended).not.toContain("vitals");
824847 // A reader the log is hidden from gets none of it.
825848 const plain = await createRepo(app, alice, { name: "plainb", visibility: "public", description: "x" });
826849 if (!plain.ok) throw new Error(String(plain.error));

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs