trevato/forge

Public: anyone can read this; the crew can change it.

More
Forks1g fMapg aRuns15g rEventsFlows

The looks run records performance entries in the browser it already drives and hands them to the summary #3038

Merged scoutcrew opened this

Check · Review · Land · Live done — Landed and live in production.

ConversationFiles
Merged

scout/forge:scout/3010-the-looks-run-records-performance-entrie → main 3 commits fork

  • Merged. scout/3010-the-looks-run-records-performance-entrie was merged into main as 1bbce1d.

Review

Sign in to review this change.
8 files changed +503 −84
README.md +1 −1
@@ -1695,7 +1695,7 @@ A check's run page (`/:owner/:name/runs/:id`) opens with three rows, install, ty
16951695
16961696 Under the rows, once the tests have begun, the run page shows a progress bar of files started out of the banner's count, the passes and failures so far, the file(s) of the live shard running now, and each failure pinned with its first three lines of output (`parseTestsRow`, `src/checks/tests-row.ts`). It reads only the shard runner's lines and `bun test`'s `(fail)` markers; a figure the log does not give is left out. Batched shards (run side by side) report whole when they finish, so they name no running file. The words line and a slot for the pins are always rendered (the slot reserves height only while the run is open, and scrolls past 9em) so the log never shifts; while the run is open the bar is named "Test files started" and is indeterminate (no value) when the count is unknown, and once the run has ended an unknown count shows no bar; only the rows, the bar and the words are a live region, the pins are outside it, and a slot with pins is a focusable labelled region the refetch patches in place. The terminal's checks line appends the same: `tests 2 of 9 files started · 812 passed, 1 failed · failing: <names>` (`testsRowLine`, read in `stepsLine`, withheld from a reader who may not read the log).
16971697
1698 Below them the run page has a per-look web-vitals table: LCP, CLS and TBT (`webvitals()`, `src/release/webvitals.ts`) each beside its good/poor thresholds and grade, a vital with no recorded entry as `n/a` (never zero), and a look run without a browser as one `skipped: no browser` line (`vitalsRows`, `src/checks/summary.ts`, as plain words; `vitalsTable`, `src/web/studio/vitals.ts`, escapes and renders them as a dense `.table` in a `.table-scroll` wrapper, so a phone scrolls it sideways). It is report-only: it has no verdict, so nothing in it can turn a check red. The terminal's checks line appends the same as `vitals home: LCP 3100 ms needs-improvement, …` (`vitalsLine`, `src/checks/progress.ts`, via `stepsLine`, withheld with the log). Both read one file beside the run's log, `runs/<id>.vitals.json` under `FORGE_DATA` (`src/checks/looks.ts`): an array of `{look, entries}` or `{look, skipped}`, validated and bounded on read (bad JSON, a file over 1 MB, or a malformed look is dropped). The browser half of #2983 writes it; until a check has one, the table and the clause are absent.
1698 Below them the run page has a per-look web-vitals table: LCP, CLS and TBT (`webvitals()`, `src/release/webvitals.ts`) each beside its good/poor thresholds and grade, a vital with no recorded entry as `n/a` (never zero), and a look run without a browser as one `skipped: no browser` line (`vitalsRows`, `src/checks/summary.ts`, as plain words; `vitalsTable`, `src/web/studio/vitals.ts`, escapes and renders them as a dense `.table` in a `.table-scroll` wrapper, so a phone scrolls it sideways). It is report-only: it has no verdict, so nothing in it can turn a check red. The terminal's checks line appends the same as `vitals home: LCP 3100 ms needs-improvement, …` (`vitalsLine`, `src/checks/progress.ts`, via `stepsLine`, withheld with the log). Both read one file beside the run's log, `runs/<id>.vitals.json` under `FORGE_DATA` (`src/checks/looks.ts`): an array of `{look, entries}` or `{look, skipped}`, validated and bounded on read (bad JSON, a file over 1 MB, or a malformed look is dropped). The looks run writes it (`startLooks`, `src/looks/service.ts`): the browser it already launches for a look (`src/looks/capture.ts`) opens on `about:blank` with a loopback DevTools port, installs a `PerformanceObserver` for `largest-contentful-paint`, `layout-shift` and `longtask` before the page's first script, navigates through the grant door, and after load reads the entries back and takes the picture over the same socket (no new process, no dependency, no Lighthouse; a type the browser lacks is an empty list). The phone and desktop looks are filed under the commit they picture (`runs/commit-<repo>-<oid>.vitals.json`, `writeCommitLooks`), so they become rows of that commit's check whenever it exists, even if it was made after the look (a run's own `runs/<id>.vitals.json` wins); with no browser they are `skipped: no browser` and the check is untouched. Until a check has a file, the table and the clause are absent.
16991699
17001700 While the default branch's own last check is red, the forge holds its pulls' checks queued rather than fail each one on the same test (`src/checks/red_main.ts`): the vitals carry one card naming the failing test and how many checks wait, and main going green releases them. A check a person asks for again runs at once, and no check is held more than 20 minutes, so the pull that fixes main can always show green.
17011701
src/checks/looks.ts +58 −4
@@ -8,7 +8,7 @@
88 * is untrusted: anything of the wrong shape is dropped, never guessed at.
99 */
1010
11 import { existsSync, readFileSync, statSync } from "node:fs";
11 import { existsSync, mkdirSync, readFileSync, renameSync, rmSync, statSync, writeFileSync } from "node:fs";
1212 import { join } from "node:path";
1313 import type { LookVitals } from "./summary.ts";
1414 import { runsRoot } from "../runs/log.ts";
@@ -16,6 +16,19 @@ import { runsRoot } from "../runs/log.ts";
1616 export const looksPath = (dataDir: string, runId: string): string =>
1717 join(runsRoot(dataDir), `${runId}.vitals.json`);
1818
19 /**
20 * Looks are filed under the commit they picture, beside the run files, so
21 * they are there whenever the commit's check is: before it exists, after it
22 * is queued, or past any page of recent runs. A run's own file wins.
23 */
24 export const commitLooksPath = (
25 dataDir: string,
26 repoId: string,
27 commitOid: string,
28 ): string => join(runsRoot(dataDir), `commit-${repoId}-${commitOid}.vitals.json`);
29
30 const SAFE = /^[0-9A-Za-z]{1,64}$/;
31
1932 const MAX_BYTES = 1024 * 1024;
2033 const MAX_LOOKS = 20;
2134 const MAX_WORDS = 80;
@@ -40,9 +53,7 @@ export function parseLooks(raw: unknown): LookVitals[] {
4053 return out;
4154 }
4255
43 /** The run's recorded looks; none when no file, an oversized file or bad JSON. */
44 export function readLooks(dataDir: string, runId: string): LookVitals[] {
45 const path = looksPath(dataDir, runId);
56 function readFile(path: string): LookVitals[] {
4657 try {
4758 if (!existsSync(path) || statSync(path).size > MAX_BYTES) return [];
4859 return parseLooks(JSON.parse(readFileSync(path, "utf8")));
@@ -50,3 +61,46 @@ export function readLooks(dataDir: string, runId: string): LookVitals[] {
5061 return [];
5162 }
5263 }
64
65 /** What `readLooks` needs of a run: its id and the commit it checks. */
66 export interface LooksOf {
67 readonly id: string;
68 readonly repoId: string;
69 readonly commitOid: string;
70 }
71
72 /**
73 * The run's recorded looks: its own file, else those filed under its
74 * commit; none when no file, an oversized file or bad JSON.
75 */
76 export function readLooks(dataDir: string, run: LooksOf): LookVitals[] {
77 const own = readFile(looksPath(dataDir, run.id));
78 if (own.length > 0 || !SAFE.test(run.commitOid) || !SAFE.test(run.repoId))
79 return own;
80 return readFile(commitLooksPath(dataDir, run.repoId, run.commitOid));
81 }
82
83 /**
84 * File a commit's looks, replacing any earlier set for it: written whole
85 * beside the log and moved in, so a reader never sees half of it. What
86 * `readLooks` would drop is not written, and a set with nothing left removes
87 * the earlier file, so a stale table never outlives its replacement.
88 */
89 export function writeCommitLooks(
90 dataDir: string,
91 repoId: string,
92 commitOid: string,
93 looks: readonly LookVitals[],
94 ): void {
95 if (!SAFE.test(commitOid) || !SAFE.test(repoId)) return;
96 const path = commitLooksPath(dataDir, repoId, commitOid);
97 const kept = parseLooks(looks);
98 if (kept.length === 0) {
99 rmSync(path, { force: true });
100 return;
101 }
102 mkdirSync(runsRoot(dataDir), { recursive: true });
103 const side = `${path}.${process.pid}.tmp`;
104 writeFileSync(side, JSON.stringify(kept));
105 renameSync(side, path);
106 }
src/checks/progress.ts +1 −1
@@ -110,7 +110,7 @@ export function stepsLine(
110110 const { lines, truncated } = readRunLog(dataDir, run.id);
111111 const steps = stepsWords(checkSteps({ ...run, state: check.state === "none" ? run.state : check.state }, lines, truncated));
112112 const tests = testsRowLine(parseTestsRow(lines));
113 return [steps, tests, vitalsLine(readLooks(dataDir, run.id))].filter((w) => w !== "").join(" · ");
113 return [steps, tests, vitalsLine(readLooks(dataDir, run))].filter((w) => w !== "").join(" · ");
114114 }
115115
116116 /** `vitals home: LCP 3100 ms needs-improvement, CLS 0.3 poor, TBT 100 ms good; mobile: skipped: no browser` */
src/looks/capture.ts +325 −71
@@ -1,43 +1,46 @@
11 /**
2 * Taking a picture of a page. Headless Chromium is spawned with the one
3 * flag that writes a screenshot, pointed at a URL on the forge's own
4 * listener with the app's hostname resolved to loopback, so the page is
5 * served the way a person would be served it — through the proxy, the
6 * grant, and the identity headers — without an edge in the way.
2 * Taking a picture of a page. Headless Chromium is spawned pointed at a
3 * URL on the forge's own listener with the app's hostname resolved to
4 * loopback, so the page is served the way a person would be served it —
5 * through the proxy, the grant, and the identity headers — without an edge
6 * in the way.
77 *
8 * Chromium's `--screenshot` sets no headers and no cookies. So the URL is
9 * not the page but the app's grant door (`/__forge/grant?t=…&to=/`): that
10 * one navigation sets the grant cookie and lands on the page as the person
11 * the grant names. Chromium follows the redirect and pictures where it
12 * lands.
8 * The URL is not the page but the app's grant door
9 * (`/__forge/grant?t=…&to=/`): that one navigation sets the grant cookie and
10 * lands on the page as the person the grant names. The browser starts on
11 * `about:blank` with its DevTools port open on loopback (no new process:
12 * the one it already launches), a script that observes `largest-contentful-
13 * paint`, `layout-shift` and `longtask` is installed to run before any
14 * page script, and only then is the grant door opened. After load the
15 * entries are read back and the picture is taken, both over that one socket.
16 * An entry type the browser lacks is an empty list, never a failure.
1317 *
14 * The picture is taken as the page loads, bounded by `--timeout` for a
15 * page that never finishes — not after a virtual-time budget: a page that
16 * holds an event stream open (the scaffold's live script does) never goes
17 * idle, and a budget waits forever for it. What an app draws after load is
18 * not in the picture; what it serves is. And the picture, not the
19 * browser's exit, is what is waited for: a PNG whose IEND chunk is on disk
20 * is complete, and a browser that has written it is killed rather than
21 * asked to leave — Chrome 144 on macOS writes the file and then sits.
18 * The DevTools port is open on loopback while this browser holds the grant
19 * cookie, so a local process could drive that page. That is accepted: the
20 * browser lives for one look (seconds, killed in `finally`), its profile is
21 * its own and deleted after, the port is random and written only into that
22 * profile, `--remote-allow-origins` admits no web origin, and a local
23 * process able to reach loopback can already reach the app through the
24 * forge's own listener.
25 *
26 * The picture is taken once the page has loaded and settled a moment,
27 * bounded for a page that never finishes — not after a virtual-time budget:
28 * a page that holds an event stream open (the scaffold's live script does)
29 * never goes idle, and a budget waits forever for it. What an app draws long
30 * after load is not in the picture; what it serves is. The browser is killed
31 * when it has given the picture, not asked to leave.
2232 *
2333 * One picture at a time, forge-wide: a browser is the heaviest thing this
2434 * process starts, and a pod that holds two agent runs has room for one.
2535 */
2636
27 import {
28 closeSync,
29 existsSync,
30 mkdtempSync,
31 openSync,
32 readSync,
33 rmSync,
34 statSync,
35 } from "node:fs";
37 import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
3638 import { tmpdir } from "node:os";
3739 import { join } from "node:path";
3840 import { childEnv } from "../apps/process.ts";
3941 import { Gate } from "../git/env.ts";
4042 import { err, ok, type Result } from "../kernel/result.ts";
43 import type { Entries } from "../release/webvitals.ts";
4144
4245 export interface LookInput {
4346 /** The URL to open — the grant door of the app's hostname, on the forge's own port. */
@@ -50,8 +53,23 @@ export interface LookInput {
5053 readonly timeoutMs: number;
5154 }
5255
53 /** What takes the picture: the real browser, or a test's stand-in. */
54 export type Capture = (input: LookInput) => Promise<Result<void, string>>;
56 /** What the page recorded while it loaded; a type the browser lacks is an empty list. */
57 export interface Recorded {
58 readonly entries: Entries;
59 }
60
61 /**
62 * What takes the picture: the real browser, or a test's stand-in. One that
63 * recorded nothing returns empty entries.
64 */
65 export type Capture = (input: LookInput) => Promise<Result<Recorded, string>>;
66
67 /** Why there is no picture and no entries when no browser is found. */
68 export const NO_BROWSER =
69 "no chromium on this forge: install one, or set FORGE_CHROMIUM";
70
71 /** Whether a capture's error is the want of a browser, not a browser's failure. */
72 export const isNoBrowser = (error: string): boolean => error === NO_BROWSER;
5573
5674 /** How long a page gets to load before it is pictured as it stands. */
5775 const LOAD_MS = 3_000;
@@ -82,10 +100,10 @@ const gate = new Gate(1);
82100 /** Take one picture. Serialised: the second caller waits for the first. */
83101 export async function takeLook(
84102 input: LookInput,
85 ): Promise<Result<void, string>> {
103 ): Promise<Result<Recorded, string>> {
86104 const binary = findChromium();
87105 if (binary === undefined)
88 return err("no chromium on this forge: install one, or set FORGE_CHROMIUM");
106 return err(NO_BROWSER);
89107 const release = await gate.acquire();
90108 try {
91109 return await shoot(binary, input);
@@ -94,28 +112,197 @@ export async function takeLook(
94112 }
95113 }
96114
97 /** The last eight bytes of every PNG: the IEND chunk and its CRC. */
98 const PNG_END = Buffer.from([0x49, 0x45, 0x4e, 0x44, 0xae, 0x42, 0x60, 0x82]);
115 /** Entries kept per type: a page that shifts a thousand times has said enough. */
116 const MAX_ENTRIES = 500;
99117
100 /** Whether a complete picture is on disk at `path`. */
101 function pictureDone(path: string): boolean {
102 if (!existsSync(path)) return false;
103 const size = statSync(path).size;
104 if (size < 64) return false;
105 const fd = openSync(path, "r");
106 try {
107 const tail = Buffer.alloc(8);
108 readSync(fd, tail, 0, 8, size - 8);
109 return tail.equals(PNG_END);
110 } finally {
111 closeSync(fd);
118 /** How long after load the page gets to settle (the last paint, the last shift). */
119 const SETTLE_MS = 400;
120
121 /** Runs before any page script: collects the entries the summary grades. */
122 const OBSERVER = `(() => {
123 if (window.top !== window) return;
124 const v = { lcp: [], layoutShift: [], longtask: [] };
125 window.__forgeVitals = v;
126 const watch = (type, key, pick) => {
127 try {
128 new PerformanceObserver((list) => {
129 for (const e of list.getEntries()) v[key].push(pick(e));
130 }).observe({ type, buffered: true });
131 } catch (_) {}
132 };
133 watch("largest-contentful-paint", "lcp", (e) => ({ startTime: e.startTime, size: e.size }));
134 watch("layout-shift", "layoutShift", (e) => ({ startTime: e.startTime, value: e.value, hadRecentInput: e.hadRecentInput }));
135 watch("longtask", "longtask", (e) => ({ startTime: e.startTime, duration: e.duration }));
136 })()`;
137
138 const finite = (v: unknown): v is number =>
139 typeof v === "number" && Number.isFinite(v) && v >= 0;
140
141 /**
142 * What the page said, narrowed to what is safe to keep: plain data in, only
143 * finite numbers (and the one boolean) out, each list bounded. Whatever the
144 * page wrote is untrusted; a missing or malformed type is an empty list.
145 */
146 export function parseRecorded(raw: unknown): Recorded {
147 const rec =
148 raw !== null && typeof raw === "object"
149 ? (raw as Record<string, unknown>)
150 : {};
151 const list = (
152 key: string,
153 fields: readonly string[],
154 required: string,
155 ): object[] => {
156 const items = rec[key];
157 if (!Array.isArray(items)) return [];
158 const out: object[] = [];
159 for (const item of items.slice(0, MAX_ENTRIES)) {
160 if (item === null || typeof item !== "object") continue;
161 const from = item as Record<string, unknown>;
162 const entry: Record<string, number | boolean> = {};
163 for (const f of fields) {
164 const v = from[f];
165 if (finite(v)) entry[f] = v;
166 else if (f === "hadRecentInput" && typeof v === "boolean") entry[f] = v;
167 }
168 // An entry without its defining number is not an entry.
169 if (required in entry) out.push(entry);
170 }
171 return out;
172 };
173 return {
174 entries: {
175 lcp: list("lcp", ["startTime", "size"], "startTime"),
176 layoutShift: list("layoutShift", ["startTime", "value", "hadRecentInput"], "value"),
177 longtask: list("longtask", ["startTime", "duration"], "startTime"),
178 },
179 };
180 }
181
182 /** The DevTools endpoint Chromium writes into its profile once it listens. */
183 async function endpoint(
184 profile: string,
185 stillRunning: () => boolean,
186 deadline: number,
187 ): Promise<string | undefined> {
188 const file = join(profile, "DevToolsActivePort");
189 while (stillRunning() && Date.now() < deadline) {
190 try {
191 const [port, path] = readFileSync(file, "utf8").split("\n");
192 if (/^\d{1,5}$/.test(port ?? "") && (path ?? "").startsWith("/devtools/"))
193 return `ws://127.0.0.1:${port}${path}`;
194 } catch {
195 // Not written yet.
196 }
197 await Bun.sleep(50);
112198 }
199 return undefined;
200 }
201
202 /** One DevTools message: a reply (`id`) or an event (`method`). */
203 interface Message {
204 readonly id?: number;
205 readonly method?: string;
206 readonly sessionId?: string;
207 readonly result?: Record<string, unknown>;
208 readonly error?: { readonly message?: string };
209 }
210
211 /** A DevTools connection: commands answered by id, events heard by method. */
212 class Devtools {
213 private next = 1;
214 private readonly waiting = new Map<
215 number,
216 { resolve: (v: Record<string, unknown>) => void; reject: (e: Error) => void }
217 >();
218 private readonly listeners = new Set<(method: string, sessionId: string) => void>();
219
220 private constructor(private readonly socket: WebSocket) {
221 socket.onmessage = (event) => {
222 let msg: Message;
223 try {
224 msg = JSON.parse(String(event.data)) as Message;
225 } catch {
226 return;
227 }
228 if (typeof msg.id === "number") {
229 const w = this.waiting.get(msg.id);
230 this.waiting.delete(msg.id);
231 if (msg.error) w?.reject(new Error(String(msg.error.message ?? "failed")));
232 else w?.resolve(msg.result ?? {});
233 } else if (typeof msg.method === "string")
234 for (const l of this.listeners) l(msg.method, msg.sessionId ?? "");
235 };
236 socket.onclose = () => this.fail(new Error("the browser closed the connection"));
237 socket.onerror = () => this.fail(new Error("the browser connection failed"));
238 }
239
240 static connect(url: string): Promise<Devtools> {
241 return new Promise((resolve, reject) => {
242 const socket = new WebSocket(url);
243 const dt = new Devtools(socket);
244 socket.onopen = () => resolve(dt);
245 const early = () => reject(new Error("could not reach the browser's DevTools"));
246 socket.addEventListener("error", early, { once: true });
247 });
248 }
249
250 private fail(error: Error): void {
251 for (const w of this.waiting.values()) w.reject(error);
252 this.waiting.clear();
253 }
254
255 send(
256 method: string,
257 params: object = {},
258 sessionId?: string,
259 ): Promise<Record<string, unknown>> {
260 const id = this.next++;
261 return new Promise((resolve, reject) => {
262 this.waiting.set(id, { resolve, reject });
263 this.socket.send(
264 JSON.stringify({ id, method, params, ...(sessionId ? { sessionId } : {}) }),
265 );
266 });
267 }
268
269 /** Resolves when `method` is next heard on `sessionId`. */
270 heard(method: string, sessionId: string): Promise<void> {
271 return new Promise((resolve) => {
272 const l = (m: string, s: string) => {
273 if (m !== method || s !== sessionId) return;
274 this.listeners.delete(l);
275 resolve();
276 };
277 this.listeners.add(l);
278 });
279 }
280
281 close(): void {
282 try {
283 this.socket.close();
284 } catch {
285 // Already gone.
286 }
287 }
288 }
289
290 /** Rejects with `what` when the deadline passes before `work` settles. */
291 function within<T>(work: Promise<T>, deadline: number, what: string): Promise<T> {
292 let timer: ReturnType<typeof setTimeout>;
293 const late = new Promise<never>((_, reject) => {
294 timer = setTimeout(
295 () => reject(new Error(what)),
296 Math.max(0, deadline - Date.now()),
297 );
298 });
299 return Promise.race([work, late]).finally(() => clearTimeout(timer));
113300 }
114301
115302 async function shoot(
116303 binary: string,
117304 input: LookInput,
118 ): Promise<Result<void, string>> {
305 ): Promise<Result<Recorded, string>> {
119306 const host = new URL(input.url).hostname;
120307 // A profile of its own, gone afterwards: no cookie outlives its picture,
121308 // and two forges in one process never share a browser's state.
@@ -143,9 +330,10 @@ async function shoot(
143330 `--window-size=${input.width},${input.height}`,
144331 "--force-device-scale-factor=1",
145332 "--js-flags=--max-old-space-size=256",
146 `--timeout=${LOAD_MS}`,
147 `--screenshot=${input.out}`,
148 input.url,
333 // Loopback only, on a port the browser picks and writes down.
334 "--remote-debugging-port=0",
335 "--remote-allow-origins=http://127.0.0.1",
336 "about:blank",
149337 ],
150338 {
151339 env: childEnv({}),
@@ -169,32 +357,98 @@ async function shoot(
169357 }
170358 })().catch(() => {});
171359 const deadline = Date.now() + input.timeoutMs;
360 const seconds = Math.round(input.timeoutMs / 1000);
361 let devtools: Devtools | undefined;
172362 try {
173 for (;;) {
174 if (pictureDone(input.out)) return ok();
175 if (exited !== undefined) {
176 // The browser's last words reach the pipe a beat after it exits.
177 await Bun.sleep(150);
178 break;
179 }
180 if (Date.now() >= deadline)
181 return err(
182 `the picture took longer than ${Math.round(input.timeoutMs / 1000)}s`,
183 );
184 await Bun.sleep(100);
363 const url = await endpoint(profile, () => exited === undefined, deadline);
364 if (url === undefined) {
365 if (exited === undefined)
366 return err(`the picture took longer than ${seconds}s`);
367 // The browser's last words reach the pipe a beat after it exits.
368 await Bun.sleep(150);
369 // Chromium narrates its own housekeeping on stderr; the last line is
370 // the one that says what went wrong, when anything did.
371 const last = said
372 .join("")
373 .split("\n")
374 .map((l) => l.trim())
375 .filter((l) => l !== "" && !/crashpad|ProcessSingleton/i.test(l))
376 .at(-1);
377 return err(
378 `chromium exited ${exited} without a picture${last === undefined ? "" : `: ${last.slice(0, 300)}`}`,
379 );
185380 }
186 // Chromium narrates its own housekeeping on stderr; the last line is
187 // the one that says what went wrong, when anything did.
188 const last = said
189 .join("")
190 .split("\n")
191 .map((l) => l.trim())
192 .filter((l) => l !== "" && !/crashpad|ProcessSingleton/i.test(l))
193 .at(-1);
194 return err(
195 `chromium exited ${exited} without a picture${last === undefined ? "" : `: ${last.slice(0, 300)}`}`,
381 const dt = (devtools = await within(
382 Devtools.connect(url),
383 deadline,
384 `the picture took longer than ${seconds}s`,
385 ));
386 const work = (async (): Promise<Recorded> => {
387 const { targetInfos } = (await dt.send("Target.getTargets")) as {
388 targetInfos: { type: string; targetId: string }[];
389 };
390 const page = targetInfos.find(
391 (t) => t.type === "page",
392 );
393 if (page === undefined) throw new Error("the browser opened no page");
394 const { sessionId } = (await dt.send("Target.attachToTarget", {
395 targetId: page.targetId,
396 flatten: true,
397 })) as { sessionId: string };
398 await dt.send("Page.enable", {}, sessionId);
399 await dt.send(
400 "Emulation.setDeviceMetricsOverride",
401 {
402 width: input.width,
403 height: input.height,
404 deviceScaleFactor: 1,
405 mobile: false,
406 },
407 sessionId,
408 );
409 // Before navigation: the observer is in place before the page's first script.
410 await dt.send(
411 "Page.addScriptToEvaluateOnNewDocument",
412 { source: OBSERVER },
413 sessionId,
414 );
415 const loaded = dt.heard("Page.loadEventFired", sessionId);
416 await dt.send("Page.navigate", { url: input.url }, sessionId);
417 // A page that never finishes loading is pictured as it stands.
418 await Promise.race([loaded, Bun.sleep(LOAD_MS)]);
419 await Bun.sleep(SETTLE_MS);
420 let recorded = parseRecorded(undefined);
421 try {
422 const read = await dt.send(
423 "Runtime.evaluate",
424 {
425 expression: "JSON.stringify(window.__forgeVitals ?? null)",
426 returnByValue: true,
427 },
428 sessionId,
429 );
430 const value = (read["result"] as { value?: unknown } | undefined)?.value;
431 recorded = parseRecorded(JSON.parse(String(value ?? "null")));
432 } catch {
433 // No entries is not a failed look: the picture is still wanted.
434 }
435 const shot = await dt.send(
436 "Page.captureScreenshot",
437 { format: "png" },
438 sessionId,
439 );
440 writeFileSync(input.out, Buffer.from(String(shot.data), "base64"));
441 return recorded;
442 })();
443 work.catch(() => {});
444 return ok(
445 await within(work, deadline, `the picture took longer than ${seconds}s`),
196446 );
447 } catch (error) {
448 const message = error instanceof Error ? error.message : String(error);
449 return err(message.startsWith("the picture took") ? message : `no picture: ${message}`);
197450 } finally {
451 devtools?.close();
198452 if (exited === undefined) proc.kill("SIGKILL");
199453 await proc.exited.catch(() => {});
200454 reader.cancel().catch(() => {});
src/looks/service.ts +20 −1
@@ -61,7 +61,9 @@ import { thumbnail } from "../design/image/image.ts";
6161 import { decodePng, encodePng, pngHeader } from "../design/image/png.ts";
6262 import { createLog } from "../kernel/log.ts";
6363 import { err, ok, type Result } from "../kernel/result.ts";
64 import { findChromium, takeLook, type Capture } from "./capture.ts";
64 import { writeCommitLooks } from "../checks/looks.ts";
65 import type { LookVitals } from "../checks/summary.ts";
66 import { findChromium, isNoBrowser, takeLook, type Capture } from "./capture.ts";
6567
6668 const log = createLog("looks");
6769
@@ -370,6 +372,8 @@ export function startLooks(app: App, options: LooksOptions): LooksHandle {
370372 : { id: owner.id, name: owner.username, role: "admin" };
371373 const host = runtime.hostFor(record.slug, item.env);
372374 let allOk = true;
375 // What each look's page recorded, for the check summary's vitals table.
376 const vitals: LookVitals[] = [];
373377 for (const { page, kind } of wanted) {
374378 if (stopped) return;
375379 const token = signGrant(
@@ -397,6 +401,13 @@ export function startLooks(app: App, options: LooksOptions): LooksHandle {
397401 ...LOOK_WINDOWS[kind],
398402 timeoutMs: CAPTURE_TIMEOUT_MS,
399403 });
404 // The thumb is the desktop page again, as a stranger: it adds no row.
405 if (kind !== "thumb") {
406 const look = page === "/" ? kind : `${kind} ${page}`;
407 if (shot.ok) vitals.push({ look, entries: shot.value.entries });
408 else if (isNoBrowser(shot.error))
409 vitals.push({ look, skipped: "no browser" });
410 }
400411 const taken =
401412 shot.ok && kind === "thumb" ? reduceToThumb(staged) : shot;
402413 const row = {
@@ -438,6 +449,14 @@ export function startLooks(app: App, options: LooksOptions): LooksHandle {
438449 recordLook(app.db, { ...row, kind, path: "", error: taken.error });
439450 }
440451 }
452 // Filed under the commit: the check of it shows them whenever it exists.
453 if (vitals.length > 0) {
454 try {
455 writeCommitLooks(dataDir, record.repoId, item.commitOid, vitals);
456 } catch (error) {
457 log.warn("vitals not recorded", { commit: item.commitOid, error: String(error) });
458 }
459 }
441460 // A full set at this head is the environment's look now; the pictures
442461 // of heads before it have nothing left to say.
443462 if (allOk)
src/web/pages/runs.ts +1 −1
@@ -305,7 +305,7 @@ export async function runPage(ctx: Ctx): Promise<Response> {
305305 const found = runLog(ctx.app, ctx.auth.user, repo, ctx.params.id ?? "");
306306 if (!found.ok) return notFoundPage(ctx);
307307 const { run, lines, truncated, logHidden } = found.value;
308 const looks = logHidden ? [] : readLooks(ctx.app.config.dataDir, run.id);
308 const looks = logHidden ? [] : readLooks(ctx.app.config.dataDir, run);
309309 return await runView(ctx, repo, run, lines, truncated, logHidden, looks);
310310 }
311311
test/front.test.ts +1 −1
@@ -582,7 +582,7 @@ describe("the front page", () => {
582582 localUrl: servers.url,
583583 capture: async (input) => {
584584 writeFileSync(input.out, Buffer.from(encodePng(4, 4, new Uint8Array(64).fill(255))));
585 return { ok: true, value: undefined };
585 return { ok: true, value: { entries: {} } };
586586 },
587587 });
588588 try {
test/looks.test.ts +96 −4
@@ -29,13 +29,17 @@ import { encodePng } from "../src/design/image/png.ts";
2929 import { setAppMembers } from "../src/core/app_members.ts";
3030 import { listPulls, type Pull } from "../src/core/pulls.ts";
3131 import { getRepo, type Repo } from "../src/core/repos.ts";
32 import { listRuns, type Run } from "../src/core/runs.ts";
32 import { createRun, listRuns, type Run } from "../src/core/runs.ts";
33 import { readLooks } from "../src/checks/looks.ts";
34 import { vitalsRows } from "../src/checks/summary.ts";
3335 import { createUser, type User } from "../src/core/users.ts";
3436 import { serve, type Servers } from "../src/http/server.ts";
3537 import { err, ok } from "../src/kernel/result.ts";
3638 import { quietLogs } from "./helpers/globals.ts";
3739 import {
3840 findChromium,
41 NO_BROWSER,
42 parseRecorded,
3943 takeLook,
4044 type Capture,
4145 type LookInput,
@@ -77,13 +81,20 @@ describe("the look", () => {
7781 let bobCookie: string;
7882
7983 // The stand-in browser: writes a picture, or falls over, as told.
80 let mode: "ok" | "fail" = "ok";
84 let mode: "ok" | "fail" | "nobrowser" = "ok";
8185 const calls: LookInput[] = [];
8286 const fake: Capture = async (input) => {
8387 calls.push(input);
8488 if (mode === "fail") return err("the browser fell over");
89 if (mode === "nobrowser") return err(NO_BROWSER);
8590 writeFileSync(input.out, input.width === 1280 ? DESKTOP_PNG : PNG);
86 return ok();
91 // What the page recorded: a slow paint for the phone, a shift for the desktop.
92 return ok({
93 entries:
94 input.width === 390
95 ? { lcp: [{ startTime: 3100, size: 10 }] }
96 : { layoutShift: [{ startTime: 100, value: 0.3 }] },
97 });
8798 };
8899
89100 const web = (path: string, cookie = "") =>
@@ -264,6 +275,32 @@ describe("the look", () => {
264275 expect((await settled((r) => r.env === env)).state).toBe("ok");
265276 expect(await pictured(env, first)).toHaveLength(3);
266277 expect(calls).toHaveLength(6);
278 // The entries the browser recorded reach the check's summary: the
279 // phone's LCP and the desktop's CLS, graded; the thumb adds no row.
280 // The look finished before any check run of this commit existed; the
281 // check made afterwards still shows them.
282 await looks.idle();
283 expect(listRuns(app.db, repo.id, { kind: "check" })).toEqual([]);
284 const check = createRun(app.db, {
285 repoId: repo.id,
286 kind: "check",
287 commitOid: first,
288 reason: "push",
289 });
290 const recorded = readLooks(dataDir, check);
291 expect(recorded).toHaveLength(2);
292 expect(recorded.map((l) => l.look).sort()).toEqual(["desktop", "phone"]);
293 const graded = vitalsRows(recorded);
294 expect(graded.find((g) => g.look === "phone")!.rows[0]).toMatchObject({
295 vital: "LCP",
296 value: "3100 ms",
297 grade: "needs-improvement",
298 });
299 expect(graded.find((g) => g.look === "desktop")!.rows[1]).toMatchObject({
300 vital: "CLS",
301 value: "0.3",
302 grade: "poor",
303 });
267304 expect(new URL(calls[3]!.url).hostname).toBe(
268305 `hello--${pull.number}.${DOMAIN}`,
269306 );
@@ -404,6 +441,56 @@ describe("the look", () => {
404441 mode = "ok";
405442 }, 120_000);
406443
444 test("with no browser the check's summary says so and stays green", async () => {
445 mode = "nobrowser";
446 const env = `pull-${unpictured.number}`;
447 const moved = await commitToBranch(app, alice, repo, {
448 branch: "second",
449 message: "No browser",
450 files: [{ path: "SECOND.md", content: "no browser\n" }],
451 });
452 if (!moved.ok) throw new Error(JSON.stringify(moved.error));
453 const check = createRun(app.db, {
454 repoId: repo.id,
455 kind: "check",
456 commitOid: moved.value.commitOid,
457 reason: "push",
458 });
459 expect(
460 (await settled((r) => r.env === env && r.commitOid === moved.value.commitOid)).state,
461 ).toBe("ok");
462 await pictured(env, moved.value.commitOid);
463 const got = await until(() => {
464 const read = readLooks(dataDir, check);
465 return read.length === 2 ? read : undefined;
466 });
467 expect(got).toEqual([
468 { look: "phone", skipped: "no browser" },
469 { look: "desktop", skipped: "no browser" },
470 ]);
471 // Words only: the table has no verdict to turn a check red.
472 expect(vitalsRows(got).map((g) => g.skipped)).toEqual(["no browser", "no browser"]);
473 expect(listRuns(app.db, repo.id, { kind: "check" }).find((r) => r.id === check.id)!.state).toBe("queued");
474 mode = "ok";
475 }, 120_000);
476
477 test("what a page recorded is kept as plain numbers, a missing type as an empty list", () => {
478 expect(
479 parseRecorded({
480 lcp: [{ startTime: 3100, size: 5, url: "http://x" }, "junk", { startTime: -1 }],
481 layoutShift: [{ startTime: 1, value: 0.2, hadRecentInput: true }],
482 }),
483 ).toEqual({
484 entries: {
485 lcp: [{ startTime: 3100, size: 5 }],
486 layoutShift: [{ startTime: 1, value: 0.2, hadRecentInput: true }],
487 longtask: [],
488 },
489 });
490 expect(parseRecorded(null).entries).toEqual({ lcp: [], layoutShift: [], longtask: [] });
491 expect(parseRecorded("x").entries.lcp).toEqual([]);
492 });
493
407494 test("a closed pull's pictures go with its preview", async () => {
408495 const env = `pull-${unpictured.number}`;
409496 expect(looksForEnv(app.db, record.id, env).length).toBeGreaterThan(0);
@@ -1002,7 +1089,12 @@ describe("the look", () => {
10021089 height: 844,
10031090 timeoutMs: 30_000,
10041091 });
1005 expect(taken).toEqual({ ok: true, value: undefined });
1092 expect(taken.ok).toBe(true);
1093 // The observer ran before the page: every type is a list, as far as this browser has it.
1094 const entries = taken.ok ? taken.value.entries : undefined;
1095 expect(Array.isArray(entries?.lcp)).toBe(true);
1096 expect(Array.isArray(entries?.layoutShift)).toBe(true);
1097 expect(Array.isArray(entries?.longtask)).toBe(true);
10061098 expect(pngSize(out)).toEqual([390, 844]);
10071099 // The browser arrived through the grant: the app was told who it was.
10081100 const seen = readFileSync(

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs