Public: anyone can read this; the crew can change it.

Migrated from github.com/Trevato/SO-ARM100

workbenchthis machine

/admin/data draws the facts: overview strip, precision, latency, people and health charts, worst sessions, on phone, tablet and desktop #557

Merged opened this

Check · Review · Land done · Live — Landed; nothing deploys from this branch, so this is the end of its path.

Merged

quill/arm:quill/530-admin-data-draws-the-facts-overview-stri → main 2 commits fork

  • Merged. quill/530-admin-data-draws-the-facts-overview-stri was merged into main as 50021dd.

Review

Sign in to review this change.
15 files changed +1409 −9
Software/so101/README.md +26 −3
@@ -62,7 +62,7 @@ Nobody can drive while it runs, and it will not start while someone does (409; 4
6262
6363 ## Facts kept for the Data view
6464
65 The relay (`src/relay.ts`, through `src/facts.ts`) keeps 90 days of what happened in `facts.sqlite` under `FORGE_APP_DATA` (`so101/facts.sqlite`), every row tagged with the app's commit. Older rows are dropped. Nothing reads it yet: there is no API or page.
65 The relay (`src/relay.ts`, through `src/facts.ts`) keeps 90 days of what happened in `facts.sqlite` under `FORGE_APP_DATA` (`so101/facts.sqlite`), every row tagged with the app's commit. Older rows are dropped. `GET /data` reads it and `/admin/data` draws it (below).
6666
6767 - **turns**: who, start, length, why it ended, wait in line; and, when the arm's `/precision` answers about a second after the turn, releases, snap-backs by cause, jams and arrive p95. That card is the arm's current session if it still has one, otherwise its pooled last ten minutes, so after a turn it usually covers the last ten minutes of driving, not that turn alone. Read those columns as a trend, not per turn.
6868 - **events**: visit (one per browser an hour, at most 600 an hour), name set, joined line, chat (counted, never the text), STOP (only when accepted), jam, block, reconnect, app start.
@@ -72,14 +72,37 @@ The relay (`src/relay.ts`, through `src/facts.ts`) keeps 90 days of what happene
7272
7373 ## Reading the facts: GET /data
7474
75 `GET /data` on the hosted app answers the kept facts as JSON (src/data-api.ts, src/data-series.ts; how each number is made is at the top of the latter). It is for the owner and moderators; anonymous visitors and plain users get 403. There is no terminal command for it.
75 `GET /data` on the hosted app answers the kept facts as JSON (src/data-api.ts, src/data-series.ts; how each number is made is at the top of the latter). It is for the owner and moderators; anonymous visitors and plain users get 403. In the terminal, `/data` reads it (below).
7676
7777 - `from`, `to`: ms since the epoch, `from` before `to`; default the last 24 hours.
7878 - `step`: seconds, one of 60, 300, 900, 1800, 3600, 10800, 21600, 86400; default about 96 steps over the range, at most 400.
7979 - `compare=1`: adds `previous`, the same length of time just before, on the same steps (`compare=0` is the default).
8080 - `turn=<id>`: instead of a range, one turn: `{ turn, events, samples }` from a minute before to a minute after, with the arm's events only (STOP, jam, block, reconnect, app start) and no one named. An unknown id is 404.
8181
82 A bad parameter is a 400 whose `error` says what to send; POST is 405. Shape of a range answer (trimmed): `{ from, to, step, t: [step starts], overview: { visitors, sessions, drivers, minutes, snapBacksPer100, arriveP95Ms, jams, uptime }, series: { sessions: [..], snapBacksPer100: [..], watchers: [..], uptime: [..], ... one array per step }, funnel, turnLengths, reasons, versions, live: { watchers, driving, queue }, oldest, commit, previous? }`.
82 A bad parameter is a 400 whose `error` says what to send; POST is 405. Shape of a range answer (trimmed): `{ from, to, step, t: [step starts], overview: { visitors, sessions, drivers, minutes, snapBacksPer100, arriveP95Ms, jams, uptime }, series: { sessions: [..], snapBacksPer100: [..], watchers: [..], uptime: [..], ... one array per step }, funnel, turnLengths, arrive: { edgesMs, counts }, reasons, versions, worst: [{ id, start, seconds, ended, releases, snapBacks, snapBacksPer100, jams, arriveP95Ms, commit }] (the ten measured turns with the most snap-backs per 100 releases, then the slowest arrive), live: { watchers, driving, queue }, oldest, commit, previous? }`.
83
84 ## The Data view: /admin/data
85
86 `/admin/data` draws the facts for the owner and moderators (anyone else: 404), linked from `/admin` and from the Room card in the app's Admin tab. It is a page of its own, so nothing of it loads with the camera and nothing covers it.
87
88 - **Overview**: visitors, driving sessions, drivers, minutes driven, snap-backs per 100 releases, p95 time to arrive, jams and arm-link uptime, each a big number with a sparkline. Swipeable cards on a portrait phone; a grid anywhere wider.
89 - **Precision**: snap-backs by cause (stacked), snap-backs per 100 releases, and the worst sessions; a session opens its numbers, its events (STOP, jam, block, reconnect, app start) on a timeline a minute either side, and the room each minute.
90 - **Latency**: p95 time to arrive over time, and the turns by their p95 arrive.
91 - **People**: in the room (watchers, driving, in line), visitors anonymous against account, where people drop off, wait in line, turn length, how turns ended, chat volume.
92 - **Health**: arm-link uptime; reconnects, app starts, STOPs, jams and blocks; camera fps.
93
94 Choose Hour, Day, Week or All; drag across any chart to zoom every chart into that stretch (Reset zoom goes back); tick "Compare to the period before" for a faint line of the span just before and a change on each card; hover, tap, or focus a chart and use the arrow keys (Home, End) for exact values. Dashed upright lines on every time axis mark the app starting on a new commit. One column on a portrait phone, two in landscape and on a tablet, three from 1280 px wide; both colour schemes; the only motion is a fade and smooth scrolling, and only under `prefers-reduced-motion: no-preference`.
95
96 The SVG is drawn by hand in `src/ui/data-view.js`, inlined in the page by `src/data-page.ts` and pinned by its hash, with the inline style, under a `default-src 'none'` Content-Security-Policy; it reads only `GET /data` and puts every text in as text. No dependency was added.
97
98 **In the terminal** (`ssh` to the app, the command `FORGE_APP_SSH` names), the owner and moderators read the same facts with `/data` (src/data-lines.ts, through `GET /data`, so the server gates it exactly as it gates the page; a guest or a plain user is told it is for the owner and moderators and sees nothing):
99
100 - `/data [hour|day|week|all]` (default `day`): the overview numbers, who is here now, and the worst sessions, one line each, `#<turn>` first.
101 - `/data #<turn>`: that turn's length, how it ended, its wait, releases, snap-backs by cause, jams, p95 arrive and commit, and the arm's events from a minute before to a minute after, in UTC.
102
103 With JavaScript off the page says so and points here.
104
105 Not drawn yet, because nothing records it: latency per hop (page → relay → Pi and back is timed live on the arm's page, not kept), per-joint settle error and arrival, Pi temperature and under-voltage (#546) and a session's joint traces (#548).
83106
84107 ## Faster limits
85108
Software/so101/src/admin-page.ts +1 −1
@@ -321,7 +321,7 @@ export function renderAdmin(p: AdminPage): string {
321321 "Admin · SO-101",
322322 `<header>
323323 <h1>Admin</h1>
324 <div class="state"><span class="pill${s.paused ? " paused" : ""}">${s.paused ? "Paused" : "Open"}</span><span>${d ? `<b>${esc(d.name)}</b> has control` : "Nobody has control"}</span><span><b>${p.snapshot.queue.length}</b> waiting</span><a href="/admin/camera-list">Cameras</a><a href="/">Back to the arm</a></div>
324 <div class="state"><span class="pill${s.paused ? " paused" : ""}">${s.paused ? "Paused" : "Open"}</span><span>${d ? `<b>${esc(d.name)}</b> has control` : "Nobody has control"}</span><span><b>${p.snapshot.queue.length}</b> waiting</span><a href="/admin/data">Data</a><a href="/admin/camera-list">Cameras</a><a href="/">Back to the arm</a></div>
325325 </header>
326326 ${flash}
327327 <section aria-labelledby="now"><h2 id="now">Right now</h2>
Software/so101/src/data-layout.test.ts added +124
@@ -0,0 +1,124 @@
1 import { describe, expect, test } from "bun:test";
2 import { readFileSync } from "node:fs";
3 import { DATA_SCRIPT, DATA_STYLE, renderDataPage } from "./data-page.ts";
4
5 // Read, not rendered: the Data view's CSS resolved at each screen the issue names, the way a browser picks its @media blocks.
6
7 type Rule = { selector: string; body: string };
8 const css = DATA_STYLE.replace(/\/\*[\s\S]*?\*\//g, "");
9
10 /** Does a media query hold at width × height (orientation, min-/max-width, prefers-* taken as their defaults)? */
11 function holds(query: string, w: number, h: number): boolean {
12 return query.split(/\s+and\s+/).every((part) => {
13 const p = part.trim().replace(/^\(|\)$/g, "");
14 const [name, value] = p.split(":").map((s) => s.trim());
15 if (name === "orientation") return value === (w > h ? "landscape" : "portrait");
16 if (name === "min-width") return w >= parseFloat(value!);
17 if (name === "max-width") return w <= parseFloat(value!);
18 if (name === "prefers-reduced-motion") return value === "no-preference";
19 if (name === "prefers-color-scheme") return value === "dark";
20 throw new Error(`unread media feature ${p}`);
21 });
22 }
23
24 /** The rules in force at width × height, in source order. */
25 function rulesAt(w: number, h: number): Rule[] {
26 const out: Rule[] = [];
27 const walk = (src: string) => {
28 let i = 0;
29 while (i < src.length) {
30 const open = src.indexOf("{", i);
31 if (open < 0) break;
32 let depth = 1;
33 let j = open + 1;
34 for (; j < src.length && depth; j++) depth += src[j] === "{" ? 1 : src[j] === "}" ? -1 : 0;
35 const head = src.slice(i, open).trim();
36 const body = src.slice(open + 1, j - 1);
37 if (head.startsWith("@media")) {
38 if (holds(head.slice(6).trim(), w, h)) walk(body);
39 } else for (const s of head.split(",")) out.push({ selector: s.trim(), body });
40 i = j;
41 }
42 };
43 walk(css);
44 return out;
45 }
46
47 /** The last value a selector's rules give a property at that screen. */
48 function prop(rules: Rule[], selector: string, name: string): string | undefined {
49 let found: string | undefined;
50 for (const r of rules) {
51 if (r.selector !== selector) continue;
52 for (const decl of r.body.split(";")) {
53 const at = decl.indexOf(":");
54 if (at > 0 && decl.slice(0, at).trim() === name) found = decl.slice(at + 1).trim();
55 }
56 }
57 return found;
58 }
59
60 const columns = (template: string | undefined) => {
61 if (!template) return 1;
62 const repeat = /^repeat\((\d+),/.exec(template);
63 return repeat ? Number(repeat[1]) : template.split(/\s+(?![^(]*\))/).length;
64 };
65
66 const SCREENS = [
67 { name: "phone portrait", w: 390, h: 844, cols: 1, swipe: true },
68 { name: "phone landscape", w: 844, h: 390, cols: 2, swipe: false },
69 { name: "tablet portrait", w: 820, h: 1180, cols: 2, swipe: false },
70 { name: "desktop", w: 1440, h: 900, cols: 3, swipe: false },
71 ] as const;
72
73 describe("Data view layout", () => {
74 for (const s of SCREENS) {
75 test(`${s.name} ${s.w}×${s.h}: ${s.cols} column${s.cols > 1 ? "s" : ""}, the overview ${s.swipe ? "as swipeable cards" : "as a grid"}`, () => {
76 const rules = rulesAt(s.w, s.h);
77 expect(columns(prop(rules, ".panels", "grid-template-columns"))).toBe(s.cols);
78 // every column may shrink to the screen: a minmax(0, …) track, never a fixed width
79 expect(prop(rules, ".panels", "grid-template-columns")).toContain("minmax(0, 1fr)");
80 if (s.swipe) {
81 expect(prop(rules, ".strip", "grid-auto-flow")).toBe("column");
82 expect(prop(rules, ".strip", "overflow-x")).toBe("auto");
83 expect(prop(rules, ".strip", "scroll-snap-type")).toBe("x mandatory");
84 } else {
85 expect(prop(rules, ".strip", "grid-auto-flow")).toBe("row");
86 expect(prop(rules, ".strip", "overflow")).toBe("visible");
87 }
88 });
89
90 test(`${s.name} ${s.w}×${s.h}: nothing overflows and nothing is 0 px`, () => {
91 const rules = rulesAt(s.w, s.h);
92 for (const sel of ["main", ".panel", ".stat", ".chart", ".plot"]) expect(prop(rules, sel, "min-width")).toBe("0");
93 expect(prop(rules, ".chart svg", "max-width")).toBe("100%");
94 expect(prop(rules, ".scroll", "overflow-x")).toBe("auto");
95 // a chart is drawn at its box's width, never under 240 px; the narrowest box must hold that
96 const pad = 12 * 2 /* main */ + 14 * 2 /* panel */ + 2;
97 expect(s.w - pad).toBeGreaterThanOrEqual(240);
98 // every button is a 40 px target, the worst sessions' links too
99 for (const sel of ["button", "button.link", "label.toggle"]) expect(parseFloat(prop(rules, sel, "min-height") ?? "0")).toBeGreaterThanOrEqual(40);
100 for (const sel of [".plot", ".stat"]) expect(parseFloat(prop(rules, sel, "min-height") ?? "0")).toBeGreaterThan(0);
101 // no width in px wider than the narrowest screen
102 for (const r of rules) for (const m of r.body.matchAll(/(?:^|[;\s])(?:min-)?width\s*:\s*(\d+)px/g)) expect(Number(m[1])).toBeLessThanOrEqual(390);
103 });
104 }
105
106 test("the camera is never covered: a page of its own, nothing fixed over another page, loaded only when opened", () => {
107 expect(css).not.toMatch(/position\s*:\s*fixed/);
108 const viewer = readFileSync(new URL("./viewer.html", import.meta.url), "utf8");
109 expect(viewer).not.toContain("data-view.js");
110 expect(viewer).toContain('href="/admin/data"');
111 expect(renderDataPage()).not.toMatch(/<script[^>]+src=|<link[^>]+stylesheet/);
112 });
113
114 test("both colour schemes and reduced motion", () => {
115 expect(css).toMatch(/color-scheme:\s*dark light/);
116 expect(css).toContain("@media (prefers-color-scheme: light)");
117 // motion only for those who did not ask for less, and the script draws without animating
118 const motion = rulesAt(390, 844).filter((r) => /(^|[;\s])(transition|animation|scroll-behavior)\s*:/.test(r.body));
119 const still = css.split("@media (prefers-reduced-motion: no-preference)")[0]!;
120 expect(still).not.toMatch(/(^|[;\s])(transition|animation|scroll-behavior)\s*:/);
121 expect(motion.length).toBeGreaterThan(0);
122 expect(DATA_SCRIPT).not.toMatch(/\.animate\(|requestAnimationFrame\(\s*function\s+tick/);
123 });
124 });
Software/so101/src/data-lines.test.ts added +106
@@ -0,0 +1,106 @@
1 import { afterEach, describe, expect, test } from "bun:test";
2 import { mkdtempSync, rmSync } from "node:fs";
3 import { tmpdir } from "node:os";
4 import { join } from "node:path";
5 import { aggregate } from "./data-series.ts";
6 import { dataLines, turnLines } from "./data-lines.ts";
7 import { Facts } from "./facts.ts";
8 import { sign } from "./identity.ts";
9 import { Lobby } from "./lobby.ts";
10 import { startRelay, type RunningRelay } from "./relay.ts";
11 import { dataReader, Door, emptyRoom, HELP, typed } from "./terminal.ts";
12
13 const KEY = "k".repeat(64);
14 const as = (name: string, role: "user" | "admin") => {
15 const identity = `id-${name}|${name}|${role}|${Date.now() + 600_000}`;
16 return { "x-forge-identity": identity, "x-forge-signature": sign(identity, KEY) };
17 };
18 /** The terminal of a person the forge signed in, as the forge sets its environment. */
19 const terminal = (port: number, name: string | null, role: "user" | "admin" = "user") =>
20 name ? { PORT: String(port), FORGE_APP_IDENTITY_KEY: KEY, FORGE_APP_USER_ID: `id-${name}`, FORGE_APP_USER: name, FORGE_APP_USER_ROLE: role } : { PORT: String(port) };
21
22 let dir = "";
23 let relay: RunningRelay | undefined;
24 let lobby: Lobby | undefined;
25 afterEach(async () => {
26 await relay?.close();
27 lobby?.close();
28 if (dir) rmSync(dir, { recursive: true, force: true });
29 dir = "";
30 relay = lobby = undefined;
31 });
32
33 describe("/data in the terminal", () => {
34 test("parses a range or a turn, and says how when it is neither", () => {
35 expect(typed(emptyRoom(), "/data")).toEqual({ data: { range: "day" } });
36 expect(typed(emptyRoom(), "/data Week")).toEqual({ data: { range: "week" } });
37 expect(typed(emptyRoom(), "/data all")).toEqual({ data: { range: "all" } });
38 expect(typed(emptyRoom(), "/data #12")).toEqual({ data: { turn: 12 } });
39 expect(typed(emptyRoom(), "/data 12")).toEqual({ data: { turn: 12 } });
40 for (const bad of ["/data month", "/data #x", "/data day week", "/data #1234567890"]) expect(typed(emptyRoom(), bad)).toEqual({ print: "usage: /data [hour|day|week|all] or /data #<turn>" });
41 expect(HELP).toContain("/data [hour|day|week|all|#<turn>]");
42 });
43
44 test("gated like /admin/data: a guest and a plain user are told no and see nothing; the owner and a moderator see aggregate()'s overview and worst sessions", async () => {
45 dir = mkdtempSync(join(tmpdir(), "so101-data-lines-"));
46 const facts = new Facts({ file: join(dir, "facts.sqlite"), commit: "abc1234def56" });
47 lobby = new Lobby({ identityKey: KEY, adminFile: join(dir, "admin.json"), countsFile: join(dir, "visitors.json") });
48 relay = await startRelay({ port: 0, simulation: "http://127.0.0.1:1", lobby, facts });
49 const port = relay.port;
50 await fetch(`http://127.0.0.1:${port}/admin`, { method: "POST", headers: { ...as("trevato", "admin"), "content-type": "application/x-www-form-urlencoded" }, body: new URLSearchParams({ action: "mod-add", name: "bob" }), redirect: "manual" });
51 const ada = { id: "a-ada", name: "ada", owner: false };
52 const snapshot = { driver: null, queue: [] } as never;
53 facts.event("app start");
54 facts.turn({ kind: "turn", ended: null, started: ada }, snapshot);
55 facts.event("jam");
56 const id = facts.turn({ kind: "turn", ended: { who: ada, reason: "time is up" }, started: null }, snapshot)!;
57 facts.card(id, { releases: 8, snapBacks: { jam: 2 }, jams: 2, arriveP95Ms: 640 });
58
59 for (const env of [terminal(port, null), terminal(port, "eve")]) {
60 for (const ask of [{ range: "day" as const }, { turn: id }]) expect(await dataReader(env)(ask)).toEqual({ error: "the data is for the owner and moderators", lines: [] });
61 }
62 const now = Date.now() + 1;
63 const truth = aggregate({ from: now - 86_400_000, to: now, step: 900, now, oldest: facts.oldest(), turns: facts.turns(), events: facts.events(), samples: facts.samples() });
64 for (const env of [terminal(port, "trevato", "admin"), terminal(port, "bob")]) {
65 const r = await dataReader(env, fetch, () => now)({ range: "day" });
66 expect(r.error).toBeNull();
67 expect(r.lines[0]).toBe("the last day, in 15 min steps · commit abc1234");
68 expect(r.lines[1]).toBe(` visitors ${truth.overview.visitors} · sessions 1 · drivers 1 · minutes driven ${truth.overview.minutes}`);
69 expect(r.lines[2]).toBe(" snap-backs per 100 releases 25 · p95 time to arrive 640 ms · jams 1 · uptime —");
70 expect(r.lines[3]).toBe(" now: 0 watching · nobody driving · 0 in line");
71 expect(r.lines[4]).toBe("worst sessions (/data #<turn> for one):");
72 expect(r.lines[5]).toStartWith(` #${id} `);
73 expect(r.lines[5]).toContain("snap-backs 2/8 releases (25 per 100) · p95 arrive 640 ms · abc1234");
74 expect(truth.worst.map((w) => w.id)).toEqual([id]);
75
76 const one = await dataReader(env)({ turn: id });
77 expect(one.error).toBeNull();
78 expect(one.lines[0]).toMatch(new RegExp(`^turn #${id} · \\d{4}-\\d\\d-\\d\\d \\d\\d:\\d\\dZ · \\d+ s · ended: time is up · waited 0 s$`));
79 expect(one.lines[1]).toBe(" releases 8 · snap-backs jam 2 · jams 2 · p95 arrive 640 ms");
80 // the jam came while ada drove: inside the turn
81 expect(one.lines.some((l) => / jam$/.test(l))).toBe(true);
82 expect(one.lines.some((l) => / app start( \(outside the turn\))?$/.test(l))).toBe(true);
83 expect((await dataReader(env)({ turn: 999 })).error).toBe("no such turn");
84 }
85 });
86
87 test("the Door prints what the reader answered, and says so when there is no reader", async () => {
88 const printed: string[] = [];
89 const asked: unknown[] = [];
90 new Door("ws://127.0.0.1:1/live", {}, (l) => printed.push(l), null, null, null, null, null, null, null, null, async (ask) => (asked.push(ask), { error: null, lines: ["a", "b"] })).input("/data hour");
91 new Door("ws://127.0.0.1:1/live", {}, (l) => printed.push(l)).input("/data");
92 await new Promise((r) => setTimeout(r, 10));
93 expect(asked).toEqual([{ range: "hour" }]);
94 // the door without a reader answers at once; the other after its read
95 expect(printed).toEqual(["error: the data is not available here", "a", "b"]);
96 });
97
98 test("text from the answer cannot move the cursor; an answer of the wrong shape is not drawn", () => {
99 const lines = turnLines({ turn: { id: 1, startedAt: "2026-10-22T10:00:00.000Z", seconds: 5, ended: "idle\x1b[2J", waitSeconds: 0, releases: null, commit: "c\x1b]0;x\x07" }, events: [{ at: 0, kind: "jam\x1b[H" }] })!;
100 expect(lines.join("\n")).not.toContain("\x1b");
101 expect(lines[1]).toBe(" not measured: the arm gave no card for it");
102 expect(dataLines({ error: "x" }, "day")).toBeNull();
103 expect(turnLines({ turn: null })).toBeNull();
104 expect(dataLines({ step: 900, overview: {}, worst: [] }, "day")).toContain("worst sessions: no turn the arm measured in this range");
105 });
106 });
Software/so101/src/data-lines.ts added +59
@@ -0,0 +1,59 @@
1 /**
2 * The Data view in words, for the terminal's /data (src/terminal.ts): GET /data's answer as lines. Plain data in,
3 * plain lines out; every text from the answer goes through `plain`, so nothing in it can move the terminal's cursor.
4 */
5
6 import { UNPRINTABLE } from "./chat.ts";
7
8 export const RANGES = { hour: 3_600_000, day: 86_400_000, week: 7 * 86_400_000, all: 90 * 86_400_000 } as const;
9 export type Range = keyof typeof RANGES;
10
11 const plain = (text: unknown): string => String(text ?? "").replace(/\r\n?|\n|\t/g, " ").replace(UNPRINTABLE, "");
12 const finite = (v: unknown): v is number => typeof v === "number" && Number.isFinite(v);
13 const num = (v: unknown, unit = ""): string => (finite(v) ? `${Number.isInteger(v) ? v : v.toFixed(1)}${unit}` : "—");
14 /** A moment as UTC to the minute (or second), the same on every terminal. */
15 const at = (ms: unknown, seconds = false): string => (finite(ms) ? `${new Date(ms).toISOString().slice(0, seconds ? 19 : 16).replace("T", " ")}Z` : "—");
16 const length = (s: unknown): string => (!finite(s) ? "—" : s < 60 ? `${Math.round(s)} s` : `${Math.floor(s / 60)} min ${Math.round(s % 60)} s`);
17 const short = (commit: unknown) => plain(commit).slice(0, 7);
18
19 const STEP_NAMES: Record<number, string> = { 60: "1 min", 300: "5 min", 900: "15 min", 1800: "30 min", 3600: "1 h", 10_800: "3 h", 21_600: "6 h", 86_400: "1 day" };
20
21 /** A range answer: the overview, who is here now, and the worst sessions; null when it is not one. */
22 export function dataLines(body: unknown, range: Range): string[] | null {
23 const d = body as { step?: unknown; overview?: Record<string, unknown>; live?: Record<string, unknown>; worst?: unknown; commit?: unknown } | null;
24 if (!d || !d.overview || typeof d.overview !== "object" || !Array.isArray(d.worst)) return null;
25 const o = d.overview;
26 const lines = [`the last ${range === "all" ? "90 days (all that is kept)" : range}, in ${STEP_NAMES[d.step as number] ?? `${plain(d.step)} s`} steps · commit ${short(d.commit)}`];
27 lines.push(` visitors ${num(o.visitors)} · sessions ${num(o.sessions)} · drivers ${num(o.drivers)} · minutes driven ${num(o.minutes)}`);
28 lines.push(` snap-backs per 100 releases ${num(o.snapBacksPer100)} · p95 time to arrive ${num(o.arriveP95Ms, " ms")} · jams ${num(o.jams)} · uptime ${finite(o.uptime) ? `${(o.uptime * 100).toFixed(1)}%` : "—"}`);
29 if (d.live) lines.push(` now: ${num(d.live.watchers)} watching · ${d.live.driving ? "someone driving" : "nobody driving"} · ${num(d.live.queue)} in line`);
30 if (!d.worst.length) {
31 lines.push("worst sessions: no turn the arm measured in this range");
32 return lines;
33 }
34 lines.push("worst sessions (/data #<turn> for one):");
35 for (const w of d.worst as Array<Record<string, unknown>>) {
36 lines.push(` #${num(w.id)} ${at(w.start)} · ${length(w.seconds)} · snap-backs ${num(w.snapBacks)}/${num(w.releases)} releases (${num(w.snapBacksPer100)} per 100) · p95 arrive ${num(w.arriveP95Ms, " ms")} · ${short(w.commit)}`);
37 }
38 return lines;
39 }
40
41 /** A ?turn=<id> answer: the turn's numbers and the arm's events around it; null when it is not one. */
42 export function turnLines(body: unknown): string[] | null {
43 const d = body as { turn?: Record<string, unknown>; events?: unknown } | null;
44 const t = d?.turn;
45 if (!t || typeof t !== "object" || !Array.isArray(d.events)) return null;
46 const start = Date.parse(String(t.startedAt));
47 const backs = t.snapBacks && typeof t.snapBacks === "object" ? Object.entries(t.snapBacks as Record<string, unknown>).filter(([, n]) => finite(n) && n > 0) : null;
48 const lines = [`turn #${num(t.id)} · ${at(start)} · ${length(t.seconds)} · ended: ${plain(t.ended)} · waited ${length(t.waitSeconds)}`];
49 if (t.releases === null) lines.push(" not measured: the arm gave no card for it");
50 else lines.push(` releases ${num(t.releases)} · snap-backs ${backs === null ? "—" : backs.length ? backs.map(([c, n]) => `${plain(c)} ${n}`).join(", ") : "none"} · jams ${num(t.jams)} · p95 arrive ${num(t.arriveP95Ms, " ms")}`);
51 lines.push(` commit ${plain(t.commit).slice(0, 12)}`);
52 const events = d.events as Array<Record<string, unknown>>;
53 if (!events.length) lines.push(" no STOP, jam, block, reconnect or app start from a minute before to a minute after");
54 for (const e of events) {
55 const inside = finite(e.at) && e.at >= start && e.at <= start + Number(t.seconds) * 1000;
56 lines.push(` ${at(e.at, true)} ${plain(e.kind)}${inside ? "" : " (outside the turn)"}`);
57 }
58 return lines;
59 }
Software/so101/src/data-page.test.ts added +100
@@ -0,0 +1,100 @@
1 import { afterEach, expect, test } from "bun:test";
2 import { createHash } from "node:crypto";
3 import { mkdtempSync, rmSync } from "node:fs";
4 import { tmpdir } from "node:os";
5 import { join } from "node:path";
6 import { DATA_SCRIPT, DATA_STYLE, renderDataPage } from "./data-page.ts";
7 import { Facts } from "./facts.ts";
8 import { sign } from "./identity.ts";
9 import { Lobby } from "./lobby.ts";
10 import { startRelay, type RunningRelay } from "./relay.ts";
11
12 const KEY = "k".repeat(64);
13 const later = Date.now() + 600_000;
14 const as = (name: string, role: "user" | "admin") => {
15 const identity = `id-${name}|${name}|${role}|${later}`;
16 return { "x-forge-identity": identity, "x-forge-signature": sign(identity, KEY) };
17 };
18 const owner = as("trevato", "admin");
19 const sha = (text: string) => createHash("sha256").update(text).digest("base64");
20
21 let dir = "";
22 let relay: RunningRelay | undefined;
23 let lobby: Lobby | undefined;
24 afterEach(async () => {
25 await relay?.close();
26 lobby?.close();
27 if (dir) rmSync(dir, { recursive: true, force: true });
28 dir = "";
29 relay = lobby = undefined;
30 });
31
32 async function boot() {
33 dir = mkdtempSync(join(tmpdir(), "so101-data-page-"));
34 const facts = new Facts({ file: join(dir, "facts.sqlite"), commit: "abc1234def56" });
35 lobby = new Lobby({ identityKey: KEY, adminFile: join(dir, "admin.json"), countsFile: join(dir, "visitors.json") });
36 relay = await startRelay({ port: 0, simulation: "http://127.0.0.1:1", lobby, facts });
37 return `http://127.0.0.1:${relay.port}`;
38 }
39
40 test("GET /admin/data: not found to anyone but the owner and moderators; theirs under a CSP that allows only its own style and script", async () => {
41 const base = await boot();
42 expect((await fetch(`${base}/admin/data`)).status).toBe(404);
43 expect((await fetch(`${base}/admin/data`, { headers: as("eve", "user") })).status).toBe(404);
44 await fetch(`${base}/admin`, { method: "POST", headers: { ...owner, "content-type": "application/x-www-form-urlencoded" }, body: new URLSearchParams({ action: "mod-add", name: "bob" }), redirect: "manual" });
45 for (const who of [owner, as("bob", "user")]) {
46 const res = await fetch(`${base}/admin/data`, { headers: who });
47 expect(res.status).toBe(200);
48 expect(res.headers.get("content-type")).toContain("text/html");
49 expect(res.headers.get("x-content-type-options")).toBe("nosniff");
50 expect(res.headers.get("referrer-policy")).toBe("same-origin");
51 const csp = res.headers.get("content-security-policy")!;
52 expect(csp).toStartWith("default-src 'none';");
53 expect(csp).toContain("frame-ancestors 'none'");
54 const page = await res.text();
55 // the hashes are of what the page carries, byte for byte
56 const style = /<style>([\s\S]*?)<\/style>/.exec(page)![1]!;
57 const script = /<script>([\s\S]*?)<\/script>/.exec(page)![1]!;
58 expect(csp).toContain(`style-src 'sha256-${sha(style)}'`);
59 expect(csp).toContain(`script-src 'sha256-${sha(script)}'`);
60 expect(page.match(/<style|<script/g)).toHaveLength(2);
61 }
62 const head = await fetch(`${base}/admin/data`, { method: "HEAD", headers: owner });
63 expect(head.status).toBe(200);
64 expect(await head.text()).toBe("");
65 });
66
67 test("/admin links to it, and the page reads only what GET /data answers", async () => {
68 const base = await boot();
69 const admin = await (await fetch(`${base}/admin`, { headers: owner })).text();
70 expect(admin).toContain('href="/admin/data"');
71
72 const body = (await (await fetch(`${base}/data`, { headers: owner })).json()) as Record<string, unknown>;
73 // besides the series: what ?turn=<id> answers, and the error of a refusal
74 const turnAnswer = ["turn", "events", "samples", "error"];
75 const read = new Set([...DATA_SCRIPT.matchAll(/\bd\.(\w+)/g)].map((m) => m[1]!));
76 for (const key of read) expect([...Object.keys(body), ...turnAnswer]).toContain(key);
77 // `s` is the series in the drawing functions, and a chart's own line in timeChart: its keys are left out
78 const line = new Set(["prev", "kind", "values", "name", "color", "unit", "dashed"]);
79 const series = new Set([...DATA_SCRIPT.matchAll(/\b(?:s|d\.series)\.(\w+)/g)].map((m) => m[1]!).filter((k) => !line.has(k)));
80 expect(series.size).toBeGreaterThan(15);
81 for (const key of series) expect(Object.keys(body.series as object)).toContain(key);
82 expect(read.has("worst")).toBe(true);
83 expect(read.has("versions")).toBe(true);
84 });
85
86 test("every element the script looks for is on the page; the ranges, compare, zoom and the session dialog are there", () => {
87 const page = renderDataPage();
88 for (const [, id] of DATA_SCRIPT.matchAll(/\$\("#([\w-]+)/g)) expect(page).toContain(`id="${id}"`);
89 for (const range of ["hour", "day", "week", "all"]) expect(page).toContain(`data-range="${range}"`);
90 expect(page).toContain('id="compare"');
91 expect(page).toContain('id="reset"');
92 expect(page).toContain('<table id="worst">');
93 expect(page).toContain('<dialog id="turn"');
94 // with JavaScript off the page says why it is empty, and where else to look
95 expect(page).toMatch(/<noscript><p class="note">The charts are drawn by JavaScript[^<]*\/data\.<\/p><\/noscript>/);
96 // drawn by hand: no chart library, nothing fetched but /data
97 expect(DATA_SCRIPT).not.toMatch(/\bimport\b|innerHTML|insertAdjacentHTML|\beval\(|new Function/);
98 expect([...DATA_SCRIPT.matchAll(/fetch\(`([^`?]+)/g)].map((m) => m[1])).toEqual(["/data", "/data"]);
99 expect(DATA_STYLE).not.toContain("@import");
100 });
Software/so101/src/data-page.ts added +196
@@ -0,0 +1,196 @@
1 /**
2 * The Data view, /admin/data: the owner's and the moderators' place to explore how the arm is doing. A page of its own,
3 * so nothing of it loads with the camera and it never covers it. The markup is fixed (no person's text in it); the
4 * charts are src/ui/data-view.js, drawn from GET /data. Style and script are inline, pinned by their hashes in the
5 * page's Content-Security-Policy, which allows nothing else. To anyone else there is no such page (404, as /admin).
6 */
7
8 import { createHash } from "node:crypto";
9 import { readFileSync } from "node:fs";
10 import type { IncomingMessage, ServerResponse } from "node:http";
11 import type { Lobby } from "./lobby.ts";
12
13 export const DATA_SCRIPT = readFileSync(new URL("./ui/data-view.js", import.meta.url), "utf8");
14
15 export const DATA_STYLE = `
16 :root { color-scheme: dark light; --bg: #0b0d10; --panel: #11161d; --fg: #e8eaed; --dim: #9aa4ae; --line: #2a313a; --accent: #58a6ff; --good: #3fb950; --bad: #f85149;
17 --c0: #58a6ff; --c1: #3fb950; --c2: #d2a8ff; --c3: #f0883e; --c4: #f85149; --c5: #e3b341; --c6: #8b949e; --c7: #39c5cf; }
18 @media (prefers-color-scheme: light) {
19 :root { --bg: #f6f7f9; --panel: #fff; --fg: #1f2328; --dim: #59636e; --line: #d8dee4; --accent: #0969da; --good: #1a7f37; --bad: #cf222e;
20 --c0: #0969da; --c1: #1a7f37; --c2: #8250df; --c3: #bc4c00; --c4: #cf222e; --c5: #9a6700; --c6: #6e7781; --c7: #1b7c83; }
21 }
22 * { box-sizing: border-box; }
23 body { margin: 0; background: var(--bg); color: var(--fg); font: 15px/1.45 ui-sans-serif, system-ui, -apple-system, "Segoe UI", sans-serif; font-variant-numeric: tabular-nums; }
24 main { max-width: 110rem; margin: 0 auto; padding: max(1rem, env(safe-area-inset-top)) max(.75rem, env(safe-area-inset-right)) 3rem max(.75rem, env(safe-area-inset-left)); display: grid; gap: 1rem; min-width: 0; }
25 header { display: grid; gap: .6rem; }
26 .title { display: flex; flex-wrap: wrap; align-items: baseline; gap: .25rem 1rem; }
27 h1 { font-size: clamp(1.5rem, 4vw, 2rem); line-height: 1.1; margin: 0; letter-spacing: -.01em; }
28 .title nav { display: flex; gap: 1rem; font-size: .9rem; }
29 a { color: var(--accent); }
30 .controls { display: flex; flex-wrap: wrap; align-items: center; gap: .5rem .9rem; }
31 .seg { display: inline-flex; border: 1px solid var(--line); border-radius: 8px; overflow: hidden; }
32 .seg button { border: 0; border-radius: 0; border-right: 1px solid var(--line); }
33 .seg button:last-child { border-right: 0; }
34 button { font: inherit; font-weight: 600; min-height: 40px; padding: .35rem .85rem; border-radius: 8px; border: 1px solid var(--line); background: var(--panel); color: var(--fg); cursor: pointer; }
35 button[aria-pressed="true"] { background: var(--accent); color: #fff; }
36 button:focus-visible, input:focus-visible, figure:focus-visible { outline: 2px solid var(--accent); outline-offset: 2px; }
37 button.link { border: 0; background: none; color: var(--accent); padding: 0; min-height: 40px; font-weight: 500; text-align: left; }
38 label.toggle { display: inline-flex; align-items: center; gap: .4rem; min-height: 40px; }
39 label.toggle input { width: 1.1rem; height: 1.1rem; accent-color: var(--accent); }
40 .range, .status, .live, .note { color: var(--dim); font-size: .85rem; margin: 0; }
41 .status:empty { display: none; }
42 .status { color: var(--bad); }
43 body.loading .panels, body.loading .strip { opacity: .6; }
44 .strip { display: grid; grid-auto-flow: column; grid-auto-columns: minmax(11rem, 72%); gap: .6rem; overflow-x: auto; scroll-snap-type: x mandatory; padding-bottom: .25rem; overscroll-behavior-x: contain; }
45 .stat { scroll-snap-align: start; background: var(--panel); border: 1px solid var(--line); border-radius: 10px; padding: .7rem .8rem .5rem; display: grid; gap: .1rem; min-width: 0; min-height: 8.5rem; }
46 .stat h3 { margin: 0; font-size: .75rem; font-weight: 600; letter-spacing: .04em; text-transform: uppercase; color: var(--dim); }
47 .stat .big { margin: 0; font-size: 1.7rem; font-weight: 650; line-height: 1.15; }
48 .stat .delta { margin: 0; font-size: .8rem; color: var(--dim); min-height: 1.2em; }
49 .delta.good { color: var(--good); }
50 .delta.bad { color: var(--bad); }
51 .spark { width: 100%; height: 24px; display: block; }
52 .panels { display: grid; grid-template-columns: minmax(0, 1fr); gap: 1rem; grid-auto-flow: row dense; }
53 .panel { background: var(--panel); border: 1px solid var(--line); border-radius: 12px; padding: .85rem; min-width: 0; display: grid; gap: .6rem; align-content: start; }
54 .panel h2 { margin: 0; font-size: .8rem; font-weight: 650; letter-spacing: .08em; text-transform: uppercase; color: var(--dim); }
55 .panel h3 { margin: .2rem 0 0; font-size: .95rem; font-weight: 600; }
56 .plot { min-height: 196px; min-width: 0; }
57 .chart { margin: 0; position: relative; min-width: 0; }
58 .chart svg { display: block; max-width: 100%; height: auto; overflow: visible; touch-action: pan-y; }
59 .legend { display: flex; flex-wrap: wrap; gap: .1rem .8rem; font-size: .75rem; color: var(--dim); min-height: 1.1rem; }
60 .key, .row { display: inline-flex; align-items: center; gap: .3rem; }
61 .key i, .row i { width: .6rem; height: .6rem; border-radius: 2px; background: var(--k); display: inline-block; }
62 .grid { stroke: var(--line); stroke-width: 1; }
63 .tick, .vlabel { fill: var(--dim); font-size: 11px; }
64 .vlabel { font-size: 10px; }
65 .val { fill: var(--fg); font-size: 11px; }
66 .none { fill: var(--dim); font-size: 13px; }
67 .line { fill: none; stroke: var(--k); stroke-width: 2; stroke-linejoin: round; stroke-linecap: round; }
68 .line.dashed { stroke-dasharray: 5 4; }
69 .line.prev { stroke-width: 1.5; stroke-dasharray: 2 3; opacity: .55; }
70 .spark .line { stroke-width: 1.6; vector-effect: non-scaling-stroke; }
71 .fill { fill: var(--k); }
72 .band { fill: var(--k); opacity: .18; }
73 .dot { fill: var(--k); }
74 .version { stroke: var(--dim); stroke-dasharray: 3 3; stroke-width: 1; }
75 .cursor { stroke: var(--fg); stroke-width: 1; opacity: .5; }
76 .brush { fill: var(--accent); opacity: .15; }
77 .c0 { --k: var(--c0); } .c1 { --k: var(--c1); } .c2 { --k: var(--c2); } .c3 { --k: var(--c3); }
78 .c4 { --k: var(--c4); } .c5 { --k: var(--c5); } .c6 { --k: var(--c6); } .c7 { --k: var(--c7); }
79 .tip { position: absolute; top: 1.4rem; right: .25rem; z-index: 1; display: grid; gap: .1rem; padding: .45rem .6rem; border-radius: 8px; background: var(--bg); border: 1px solid var(--line); font-size: .78rem; pointer-events: none; box-shadow: 0 4px 16px rgb(0 0 0 / .25); max-width: min(18rem, 90%); }
80 .tip.left { right: auto; left: 2.6rem; }
81 .tip .prev { opacity: .7; }
82 .tip[hidden] { display: none; }
83 table { width: 100%; border-collapse: collapse; font-size: .85rem; }
84 th, td { text-align: right; padding: .35rem .4rem; border-bottom: 1px solid var(--line); white-space: nowrap; }
85 th:first-child, td:first-child { text-align: left; white-space: normal; }
86 th { color: var(--dim); font-weight: 600; font-size: .75rem; }
87 .scroll { overflow-x: auto; }
88 .mono { font-family: ui-monospace, SFMono-Regular, Menlo, monospace; font-size: .8rem; }
89 .empty { color: var(--dim); font-size: .85rem; margin: 0; }
90 dialog { width: min(44rem, calc(100vw - 1.5rem)); max-height: calc(100dvh - 1.5rem); border: 1px solid var(--line); border-radius: 12px; background: var(--panel); color: var(--fg); padding: 1rem; }
91 dialog::backdrop { background: rgb(0 0 0 / .5); }
92 dialog h2 { margin: 0 0 .6rem; font-size: 1.1rem; }
93 dialog .close { float: right; }
94 .facts { display: grid; grid-template-columns: max-content 1fr; gap: .2rem 1rem; margin: 0; font-size: .9rem; }
95 .facts dt { color: var(--dim); }
96 .facts dd { margin: 0; }
97 .timeline { display: block; max-width: 100%; height: auto; }
98 .turnbar { fill: var(--accent); opacity: .3; }
99 .mark { stroke: var(--dim); stroke-width: 2; }
100 .mark.stop { stroke: var(--c4); } .mark.jam { stroke: var(--c3); } .mark.block { stroke: var(--c5); } .mark.reconnect { stroke: var(--c2); } .mark.app-start { stroke: var(--c6); }
101 .events { margin: 0; padding-left: 1.1rem; font-size: .85rem; }
102 @media (orientation: landscape) and (min-width: 560px) {
103 .strip { grid-auto-flow: row; grid-template-columns: repeat(auto-fit, minmax(9.5rem, 1fr)); grid-auto-columns: auto; overflow: visible; }
104 .panels { grid-template-columns: repeat(2, minmax(0, 1fr)); }
105 .wide { grid-column: 1 / -1; }
106 }
107 @media (min-width: 760px) {
108 main { padding-left: 1.5rem; padding-right: 1.5rem; }
109 .strip { grid-auto-flow: row; grid-template-columns: repeat(auto-fit, minmax(9.5rem, 1fr)); grid-auto-columns: auto; overflow: visible; }
110 .panels { grid-template-columns: repeat(2, minmax(0, 1fr)); }
111 .wide { grid-column: 1 / -1; }
112 }
113 @media (min-width: 1280px) {
114 .panels { grid-template-columns: repeat(3, minmax(0, 1fr)); }
115 .wide { grid-column: span 2; }
116 }
117 @media (prefers-reduced-motion: no-preference) {
118 .panels, .strip { transition: opacity .15s ease; }
119 .strip { scroll-behavior: smooth; }
120 }
121 `;
122
123 const sha = (text: string) => createHash("sha256").update(text).digest("base64");
124 export const dataCsp = (): string => `default-src 'none'; style-src 'sha256-${sha(DATA_STYLE)}'; script-src 'sha256-${sha(DATA_SCRIPT)}'; connect-src 'self'; img-src 'none'; form-action 'none'; frame-ancestors 'none'; base-uri 'none'`;
125
126 const plot = (id: string) => `<div class="plot" id="${id}"></div>`;
127
128 export function renderDataPage(): string {
129 return `<!doctype html>
130 <html lang="en">
131 <head>
132 <meta charset="utf-8">
133 <meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover">
134 <meta name="referrer" content="same-origin">
135 <title>Data · the arm</title>
136 <style>${DATA_STYLE}</style>
137 </head>
138 <body>
139 <main>
140 <header>
141 <div class="title"><h1>Data</h1><nav><a href="/">The arm</a><a href="/admin">Admin</a></nav></div>
142 <div class="controls">
143 <div class="seg" role="group" aria-label="Time range">
144 <button type="button" data-range="hour" aria-pressed="false">Hour</button><button type="button" data-range="day" aria-pressed="true">Day</button><button type="button" data-range="week" aria-pressed="false">Week</button><button type="button" data-range="all" aria-pressed="false">All</button>
145 </div>
146 <label class="toggle"><input type="checkbox" id="compare"> Compare to the period before</label>
147 <button type="button" id="reset" hidden>Reset zoom</button>
148 </div>
149 <p class="range" id="range">Loading…</p>
150 <noscript><p class="note">The charts are drawn by JavaScript, which is off here. The same facts are in the terminal: /data.</p></noscript>
151 <p class="status" id="status" role="status" aria-live="polite"></p>
152 </header>
153 <section class="strip" id="strip" aria-label="Overview"></section>
154 <p class="note">Drag across any chart to zoom into that stretch; hover, tap or use the arrow keys on a chart for exact values. Dashed upright lines mark a deployed commit.</p>
155 <div class="panels">
156 <section class="panel" aria-labelledby="h-causes"><h2 id="h-causes">Precision · snap-backs by cause</h2>${plot("c-causes")}</section>
157 <section class="panel" aria-labelledby="h-per100"><h2 id="h-per100">Precision · snap-backs per 100 releases</h2>${plot("c-per100")}</section>
158 <section class="panel wide" aria-labelledby="h-worst"><h2 id="h-worst">Precision · worst sessions</h2>
159 <p class="note">A turn's numbers are the arm's card read just after it: that session while the arm still had it, else its last ten minutes of driving.</p>
160 <div class="scroll"><table id="worst"><thead><tr><th>Session</th><th>Length</th><th>Snap-backs / releases</th><th>Per 100</th><th>p95 arrive</th><th>Commit</th></tr></thead><tbody></tbody></table></div></section>
161 <section class="panel" aria-labelledby="h-arrive"><h2 id="h-arrive">Latency · time to arrive, p95</h2>${plot("c-arrive")}</section>
162 <section class="panel" aria-labelledby="h-arrivehist"><h2 id="h-arrivehist">Latency · turns by p95 arrive</h2>${plot("c-arrivehist")}</section>
163 <section class="panel" aria-labelledby="h-room"><h2 id="h-room">People · in the room</h2><p class="live" id="live"></p>${plot("c-room")}</section>
164 <section class="panel" aria-labelledby="h-visitors"><h2 id="h-visitors">People · visitors</h2>${plot("c-visitors")}</section>
165 <section class="panel" aria-labelledby="h-funnel"><h2 id="h-funnel">People · where they drop off</h2>${plot("c-funnel")}</section>
166 <section class="panel" aria-labelledby="h-wait"><h2 id="h-wait">People · wait in line</h2>${plot("c-wait")}</section>
167 <section class="panel" aria-labelledby="h-turns"><h2 id="h-turns">People · turn length</h2>${plot("c-turns")}</section>
168 <section class="panel" aria-labelledby="h-ended"><h2 id="h-ended">People · how turns ended</h2>${plot("c-ended")}</section>
169 <section class="panel" aria-labelledby="h-chat"><h2 id="h-chat">People · chat volume</h2>${plot("c-chat")}</section>
170 <section class="panel" aria-labelledby="h-uptime"><h2 id="h-uptime">Health · arm link uptime</h2>${plot("c-uptime")}</section>
171 <section class="panel" aria-labelledby="h-events"><h2 id="h-events">Health · reconnects, starts, STOPs, jams</h2><p class="note" id="starts"></p>${plot("c-events")}</section>
172 <section class="panel" aria-labelledby="h-fps"><h2 id="h-fps">Health · camera fps</h2>${plot("c-fps")}</section>
173 </div>
174 </main>
175 <dialog id="turn" aria-labelledby="turn-title"><button type="button" class="close" id="turn-close">Close</button><div id="turn-body"></div></dialog>
176 <script>${DATA_SCRIPT}</script>
177 </body>
178 </html>
179 `;
180 }
181
182 /** GET /admin/data: the page for the owner and moderators; to anyone else, the ordinary not-found. */
183 export function serveDataPage(lobby: Lobby, req: IncomingMessage, res: ServerResponse): void {
184 if (!lobby.who(req) || !lobby.moderates(req)) {
185 res.writeHead(404, { "content-type": "application/json", "cache-control": "no-store" });
186 res.end(JSON.stringify({ error: "no such route" }));
187 return;
188 }
189 if (req.method !== "GET" && req.method !== "HEAD") {
190 res.writeHead(405, { "content-type": "application/json", "cache-control": "no-store", allow: "GET, HEAD" });
191 res.end(JSON.stringify({ error: "GET" }));
192 return;
193 }
194 res.writeHead(200, { "content-type": "text/html; charset=utf-8", "cache-control": "no-store", "content-security-policy": dataCsp() });
195 res.end(req.method === "HEAD" ? undefined : renderDataPage());
196 }
Software/so101/src/data-series.test.ts +23
@@ -93,6 +93,29 @@ describe("aggregate", () => {
9393 expect(a.series.stops).toEqual([1, 0, 0, 0]);
9494 expect(a.overview.jams).toBe(1);
9595 });
96
97 test("the worst sessions: measured turns only, most snap-backs per 100 releases first, then the slowest arrive; at most ten", () => {
98 const turns = [
99 turn({ id: 1, releases: 10, snapBacks: { jam: 1 }, jams: 1, arriveP95Ms: 400 }),
100 turn({ id: 2, start: T0 + 60_000, releases: 4, snapBacks: { jam: 1, zone: 1 }, jams: 1, arriveP95Ms: 300, commit: "def5678" }),
101 turn({ id: 3, start: T0 + 120_000, releases: 10, snapBacks: { jam: 1 }, jams: 1, arriveP95Ms: 900 }),
102 turn({ id: 4, start: T0 + 180_000 }),
103 ...Array.from({ length: 12 }, (_, k) => turn({ id: 10 + k, start: T0 + H + k * 60_000, releases: 5, snapBacks: {}, jams: 0, arriveP95Ms: 100 })),
104 ];
105 const a = aggregate(input({ turns }));
106 expect(a.worst.length).toBe(10);
107 expect(a.worst.slice(0, 3).map((w) => w.id)).toEqual([2, 3, 1]);
108 expect(a.worst.map((w) => w.id)).not.toContain(4);
109 expect(a.worst[0]).toEqual({ id: 2, start: T0 + 60_000, seconds: 60, ended: "time is up", releases: 4, snapBacks: 2, snapBacksPer100: 50, jams: 1, arriveP95Ms: 300, commit: "def5678" });
110 expect(a.worst[0]).not.toHaveProperty("who");
111 });
112
113 test("turns by their p95 time to arrive, in bins; unmeasured turns are not counted", () => {
114 const turns = [turn({ id: 1, releases: 1, snapBacks: {}, arriveP95Ms: 120 }), turn({ id: 2, releases: 1, snapBacks: {}, arriveP95Ms: 750 }), turn({ id: 3, releases: 1, snapBacks: {}, arriveP95Ms: 5000 }), turn({ id: 4 })];
115 const a = aggregate(input({ turns }));
116 expect(a.arrive.edgesMs).toEqual([250, 500, 1000, 2000, 4000]);
117 expect(a.arrive.counts).toEqual([1, 0, 1, 0, 0, 1]);
118 });
96119 });
97120
98121 describe("parseRange", () => {
Software/so101/src/data-series.ts +42
@@ -11,6 +11,7 @@
1111 * - uptime: the minutes the arm link was sampled live over the minutes that have passed since the range (or the
1212 * record) began, in a step and in all.
1313 * - counts of chat, STOP, jam, block and reconnect events are what happened in the step, nothing smoothed.
14 * - the worst sessions: the turns the arm measured, most snap-backs per 100 releases first, then the slowest p95 arrive.
1415 */
1516
1617 import type { EventRow, SampleRow, TurnRow } from "./facts.ts";
@@ -41,6 +42,10 @@ const sum = (values: readonly number[]): number => values.reduce((a, b) => a + b
4142 const mean = (values: readonly number[]): number | null => (values.length ? sum(values) / values.length : null);
4243
4344 export const TURN_BINS = [10, 30, 60, 120, 300] as const;
45 /** Edges, ms, for the turns by their p95 time to arrive. */
46 export const ARRIVE_BINS = [250, 500, 1000, 2000, 4000] as const;
47 /** The most worst sessions one answer lists. */
48 export const WORST = 10;
4449 /** Counts per bin: below the first edge, between each pair, and at or past the last. */
4550 export function histogram(values: readonly number[], edges: readonly number[]): number[] {
4651 const out = new Array<number>(edges.length + 1).fill(0);
@@ -96,6 +101,21 @@ export interface Marker {
96101 commit: string;
97102 }
98103
104 /** A measured turn, as the worst-sessions table lists it; who drove is not in it. */
105 export interface Worst {
106 id: number;
107 /** When it began, ms. */
108 start: number;
109 seconds: number;
110 ended: string;
111 releases: number;
112 snapBacks: number;
113 snapBacksPer100: number | null;
114 jams: number;
115 arriveP95Ms: number | null;
116 commit: string;
117 }
118
99119 export interface Aggregate {
100120 from: number;
101121 to: number;
@@ -107,10 +127,13 @@ export interface Aggregate {
107127 /** Who got how far: opened the page, set a name, joined the line, drove, drove under 10 s (each distinct people). */
108128 funnel: { visitors: number; named: number; queued: number; drove: number; under10s: number };
109129 turnLengths: { edgesS: readonly number[]; counts: number[] };
130 /** The measured turns by their own p95 time to arrive. */
131 arrive: { edgesMs: readonly number[]; counts: number[] };
110132 /** How turns ended, by the reason kept. */
111133 reasons: Record<string, number>;
112134 /** Each time the app started on a commit other than the one before. */
113135 versions: Marker[];
136 worst: Worst[];
114137 }
115138
116139 export interface Input {
@@ -163,6 +186,7 @@ export function aggregate(input: Input): Aggregate {
163186 const drove = new Map<string, number>();
164187 const reasons: Record<string, number> = {};
165188 const arrives: number[] = [];
189 const measured: Worst[] = [];
166190 for (const turn of input.turns) {
167191 const i = at(Date.parse(turn.startedAt));
168192 sessions[i]!++;
@@ -182,7 +206,23 @@ export function aggregate(input: Input): Aggregate {
182206 arrive[i]!.push(turn.arriveP95Ms);
183207 arrives.push(turn.arriveP95Ms);
184208 }
209 const backs = sum(Object.values(turn.snapBacks ?? {}));
210 measured.push({
211 id: turn.id,
212 start: Date.parse(turn.startedAt),
213 seconds: turn.seconds,
214 ended: turn.ended,
215 releases: turn.releases,
216 snapBacks: backs,
217 snapBacksPer100: turn.releases > 0 ? round((100 * backs) / turn.releases) : null,
218 jams: turn.jams ?? 0,
219 arriveP95Ms: turn.arriveP95Ms,
220 commit: turn.commit,
221 });
185222 }
223 const worst = measured
224 .sort((a, b) => (b.snapBacksPer100 ?? 0) - (a.snapBacksPer100 ?? 0) || (b.arriveP95Ms ?? 0) - (a.arriveP95Ms ?? 0) || b.start - a.start)
225 .slice(0, WORST);
186226
187227 // ---- the minute samples
188228 const sampled = { watchers: lists(), drivers: lists(), queue: lists() };
@@ -256,8 +296,10 @@ export function aggregate(input: Input): Aggregate {
256296 under10s: [...drove.values()].filter((s) => s < 10).length,
257297 },
258298 turnLengths: { edgesS: TURN_BINS, counts: histogram(input.turns.map((x) => x.seconds), TURN_BINS) },
299 arrive: { edgesMs: ARRIVE_BINS, counts: histogram(arrives, ARRIVE_BINS) },
259300 reasons,
260301 versions,
302 worst,
261303 };
262304 }
263305
Software/so101/src/data-view.test.ts added +209
@@ -0,0 +1,209 @@
1 import { afterEach, expect, test } from "bun:test";
2 import { mkdtempSync, rmSync } from "node:fs";
3 import { tmpdir } from "node:os";
4 import { join } from "node:path";
5 import { DATA_SCRIPT, renderDataPage } from "./data-page.ts";
6 import { Facts } from "./facts.ts";
7 import { sign } from "./identity.ts";
8 import { Lobby } from "./lobby.ts";
9 import { startRelay, type RunningRelay } from "./relay.ts";
10
11 // The Data view's script run against a real GET /data, in the smallest DOM it needs: no browser, no new dependency.
12
13 const KEY = "k".repeat(64);
14 const identity = `id-trevato|trevato|admin|${Date.now() + 600_000}`;
15 const owner = { "x-forge-identity": identity, "x-forge-signature": sign(identity, KEY) };
16
17 type Listener = (ev: Record<string, unknown>) => void;
18 class Node {
19 children: Node[] = [];
20 attrs: Record<string, string> = {};
21 listeners: Record<string, Listener[]> = {};
22 classes = new Set<string>();
23 dataset: Record<string, string> = {};
24 text = "";
25 hidden = false;
26 open = false;
27 [key: string]: unknown;
28 readonly tag: string;
29 constructor(tag: string) {
30 this.tag = tag;
31 }
32 set className(v: string) {
33 this.classes = new Set(v.split(/\s+/).filter(Boolean));
34 }
35 get classList() {
36 const c = this.classes;
37 return { add: (k: string) => c.add(k), remove: (k: string) => c.delete(k), toggle: (k: string, on: boolean) => (on ? c.add(k) : c.delete(k)) };
38 }
39 set textContent(v: string) {
40 this.children = [];
41 this.text = v;
42 }
43 get textContent(): string {
44 return this.text + this.children.map((c) => c.textContent).join("");
45 }
46 setAttribute(k: string, v: string) {
47 this.attrs[k] = v;
48 if (k === "class") this.className = v;
49 }
50 getAttribute(k: string) {
51 return this.attrs[k] ?? null;
52 }
53 appendChild(c: Node) {
54 this.children.push(c);
55 return c;
56 }
57 append(...items: Array<Node | string>) {
58 for (const i of items) typeof i === "string" ? (this.text += i) : this.children.push(i);
59 }
60 replaceChildren(...items: Node[]) {
61 this.children = [...items];
62 this.text = "";
63 }
64 addEventListener(type: string, fn: Listener) {
65 (this.listeners[type] ??= []).push(fn);
66 }
67 fire(type: string, ev: Record<string, unknown> = {}) {
68 for (const fn of this.listeners[type] ?? []) fn({ preventDefault() {}, target: this, ...ev });
69 }
70 getBoundingClientRect() {
71 return { left: 0, width: 600 };
72 }
73 showModal() {
74 this.open = true;
75 }
76 close() {
77 this.open = false;
78 }
79 all(pred: (n: Node) => boolean): Node[] {
80 return [...(pred(this) ? [this] : []), ...this.children.flatMap((c) => c.all(pred))];
81 }
82 }
83
84 function dom() {
85 const page = renderDataPage();
86 const byId = new Map<string, Node>();
87 for (const [, tag, id] of page.matchAll(/<(\w+)[^>]*\bid="([\w-]+)"/g)) byId.set(id!, new Node(tag!));
88 const tbody = new Node("tbody");
89 const ranges = ["hour", "day", "week", "all"].map((r) => Object.assign(new Node("button"), { dataset: { range: r } }));
90 const body = new Node("body");
91 const document = {
92 body,
93 documentElement: { clientWidth: 1440 },
94 querySelector: (sel: string) => (sel === "#worst tbody" ? tbody : byId.get(sel.slice(1)) ?? null),
95 querySelectorAll: (sel: string) => (sel === "[data-range]" ? ranges : []),
96 createElement: (tag: string) => new Node(tag),
97 createElementNS: (_ns: string, tag: string) => new Node(tag),
98 };
99 return { document, byId, tbody, ranges, body };
100 }
101
102 const settle = () => new Promise((r) => setTimeout(r, 120));
103
104 let dir = "";
105 let relay: RunningRelay | undefined;
106 let lobby: Lobby | undefined;
107 afterEach(async () => {
108 await relay?.close();
109 lobby?.close();
110 if (dir) rmSync(dir, { recursive: true, force: true });
111 dir = "";
112 relay = lobby = undefined;
113 });
114
115 test("the script draws every chart from a real answer, opens a worst session, and keeps what a tap showed", async () => {
116 dir = mkdtempSync(join(tmpdir(), "so101-data-view-"));
117 const facts = new Facts({ file: join(dir, "facts.sqlite"), commit: "abc1234def56" });
118 lobby = new Lobby({ identityKey: KEY, adminFile: join(dir, "admin.json"), countsFile: join(dir, "visitors.json") });
119 relay = await startRelay({ port: 0, simulation: "http://127.0.0.1:1", lobby, facts });
120 const base = `http://127.0.0.1:${relay.port}`;
121 const ada = { id: "a-ada", name: "ada", owner: false };
122 const snapshot = { driver: null, queue: [] } as never;
123 facts.event("app start");
124 facts.visit("browser-1");
125 facts.sample({ watchers: 2, drivers: 1, queue: 0, linkLive: true, fps: { "1": 24 } });
126 facts.turn({ kind: "turn", ended: null, started: ada }, snapshot);
127 facts.event("jam");
128 const id = facts.turn({ kind: "turn", ended: { who: ada, reason: "taken over by the owner" }, started: null }, snapshot)!;
129 facts.card(id, { releases: 8, snapBacks: { jam: 2 }, jams: 2, arriveP95Ms: 640 });
130
131 const { document, byId, tbody, ranges, body } = dom();
132 const asked: string[] = [];
133 const fetchAs = (url: string) => {
134 asked.push(url);
135 return fetch(base + url, { headers: owner });
136 };
137 let timers = 0;
138 new Function("document", "fetch", "addEventListener", "setTimeout", "clearTimeout", DATA_SCRIPT)(document, fetchAs, () => {}, () => ++timers, () => {});
139 await settle();
140
141 expect(byId.get("status")!.textContent).toBe("");
142 expect(body.classes.has("loading")).toBe(false);
143 expect(asked[0]).toMatch(/^\/data\?from=\d+&to=\d+$/);
144 // the overview: eight cards, each a big number and a sparkline
145 const strip = byId.get("strip")!;
146 expect(strip.children).toHaveLength(8);
147 expect(strip.children.map((c) => c.children[0]!.textContent)).toContain("Snap-backs / 100 releases");
148 // every chart box holds a drawn svg
149 for (const [key, node] of byId) if (key.startsWith("c-")) expect(node.all((n) => n.tag === "svg").length).toBeGreaterThan(0);
150 // the deployed commit marks the time axes
151 const causes = byId.get("c-causes")!;
152 expect(causes.all((n) => n.classes.has("version")).length).toBe(1);
153 expect(causes.all((n) => n.classes.has("vlabel"))[0]!.textContent).toBe("abc1234");
154 // the worst session is listed, and opens its numbers and events
155 expect(tbody.children).toHaveLength(1);
156 expect(tbody.textContent).toContain("2 / 8");
157 expect(tbody.textContent).toContain("640 ms");
158 const open = tbody.all((n) => n.tag === "button")[0]!;
159 open.fire("click");
160 await settle();
161 const dialog = byId.get("turn")!;
162 expect(dialog.open).toBe(true);
163 const detail = byId.get("turn-body")!.textContent;
164 expect(detail).toContain(`Turn ${id}`);
165 expect(detail).toContain("taken over by the owner");
166 expect(detail).toContain("jam 2");
167 expect(detail).toContain("· jam");
168 expect(asked).toContain(`/data?turn=${id}`);
169 expect(byId.get("turn-body")!.all((n) => n.classes.has("mark")).map((n) => [...n.classes].join(" "))).toEqual(expect.arrayContaining(["mark app-start", "mark jam"]));
170
171 // a tap shows exact values, and the finger lifting (which also leaves) does not hide them
172 const fig = byId.get("c-room")!.children[0]!;
173 const svg = fig.children[0]!;
174 const tip = fig.all((n) => n.classes.has("tip"))[0]!;
175 svg.fire("pointerdown", { clientX: 590, buttons: 1 });
176 svg.fire("pointerup", { clientX: 590 });
177 svg.fire("pointerleave", { pointerType: "touch" });
178 expect(tip.hidden).toBe(false);
179 expect(tip.textContent).toContain("watchers");
180 svg.fire("pointerleave", { pointerType: "mouse" });
181 expect(tip.hidden).toBe(true);
182 // arrow keys walk the steps
183 fig.fire("keydown", { key: "End" });
184 expect(tip.hidden).toBe(false);
185
186 // compare asks for the period before, and the cards say how it moved
187 byId.get("compare")!.fire("change", { target: { checked: true } });
188 await settle();
189 expect(asked.at(-1)).toEndWith("&compare=1");
190 expect(strip.textContent).toContain("vs before");
191
192 // a drag across a chart zooms every chart into that stretch; Reset goes back
193 const room = byId.get("c-room")!.children[0]!.children[0]!;
194 room.fire("pointerdown", { clientX: 100, buttons: 1 });
195 room.fire("pointermove", { clientX: 300, buttons: 1 });
196 room.fire("pointerup", { clientX: 300 });
197 await settle();
198 const zoomed = new URLSearchParams(asked.at(-1)!.split("?")[1]);
199 expect(Number(zoomed.get("to")) - Number(zoomed.get("from"))).toBeLessThan(12 * 3_600_000);
200 expect(byId.get("reset")!.hidden).toBe(false);
201
202 // the ranges: Hour asks for one hour
203 ranges[0]!.fire("click");
204 await settle();
205 const hour = new URLSearchParams(asked.at(-1)!.split("?")[1]);
206 expect(Number(hour.get("to")) - Number(hour.get("from"))).toBe(3_600_000);
207 expect(ranges.map((r) => r.attrs["aria-pressed"])).toEqual(["true", "false", "false", "false"]);
208 expect(byId.get("status")!.textContent).toBe("");
209 });
Software/so101/src/relay.ts +3
@@ -49,6 +49,7 @@
4949 * the "camera offline" picture when its source is down; stream?fps=1-30 holds one viewer to that frame rate
5050 * WS /camera/<id>/ws the same feed as whole binary frames (JPEG; the offline PNG when its source is down), latest frame wins
5151 * GET /data?from&to&step[&compare=1], ?turn=<id> the facts as series, for the owner and moderators (403 to anyone else)
52 * GET /admin/data the Data view: the facts drawn, for the owner and moderators (404 to anyone else)
5253 * GET /admin/camera-list the owner's page for the camera list
5354 * POST /admin/cameras {"name", "source"}, DELETE /admin/cameras/<id> the owner changes the list; viewers see it within seconds
5455 * GET|PUT|DELETE /leader/calibration the signed-in person's own leader calibration (src/leader-map.js), any time,
@@ -80,6 +81,7 @@ import { OFFLINE_FRAME } from "./offline-frame.ts";
8081 import { ServoBaseline } from "./servo-baseline.ts";
8182 import type { PiSync } from "./pi-deploy.ts";
8283 import { serveData } from "./data-api.ts";
84 import { serveDataPage } from "./data-page.ts";
8385 import { handleAdmin, handleLift, type TuneP } from "./admin-routes.ts";
8486 import { readPrecision } from "./precision-panel.ts";
8587 import { readTuning } from "./tuning-panel.ts";
@@ -774,6 +776,7 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
774776 if (lobby && url.pathname === "/admin/lift") return await handleLift(lobby, req, res, readBody);
775777 if (lobby && (url.pathname === "/admin" || url.pathname === "/admin/health")) return await handleAdmin(lobby, req, res, url, readBody, askPrecision, askServos, videoCards, options.piSync ? () => options.piSync!.versions() : undefined, tuneP);
776778 if (url.pathname === "/admin/versions") return versionsRead(req, res);
779 if (lobby && url.pathname === "/admin/data") return serveDataPage(lobby, req, res);
777780 if (url.pathname === "/data") return serveData(facts ?? undefined, lobby ?? undefined, req, res, url);
778781 if (url.pathname === "/precision/trace") return await traceRead(req, res, url);
779782 if (url.pathname === "/precision") return await precisionRead(req, res, url);
Software/so101/src/terminal.ts +45 −3
@@ -36,6 +36,8 @@
3636 * /health owner or moderator: each open page's link as it last reported it (GET /admin/health, the page's Admin -> Health)
3737 * /versions owner or moderator: the app's and the Pi's commits and the last sync (GET /admin/versions)
3838 * /trace <session> owner or moderator: a kept session's 5 Hz trace, per joint (GET /precision/trace)
39 * /data [hour|day|week|all] owner or moderator: the Data view's overview and worst sessions (GET /data, src/data-lines.ts); default day
40 * /data #<turn> owner or moderator: one turn's numbers and the arm's events around it (GET /data?turn=<id>)
3941 * /precision owner or moderator: the precision scorecard /admin shows (GET /precision, src/precision-panel.ts)
4042 * /servos owner: each joint's servo control registers /admin shows (GET /servo-params, src/servo-panel.ts); 404 to anyone else
4143 * /tune <joint> <P> owner: the P experiment /admin's Servos "Try this P" runs (POST /tuning/p): sweep, write, sweep, keep or put back; 404 to anyone else
@@ -49,6 +51,7 @@
4951 import WebSocket from "ws";
5052 import { REFERENCES, referenceLines, verdictLine } from "./joint-reference.ts";
5153 import { UNPRINTABLE } from "./chat.ts";
54 import { dataLines, RANGES, turnLines, type Range } from "./data-lines.ts";
5255 import { ID, NAME, sign } from "./identity.ts";
5356 import { readServos, type ServoPanel } from "./servo-panel.ts";
5457 import { readTuning, tuningSentence, type TuningView } from "./tuning-panel.ts";
@@ -241,6 +244,27 @@ export function traceReader(env: Env, fetcher: typeof fetch = fetch): ReadTrace
241244 };
242245 }
243246
247 /** What /data asks: a range back from now, or one turn. */
248 export type DataAsk = { range: Range } | { turn: number };
249 export type ReadData = (ask: DataAsk) => Promise<{ error: string | null; lines: string[] }>;
250
251 /** GET /data, signed afresh; the server decides who may, as it does for /admin/data. */
252 export function dataReader(env: Env, fetcher: typeof fetch = fetch, now: () => number = Date.now): ReadData {
253 return async (ask) => {
254 const t = now();
255 const query = "turn" in ask ? `turn=${ask.turn}` : `from=${t - RANGES[ask.range]}&to=${t}`;
256 try {
257 const r = await fetcher(`${appUrl(env)}/data?${query}`, { headers: terminalHeaders(env), signal: AbortSignal.timeout(5000) });
258 const body = (await r.json().catch(() => null)) as { error?: unknown } | null;
259 if (r.status === 403) return { error: "the data is for the owner and moderators", lines: [] };
260 const lines = r.ok ? ("turn" in ask ? turnLines(body) : dataLines(body, ask.range)) : null;
261 return lines ? { error: null, lines } : { error: plain(body?.error ?? `status ${r.status}`), lines: [] };
262 } catch (err) {
263 return { error: plain((err as Error).message), lines: [] };
264 }
265 };
266 }
267
244268 /** The commits and the last sync outcome, one line each; a mismatch is flagged. */
245269 export function versionLines(v: Record<string, unknown> | null): string[] {
246270 if (!v) return ["the Pi is not kept in step from here (no deploy key, or not production)"];
@@ -354,7 +378,8 @@ const said = (room: Room, m: unknown): string => {
354378
355379 const TUNE_USAGE = "usage: /tune <joint, e.g. shoulder_pan or 1> <P, 1 to 64>, or /tune alone for the run so far";
356380 const ZERO_USAGE = "usage: /zero <joint, e.g. shoulder_pan> [degrees the model should read, default 0]";
357 export const HELP = "type to chat · /join /leave /ready /extend · /stop /resume (driver, owner) · /end [who] /table here|touch 1|2|3|<mm> /zero <joint> [deg] /align [joint] (owner) · /mute <who> /unmute <who> (owner, moderator) · /delete <who|#id> (owner, moderator) · /report <who|#id> · /reports /dismiss #<id> (owner, moderator) · /removed /restore #<id> /visitors (owner) · /health /precision /trace <session> /versions (owner, moderator) · /servos /tune [<joint> <P>] (owner) · /machine [off] · /who · /quit";
381 const DATA_USAGE = "usage: /data [hour|day|week|all] or /data #<turn>";
382 export const HELP = "type to chat · /join /leave /ready /extend · /stop /resume (driver, owner) · /end [who] /table here|touch 1|2|3|<mm> /zero <joint> [deg] /align [joint] (owner) · /mute <who> /unmute <who> (owner, moderator) · /delete <who|#id> (owner, moderator) · /report <who|#id> · /reports /dismiss #<id> (owner, moderator) · /removed /restore #<id> /visitors (owner) · /health /precision /trace <session> /data [hour|day|week|all|#<turn>] /versions (owner, moderator) · /servos /tune [<joint> <P>] (owner) · /machine [off] · /who · /quit";
358383
359384 /** The Machine feed's socket beside the lobby's: same app, same port. */
360385 export const machineUrl = (live: string): string => live.replace(/\/live$/, "/machine");
@@ -497,7 +522,7 @@ function target(room: Room, word: string): string | null {
497522 return ID.test(word) ? word : null;
498523 }
499524
500 export type Typed = { send: Frame } | { post: "/stop" | "/resume" } | { table: "here" | number | { touch: 1 | 2 | 3 } } | { zero: { joint: string; referenceDeg: number } } | { align: string | null } | { print: string } | { lines: string[] } | { machine: boolean } | { health: true } | { precision: true } | { trace: number } | { servos: true } | { tune: { joint: number; p: number } | null } | { versions: true } | { quit: true } | null;
525 export type Typed = { send: Frame } | { post: "/stop" | "/resume" } | { table: "here" | number | { touch: 1 | 2 | 3 } } | { zero: { joint: string; referenceDeg: number } } | { align: string | null } | { print: string } | { lines: string[] } | { machine: boolean } | { health: true } | { precision: true } | { trace: number } | { data: DataAsk } | { servos: true } | { tune: { joint: number; p: number } | null } | { versions: true } | { quit: true } | null;
501526
502527 /** What a typed line asks for. Validation the server repeats anyway; this only saves a round trip on a typo. */
503528 export function typed(room: Room, input: string): Typed {
@@ -520,6 +545,7 @@ export function typed(room: Room, input: string): Typed {
520545 if (joint === undefined || rest.length !== 1 || !/^\d{1,3}$/.test(rest[0]!)) return { print: TUNE_USAGE };
521546 return { tune: { joint: Number(joint), p: Number(rest[0]) } };
522547 }
548 if (/^data$/i.test(verb) && rest.length) return { print: DATA_USAGE };
523549 const timed = /^mute$/i.test(verb) && rest.length === 1 && /^(10m|1h|24h|forever)$/i.test(rest[0]!);
524550 if (rest.length && !timed && /^unmute$/i.test(verb)) return { print: "usage: /unmute <name>" };
525551 const touching = /^table$/i.test(verb) && /^touch$/i.test(arg) && rest.length === 1;
@@ -586,6 +612,12 @@ export function typed(room: Room, input: string): Typed {
586612 return { precision: true };
587613 case "trace":
588614 return /^\d{1,9}$/.test(arg) ? { trace: Number(arg) } : { print: "usage: /trace <session number, from /precision>" };
615 case "data": {
616 if (!arg) return { data: { range: "day" } };
617 if (Object.hasOwn(RANGES, arg.toLowerCase())) return { data: { range: arg.toLowerCase() as Range } };
618 const turn = /^#?(\d{1,9})$/.exec(arg);
619 return turn ? { data: { turn: Number(turn[1]) } } : { print: DATA_USAGE };
620 }
589621 case "servos":
590622 return { servos: true };
591623 case "versions":
@@ -622,9 +654,11 @@ export class Door {
622654 private readonly readTrace: ReadTrace | null;
623655 private readonly readVersions: ReadVersions | null;
624656 private readonly readAlign: ReadAlign | null;
657 private readonly readData: ReadData | null;
625658
626659 /** `post` reaches the app's /stop and /resume, `read` its /admin/health, `readCard` its /precision; without them, the door says it cannot. */
627 constructor(url: string, headers: Record<string, string>, print: (line: string) => void, post: Post | null = null, read: Read | null = null, readCard: ReadCard | null = null, readServos: ReadServos | null = null, readVersions: ReadVersions | null = null, readAlign: ReadAlign | null = null, readTuning: ReadTuning | null = null, readTrace: ReadTrace | null = null) {
660 constructor(url: string, headers: Record<string, string>, print: (line: string) => void, post: Post | null = null, read: Read | null = null, readCard: ReadCard | null = null, readServos: ReadServos | null = null, readVersions: ReadVersions | null = null, readAlign: ReadAlign | null = null, readTuning: ReadTuning | null = null, readTrace: ReadTrace | null = null, readData: ReadData | null = null) {
661 this.readData = readData;
628662 this.readTrace = readTrace;
629663 this.readAlign = readAlign;
630664 this.readTuning = readTuning;
@@ -677,6 +711,7 @@ export class Door {
677711 else if ("health" in t) void this.health();
678712 else if ("precision" in t) void this.precision();
679713 else if ("trace" in t) void this.trace(t.trace);
714 else if ("data" in t) void this.data(t.data);
680715 else if ("servos" in t) void this.servos();
681716 else if ("tune" in t) void this.tune(t.tune);
682717 else if ("versions" in t) void this.versions();
@@ -722,6 +757,13 @@ export class Door {
722757 for (const line of r.error ? [`error: ${r.error}`] : r.lines) this.print(line);
723758 }
724759
760 /** The server decides who may; a refusal is printed, else the overview and worst sessions, or the one turn. */
761 private async data(ask: DataAsk): Promise<void> {
762 if (!this.readData) return this.print("error: the data is not available here");
763 const r = await this.readData(ask);
764 for (const line of r.error ? [`error: ${r.error}`] : r.lines) this.print(line);
765 }
766
725767 /** The server decides who may; a refusal is printed, else the joints' registers. */
726768 private async servos(): Promise<void> {
727769 if (!this.readServos) return this.print("error: servos are not available here");
Software/so101/src/tty.ts +2 −2
@@ -4,7 +4,7 @@
44 */
55
66 import { createInterface, type Interface } from "node:readline";
7 import { alignReader, cardReader, Door, HELP, liveUrl, poster, reader, servosReader, terminalHeaders, traceReader, tuningReader, versionsReader } from "./terminal.ts";
7 import { alignReader, cardReader, dataReader, Door, HELP, liveUrl, poster, reader, servosReader, terminalHeaders, traceReader, tuningReader, versionsReader } from "./terminal.ts";
88
99 const env = process.env;
1010 const where = env.FORGE_APP_ENV === "preview" ? `preview of #${env.FORGE_APP_PULL ?? "?"}` : (env.FORGE_APP_ENV ?? "local");
@@ -17,7 +17,7 @@ const print = (line: string) => {
1717 rl?.prompt(true);
1818 };
1919
20 const door = new Door(liveUrl(env), terminalHeaders(env), print, poster(env), reader(env), cardReader(env), servosReader(env), versionsReader(env), alignReader(env), tuningReader(env), traceReader(env));
20 const door = new Door(liveUrl(env), terminalHeaders(env), print, poster(env), reader(env), cardReader(env), servosReader(env), versionsReader(env), alignReader(env), tuningReader(env), traceReader(env), dataReader(env));
2121 print(`\x1b[1mSO-101 arm\x1b[0m · ${where} · ${HELP}`);
2222 try {
2323 await door.open(() => {
Software/so101/src/ui/data-view.js added +472
@@ -0,0 +1,472 @@
1 // @ts-nocheck: a classic script inlined in the page, typed by its use (no DOM lib in this tsconfig).
2 // The Data view's charts (/admin/data): reads GET /data and draws every chart by hand in SVG. Inlined in the page
3 // and pinned by its hash (src/data-page.ts). Nothing from the answer is ever parsed as HTML: text goes in as text.
4 (() => {
5 "use strict";
6 const NS = "http://www.w3.org/2000/svg";
7 const HOUR = 3_600_000;
8 const RANGES = { hour: HOUR, day: 24 * HOUR, week: 7 * 24 * HOUR, all: 90 * 24 * HOUR };
9 const state = { range: "day", custom: null, compare: false, data: null, seq: 0 };
10 const $ = (sel) => document.querySelector(sel);
11 const svg = (tag, attrs, parent) => {
12 const e = document.createElementNS(NS, tag);
13 for (const k in attrs) e.setAttribute(k, String(attrs[k]));
14 if (parent) parent.appendChild(e);
15 return e;
16 };
17 const html = (tag, cls, text, parent) => {
18 const e = document.createElement(tag);
19 if (cls) e.className = cls;
20 if (text !== undefined && text !== null) e.textContent = String(text);
21 if (parent) parent.appendChild(e);
22 return e;
23 };
24 const finite = (v) => typeof v === "number" && Number.isFinite(v);
25
26 // ---------------------------------------------------------------- numbers and times
27 const fmt = (v, unit) => {
28 if (!finite(v)) return "—";
29 const a = Math.abs(v);
30 const s = a >= 10_000 ? `${(v / 1000).toFixed(a >= 100_000 ? 0 : 1)}k` : a >= 100 || Number.isInteger(v) ? Math.round(v).toLocaleString() : v.toFixed(a >= 10 ? 1 : 2);
31 return unit ? `${s} ${unit}` : s;
32 };
33 const pct = (v) => (finite(v) ? `${(v * 100).toFixed(v >= 0.999 ? 0 : 1)}%` : "—");
34 const when = (ms, step) => {
35 const date = new Date(ms);
36 const day = date.toLocaleDateString(undefined, { month: "short", day: "numeric" });
37 const time = date.toLocaleTimeString(undefined, { hour: "2-digit", minute: "2-digit" });
38 return step >= 86_400 ? day : step >= 3 * 3600 ? `${day} ${time}` : time;
39 };
40 const duration = (ms) => (ms < 60_000 ? `${Math.round(ms / 1000)} s` : ms < HOUR ? `${Math.floor(ms / 60_000)} min ${Math.round((ms % 60_000) / 1000)} s` : `${(ms / HOUR).toFixed(1)} h`);
41 const niceMax = (v) => {
42 if (!(v > 0)) return 1;
43 const p = 10 ** Math.floor(Math.log10(v));
44 for (const m of [1, 2, 2.5, 5, 10]) if (v <= m * p) return m * p;
45 return 10 * p;
46 };
47 const sum = (a) => a.reduce((x, y) => x + (finite(y) ? y : 0), 0);
48
49 // ---------------------------------------------------------------- one time chart
50 const CH = 170, L = 40, R = 10, T = 14, B = 22;
51 /** Drawn at the box's own width, one unit a pixel, so text is never stretched. */
52 const width = (box) => Math.max(240, Math.round(box.getBoundingClientRect().width || 600));
53 /**
54 * series: [{name, values, color (a palette index), kind: "line"|"bar", unit, dashed?}], stacked bars when stack is true;
55 * band: [lowName, highName] fills between two lines. Previous period: series with prev: true are drawn faint.
56 */
57 function timeChart(box, spec) {
58 const d = state.data;
59 const t = d.t, n = t.length, step = d.step;
60 box.replaceChildren();
61 const W = width(box), H = spec.h || CH;
62 const fig = html("figure", "chart", null, box);
63 const root = svg("svg", { viewBox: `0 0 ${W} ${H}`, width: W, height: H, role: "img", "aria-label": spec.label }, fig);
64 const legend = html("figcaption", "legend", null, fig);
65 for (const s of spec.series.filter((s) => !s.prev)) {
66 const k = html("span", `key c${s.color}`, null, legend);
67 html("i", null, null, k);
68 k.append(s.name);
69 }
70 const bw = (W - L - R) / n;
71 const x = (i) => L + i * bw + bw / 2;
72 const stackTotals = t.map((_, i) => sum(spec.series.filter((s) => s.kind === "bar" && !s.prev).map((s) => s.values[i])));
73 const all = spec.series.flatMap((s) => (s.kind === "bar" && spec.stack ? [] : s.values)).filter(finite);
74 const top = niceMax(Math.max(spec.min ?? 0, ...all, ...(spec.stack ? stackTotals : []), ...spec.series.filter((s) => s.kind === "bar" && !spec.stack).flatMap((s) => s.values.filter(finite))));
75 const y = (v) => T + (H - T - B) * (1 - v / top);
76 for (const f of [0, 0.5, 1]) {
77 svg("line", { x1: L, x2: W - R, y1: y(top * f), y2: y(top * f), class: "grid" }, root);
78 svg("text", { x: L - 6, y: y(top * f) + 4, class: "tick", "text-anchor": "end" }, root).textContent = spec.percent ? pct(top * f) : fmt(top * f);
79 }
80 for (const i of [0, Math.floor(n / 2), n - 1]) {
81 if (i < 0 || (i > 0 && i === Math.floor(n / 2) && n < 3)) continue;
82 svg("text", { x: x(i), y: H - 6, class: "tick", "text-anchor": i === 0 ? "start" : i === n - 1 ? "end" : "middle" }, root).textContent = when(t[i], step);
83 }
84 // bars, stacked or side by side
85 const base = new Array(n).fill(0);
86 const bars = spec.series.filter((s) => s.kind === "bar" && !s.prev);
87 bars.forEach((s, k) => {
88 for (let i = 0; i < n; i++) {
89 const v = s.values[i];
90 if (!finite(v) || v <= 0) continue;
91 const w = spec.stack ? bw * 0.78 : (bw * 0.78) / bars.length;
92 const x0 = L + i * bw + bw * 0.11 + (spec.stack ? 0 : k * w);
93 const y1 = spec.stack ? y(base[i] + v) : y(v);
94 const y0 = spec.stack ? y(base[i]) : y(0);
95 svg("rect", { x: x0, y: y1, width: Math.max(1, w), height: Math.max(0.5, y0 - y1), class: `fill c${s.color}` }, root);
96 if (spec.stack) base[i] += v;
97 }
98 });
99 // a band between two lines
100 if (spec.band) {
101 const [lo, hi] = spec.band.map((name) => spec.series.find((s) => s.name === name));
102 let seg = [];
103 const flush = () => {
104 if (seg.length > 1) svg("path", { d: `M${seg.map((i) => `${x(i)},${y(hi.values[i])}`).join("L")}L${[...seg].reverse().map((i) => `${x(i)},${y(lo.values[i])}`).join("L")}Z`, class: `band c${hi.color}` }, root);
105 seg = [];
106 };
107 for (let i = 0; i < n; i++) (finite(lo.values[i]) && finite(hi.values[i]) ? seg.push(i) : flush());
108 flush();
109 }
110 // lines, with gaps where nothing was measured
111 for (const s of spec.series.filter((s) => s.kind === "line" || (s.prev && s.kind === "bar"))) {
112 let d = "";
113 let pen = false;
114 for (let i = 0; i < n; i++) {
115 const v = s.values[i];
116 if (!finite(v)) {
117 pen = false;
118 continue;
119 }
120 d += `${pen ? "L" : "M"}${x(i).toFixed(1)},${y(v).toFixed(1)}`;
121 pen = true;
122 }
123 if (d) svg("path", { d, class: `line c${s.color}${s.prev ? " prev" : ""}${s.dashed ? " dashed" : ""}` }, root);
124 const lone = s.values.map((v, i) => (finite(v) && !finite(s.values[i - 1]) && !finite(s.values[i + 1]) ? i : -1)).filter((i) => i >= 0);
125 if (!s.prev) for (const i of lone) svg("circle", { cx: x(i), cy: y(s.values[i]), r: 2.5, class: `dot c${s.color}` }, root);
126 }
127 // the deployed commits
128 for (const v of d.versions) {
129 if (v.at < d.from || v.at >= d.to) continue;
130 const vx = L + ((v.at - d.from) / (d.to - d.from)) * (W - L - R);
131 const mark = svg("line", { x1: vx, x2: vx, y1: T - 4, y2: H - B, class: "version" }, root);
132 svg("title", {}, mark).textContent = `deployed ${v.commit} at ${new Date(v.at).toLocaleString()}`;
133 svg("text", { x: vx + 3, y: T - 4, class: "vlabel" }, root).textContent = v.commit.slice(0, 7);
134 }
135 if (!spec.series.some((s) => !s.prev && s.values.some((v) => finite(v) && v !== 0))) svg("text", { x: W / 2, y: H / 2, class: "none", "text-anchor": "middle" }, root).textContent = spec.empty || "Nothing in this range";
136 // hover, tap, arrow keys: exact values; drag: zoom
137 const cursor = svg("line", { y1: T, y2: H - B, class: "cursor", visibility: "hidden" }, root);
138 const brush = svg("rect", { y: T, height: H - T - B, class: "brush", visibility: "hidden" }, root);
139 const tip = html("div", "tip", null, fig);
140 tip.hidden = true;
141 fig.tabIndex = 0;
142 let at = -1;
143 const show = (i) => {
144 at = Math.max(0, Math.min(n - 1, i));
145 cursor.setAttribute("x1", x(at));
146 cursor.setAttribute("x2", x(at));
147 cursor.setAttribute("visibility", "visible");
148 tip.replaceChildren();
149 html("b", null, `${when(t[at], step)} – ${when(t[at] + step * 1000, step)}`, tip);
150 for (const s of spec.series) {
151 const row = html("span", `row c${s.color}${s.prev ? " prev" : ""}`, null, tip);
152 html("i", null, null, row);
153 row.append(`${s.prev ? "before: " : ""}${s.name} `);
154 html("b", null, spec.percent ? pct(s.values[at]) : fmt(s.values[at], s.unit), row);
155 }
156 tip.hidden = false;
157 tip.classList.toggle("left", at > n / 2);
158 };
159 const hide = () => {
160 cursor.setAttribute("visibility", "hidden");
161 tip.hidden = true;
162 };
163 const index = (ev) => {
164 const r = root.getBoundingClientRect();
165 return Math.floor((((ev.clientX - r.left) / r.width) * W - L) / bw);
166 };
167 let from = null;
168 root.addEventListener("pointerdown", (ev) => {
169 from = index(ev);
170 show(from);
171 });
172 root.addEventListener("pointermove", (ev) => {
173 const i = index(ev);
174 show(i);
175 if (from !== null && ev.buttons && Math.abs(i - from) >= 1) {
176 const [a, b] = [Math.max(0, Math.min(from, i)), Math.min(n - 1, Math.max(from, i))];
177 brush.setAttribute("x", L + a * bw);
178 brush.setAttribute("width", (b - a + 1) * bw);
179 brush.setAttribute("visibility", "visible");
180 }
181 });
182 root.addEventListener("pointerup", (ev) => {
183 const i = index(ev);
184 brush.setAttribute("visibility", "hidden");
185 if (from !== null && Math.abs(i - from) >= 2) {
186 const [a, b] = [Math.max(0, Math.min(from, i)), Math.min(n - 1, Math.max(from, i))];
187 state.custom = { from: t[a], to: Math.min(d.to, t[b] + step * 1000) };
188 load();
189 }
190 from = null;
191 });
192 // a finger lifting also leaves: what a tap showed stays until the next tap or a blur
193 root.addEventListener("pointerleave", (ev) => {
194 from = null;
195 brush.setAttribute("visibility", "hidden");
196 if (ev.pointerType === "mouse") hide();
197 });
198 root.addEventListener("pointercancel", () => {
199 from = null;
200 brush.setAttribute("visibility", "hidden");
201 });
202 fig.addEventListener("keydown", (ev) => {
203 if (ev.key === "ArrowLeft" || ev.key === "ArrowRight") {
204 ev.preventDefault();
205 show(at < 0 ? n - 1 : at + (ev.key === "ArrowLeft" ? -1 : 1));
206 } else if (ev.key === "Home" || ev.key === "End") {
207 ev.preventDefault();
208 show(ev.key === "Home" ? 0 : n - 1);
209 } else if (ev.key === "Escape") hide();
210 });
211 fig.addEventListener("blur", hide);
212 }
213
214 /** Bars over named bins, for the histograms and the funnel. */
215 function binChart(box, label, names, counts, color, empty) {
216 box.replaceChildren();
217 const W = width(box), H = CH;
218 const fig = html("figure", "chart", null, box);
219 const root = svg("svg", { viewBox: `0 0 ${W} ${H}`, width: W, height: H, role: "img", "aria-label": label }, fig);
220 const top = niceMax(Math.max(1, ...counts));
221 const bw = (W - L - R) / counts.length;
222 const y = (v) => T + (H - T - B) * (1 - v / top);
223 svg("line", { x1: L, x2: W - R, y1: y(0), y2: y(0), class: "grid" }, root);
224 svg("text", { x: L - 6, y: y(top) + 4, class: "tick", "text-anchor": "end" }, root).textContent = fmt(top);
225 counts.forEach((c, i) => {
226 const r = svg("rect", { x: L + i * bw + bw * 0.12, y: y(c), width: bw * 0.76, height: Math.max(0.5, y(0) - y(c)), class: `fill c${color}` }, root);
227 svg("title", {}, r).textContent = `${names[i]}: ${c}`;
228 svg("text", { x: L + i * bw + bw / 2, y: H - 6, class: "tick", "text-anchor": "middle" }, root).textContent = names[i];
229 if (c > 0) svg("text", { x: L + i * bw + bw / 2, y: y(c) - 4, class: "val", "text-anchor": "middle" }, root).textContent = String(c);
230 });
231 if (!counts.some((c) => c > 0)) svg("text", { x: W / 2, y: H / 2, class: "none", "text-anchor": "middle" }, root).textContent = empty || "Nothing in this range";
232 }
233 /** One bar a row, the label beside it, for names too long to stand under a bar. */
234 function rowChart(box, label, entries, color, empty) {
235 box.replaceChildren();
236 const W = width(box), rowH = 26, H = Math.max(rowH * 2, entries.length * rowH + 8);
237 const fig = html("figure", "chart", null, box);
238 const root = svg("svg", { viewBox: `0 0 ${W} ${H}`, width: W, height: H, role: "img", "aria-label": label }, fig);
239 if (!entries.length) {
240 svg("text", { x: W / 2, y: H / 2, class: "none", "text-anchor": "middle" }, root).textContent = empty || "Nothing in this range";
241 return;
242 }
243 const left = Math.min(176, Math.round(W * 0.48)), right = 40;
244 const top = Math.max(1, ...entries.map(([, n]) => n));
245 entries.forEach(([name, n], i) => {
246 const y = 4 + i * rowH;
247 svg("text", { x: left - 8, y: y + rowH / 2 + 4, class: "tick", "text-anchor": "end" }, root).textContent = name;
248 const w = Math.max(1, ((W - left - right) * n) / top);
249 const bar = svg("rect", { x: left, y: y + 5, width: w, height: rowH - 10, rx: 2, class: `fill c${color}` }, root);
250 svg("title", {}, bar).textContent = `${name}: ${n}`;
251 svg("text", { x: left + w + 6, y: y + rowH / 2 + 4, class: "val" }, root).textContent = String(n);
252 });
253 }
254 const binNames = (edges, unit) => [`<${edges[0]}`, ...edges.slice(1).map((e, i) => `${edges[i]}–${e}`), `${edges[edges.length - 1]}+`].map((s, i) => (i === edges.length ? `${s} ${unit}` : s));
255
256 // ---------------------------------------------------------------- the page
257 function spark(parent, values, color) {
258 const v = values.map((x) => (finite(x) ? x : null));
259 const top = niceMax(Math.max(0, ...v.filter(finite)));
260 const root = svg("svg", { viewBox: "0 0 100 24", class: "spark", "aria-hidden": "true", preserveAspectRatio: "none" }, parent);
261 let d = "";
262 let pen = false;
263 v.forEach((x, i) => {
264 if (x === null) return void (pen = false);
265 d += `${pen ? "L" : "M"}${((i + 0.5) / v.length) * 100},${22 - (x / top) * 20}`;
266 pen = true;
267 });
268 if (d) svg("path", { d, class: `line c${color}` }, root);
269 }
270
271 function overview(d) {
272 const o = d.overview, s = d.series, p = d.previous && d.previous.overview;
273 const plus = (a, b) => a.map((x, i) => x + b[i]);
274 const cards = [
275 ["Visitors", o.visitors, p && p.visitors, plus(s.visitorsAnon, s.visitorsNamed), 0, fmt, true],
276 ["Driving sessions", o.sessions, p && p.sessions, s.sessions, 1, fmt, true],
277 ["Drivers", o.drivers, p && p.drivers, s.drivers, 1, fmt, true],
278 ["Minutes driven", o.minutes, p && p.minutes, s.minutes, 2, fmt, true],
279 ["Snap-backs / 100 releases", o.snapBacksPer100, p && p.snapBacksPer100, s.snapBacksPer100, 3, fmt, false],
280 ["p95 time to arrive", o.arriveP95Ms, p && p.arriveP95Ms, s.arriveP95Ms, 5, (v) => fmt(v, "ms"), false],
281 ["Jams", o.jams, p && p.jams, s.jams, 3, fmt, false],
282 ["Uptime", o.uptime, p && p.uptime, s.uptime, 1, pct, true],
283 ];
284 const strip = $("#strip");
285 strip.replaceChildren();
286 for (const [label, v, before, series, color, f, upIsGood] of cards) {
287 const card = html("article", "stat", null, strip);
288 html("h3", null, label, card);
289 html("p", "big", f(v), card);
290 if (state.compare && finite(before) && finite(v)) {
291 const delta = v - before;
292 const good = delta === 0 ? "same" : delta > 0 === upIsGood ? "good" : "bad";
293 html("p", `delta ${good}`, `${delta > 0 ? "▲" : delta < 0 ? "▼" : "="} ${f(Math.abs(delta))} vs before`, card);
294 } else html("p", "delta", state.compare ? "nothing before" : "", card);
295 spark(card, series, color);
296 }
297 }
298
299 const prevOf = (key, name, color) => (state.compare && state.data.previous ? [{ name, values: state.data.previous.series[key], color, kind: "line", prev: true }] : []);
300
301 function precision(d) {
302 const s = d.series, causes = Object.keys(s.byCause);
303 timeChart($("#c-causes"), { label: "Snap-backs by cause", stack: true, series: causes.map((c, i) => ({ name: c, values: s.byCause[c], color: i % 8, kind: "bar" })), empty: "No snap-backs in this range" });
304 timeChart($("#c-per100"), { label: "Snap-backs per 100 releases", series: [{ name: "per 100 releases", values: s.snapBacksPer100, color: 3, kind: "line" }, ...prevOf("snapBacksPer100", "per 100 releases", 3)], empty: "No measured releases in this range" });
305 const body = $("#worst tbody");
306 body.replaceChildren();
307 if (!d.worst.length) {
308 const tr = html("tr", null, null, body);
309 html("td", "empty", "No turn the arm measured in this range.", tr).colSpan = 6;
310 }
311 for (const w of d.worst) {
312 const tr = html("tr", null, null, body);
313 const td = html("td", null, null, tr);
314 const btn = html("button", "link", `${when(w.start, 3600)} · turn ${w.id}`, td);
315 btn.type = "button";
316 btn.addEventListener("click", () => turn(w.id));
317 html("td", null, duration(w.seconds * 1000), tr);
318 html("td", null, `${fmt(w.snapBacks)} / ${fmt(w.releases)}`, tr);
319 html("td", null, fmt(w.snapBacksPer100), tr);
320 html("td", null, fmt(w.arriveP95Ms, "ms"), tr);
321 html("td", "mono", w.commit.slice(0, 7), tr);
322 }
323 }
324
325 function latency(d) {
326 timeChart($("#c-arrive"), { label: "Time to arrive, p95 of the turns", series: [{ name: "p95 arrive", values: d.series.arriveP95Ms, color: 5, kind: "line", unit: "ms" }, ...prevOf("arriveP95Ms", "p95 arrive", 5)], empty: "No turn the arm timed in this range" });
327 binChart($("#c-arrivehist"), "Turns by their p95 time to arrive", binNames(d.arrive.edgesMs, "ms"), d.arrive.counts, 5);
328 }
329
330 function people(d) {
331 const s = d.series;
332 $("#live").textContent = `Now: ${d.live.watchers} watching · ${d.live.driving ? "someone driving" : "nobody driving"} · ${d.live.queue} in line`;
333 timeChart($("#c-visitors"), { label: "Visitors", stack: true, series: [{ name: "named", values: s.visitorsNamed, color: 0, kind: "bar" }, { name: "anonymous", values: s.visitorsAnon, color: 6, kind: "bar" }] });
334 timeChart($("#c-room"), { label: "Watchers, drivers and the line", series: [{ name: "watchers", values: s.watchers, color: 0, kind: "line" }, { name: "driving", values: s.driving, color: 1, kind: "line" }, { name: "in line", values: s.queue, color: 3, kind: "line" }, ...prevOf("watchers", "watchers", 0)] });
335 timeChart($("#c-wait"), { label: "Wait in line", series: [{ name: "p50 wait", values: s.waitP50S, color: 2, kind: "line", unit: "s" }, { name: "p95 wait", values: s.waitP95S, color: 3, kind: "line", unit: "s", dashed: true }] });
336 binChart($("#c-turns"), "Turn length", binNames(d.turnLengths.edgesS, "s"), d.turnLengths.counts, 1);
337 timeChart($("#c-chat"), { label: "Chat lines", series: [{ name: "lines", values: s.chat, color: 7, kind: "bar" }, ...prevOf("chat", "lines", 7)] });
338 const f = d.funnel;
339 binChart($("#c-funnel"), "Where people drop off", ["opened", "named", "in line", "drove", "drove <10 s"], [f.visitors, f.named, f.queued, f.drove, f.under10s], 0);
340 const ended = Object.entries(d.reasons).sort((a, b) => b[1] - a[1]);
341 rowChart($("#c-ended"), "How turns ended", ended, 2, "No turn ended in this range");
342 }
343
344 function health(d) {
345 const s = d.series;
346 timeChart($("#c-uptime"), { label: "Arm link uptime", percent: true, min: 1, series: [{ name: "live", values: s.uptime, color: 1, kind: "line" }], empty: "No link samples: the app is simulating, or just started" });
347 timeChart($("#c-events"), { label: "Reconnects, app starts, STOPs, jams and blocks", series: [{ name: "reconnects", values: s.reconnects, color: 2, kind: "bar" }, { name: "app starts", values: s.restarts, color: 4, kind: "bar" }, { name: "STOP", values: s.stops, color: 6, kind: "bar" }, { name: "jams", values: s.jams, color: 3, kind: "bar" }, { name: "blocks", values: s.blocks, color: 5, kind: "bar" }], empty: "No reconnect, app start, STOP, jam or block" });
348 const cams = Object.keys(s.fps);
349 timeChart($("#c-fps"), { label: "Camera fps", series: cams.map((c, i) => ({ name: `camera ${c}`, values: s.fps[c], color: i % 8, kind: "line", unit: "fps" })), empty: "No H.264 camera reported fps" });
350 const starts = sum(s.restarts);
351 const deploys = d.versions.filter((v) => v.at >= d.from && v.at < d.to).length;
352 $("#starts").textContent = starts ? `The app started ${starts} time${starts === 1 ? "" : "s"} in this range, ${deploys} on a new commit.` : "The app did not start in this range.";
353 }
354
355 function render() {
356 const d = state.data;
357 const range = $("#range");
358 range.textContent = `${new Date(d.from).toLocaleString()} – ${new Date(d.to).toLocaleString()} · per ${duration(d.step * 1000)}`;
359 $("#reset").hidden = !state.custom;
360 overview(d);
361 precision(d);
362 latency(d);
363 people(d);
364 health(d);
365 }
366
367 async function load() {
368 const seq = ++state.seq;
369 const now = Date.now();
370 const span = state.custom ? null : RANGES[state.range];
371 const to = state.custom ? state.custom.to : now;
372 const from = state.custom ? state.custom.from : now - span;
373 document.body.classList.add("loading");
374 $("#status").textContent = "";
375 try {
376 const r = await fetch(`/data?from=${Math.floor(from)}&to=${Math.ceil(to)}${state.compare ? "&compare=1" : ""}`, { credentials: "same-origin", cache: "no-store" });
377 const body = await r.json();
378 if (seq !== state.seq) return;
379 if (!r.ok) throw new Error(body && body.error ? body.error : `the app answered ${r.status}`);
380 state.data = body;
381 render();
382 } catch (err) {
383 if (seq === state.seq) $("#status").textContent = `Could not load: ${err.message}`;
384 } finally {
385 if (seq === state.seq) document.body.classList.remove("loading");
386 }
387 }
388
389 /** An event's kind as a class name: "app start" is app-start, STOP is stop. */
390 const markOf = (kind) => String(kind).toLowerCase().replace(/[^a-z]+/g, "-");
391
392 async function turn(id) {
393 const dialog = $("#turn");
394 const body = $("#turn-body");
395 body.replaceChildren(html("p", "empty", "Loading…"));
396 dialog.showModal();
397 try {
398 const r = await fetch(`/data?turn=${encodeURIComponent(id)}`, { credentials: "same-origin", cache: "no-store" });
399 const d = await r.json();
400 if (!r.ok) throw new Error(d.error || r.status);
401 const t = d.turn;
402 const start = Date.parse(t.startedAt), ms = t.seconds * 1000;
403 body.replaceChildren();
404 html("h2", null, `Turn ${t.id} · ${new Date(start).toLocaleString()}`, body).id = "turn-title";
405 const facts = html("dl", "facts", null, body);
406 const pair = (k, v) => {
407 html("dt", null, k, facts);
408 html("dd", null, v, facts);
409 };
410 const backs = t.snapBacks ? Object.entries(t.snapBacks).filter(([, n]) => n > 0) : null;
411 pair("Length", duration(ms));
412 pair("Ended", t.ended);
413 pair("Waited", t.waitSeconds > 0 ? duration(t.waitSeconds * 1000) : "no wait");
414 pair("Releases", t.releases === null ? "not measured" : fmt(t.releases));
415 pair("Snap-backs", backs === null ? "not measured" : backs.length ? backs.map(([c, n]) => `${c} ${n}`).join(", ") : "none");
416 pair("Jams", t.jams === null ? "not measured" : fmt(t.jams));
417 pair("p95 time to arrive", fmt(t.arriveP95Ms, "ms"));
418 pair("Commit", t.commit.slice(0, 12));
419 // the events on the turn's timeline, a minute either side
420 html("h3", null, "Timeline", body);
421 const span = ms + 120_000, t0 = start - 60_000;
422 const W = width(body);
423 const root = svg("svg", { viewBox: `0 0 ${W} 60`, width: W, height: 60, class: "timeline", role: "img", "aria-label": "Events around the turn" }, body);
424 const x = (at) => L + ((at - t0) / span) * (W - L - R);
425 svg("rect", { x: x(start), y: 18, width: Math.max(2, x(start + ms) - x(start)), height: 16, class: "turnbar" }, root);
426 for (const e of d.events) {
427 const m = svg("line", { x1: x(e.at), x2: x(e.at), y1: 8, y2: 44, class: `mark ${markOf(e.kind)}` }, root);
428 svg("title", {}, m).textContent = `${e.kind} at ${new Date(e.at).toLocaleTimeString()}`;
429 }
430 svg("text", { x: L, y: 56, class: "tick" }, root).textContent = new Date(t0).toLocaleTimeString();
431 svg("text", { x: W - R, y: 56, class: "tick", "text-anchor": "end" }, root).textContent = new Date(t0 + span).toLocaleTimeString();
432 const list = html("ul", "events", null, body);
433 if (!d.events.length) html("li", "empty", "No STOP, jam, block, reconnect or app start.", list);
434 for (const e of d.events) html("li", null, `${new Date(e.at).toLocaleTimeString()} · ${e.kind}${e.at >= start && e.at <= start + ms ? "" : " (outside the turn)"}`, list);
435 const seen = d.samples.filter((m) => m.at >= start && m.at < start + ms);
436 if (seen.length) {
437 html("h3", null, "The room each minute", body);
438 const sl = html("ul", "events", null, body);
439 for (const m of seen) html("li", null, `${new Date(m.at).toLocaleTimeString()} · ${m.watchers} watching · ${m.queue} in line · arm link ${m.linkLive ? "live" : "down"}`, sl);
440 }
441 } catch (err) {
442 body.replaceChildren(html("p", "empty", `Could not load that turn: ${err.message}`));
443 }
444 }
445
446 for (const b of document.querySelectorAll("[data-range]")) {
447 b.addEventListener("click", () => {
448 state.range = b.dataset.range;
449 state.custom = null;
450 for (const o of document.querySelectorAll("[data-range]")) o.setAttribute("aria-pressed", String(o === b));
451 load();
452 });
453 }
454 $("#compare").addEventListener("change", (ev) => {
455 state.compare = ev.target.checked;
456 load();
457 });
458 $("#reset").addEventListener("click", () => {
459 state.custom = null;
460 load();
461 });
462 $("#turn-close").addEventListener("click", () => $("#turn").close());
463 // a new width (a phone turned, a window resized) draws again at it
464 let wide = document.documentElement.clientWidth, redraw = 0;
465 addEventListener("resize", () => {
466 if (document.documentElement.clientWidth === wide) return;
467 wide = document.documentElement.clientWidth;
468 clearTimeout(redraw);
469 redraw = setTimeout(() => state.data && render(), 150);
470 });
471 load();
472 })();
Software/so101/src/viewer.html +1
@@ -357,6 +357,7 @@
357357 <div class="card" id="modCard">
358358 <h3>Room</h3>
359359 <div class="row" id="modTurn"></div>
360 <p><a href="/admin/data">Data</a>: people, precision, latency and health over time.</p>
360361 <p>Long-press a chat message (or right-click it) to delete it, or mute or ban its author. In the Queue tab, × takes someone out of the line.</p>
361362 <div id="mutedLine"></div>
362363 <div id="bannedLine"></div>

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs