Public: anyone can read this; the crew can change it.

Migrated from github.com/Trevato/SO-ARM100

workbenchthis machine

Frame age p95 and bytes per viewer for H.264 versus MJPEG are measured on the public URL and written in the README #431

Closed opened this
Closed without merging

vesper/arm:vesper/403-frame-age-p95-and-bytes-per-viewer-for-h → main 3 commits fork

  • Closed without merging. Nothing was applied to main.

Review

Sign in to review this change.
3 files changed +497 −0
Software/README.md +62
@@ -925,6 +925,68 @@ curl https://arm--forge-trevato-apps.open-platform.sh/metrics.prom
925925 The endpoint is unauthenticated, like `/link` and `/status`: it exposes only timing aggregates, no names, tokens or
926926 commands, and cannot move the arm. `GET /metrics` is unchanged and is still the arm server's own, proxied.
927927
928 ## Video trial
929
930 `so101/bench/video-trial.ts` watches one camera of a deployed page the way a
931 viewer does and prints one line per feed: pictures and fps, bytes per second
932 to this one viewer, freezes (a gap of 1 s or more without a new picture, the
933 start and end of the run included), the longest gap, and for H.264 the frame
934 age p50/p95. With `--net` it adds the verdict against the target: p95 under
935 250 ms on wifi and under 500 ms on 4G, with 0 freezes, and a miss is printed
936 as `target MISSED`. Run it from the network being measured: a laptop on the
937 home wifi, then the same laptop tethered to a phone on 4G.
938
939 ```
940 bun run so101/bench/video-trial.ts https://arm--forge-trevato-apps.open-platform.sh --net wifi
941 bun run so101/bench/video-trial.ts https://arm--forge-trevato-apps.open-platform.sh --net 4g --seconds 600 --feed frames,h264,stream --camera 1
942 ```
943
944 The feeds are `frames` (WS `/camera/<id>/ws`, what the page draws behind the
945 relay), `h264` (WS `/camera/<id>/video`, what it decodes when the camera
946 offers H.264) and `stream` (the multipart `/camera/<id>/stream` of a plain
947 img). The default is frames then h264, 600 s each, one after the other.
948
949 What it can and cannot measure:
950
951 - The H.264 frame age is the one the page takes: each picture's delay past
952 the least delay seen, from the relay's stamp to arrival. That is the
953 queueing on the way, not capture to shown: the Pi→relay leg and the decode
954 are not in it. For capture to shown, film a running clock with the camera
955 and photograph the screen next to the clock.
956 - The JPEG feeds carry no timestamp, so their age is not measured.
957 - A feed that closes ends that feed's run early. The page reopens it after
958 0.5 s; the tool does not, so its seconds say how long the socket lasted.
959
960 ### Measured
961
962 | Date (UTC) | From | Feed | Seconds | fps | Bytes per viewer | Frame age p95 | Freezes (longest gap) | Target |
963 | --- | --- | --- | --- | --- | --- | --- | --- | --- |
964 | 2026-10-21 07:28 | forge sandbox (datacenter, not wifi or 4G) | frames | 35.7 (socket closed) | 10.1 | 398.7 KiB/s | not measured | 1 (1648 ms) | **missed** (1 freeze) |
965 | 2026-10-21 07:28 | forge sandbox | h264 | — | — | — | — | — | **not measurable**: production serves no H.264 for camera 1 (`/cameras` says `"h264": false`, the socket is refused) |
966 | 2026-10-21 07:29 | forge sandbox | stream | 120 | 9.0 | 356.8 KiB/s | not measured | 3 (11 970 ms) | **missed** (3 freezes) |
967 | 2026-10-21 07:21 | forge sandbox | stream | 79.9 (stream closed) | 10.0 | 396.5 KiB/s | not measured | 2 (1657 ms) | **missed** (2 freezes) |
968
969 Command for the 07:28 rows:
970 `bun run so101/bench/video-trial.ts https://arm--forge-trevato-apps.open-platform.sh --seconds 120 --feed frames,h264,stream --net wifi`.
971 `--net wifi` only picks the target to compare against. The run went from
972 the forge's sandbox through its proxy, not over home wifi, and ran for
973 120 s, not 10 minutes.
974
975 What this says so far: an MJPEG viewer costs about 360–400 KiB/s
976 (≈3 Mb/s) at 9–10 fps. That is about twice the 720p H.264 rung's ~1.5 Mb/s
977 and 5× the 480p rung a watcher starts on. Even from a datacenter link, the
978 JPEG feeds froze 1–3 times in two minutes and their sockets closed after
979 36–80 s. The H.264 numbers, the 10-minute runs, and the wifi and 4G runs
980 are still owed. They need the Pi's renditions on 8082–8084 running and
981 `ARM_H264_1` set on the relay, then a person on those two networks.
982
983 | Network | Feed | Frame age p95 | Freezes in 10 min | Bytes per viewer | Target |
984 | --- | --- | --- | --- | --- | --- |
985 | Home wifi | h264 | not measured yet | not measured yet | not measured yet | p95 < 250 ms, 0 freezes |
986 | Home wifi | frames | not measured (no timestamp) | not measured yet | not measured yet | 0 freezes |
987 | 4G | h264 | not measured yet | not measured yet | not measured yet | p95 < 500 ms, 0 freezes |
988 | 4G | frames | not measured (no timestamp) | not measured yet | not measured yet | 0 freezes |
989
928990 ## Launch check
929991
930992 `so101/bench/launch-check.ts` checks a deployed page as an anonymous visitor and
Software/so101/bench/video-trial.test.ts added +155
@@ -0,0 +1,155 @@
1 import { afterEach, expect, test } from "bun:test";
2 import { mkdtempSync, rmSync } from "node:fs";
3 import { createServer, type Server } from "node:http";
4 import { tmpdir } from "node:os";
5 import { join } from "node:path";
6 import { CameraHub } from "../src/cameras.ts";
7 import { clip } from "../src/h264-clip.ts";
8 import { Lobby } from "../src/lobby.ts";
9 import { startRelay, type RunningRelay } from "../src/relay.ts";
10 import { ages, args, framesTrial, h264Trial, line, percentile, proxyFor, streamTrial, summarize } from "./video-trial.ts";
11
12 test("percentile is nearest-rank over unsorted values, null for none", () => {
13 expect(percentile([], 0.95)).toBeNull();
14 expect(percentile([5, 1, 3, 2, 4], 0.5)).toBe(3);
15 expect(percentile(Array.from({ length: 100 }, (_, i) => 100 - i), 0.95)).toBe(95);
16 });
17
18 test("a gap of a second or more is a freeze, the start and the end of the run included", () => {
19 const steady = summarize("x", [100, 200, 300].map((at) => ({ at })), 3000, 0, 400);
20 expect(steady).toMatchObject({ frames: 3, freezes: 0, longestGapMs: 100, bytes: 3000, bytesPerS: 7500, fps: 7.5, ageP95: null });
21 const late = summarize("x", [{ at: 1200 }, { at: 1300 }], 0, 0, 1400);
22 expect(late.freezes).toBe(1);
23 const stopped = summarize("x", [{ at: 100 }, { at: 200 }], 0, 0, 1500);
24 expect(stopped).toMatchObject({ freezes: 1, longestGapMs: 1300 });
25 expect(summarize("x", [], 0, 0, 3000).freezes).toBe(1);
26 });
27
28 test("frame ages are each delay past the least seen so far, as the page takes them", () => {
29 expect(ages([500, 520, 480, 700, 480])).toEqual([0, 20, 0, 220, 0]);
30 const s = summarize("h264", [10, 20, 30, 40].map((at, i) => ({ at, ageMs: [0, 100, 200, 300][i]! })), 0, 0, 50);
31 expect([s.ageP50, s.ageP95]).toEqual([100, 300]);
32 });
33
34 test("the line writes a missed target as missed, and an unmeasured age as unmeasured", () => {
35 const ok = summarize("h264", [{ at: 100, ageMs: 40 }, { at: 200, ageMs: 60 }], 2048, 0, 300);
36 expect(line(ok, "wifi")).toContain("target met");
37 const slow = summarize("h264", [{ at: 100, ageMs: 300 }, { at: 200, ageMs: 400 }], 2048, 0, 300);
38 expect(line(slow, "wifi")).toContain("target MISSED (p95 400 ms over 250 ms)");
39 expect(line(slow, "4g")).toContain("target met");
40 const frozen = summarize("mjpeg", [{ at: 1500 }], 2048, 0, 1600);
41 expect(line(frozen, "4g")).toContain("target MISSED (1 freezes)");
42 expect(line(summarize("mjpeg", [{ at: 100 }], 1, 0, 200), "wifi")).toContain("target: age not measured");
43 expect(line({ ...ok, error: "status 404" })).toBe("h264: no feed (status 404)");
44 });
45
46 let dir = "";
47 let relay: RunningRelay | undefined;
48 let lobby: Lobby | undefined;
49 const sources: Server[] = [];
50 afterEach(async () => {
51 await Promise.race([relay?.close(), new Promise((r) => setTimeout(r, 500))]);
52 lobby?.close();
53 for (const s of sources.splice(0)) s.closeAllConnections(), s.close();
54 if (dir) rmSync(dir, { recursive: true, force: true });
55 relay = lobby = undefined;
56 dir = "";
57 });
58
59 const listen = async (handler: Parameters<typeof createServer>[1]) => {
60 const s = createServer(handler);
61 sources.push(s);
62 await new Promise<void>((r) => s.listen(0, "127.0.0.1", r));
63 return (s.address() as { port: number }).port;
64 };
65
66 /** A relay whose camera 1 has an MJPEG source at 20 fps and a 720p H.264 rung at 20 fps; camera 2 has neither. */
67 async function boot() {
68 dir = mkdtempSync(join(tmpdir(), "so101-video-trial-"));
69 const jpeg = Buffer.concat([Buffer.from([0xff, 0xd8]), Buffer.alloc(1000, 7), Buffer.from([0xff, 0xd9])]);
70 const mjpeg = await listen((_req, res) => {
71 res.writeHead(200, { "content-type": "multipart/x-mixed-replace; boundary=f" });
72 const timer = setInterval(() => res.write(Buffer.concat([Buffer.from(`--f\r\nContent-Type: image/jpeg\r\nContent-Length: ${jpeg.length}\r\n\r\n`), jpeg, Buffer.from("\r\n")])), 50);
73 res.on("close", () => clearInterval(timer));
74 });
75 const pictures = clip(2, 20, 10, 500);
76 const pi = await listen((_req, res) => {
77 res.writeHead(200, { "content-type": "video/h264" });
78 let i = 0;
79 const timer = setInterval(() => res.write(pictures[i++ % pictures.length]!), 50);
80 res.on("close", () => clearInterval(timer));
81 });
82 lobby = new Lobby({ identityKey: "k".repeat(64), adminFile: join(dir, "admin.json") });
83 const cameras = new CameraHub({ seed: [{ name: "One", source: `http://127.0.0.1:${mjpeg}/stream` }, { name: "Two", source: "http://127.0.0.1:9/stream" }] });
84 relay = await startRelay({ port: 0, simulation: "http://127.0.0.1:1", lobby, cameras, video: { 1: { "720p": `http://127.0.0.1:${pi}/720p` } }, env: {} });
85 return `http://127.0.0.1:${relay.port}`;
86 }
87
88 test("against a relay, every feed is measured: pictures, bytes per viewer, no freezes, and an H.264 frame age", async () => {
89 const base = await boot();
90 for (const trial of [framesTrial, streamTrial]) {
91 const m = await trial(base, 1, 900);
92 expect(m.error).toBeUndefined();
93 expect(m.frames).toBeGreaterThan(5);
94 expect(m.bytes).toBeGreaterThan(m.frames * 1000);
95 expect(m.freezes).toBe(0);
96 }
97 const h = await h264Trial(base, 1, 900);
98 expect(h.error).toBeUndefined();
99 expect(h.feed).toBe("h264 (720p)");
100 expect(h.frames).toBeGreaterThan(3);
101 expect(h.bytes).toBeGreaterThan(0);
102 expect(h.ageP95).not.toBeNull();
103 }, 15_000);
104
105 test("a camera without H.264 is reported as no feed, not as numbers", async () => {
106 const base = await boot();
107 const h = await h264Trial(base, 2, 900);
108 expect(h.error).toBeDefined();
109 expect(h.frames).toBe(0);
110 }, 10_000);
111
112 test("the arguments take a URL, known feeds and a named network, and refuse anything else", () => {
113 expect(args(["https://arm.example/x?y"])).toMatchObject({ base: "https://arm.example", camera: 1, seconds: 600, feeds: ["frames", "h264"] });
114 expect(args(["http://a", "--feed", "h264,stream", "--net", "4g", "--seconds", "60", "--camera", "2"])).toMatchObject({ feeds: ["h264", "stream"], net: "4g", seconds: 60, camera: 2 });
115 for (const bad of [[], ["ftp://a"], ["http://a", "--feed", "toString"], ["http://a", "--net", "5g"], ["http://a", "--seconds", "0"], ["http://a", "--camera", "-1"]]) expect(() => args(bad)).toThrow();
116 });
117
118 test("a WebSocket goes through the proxy the environment names, except to a loopback or NO_PROXY host", () => {
119 const env = { HTTPS_PROXY: "http://p:3128", http_proxy: "http://q:3128", NO_PROXY: "lan.example,.corp" };
120 expect(proxyFor(new URL("wss://arm.example/x"), env)).toBe("http://p:3128");
121 expect(proxyFor(new URL("ws://arm.example/x"), env)).toBe("http://q:3128");
122 for (const u of ["ws://127.0.0.1:9/x", "ws://localhost/x", "ws://[::1]/x", "wss://lan.example/x", "wss://a.corp/x"]) expect(proxyFor(new URL(u), env)).toBeUndefined();
123 expect(proxyFor(new URL("wss://arm.example/x"), {})).toBeUndefined();
124 });
125
126 test("a socket the far end closes is opened again as the page does, and the reconnect is counted", async () => {
127 let opens = 0;
128 const server = Bun.serve({
129 hostname: "127.0.0.1",
130 port: 0,
131 fetch: (req, srv) => (srv.upgrade(req) ? undefined : new Response("no", { status: 400 })),
132 websocket: {
133 open(ws) {
134 const first = ++opens === 1;
135 let n = 0;
136 const timer = setInterval(() => {
137 if (ws.readyState !== 1) return clearInterval(timer);
138 ws.send(new Uint8Array(100));
139 if (first && ++n === 5) clearInterval(timer), ws.close();
140 }, 50);
141 },
142 message() {},
143 },
144 });
145 try {
146 const s = await framesTrial(`http://127.0.0.1:${server.port}`, 1, 1500);
147 expect(s.error).toBeUndefined();
148 expect(opens).toBe(2);
149 expect(s.reconnects).toBe(1);
150 expect(s.frames).toBeGreaterThan(5);
151 expect(s.bytes).toBe(s.frames * 100);
152 } finally {
153 server.stop(true);
154 }
155 });
Software/so101/bench/video-trial.ts added +280
@@ -0,0 +1,280 @@
1 /**
2 * Watches one camera of a deployed arm the way a viewer does and prints what it measured, one line per feed:
3 * bun run bench/video-trial.ts <url> [--camera 1] [--seconds 600] [--feed frames,h264,stream] [--net wifi|4g]
4 * Run it from the network being measured (a laptop on the home wifi, the same laptop tethered to a phone on 4G).
5 *
6 * "frames" is WS /camera/<id>/ws, the JPEGs the page draws behind the relay (src/ui/video.js); "h264" is the
7 * /camera/<id>/video WebSocket it decodes when the camera has H.264 (src/ui/video-player.js); "stream" is the multipart
8 * /camera/<id>/stream a plain img reads. The default is frames then h264, `--seconds` each.
9 *
10 * For each it counts the bytes that reached this machine, the pictures, and the freezes: gaps of FREEZE_MS or more
11 * without a new picture, the start and end of the run included. For H.264 it also takes the frame age as the page
12 * does, the delay of each picture past the least delay seen (the relay stamps the time it got the picture; the clocks
13 * differ, so the queueing is what can be measured, not the distance). The JPEG feeds carry no timestamp, so their age
14 * is not measured here. Nothing is decoded: "shown" is "arrived" plus decode time.
15 * A socket that closes or goes quiet is opened again as the page does, and counted. It sends nothing but the requests
16 * a viewer makes, and closes them at the end.
17 */
18 import { boundaryOf, MjpegParser } from "../src/mjpeg.ts";
19 import { parseFrame } from "../src/ui/video-player.js";
20 import { QUIET_MS, RETRY_MS } from "../src/ui/video.js";
21
22 export const FREEZE_MS = 1000;
23 export const TARGETS = { wifi: 250, "4g": 500 } as const;
24 export type Net = keyof typeof TARGETS;
25
26 export interface Picture {
27 /** When it arrived, ms. */
28 at: number;
29 /** Its frame age, ms, when the feed carries one. */
30 ageMs?: number;
31 }
32
33 export interface Summary {
34 feed: string;
35 seconds: number;
36 frames: number;
37 fps: number;
38 bytes: number;
39 /** Bytes per second to this one viewer, averaged over the run. */
40 bytesPerS: number;
41 freezes: number;
42 longestGapMs: number;
43 /** Sockets opened again during the run. */
44 reconnects: number;
45 ageP50: number | null;
46 ageP95: number | null;
47 /** Why the feed gave nothing, when it did not open. */
48 error?: string;
49 }
50
51 /** Nearest-rank percentile of unsorted numbers; null for none. */
52 export function percentile(values: number[], p: number): number | null {
53 if (!values.length) return null;
54 const sorted = [...values].sort((a, b) => a - b);
55 return sorted[Math.max(0, Math.ceil(p * sorted.length) - 1)]!;
56 }
57
58 /** The numbers of one run from `start` to `end` (ms) over the pictures that arrived in it, in order. */
59 export function summarize(feed: string, pictures: Picture[], bytes: number, start: number, end: number, reconnects = 0, freezeMs = FREEZE_MS): Summary {
60 const seconds = Math.max(0, end - start) / 1000;
61 let freezes = 0;
62 let longest = 0;
63 let last = start;
64 for (const t of [...pictures.map((p) => p.at), end]) {
65 const gap = t - last;
66 if (gap >= freezeMs) freezes++;
67 longest = Math.max(longest, gap);
68 last = t;
69 }
70 const ages = pictures.flatMap((p) => (p.ageMs === undefined ? [] : [p.ageMs]));
71 const round = (n: number | null) => (n === null ? null : Math.round(n));
72 return {
73 feed,
74 seconds: +seconds.toFixed(1),
75 frames: pictures.length,
76 fps: seconds ? +(pictures.length / seconds).toFixed(1) : 0,
77 bytes,
78 bytesPerS: seconds ? Math.round(bytes / seconds) : 0,
79 freezes,
80 longestGapMs: Math.round(longest),
81 reconnects,
82 ageP50: round(percentile(ages, 0.5)),
83 ageP95: round(percentile(ages, 0.95)),
84 };
85 }
86
87 /** Frame ages of H.264 pictures as the page takes them: each delay past the least seen so far. */
88 export function ages(delays: number[]): number[] {
89 let lowest = Infinity;
90 return delays.map((d) => {
91 lowest = Math.min(lowest, d);
92 return Math.max(0, d - lowest);
93 });
94 }
95
96 /** One line for a person, with the verdict against the target when the network is named. */
97 export function line(s: Summary, net?: Net): string {
98 if (s.error) return `${s.feed}: no feed (${s.error})`;
99 const kb = (n: number) => `${(n / 1024).toFixed(1)} KiB/s`;
100 const age = s.ageP95 === null ? "frame age not measured" : `frame age p50 ${s.ageP50} ms p95 ${s.ageP95} ms`;
101 let verdict = "";
102 if (net) {
103 const missed: string[] = [];
104 if (s.ageP95 !== null && s.ageP95 >= TARGETS[net]) missed.push(`p95 ${s.ageP95} ms over ${TARGETS[net]} ms`);
105 if (s.freezes > 0) missed.push(`${s.freezes} freezes`);
106 verdict = s.ageP95 === null && !missed.length ? " · target: age not measured" : missed.length ? ` · target MISSED (${missed.join(", ")})` : " · target met";
107 }
108 return `${s.feed}: ${s.seconds} s · ${s.frames} pictures (${s.fps} fps) · ${kb(s.bytesPerS)} per viewer · ${age} · ${s.freezes} freezes, longest gap ${s.longestGapMs} ms · ${s.reconnects} reconnects${verdict}`;
109 }
110
111 const empty = (feed: string, error: string): Summary => ({ ...summarize(feed, [], 0, 0, 0), error });
112
113 /** Read the multipart /stream (the plain img) for `ms`, counting a picture when its part ends. */
114 export async function streamTrial(base: string, camera: number, ms: number, now: () => number = performance.now.bind(performance)): Promise<Summary> {
115 const stop = new AbortController();
116 const pictures: Picture[] = [];
117 let bytes = 0;
118 const start = now();
119 const timer = setTimeout(() => stop.abort(), ms);
120 try {
121 const res = await fetch(`${base}/camera/${camera}/stream`, { signal: stop.signal });
122 const boundary = boundaryOf(res.headers.get("content-type"));
123 if (res.status !== 200 || !boundary || !res.body) return empty("stream", `status ${res.status}`);
124 const parser = new MjpegParser(boundary, { start() {}, data() {}, end: () => void pictures.push({ at: now() }) });
125 for await (const chunk of res.body) {
126 bytes += chunk.byteLength;
127 parser.push(chunk);
128 }
129 } catch (err) {
130 if (!stop.signal.aborted) return empty("stream", (err as Error).message);
131 } finally {
132 clearTimeout(timer);
133 }
134 return summarize("stream", pictures, bytes, start, now());
135 }
136
137 interface Watched {
138 start: number;
139 end: number;
140 bytes: number;
141 /** Sockets opened again after the first: closed by the far end, or replaced after QUIET_MS without a picture. */
142 reconnects: number;
143 error?: string;
144 }
145
146 /** The proxy the environment names for a ws:/wss: URL, or undefined when there is none or NO_PROXY lists its host. */
147 export function proxyFor(url: URL, env: Record<string, string | undefined>): string | undefined {
148 const proxy = url.protocol === "wss:" ? (env.HTTPS_PROXY ?? env.https_proxy) : (env.HTTP_PROXY ?? env.http_proxy);
149 if (!proxy) return undefined;
150 const host = url.hostname.replace(/^\[|\]$/g, "").toLowerCase();
151 const skip = (env.NO_PROXY ?? env.no_proxy ?? "").split(",").map((s) => s.trim().toLowerCase().replace(/^\*?\./, "")).filter(Boolean);
152 if (["localhost", "127.0.0.1", "::1"].includes(host) || skip.some((s) => s === "*" || host === s || host.endsWith(`.${s}`))) return undefined;
153 return proxy;
154 }
155
156 /**
157 * Keep the same-origin WebSocket at `path` open for `ms` as the page does (src/ui/video.js): a socket that closes is
158 * opened again after RETRY_MS, one with no binary message for QUIET_MS is replaced at once. Each message goes to `on`.
159 * Resolves at the end, or with `error` when the first socket never opened.
160 */
161 function watch(base: string, path: string, ms: number, now: () => number, on: (data: string | ArrayBuffer) => void): Promise<Watched> {
162 return new Promise((resolve) => {
163 const url = new URL(path, base);
164 url.protocol = url.protocol === "https:" ? "wss:" : "ws:";
165 // fetch follows HTTPS_PROXY by itself; Bun's WebSocket only when it is handed the proxy.
166 const proxy = proxyFor(url, process.env);
167 const start = now();
168 let bytes = 0;
169 let opened = false;
170 let reconnects = 0;
171 let ws: WebSocket | null = null;
172 let quiet: ReturnType<typeof setTimeout> | undefined;
173 let retry: ReturnType<typeof setTimeout> | undefined;
174 const drop = () => {
175 clearTimeout(quiet);
176 const w = ws;
177 ws = null;
178 if (!w) return;
179 w.onopen = w.onclose = w.onerror = w.onmessage = null;
180 try { w.close(); } catch {} // already closing or closed: nothing left to release
181 };
182 const finish = (error?: string) => {
183 clearTimeout(timer);
184 clearTimeout(retry);
185 drop();
186 resolve({ start, end: now(), bytes, reconnects, ...(error ? { error } : {}) });
187 };
188 const again = (wait: number) => {
189 drop();
190 if (!opened) return finish("did not open");
191 reconnects++;
192 retry = setTimeout(connect, wait);
193 };
194 function connect() {
195 const w = (ws = new WebSocket(url, { headers: { origin: new URL(base).origin }, ...(proxy ? { proxy } : {}) } as never));
196 w.binaryType = "arraybuffer";
197 w.onopen = () => ((opened = true), (quiet = setTimeout(() => again(0), QUIET_MS)));
198 w.onerror = w.onclose = () => again(RETRY_MS);
199 w.onmessage = (e) => {
200 if (typeof e.data !== "string") {
201 bytes += (e.data as ArrayBuffer).byteLength;
202 clearTimeout(quiet);
203 quiet = setTimeout(() => again(0), QUIET_MS);
204 }
205 on(e.data as string | ArrayBuffer);
206 };
207 }
208 const timer = setTimeout(() => finish(), ms);
209 connect();
210 });
211 }
212
213 /** Watch WS /camera/<id>/ws, the JPEG frames the page draws behind the relay, for `ms`: each binary message is one picture. */
214 export async function framesTrial(base: string, camera: number, ms: number, now: () => number = performance.now.bind(performance)): Promise<Summary> {
215 const pictures: Picture[] = [];
216 const w = await watch(base, `/camera/${camera}/ws`, ms, now, (data) => typeof data !== "string" && pictures.push({ at: now() }));
217 return w.error ? empty("frames", w.error) : summarize("frames", pictures, w.bytes, w.start, w.end, w.reconnects);
218 }
219
220 /**
221 * Watch the H.264 WebSocket for `ms`, as a watcher (the ladder's middle rung, stepping as the link allows). Where the
222 * page would fall back to the frames, this opens the video again, so the run stays on H.264 and the fall counts.
223 */
224 export async function h264Trial(base: string, camera: number, ms: number, now: () => number = performance.now.bind(performance)): Promise<Summary> {
225 const delays: number[] = [];
226 const at: number[] = [];
227 const renditions = new Set<string>();
228 const w = await watch(base, `/camera/${camera}/video`, ms, now, (data) => {
229 if (typeof data === "string") {
230 try {
231 const m = JSON.parse(data) as { rendition?: unknown };
232 if (typeof m.rendition === "string") renditions.add(m.rendition);
233 } catch {} // a text message that is not JSON carries no rendition; the pictures are what is measured
234 return;
235 }
236 const f = parseFrame(data);
237 if (!f) return;
238 // Wall clocks for the delay, as the page does; the arrival times on the monotonic clock for the gaps.
239 delays.push(Date.now() - f.sentAt);
240 at.push(now());
241 });
242 if (w.error) return empty("h264", `${w.error} (refused when the camera has no H.264)`);
243 const a = ages(delays);
244 const s = summarize("h264", at.map((t, i) => ({ at: t, ageMs: a[i]! })), w.bytes, w.start, w.end, w.reconnects);
245 return renditions.size ? { ...s, feed: `h264 (${[...renditions].join(", ")})` } : s;
246 }
247
248 const TRIALS = { frames: framesTrial, h264: h264Trial, stream: streamTrial };
249 type Feed = keyof typeof TRIALS;
250
251 export function args(argv: string[]) {
252 const o = { base: "", camera: 1, seconds: 600, feeds: ["frames", "h264"] as Feed[], net: undefined as Net | undefined };
253 const rest = [...argv];
254 const url = new URL(rest.shift() ?? "");
255 if (url.protocol !== "http:" && url.protocol !== "https:") throw new Error("not http");
256 o.base = url.origin;
257 while (rest.length) {
258 const k = rest.shift();
259 const v = rest.shift() ?? "";
260 if (k === "--camera" && /^\d{1,9}$/.test(v)) o.camera = Number(v);
261 else if (k === "--seconds" && /^\d{1,5}$/.test(v) && Number(v) > 0) o.seconds = Number(v);
262 else if (k === "--feed" && v.split(",").every((x) => Object.hasOwn(TRIALS, x))) o.feeds = v.split(",") as Feed[];
263 else if (k === "--net" && (v === "wifi" || v === "4g")) o.net = v;
264 else throw new Error(`bad argument ${k}`);
265 }
266 return o;
267 }
268
269 if (import.meta.main) {
270 let o: ReturnType<typeof args>;
271 try {
272 o = args(process.argv.slice(2));
273 } catch {
274 console.error("usage: bun run bench/video-trial.ts <http(s)://url> [--camera 1] [--seconds 600] [--feed frames,h264,stream] [--net wifi|4g]");
275 process.exit(2);
276 }
277 const ms = o.seconds * 1000;
278 // One after the other, so the feeds do not share the link.
279 for (const feed of o.feeds) console.log(line(await TRIALS[feed](o.base, o.camera, ms), o.net));
280 }

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs