Public: anyone can read this; the crew can change it.

Migrated from github.com/Trevato/SO-ARM100

workbenchthis machine

TOP: smooth driving and a camera that never freezes, through the public URL - one WebSocket, measured end to end #80

Closed opened this in answer to
Closed without merging

quill/arm:quill/54-top-smooth-driving-and-a-camera-that-nev → main 17 commits fork

  • Closed without merging. Nothing was applied to main.

Review

Sign in to review this change.
30 files changed +2607 −101
Software/README.md +80
@@ -229,6 +229,80 @@ and shared by everyone watching, shows a grey offline frame while its source
229229 is down (the other cameras keep playing) and reconnects by itself. Viewers
230230 get a tab per camera and can turn several on for a grid; that choice is
231231 their own browser's. A camera added or removed shows up within a few seconds.
232
233 **One socket per viewer: `WS /arm`.** Through the forge's front door every
234 request pays a round trip of about 0.4 s, so behind the app the page doesn't
235 poll. It opens one WebSocket, `/arm` (`src/viewer-socket.ts` on the app,
236 `src/ui/arm-socket.js` on the page), and everything that has to feel live
237 goes over it. The module isn't part of the first load: it comes once the
238 first view is up, and until then the page uses the old paths below. Over
239 the socket go:
240
241 - the arm's pose, up to 30 times a second. The app reads the arm server once
242 for everyone (`src/pose-feed.ts`), and the page paints at most once a frame
243 (none at all for cameras while the tab is hidden);
244 - the driver's joint targets (`{"t":"cmd","seq","targets":{"1":2048}}`).
245 Each joint keeps only its latest target, and the targets go to the arm's
246 `/follow` at most 30 times a second, so a burst never queues. Every
247 command is checked against the turn as it arrives, as on `/follow`.
248 `{"t":"end"}` lets go, and the arm holds;
249 - the camera frames as binary messages: a 20-byte header (camera id, frame
250 number, the app's receipt time), then the JPEG. The page acknowledges
251 each frame it has shown (`{"t":"seen","n"}`), and at most 4 are
252 unacknowledged at once. A viewer still busy with earlier frames gets the
253 newest frame next, never a backlog. A slow phone sees fewer frames, and
254 nobody else is slowed;
255 - every 5 s, what the page measured (`{"t":"report",…}`): command → pose
256 round trips, frame ages when shown, frames shown, stalls and reconnects.
257 The app takes at most one report per socket every 4 s; the rest are
258 dropped and counted (`arm_reports_dropped_total`). Each report is held to
259 what crossed that socket: over the socket's life, no more commands or
260 round trips than commands it received, and no more frames or frame ages
261 than frames it was sent. What one report doesn't use carries over to the
262 next, because a round trip often lands a report after its command. A page
263 can't fill the numbers with work that never happened.
264
265 The page reconnects by itself (after 0.25, 0.5, 1, 2, then every 4 s), and
266 it also reconnects when the socket goes quiet for 2 s. A drive doesn't
267 survive a reconnect: the arm holds until the next press. While the socket is
268 down the page uses the old paths: `/status` polled 5 times a second,
269 `/follow` for a drive, and the MJPEG `<img>` for the cameras. When the
270 socket is back the page moves over to it again. The top bar's camera
271 readout then shows the page's own medians, e.g.
272 `live · 24 fps · frame 90 ms · command 140 ms`. The leader arm still
273 streams on its own `/follow`. With no lobby (a run on your own machine)
274 every viewer may drive on `/arm`, as on `/follow`.
275
276 **How the link feels: `/metrics` and `/health`.** The app combines what every
277 page reports with what it measures itself: the arm server's round trip,
278 camera frames in, frames skipped for a slow viewer, bytes out, and commands
279 merged away.
280
281 - `GET /metrics` is Prometheus text, open to anyone, like a scrape target:
282 current numbers and totals only, no names. Scrape it with
283 `curl https://<app>/metrics` or a Prometheus `scrape_configs` job with
284 `metrics_path: /metrics`. With `accept: application/json` the same path
285 answers the arm server's own metrics (jams, stalls, peak loads), which the
286 page's Metrics panel shows.
287 It is open on purpose, so a scraper needs no account. What it shows is
288 already visible to anyone who opens the page: how many people are watching
289 and how quick the arm feels. It has no names, addresses, history or
290 settings, and it changes nothing. The one thing a stranger could do is skew
291 the viewer-reported numbers, and the per-socket limit above bounds that to
292 their own share of real traffic.
293 - `GET /health` is the same as JSON (`commandRttMs`, `frameAgeMs`,
294 `armRttMs` as `{p50, p95}`, `viewerFps`, `cameraFpsIn`, `bytesOutPerS`,
295 `commandsPerS`, `totals`) plus two minutes of history, one point every 2 s.
296 Only the admin may read it (401 or 403 for anyone else). Run on your own
297 machine, every visitor is the admin. On the page, **Admin → Health** draws
298 it (`src/ui/health.js`, fetched the first time it is opened): one
299 sparkline per number over those two minutes (`src/ui/sparkline.js`),
300 a dashed line at the targets (command→pose p95 250 ms, frame age p95
301 400 ms), the current value turning amber above them, and the totals since
302 the app started. It refreshes every 2 s while it is open.
303 - The app logs the same numbers as one line every minute:
304 `link: 2 viewers (1 watching) · command→pose p50/p95 140/310ms · frame age 90/180ms · …`.
305
232306 **Egress in `forge.json`.** `forge.json` lists `100.89.95.68:8101` (the arm)
233307 and `100.89.95.68:8080` (the camera) under `egress`: the destinations the app
234308 may reach on the Pi. The declaration does nothing until a site admin grants
@@ -421,6 +495,8 @@ watches. A line you type is said in the chat; commands start with `/`:
421495 - `/end [name]`, owner: end that turn (no name: whoever drives), or take
422496 them out of the line.
423497 - `/mute <name>`, `/unmute <name>`, owner.
498 - `/health`, owner: the link's numbers from `GET /health` on one line
499 (command → pose round trips, frame ages, fps, stalls, reconnects).
424500 - `/machine`: the Machine panel, the same rows the page's Machine tab shows
425501 (one line per joint, then the link and the server), followed by every
426502 command in and out and every clamp, line by line as it happens. `/machine`
@@ -428,6 +504,10 @@ watches. A line you type is said in the chat; commands start with `/`:
428504 guests too: it is read-only.
429505 - `/who` (who drives, who waits), `/help`, `/quit`.
430506
507 The terminal doesn't drive. Driving means joint targets many times a
508 second from a slider, the 3D model or a leader arm, and a typed line can't
509 hold a joint. A Stop from the page (or the owner's `/end`) stops the arm.
510
431511 The terminal is a client of the running server, like the page: it opens
432512 the same `/live` socket on `127.0.0.1:$PORT`, as the forge's identity for
433513 the person at the keyboard (`FORGE_APP_USER*`, signed with
Software/so101/bench/fake-serve.ts +1 −1
@@ -36,7 +36,7 @@ const dir = appData ? join(appData, "so101") : mkdtempSync(join(tmpdir(), "so101
3636 mkdirSync(dir, { recursive: true });
3737 const port = Number(process.env.PORT ?? process.argv[2] ?? 8102);
3838 const host = process.env.HOST ?? "127.0.0.1";
39 const simulation = await startServer(new FeetechBus(fake), { host: "127.0.0.1", port: 0, posesFile: join(dir, "poses.json"), modelDir: new URL("../../../Simulation/SO101", import.meta.url).pathname });
39 const simulation = await startServer(new FeetechBus(fake), { host: "127.0.0.1", port: 0, posesFile: join(dir, "poses.json"), modelDir: new URL("../../../Simulation/SO101", import.meta.url).pathname, warmModel: true });
4040 let remote: Remote | null = null;
4141 let problem: string | null = null;
4242 try {
Software/so101/bench/first-load.ts added +58
@@ -0,0 +1,58 @@
1 /**
2 * Weigh the page: what a first visit downloads (bytes on the wire, compressed as the browser would get them) and what
3 * a second visit still asks for, file by file, with each answer's cache-control.
4 * bun run bench/first-load.ts [base] base defaults to http://127.0.0.1:8102; the preview's URL works too
5 * "shell" is what the first paint waits for (the HTML and the modules it preloads); "3d" is three.js, the loader,
6 * the URDF and its meshes, which come after.
7 */
8 import { allImports, pageLinks, repeatCost, urdfFiles, type Repeat } from "../src/weigh.ts";
9
10 const base = (process.argv[2] ?? "http://127.0.0.1:8102").replace(/\/$/, "");
11 interface Row { part: "shell" | "3d"; path: string; bytes: number; ms: number; cache: string; repeat: Repeat }
12 const rows: Row[] = [];
13 const seen = new Set<string>();
14
15 async function get(path: string, part: Row["part"]): Promise<string> {
16 const started = performance.now();
17 const r = await fetch(base + path, { headers: { "accept-encoding": "br, gzip" }, decompress: false } as RequestInit);
18 const raw = new Uint8Array(await r.arrayBuffer());
19 const ms = performance.now() - started;
20 if (!r.ok) throw new Error(`${path}: ${r.status}`);
21 const cache = r.headers.get("cache-control") ?? "";
22 rows.push({ part, path, bytes: raw.length, ms, cache, repeat: repeatCost(cache, r.headers.get("etag")) });
23 const enc = r.headers.get("content-encoding");
24 const plain = enc === "gzip" ? Bun.gunzipSync(raw) : enc === "br" ? (await import("node:zlib")).brotliDecompressSync(raw) : raw;
25 return new TextDecoder().decode(plain);
26 }
27
28 /** A module and everything it imports, relative or absolute (bare names are the import map's, already listed). */
29 async function crawl(path: string, part: Row["part"]): Promise<void> {
30 if (seen.has(path)) return;
31 seen.add(path);
32 const js = await get(path, part);
33 for (const spec of allImports(js)) {
34 if (spec.startsWith("./") || spec.startsWith("../")) await crawl(new URL(spec, `http://x${path}`).pathname, part);
35 else if (spec.startsWith("/") && part === "3d") await crawl(spec, part);
36 }
37 }
38
39 const html = await get("/", "shell");
40 seen.add("/");
41 const { shell, mapped } = pageLinks(html);
42 for (const path of shell) await crawl(path, "shell");
43 for (const path of mapped) await crawl(path, "3d");
44 for (const m of html.matchAll(/import\("(\/[^"]+\.js)"\)/g)) await crawl(m[1]!, "3d");
45 const info = JSON.parse(await get("/model.json", "3d")) as { urdf?: string };
46 if (info.urdf) {
47 const urdf = await get(info.urdf, "3d");
48 for (const f of urdfFiles(urdf)) await get(new URL(f, `http://x${info.urdf}`).pathname, "3d");
49 }
50
51 const kb = (n: number) => `${(n / 1024).toFixed(1)} KB`;
52 for (const r of rows) console.log(`${r.part.padEnd(5)} ${kb(r.bytes).padStart(10)} ${`${r.ms.toFixed(0)} ms`.padStart(7)} ${r.repeat.padEnd(10)} ${r.cache.padEnd(36)} ${r.path}`);
53 for (const part of ["shell", "3d"] as const) {
54 const mine = rows.filter((r) => r.part === part);
55 const again = mine.filter((r) => r.repeat !== "free");
56 console.log(`${part}: first visit ${kb(mine.reduce((s, r) => s + r.bytes, 0))} in ${mine.length} requests; ` +
57 `a second visit asks ${again.length} again (${kb(again.filter((r) => r.repeat === "refetch").reduce((s, r) => s + r.bytes, 0))} refetched)`);
58 }
Software/so101/src/camera-feed.ts +55 −5
@@ -30,7 +30,18 @@ export interface FeedOptions {
3030
3131 export type FeedState = "idle" | "connecting" | "live" | "offline";
3232
33 /** One whole picture: what a WebSocket viewer gets (src/viewer-socket.ts), with the moment the app had all of it. */
34 export interface Picture {
35 type: string;
36 bytes: Buffer;
37 /** Date.now() when its last byte came in from the camera. */
38 at: number;
39 }
40 export type PictureListener = (picture: Picture) => void;
41
3342 const BOUNDARY = "armframe";
43 /** A picture larger than this is not gathered whole (a 1080p JPEG is well under 1 MB). */
44 const MAX_PICTURE = 4 << 20;
3445 const NEXT = Buffer.from(`\r\n--${BOUNDARY}\r\n`);
3546 const head = (type: string) => Buffer.from(`Content-Type: ${type}\r\n\r\n`);
3647
@@ -56,6 +67,11 @@ export function parseFps(text: string | null): number | null {
5667
5768 export class CameraFeed {
5869 private readonly viewers = new Set<Viewer>();
70 private readonly listeners = new Set<PictureListener>();
71 /** The part being gathered whole, only while someone wants whole pictures. */
72 private gathering: { type: string; chunks: Buffer[]; size: number } | null = null;
73 /** Whole pictures the source sent, since the start. */
74 framesIn = 0;
5975 private _state: FeedState;
6076 private attempt = 0;
6177 private controller: AbortController | null = null;
@@ -81,9 +97,26 @@ export class CameraFeed {
8197 return this._state;
8298 }
8399
84 /** How many viewers are watching right now. */
100 /** How many viewers are watching right now, by either way. */
85101 get watching(): number {
86 return this.viewers.size;
102 return this.viewers.size + this.listeners.size;
103 }
104
105 /**
106 * Whole pictures, one call each, for as long as the returned function is not called: the offline picture at once
107 * when the source is down, then every live frame. Counts as a viewer, so the upstream stays open meanwhile.
108 */
109 pictures(listener: PictureListener): () => void {
110 if (this.closed) return () => {};
111 this.listeners.add(listener);
112 clearTimeout(this.linger);
113 if (this._state === "offline") {
114 listener({ type: "image/png", bytes: OFFLINE_FRAME, at: Date.now() });
115 this.offline();
116 } else if (this._state === "idle") this.connect();
117 return () => {
118 if (this.listeners.delete(listener)) this.left();
119 };
87120 }
88121
89122 /**
@@ -115,14 +148,19 @@ export class CameraFeed {
115148 this.stop();
116149 for (const v of this.viewers) v.res.end();
117150 this.viewers.clear();
151 this.listeners.clear();
118152 }
119153
120154 // ------------------------------------------------------------------ viewers
121155 private leave(viewer: Viewer) {
122 if (!this.viewers.delete(viewer) || this.viewers.size) return;
156 if (this.viewers.delete(viewer)) this.left();
157 }
158
159 private left() {
160 if (this.watching) return;
123161 clearTimeout(this.linger);
124162 this.linger = setTimeout(() => {
125 if (!this.viewers.size) this.stop();
163 if (!this.watching) this.stop();
126164 }, this.o.lingerMs);
127165 this.linger.unref?.();
128166 }
@@ -145,6 +183,11 @@ export class CameraFeed {
145183 v.inPart = false;
146184 this.whole(v, "image/png", OFFLINE_FRAME);
147185 }
186 this.tell({ type: "image/png", bytes: OFFLINE_FRAME, at: Date.now() });
187 }
188
189 private tell(picture: Picture) {
190 for (const l of this.listeners) l(picture);
148191 }
149192
150193 // ------------------------------------------------------------------ the source
@@ -167,7 +210,7 @@ export class CameraFeed {
167210 if (this.source && !this.retry && !this.closed) {
168211 this.retry = setTimeout(() => {
169212 this.retry = undefined;
170 if (this.viewers.size && this._state === "offline" && !this.controller) this.connect();
213 if (this.watching && this._state === "offline" && !this.controller) this.connect();
171214 }, this.o.retryMs);
172215 this.retry.unref?.();
173216 }
@@ -210,6 +253,7 @@ export class CameraFeed {
210253 }
211254 const parser = new MjpegParser(boundary, {
212255 start: (type) => {
256 this.gathering = this.listeners.size ? { type, chunks: [], size: 0 } : null;
213257 const now = performance.now();
214258 for (const v of this.viewers) {
215259 // A frame a little early still counts as on time: a 10 fps source held to 5 sends every other frame, not every third.
@@ -221,6 +265,8 @@ export class CameraFeed {
221265 },
222266 data: (bytes) => {
223267 for (const v of this.viewers) if (v.inPart) this.write(v, bytes);
268 const g = this.gathering;
269 if (g && (g.size += bytes.length) <= MAX_PICTURE) g.chunks.push(Buffer.from(bytes));
224270 },
225271 end: () => {
226272 if (this._state !== "live") {
@@ -230,6 +276,10 @@ export class CameraFeed {
230276 this.repeat = undefined;
231277 }
232278 for (const v of this.viewers) if (v.inPart) this.write(v, NEXT), (v.inPart = false);
279 this.framesIn++;
280 const g = this.gathering;
281 this.gathering = null;
282 if (g && g.size <= MAX_PICTURE) this.tell({ type: g.type, bytes: Buffer.concat(g.chunks), at: Date.now() });
233283 },
234284 });
235285 this.lastByte = Date.now();
Software/so101/src/cameras.ts +7
@@ -128,6 +128,13 @@ export class CameraHub {
128128 return this.specs.find((c) => c.id === id)?.source ?? null;
129129 }
130130
131 /** Whole pictures the cameras have sent the app, all feeds together, since the start. */
132 get framesIn(): number {
133 let n = 0;
134 for (const feed of this.feeds.values()) n += feed.framesIn;
135 return n;
136 }
137
131138 close(): void {
132139 for (const feed of this.feeds.values()) feed.close();
133140 this.feeds.clear();
Software/so101/src/fake-pi.ts added +47
@@ -0,0 +1,47 @@
1 /**
2 * A Pi stand-in for the tests: an arm server's /status (answered after `delayMs`), an MJPEG /stream with a 20 kB frame
3 * every 30 ms, and a /follow that keeps every frame it is sent and moves joint 1 to it.
4 */
5
6 import { createServer, type ServerResponse } from "node:http";
7 import { WebSocketServer } from "ws";
8
9 const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
10
11 export async function fakePi(delayMs = 0) {
12 const follow: string[] = [];
13 const cams = new Set<ServerResponse>();
14 let pan = 2048;
15 const http = createServer(async (req, res) => {
16 if (req.url?.startsWith("/status")) {
17 await sleep(delayMs);
18 res.writeHead(200, { "content-type": "application/json" });
19 return res.end(JSON.stringify({ joints: [{ id: 1, joint: "shoulder_pan", position: pan }], driving: null }));
20 }
21 if (req.url === "/stream") {
22 res.writeHead(200, { "content-type": "multipart/x-mixed-replace;boundary=b" });
23 cams.add(res);
24 res.socket?.once("close", () => cams.delete(res));
25 return;
26 }
27 res.writeHead(404).end();
28 });
29 const body = Buffer.concat([Buffer.from([0xff, 0xd8]), Buffer.alloc(20_000, 7), Buffer.from([0xff, 0xd9])]);
30 const camera = setInterval(() => {
31 for (const res of cams) res.write(Buffer.concat([Buffer.from(`--b\r\nContent-Type: image/jpeg\r\nContent-Length: ${body.length}\r\n\r\n`), body, Buffer.from("\r\n")]));
32 }, 30);
33 const wss = new WebSocketServer({ server: http, path: "/follow" });
34 wss.on("connection", (ws) =>
35 ws.on("message", (data) => {
36 follow.push(String(data));
37 const p = (JSON.parse(String(data)) as Record<string, number>)["1"];
38 if (p !== undefined) pan = p;
39 }),
40 );
41 const port = await new Promise<number>((r) => http.listen(0, "127.0.0.1", () => r((http.address() as { port: number }).port)));
42 return {
43 port,
44 follow,
45 close: () => (clearInterval(camera), wss.close(), http.closeAllConnections(), http.close()),
46 };
47 }
Software/so101/src/latest.ts added +60
@@ -0,0 +1,60 @@
1 /**
2 * Latest wins: at most `capacity` messages in flight to a viewer at a time; whatever comes meanwhile replaces what was waiting under
3 * the same key, so a slow viewer is handed fewer, fresher messages and never a queue of stale ones. Nothing here waits
4 * on one viewer for another's sake.
5 */
6
7 /** Sends `data`, calling `done` once it has left (or failed); returns false when it could not be sent at all. */
8 export type Send = (data: Buffer | string, done: () => void) => boolean;
9
10 export class Latest {
11 private busy = 0;
12 private readonly waiting = new Map<string, Buffer | string>();
13 /** Messages replaced before they could be sent. */
14 skipped = 0;
15 sent = 0;
16 bytes = 0;
17
18 /** `capacity` is how many may be in flight at once: one viewer's round trip times it is the frame rate it can get. */
19 private readonly send: Send;
20 private readonly capacity: number;
21
22 constructor(send: Send, capacity = 1) {
23 this.send = send;
24 this.capacity = capacity;
25 }
26
27 offer(key: string, data: Buffer | string): void {
28 if (this.waiting.has(key)) this.skipped++;
29 this.waiting.set(key, data);
30 if (this.busy < this.capacity) this.next();
31 }
32
33 /** Messages sent and not yet done. */
34 get inFlight(): number {
35 return this.busy;
36 }
37
38 private next(): void {
39 const first = this.waiting.entries().next();
40 if (first.done) return;
41 const [key, data] = first.value;
42 this.waiting.delete(key);
43 this.busy++;
44 let finished = false;
45 const done = () => {
46 if (finished) return;
47 finished = true;
48 this.busy--;
49 this.next();
50 };
51 if (!this.send(data, done)) {
52 finished = true;
53 this.busy--;
54 this.waiting.clear();
55 return;
56 }
57 this.sent++;
58 this.bytes += typeof data === "string" ? Buffer.byteLength(data) : data.length;
59 }
60 }
Software/so101/src/mesh-lite.test.ts added +61
@@ -0,0 +1,61 @@
1 import { describe, expect, test } from "bun:test";
2 import { readFileSync } from "node:fs";
3 import { gzipSync } from "node:zlib";
4 import { isBinaryStl, lighten, urdfFiles } from "./mesh-lite.ts";
5
6 /** A binary STL from triangles given as nine numbers each. */
7 function stl(triangles: number[][]): Uint8Array {
8 const out = new Uint8Array(84 + triangles.length * 50);
9 const v = new DataView(out.buffer);
10 v.setUint32(80, triangles.length, true);
11 triangles.forEach((t, i) => t.forEach((x, j) => v.setFloat32(84 + i * 50 + 12 + j * 4, x, true)));
12 return out;
13 }
14 const read = (bytes: Uint8Array) => {
15 const v = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
16 const n = v.getUint32(80, true);
17 return Array.from({ length: n }, (_, i) => ({
18 normal: [0, 1, 2].map((j) => Math.round(v.getFloat32(84 + i * 50 + j * 4, true) * 1000) / 1000 + 0),
19 corners: Array.from({ length: 9 }, (_, j) => Math.round(v.getFloat32(84 + i * 50 + 12 + j * 4, true) * 1e6) / 1e6 + 0),
20 }));
21 };
22
23 describe("lightened meshes", () => {
24 test("corners snap to the grid; a sliver that collapses is dropped, a repeat too, and the faces get their normals", () => {
25 const out = read(lighten(stl([
26 [0, 0, 0, 0.01, 0, 0, 0, 0.01, 0.00004], // stays, its last corner snaps to z = 0
27 [0, 0, 0, 0.00005, 0, 0, 0, 0.01, 0], // two corners within the grid: gone
28 [0.01, 0, 0, 0, 0.01, 0, 0, 0, 0], // the first face again, turned the same way round: gone
29 [0, 0, 0, 0, 0.01, 0, 0.01, 0, 0], // the same corners turned the other way: the back face, kept
30 ])));
31 expect(out).toEqual([
32 { normal: [0, 0, 1], corners: [0, 0, 0, 0.01, 0, 0, 0, 0.01, 0] },
33 { normal: [0, 0, -1], corners: [0, 0, 0, 0, 0.01, 0, 0.01, 0, 0] },
34 ]);
35 });
36
37 test("anything but a well-formed binary STL passes through untouched", () => {
38 const ascii = new TextEncoder().encode("solid x\nendsolid x\n".padEnd(120, " "));
39 expect(isBinaryStl(ascii)).toBe(false);
40 expect(lighten(ascii)).toBe(ascii);
41 });
42
43 test("the arm's heaviest mesh gets several times smaller on the wire and keeps its size", () => {
44 const original = new Uint8Array(readFileSync(new URL("../../../Simulation/SO101/assets/wrist_roll_pitch_so101_v2.stl", import.meta.url)));
45 const light = lighten(original);
46 expect(gzipSync(light).length * 4).toBeLessThan(gzipSync(original).length);
47 const extent = (b: Uint8Array) => {
48 const v = new DataView(b.buffer, b.byteOffset, b.byteLength), n = v.getUint32(80, true);
49 let lo = Infinity, hi = -Infinity;
50 for (let i = 0; i < n; i++) for (let k = 0; k < 3; k++) { const x = v.getFloat32(84 + i * 50 + 12 + k * 12, true); lo = Math.min(lo, x); hi = Math.max(hi, x); }
51 return [lo, hi];
52 };
53 const [a, b] = [extent(original), extent(light)];
54 expect(Math.abs(a[0]! - b[0]!)).toBeLessThanOrEqual(0.0001 + 1e-7);
55 expect(Math.abs(a[1]! - b[1]!)).toBeLessThanOrEqual(0.0001 + 1e-7);
56 });
57
58 test("a URDF's mesh files, once each, package:// prefixes taken off", () => {
59 expect(urdfFiles('<mesh filename="assets/a.stl"/><mesh filename="package://so101/assets/b.stl"/><mesh filename="assets/a.stl"/>')).toEqual(["assets/a.stl", "assets/b.stl"]);
60 });
61 });
Software/so101/src/mesh-lite.ts added +73
@@ -0,0 +1,73 @@
1 /**
2 * The arm's meshes, lightened for the browser: a binary STL with its vertices snapped to a grid (0.2 mm by default, in
3 * the URDF's metres), the triangles that collapse or repeat dropped, and each face's normal worked out again. The
4 * vendor's meshes are tessellated far finer than a phone can show; this is 6-8 times smaller on the wire and looks
5 * the same. Only what the page downloads changes: the server's own geometry (src/kinematics.ts) reads the originals.
6 */
7
8 const HEADER = 84;
9 const TRIANGLE = 50;
10 /** Bump when the output changes, so the model's cached URLs change with it. */
11 export const LIGHTEN_VERSION = 1;
12
13 /** The mesh files a URDF names, relative to it. */
14 export function urdfFiles(urdf: string): string[] {
15 return [...new Set([...urdf.matchAll(/filename="([^"]+)"/g)].map((m) => m[1]!.replace(/^package:\/\/[^/]+\//, "")))];
16 }
17
18 /** Is this a binary STL whose length matches its triangle count? (ASCII STL and anything else pass through.) */
19 export function isBinaryStl(bytes: Uint8Array): boolean {
20 if (bytes.length < HEADER) return false;
21 const n = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength).getUint32(80, true);
22 return bytes.length === HEADER + n * TRIANGLE;
23 }
24
25 export function lighten(bytes: Uint8Array, grid = 0.0002): Uint8Array {
26 if (!isBinaryStl(bytes) || !(grid > 0)) return bytes;
27 const src = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
28 const n = src.getUint32(80, true);
29 const ids = new Map<string, number>();
30 const points: number[] = [];
31 const vertex = (o: number) => {
32 const x = Math.round(src.getFloat32(o, true) / grid), y = Math.round(src.getFloat32(o + 4, true) / grid), z = Math.round(src.getFloat32(o + 8, true) / grid);
33 const key = `${x},${y},${z}`;
34 let id = ids.get(key);
35 if (id === undefined) {
36 id = points.length / 3;
37 ids.set(key, id);
38 points.push(x * grid, y * grid, z * grid);
39 }
40 return id;
41 };
42 const kept: number[] = [];
43 const seen = new Set<string>();
44 for (let i = 0; i < n; i++) {
45 const o = HEADER + i * TRIANGLE + 12;
46 const a = vertex(o), b = vertex(o + 12), c = vertex(o + 24);
47 if (a === b || b === c || a === c) continue;
48 // The same three corners in the same turn are the same face; the opposite turn is the other side, and stays.
49 const r = Math.min(a, b, c);
50 const key = r === a ? `${a},${b},${c}` : r === b ? `${b},${c},${a}` : `${c},${a},${b}`;
51 if (seen.has(key)) continue;
52 seen.add(key);
53 kept.push(a, b, c);
54 }
55 const m = kept.length / 3;
56 const out = new Uint8Array(HEADER + m * TRIANGLE);
57 out.set(bytes.subarray(0, 80));
58 const dst = new DataView(out.buffer);
59 dst.setUint32(80, m, true);
60 for (let i = 0; i < m; i++) {
61 const o = HEADER + i * TRIANGLE;
62 const [a, b, c] = [kept[3 * i]! * 3, kept[3 * i + 1]! * 3, kept[3 * i + 2]! * 3];
63 const ux = points[b]! - points[a]!, uy = points[b + 1]! - points[a + 1]!, uz = points[b + 2]! - points[a + 2]!;
64 const vx = points[c]! - points[a]!, vy = points[c + 1]! - points[a + 1]!, vz = points[c + 2]! - points[a + 2]!;
65 const nx = uy * vz - uz * vy, ny = uz * vx - ux * vz, nz = ux * vy - uy * vx;
66 const len = Math.hypot(nx, ny, nz) || 1;
67 dst.setFloat32(o, nx / len, true);
68 dst.setFloat32(o + 4, ny / len, true);
69 dst.setFloat32(o + 8, nz / len, true);
70 for (const [k, p] of [a, b, c].entries()) for (let j = 0; j < 3; j++) dst.setFloat32(o + 12 + k * 12 + j * 4, points[p + j]!, true);
71 }
72 return out;
73 }
Software/so101/src/pose-feed.ts added +87
@@ -0,0 +1,87 @@
1 /**
2 * The arm's pose for everyone, read once: one reader asks the arm server's /status as fast as it answers, up to `hz`
3 * times a second, and hands every answer to every listener. Nobody listening, nobody asks. A reader that fails waits
4 * a little and tries again; listeners keep the last pose they had meanwhile.
5 */
6
7 export interface Pose {
8 status: Record<string, unknown>;
9 /** Date.now() when the read that produced it was started, and when its answer came. */
10 askedAt: number;
11 at: number;
12 }
13 export type PoseListener = (pose: Pose) => void;
14
15 export interface PoseFeedOptions {
16 /** Reads the arm's status once; throws when the arm server does not answer. */
17 read: () => Promise<Record<string, unknown>>;
18 /** Most reads a second. */
19 hz?: number;
20 /** How long to wait after a failed read. */
21 retryMs?: number;
22 /** Told each round trip, ms, and each failure. */
23 onRead?: (ms: number) => void;
24 onError?: (err: unknown) => void;
25 }
26
27 export class PoseFeed {
28 private readonly listeners = new Set<PoseListener>();
29 private running = false;
30 private closed = false;
31 private wake: (() => void) | null = null;
32 last: Pose | null = null;
33
34 private readonly o: PoseFeedOptions;
35
36 constructor(options: PoseFeedOptions) {
37 this.o = options;
38 }
39
40 listen(listener: PoseListener): () => void {
41 this.listeners.add(listener);
42 if (this.last) listener(this.last);
43 if (!this.running && !this.closed) void this.loop();
44 return () => void this.listeners.delete(listener);
45 }
46
47 get listening(): number {
48 return this.listeners.size;
49 }
50
51 close(): void {
52 this.closed = true;
53 this.listeners.clear();
54 this.wake?.();
55 }
56
57 private sleep(ms: number) {
58 return new Promise<void>((resolve) => {
59 const timer = setTimeout(() => ((this.wake = null), resolve()), ms);
60 this.wake = () => (clearTimeout(timer), (this.wake = null), resolve());
61 });
62 }
63
64 private async loop() {
65 this.running = true;
66 const period = 1000 / (this.o.hz ?? 30);
67 try {
68 while (this.listeners.size && !this.closed) {
69 const askedAt = Date.now();
70 const started = performance.now();
71 try {
72 const status = await this.o.read();
73 const ms = performance.now() - started;
74 this.o.onRead?.(ms);
75 this.last = { status, askedAt, at: Date.now() };
76 for (const l of this.listeners) l(this.last);
77 await this.sleep(Math.max(0, period - ms));
78 } catch (err) {
79 this.o.onError?.(err);
80 await this.sleep(this.o.retryMs ?? 500);
81 }
82 }
83 } finally {
84 this.running = false;
85 }
86 }
87 }
Software/so101/src/relay.ts +107 −40
@@ -22,8 +22,11 @@
2222 * The owner's speed cap (src/speed-cap.ts) is applied here too, to moves, poses, grips, playbacks and every stream frame.
2323 *
2424 * WS /live the lobby: the turn, the queue, the chat, and the arm's /status pushed when it changes
25 * WS /arm the viewer's live link: the pose at 30 Hz, the driver's commands, camera frames (src/viewer-socket.ts)
2526 * WS /machine read-only, for anyone: the arm's raw status, the link's round trip and frame rate, and every
2627 * command in and out with every safety clamp, pushed as it happens (src/machine-feed.ts)
28 * GET /metrics how the link feels, as Prometheus text (src/telemetry.ts); with accept: application/json, the arm's own metrics
29 * GET /health the same numbers as JSON, with two minutes of history for the admin's sparklines; the admin's only
2730 * POST /join {"name": "…"}: become someone here, no forge account needed (src/accounts.ts); the answer
2831 * sets the HttpOnly session cookie and names you: {"you": {"id", "name", "owner"}}
2932 * GET /admin, POST /admin the owner's page: pause, end a turn, take control, the settings, the log
@@ -61,7 +64,10 @@ import { LeaderCalibrations } from "./leader-calibrations.ts";
6164 import { proxyFor, whyNot } from "./egress.ts";
6265 import { CalibrationError } from "./leader-map.js";
6366 import type { Lobby } from "./lobby.ts";
67 import { PoseFeed } from "./pose-feed.ts";
6468 import { capBody, FrameCap } from "./speed-cap.ts";
69 import { Telemetry } from "./telemetry.ts";
70 import { ViewerSockets, type ViewerSocketOptions } from "./viewer-socket.ts";
6571
6672 export interface Remote {
6773 /** The arm server's HTTP base, no trailing slash. */
@@ -111,10 +117,17 @@ export interface RelayOptions {
111117 statusEveryMs?: number;
112118 /** Where HTTP_PROXY and NO_PROXY are read from; process.env by default. */
113119 env?: Record<string, string | undefined>;
120 /** Most pose reads a second from the arm server, shared by every viewer. */
121 poseHz?: number;
122 /** Timings of the viewer sockets on /arm. */
123 viewer?: Pick<ViewerSocketOptions, "commandHz" | "endAfterMs" | "heartbeatMs" | "frameWindow" | "seenTimeoutMs" | "reportEveryMs">;
124 /** How often the one-line link summary is logged; 0 never. */
125 logEveryMs?: number;
114126 }
115127
116128 export interface RunningRelay {
117129 readonly port: number;
130 readonly telemetry: Telemetry;
118131 link(): Link;
119132 close(): Promise<void>;
120133 }
@@ -599,6 +612,16 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
599612 by = who!.name;
600613 }
601614 if (req.method === "GET" && url.pathname === "/link") return json(res, 200, link());
615 if (req.method === "GET" && url.pathname === "/health") {
616 // The history behind the admin's sparklines is the admin's; /metrics stays open for a scraper, current numbers only.
617 const who = lobby?.who(req);
618 if (lobby && !who?.owner) return json(res, who ? 403 : 401, { error: "only an admin can see the link's health" });
619 return json(res, 200, telemetry.json());
620 }
621 if (req.method === "GET" && url.pathname === "/metrics" && !String(req.headers.accept ?? "").includes("application/json")) {
622 res.writeHead(200, { "content-type": "text/plain; version=0.0.4; charset=utf-8", "cache-control": "no-store" });
623 return res.end(telemetry.prometheus());
624 }
602625 if (req.method === "GET" && url.pathname === "/status") return await status(res, path);
603626
604627 const camera = CAMERA.exec(url.pathname);
@@ -672,24 +695,15 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
672695 });
673696 /** Close codes a server may send on; the rest (1005, 1006, 1015) only describe a connection that died. */
674697 const sendable = (code: number) => (code >= 1000 && code <= 1003) || (code >= 1007 && code <= 1014) || (code >= 3000 && code <= 4999);
675 /** Each open /follow stream and whose it is, so a turn that ends can close it. */
676 const streams = new Map<WebSocket, string>();
677698 /**
678 * `base` is the arm server's HTTP side, asked where the joints are so the speed cap starts from there. On the real arm,
679 * frames are not passed on while the link reconnects: the stream goes quiet, and the arm server's dead-man holds the arm.
699 * The owner's speed cap for one stream to the arm at `base`: each frame as it may go on, or null for none.
700 * It asks where the joints are once at the start, and again (at most once a second) while a cap holds a joint back for want of it.
680701 */
681 const bridge = (browser: WebSocket, arm: WebSocket, base: string, driver: string | null) => {
682 const real = base !== simulation;
683 if (driver !== null) {
684 streams.set(browser, driver);
685 browser.on("close", () => streams.delete(browser));
686 }
687 arm.on("message", (data, binary) => browser.readyState === WebSocket.OPEN && browser.send(data, { binary }));
702 const capper = (base: string) => {
688703 const cap = new FrameCap();
689704 let seeded = false;
690705 let asking = false;
691706 let askedAt = 0;
692 /** Where the arm is, once at the start and again (at most once a second) while a cap holds a joint back for want of it. */
693707 const seed = () => {
694708 if (!lobby || seeded || asking || Date.now() - askedAt < 1000) return;
695709 asking = true;
@@ -705,30 +719,78 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
705719 .finally(() => (asking = false));
706720 };
707721 seed();
708 /** A frame as the panel shows it: servo id → ticks. */
722 const real = base !== simulation;
723 /** A frame as the Machine panel shows it: servo id → ticks. */
709724 const values = (text: string) => perServo(parseFrame(text) ?? []);
710 browser.on("message", (data, binary) => {
725 return (frame: string): string | null => {
726 // On the real arm, frames are not passed on while the link reconnects: the stream goes quiet, and the arm server's dead-man holds the arm.
727 if (real && mode() !== "live") return void machine.note("clamp", "not passed on", { why: RECONNECTING }), null;
728 const percent = lobby?.maxSpeedPercent ?? 100;
729 if (percent >= 100) {
730 if (lobby) cap.cap(frame, percent); // keeps each joint's last target, so a cap set mid-stream starts from there
731 machine.note("sent", "follow frame", { values: values(frame) });
732 return frame;
733 }
734 if (!seeded) seed();
735 const capped = cap.cap(frame, percent);
736 if (capped !== frame) machine.note("clamp", "speed cap", { percent, asked: values(frame), sent: capped === null ? {} : values(capped) });
737 if (capped !== null) machine.note("sent", "follow frame", { values: values(capped) });
738 return capped;
739 };
740 };
741 /**
742 * The arm's /follow: the live arm server's while it is live, the simulation's when there is none or it has been gone 60 s.
743 * Reconnecting, or a live arm server that does not take the stream, is refused: driving never jumps to the simulation.
744 */
745 const openFollow = async (): Promise<{ arm: WebSocket; base: string }> => {
746 if (remote && mode() === "reconnecting") throw new Error(`${RECONNECTING}…`);
747 if (remote && mode() === "live") {
748 try {
749 return { arm: await open(`${remote.ws}/follow`, readTimeoutMs), base: remote.http };
750 } catch (err) {
751 throw new Error(`the arm server at ${remote.host} did not take the stream (${reason(err)})`);
752 }
753 }
754 return { arm: await open(`${simulationWs}/follow`, readTimeoutMs), base: simulation };
755 };
756
757 // ------------------------------------------------------------------ WS /arm: the pose, the commands and the cameras, one socket per viewer
758 let viewers: ViewerSockets | null = null;
759 const telemetry = new Telemetry({ viewers: () => viewers?.count ?? 0, watching: () => viewers?.watching ?? 0, framesIn: () => cameras.framesIn });
760 const poses = new PoseFeed({
761 hz: options.poseHz ?? 30,
762 read: async () => {
763 // Reconnecting, the last known pose stands; the link is retrying, so nothing more is asked of the arm server meanwhile.
764 if (remote && mode() === "reconnecting" && armLink?.last) return { ...armLink.last, source: source(), reconnecting: true };
765 const base = remote && mode() === "live" ? remote.http : simulation;
766 const r = await fetch(`${base}/status`, { signal: AbortSignal.timeout(readTimeoutMs), cache: "no-store" });
767 if (!r.ok) throw new Error(`/status answered ${r.status}`);
768 return { ...((await r.json()) as Record<string, unknown>), source: source() };
769 },
770 onRead: (ms) => telemetry.arm.add(ms),
771 onError: () => telemetry.count("armErrors"),
772 });
773 viewers = new ViewerSockets({ ...options.viewer, poses, cameras, telemetry, lobby, openFollow, capper, lostCode: LOST_CLOSE });
774
775 /** Each open /follow stream and whose it is, so a turn that ends can close it. */
776 const streams = new Map<WebSocket, string>();
777 /** `base` is the arm server's HTTP side, asked where the joints are so the speed cap starts from there. */
778 const bridge = (browser: WebSocket, arm: WebSocket, base: string, driver: string | null) => {
779 if (driver !== null) {
780 streams.set(browser, driver);
781 browser.on("close", () => streams.delete(browser));
782 }
783 arm.on("message", (data, binary) => browser.readyState === WebSocket.OPEN && browser.send(data, { binary }));
784 const cap = capper(base);
785 browser.on("message", (data) => {
711786 if (driver !== null) {
712787 // The turn may have ended while this frame was on its way, before the close reaches the page: it is not passed on.
713788 if (lobby && !lobby.turns.isDriver(driver) && !lobby.local) return void machine.note("clamp", "not passed on", { why: "the turn ended" });
714789 lobby?.activity(driver);
715790 }
716791 if (arm.readyState !== WebSocket.OPEN) return;
717 if (real && mode() !== "live") return void machine.note("clamp", "not passed on", { why: RECONNECTING });
718 const percent = lobby?.maxSpeedPercent ?? 100;
719 const text = String(data);
720 if (percent >= 100) {
721 if (lobby) cap.cap(text, percent); // keeps each joint's last target, so a cap set mid-stream starts from there
722 machine.note("sent", "follow frame", { values: values(text) });
723 arm.send(data, { binary });
724 return;
725 }
726 if (!seeded) seed();
727 const capped = cap.cap(text, percent);
728 if (capped !== text) machine.note("clamp", "speed cap", { percent, asked: values(text), sent: capped === null ? {} : values(capped) });
729 if (capped === null) return;
730 machine.note("sent", "follow frame", { values: values(capped) });
731 arm.send(capped);
792 const capped = cap(String(data));
793 if (capped !== null) arm.send(capped);
732794 });
733795 arm.on("close", (code, why) => browser.close(sendable(code) ? code : 1011, why.length ? why.toString() : "the arm server went away"));
734796 // A browser that closed the stream on purpose leaves the arm holding; one that vanished leaves it parking.
@@ -749,10 +811,11 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
749811 };
750812 http.on("upgrade", (req: IncomingMessage, socket: Duplex, head: Buffer) => {
751813 const url = new URL(req.url ?? "/", "http://x");
752 if (url.pathname !== "/follow" && url.pathname !== "/machine" && !(url.pathname === "/live" && lobby)) return refuse(socket, "404 Not Found");
814 if (url.pathname !== "/follow" && url.pathname !== "/arm" && url.pathname !== "/machine" && !(url.pathname === "/live" && lobby)) return refuse(socket, "404 Not Found");
753815 if (!sameOrigin(req)) return refuse(socket, "403 Forbidden");
754816 socket.on("error", () => socket.destroy());
755817 if (url.pathname === "/live") return lobby!.upgrade(req, socket, head);
818 if (url.pathname === "/arm") return viewers!.upgrade(req, socket, head);
756819 if (url.pathname === "/machine") return machine.upgrade(req, socket, head);
757820 let driver: string | null = null;
758821 if (lobby) {
@@ -769,17 +832,10 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
769832 return wss.handleUpgrade(req, socket, head, (browser) => browser.close(1013, `${RECONNECTING}…`));
770833 }
771834 try {
772 if (!remote || mode() !== "live") throw null;
773 arm = await open(`${remote.ws}/follow`, readTimeoutMs);
774 base = remote.http;
835 ({ arm, base } = await openFollow());
775836 } catch (err) {
776 if (err) return wss.handleUpgrade(req, socket, head, (browser) => browser.close(1013, `the arm server at ${remote!.host} did not take the stream (${reason(err)})`));
777 try {
778 arm = await open(`${simulationWs}/follow`, readTimeoutMs);
779 base = simulation;
780 } catch {
781 return refuse(socket, "502 Bad Gateway");
782 }
837 if (remote && mode() !== "offline") return wss.handleUpgrade(req, socket, head, (browser) => browser.close(1013, (err as Error).message));
838 return refuse(socket, "502 Bad Gateway");
783839 }
784840 if (socket.destroyed) return arm.close();
785841 wss.handleUpgrade(req, socket, head, (browser) => bridge(browser, arm, base, driver));
@@ -792,6 +848,7 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
792848 const arrived = lobby?.onArrive((tell) => (pushed ? tell(pushed) : void push()));
793849 const turnEnded = lobby?.onTurnEnd((who) => {
794850 for (const [browser, id] of streams) if (id === who.id) browser.close(1000, "your turn ended");
851 viewers!.endTurn(who.id);
795852 const base = remote && onArm() ? remote.http : simulation;
796853 fetch(`${base}/stop`, { method: "POST", signal: AbortSignal.timeout(readTimeoutMs) }).then(
797854 (r) => r.body?.cancel(),
@@ -805,16 +862,26 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
805862 await (armLink as ArmLink | null)?.ready; // bounded by the first attempt's timeout: the first page already knows which arm it has
806863 const timer = lobby ? setInterval(follow, followEveryMs) : undefined;
807864 timer?.unref?.();
865 const sampler = setInterval(() => telemetry.sample(), 2000);
866 sampler.unref?.();
867 const logEveryMs = options.logEveryMs ?? 60_000;
868 const logger = logEveryMs > 0 ? setInterval(() => console.log(telemetry.line()), logEveryMs) : undefined;
869 logger?.unref?.();
808870 const pusher = lobby ? setInterval(() => void push(), statusEveryMs) : undefined;
809871 pusher?.unref?.();
810872 await new Promise<void>((resolve) => http.listen(options.port ?? 8102, options.host ?? "127.0.0.1", resolve));
811873 const address = http.address();
812874 return {
813875 port: typeof address === "object" && address ? address.port : options.port ?? 8102,
876 telemetry,
814877 link,
815878 close: () =>
816879 new Promise<void>((resolve) => {
817880 clearInterval(timer);
881 clearInterval(sampler);
882 clearInterval(logger);
883 viewers!.close();
884 poses.close();
818885 clearInterval(pusher);
819886 settingsChanged?.();
820887 turnEnded?.();
Software/so101/src/server.test.ts +73 −4
@@ -1,5 +1,5 @@
11 import { afterAll, beforeAll, describe, expect, test } from "bun:test";
2 import { existsSync, mkdtempSync, readFileSync, writeFileSync } from "node:fs";
2 import { existsSync, mkdtempSync, readFileSync, statSync, writeFileSync } from "node:fs";
33 import { connect } from "node:net";
44 import { tmpdir } from "node:os";
55 import { join } from "node:path";
@@ -128,13 +128,59 @@ describe("viewer page", () => {
128128 });
129129
130130 test("never builds HTML from people's text: event details and E-stop names go in as text nodes", async () => {
131 const html = await (await fetch(`${base}/`)).text();
131 // The page, and the Metrics panel it loads when opened (where the events are drawn).
132 const page = await (await fetch(`${base}/`)).text();
133 const metrics = /\/ui\/[0-9a-f]{12}\/metrics\.js/.exec(page)?.[0];
134 expect(metrics).toBeDefined();
135 const html = page + "\n" + (await (await fetch(`${base}${metrics}`)).text());
132136 const sinks = html.split("\n").filter((line) => /\.innerHTML\s*=/.test(line));
133137 expect(sinks.length).toBeGreaterThan(0);
134138 for (const line of sinks) expect(line).not.toMatch(/detail|\.by\b|estop|chat|\bname\b/);
135139 expect(html).toContain('$("events").replaceChildren(');
136140 });
137141
142 test("the admin's Health panel draws GET /health as sparklines, with the targets, only behind the relay, and loads when opened", async () => {
143 const page = await (await fetch(`${base}/`)).text();
144 const path = /\/ui\/[0-9a-f]{12}\/health\.js/.exec(page)?.[0];
145 expect(path).toBeDefined();
146 expect(page).toContain(`const { openHealth } = await import("${path}")`);
147 expect(page).not.toContain("sparkline.js");
148 expect(page).toContain('<button id="healthBtn" hidden>Health</button>');
149 expect(page).toContain('$("healthBtn").hidden = !relayed;');
150 const js = await (await fetch(`${base}${path}`)).text();
151 expect(js).toContain('import { sparkline } from "./sparkline.js";');
152 expect(js).toContain('const h = await api("/health");');
153 expect(js).toContain('["command → pose p95", "ms", (h) => h.commandRttMs?.p95, 250]');
154 expect(js).toContain('["camera frame age p95", "ms", (h) => h.frameAgeMs?.p95, 400]');
155 expect((await fetch(`${base}${path!.replace("health.js", "sparkline.js")}`)).status).toBe(200);
156 });
157
158 test("the page drives, watches and shows the cameras over WS /arm, and falls back to the old paths while it is down", async () => {
159 const page = await (await fetch(`${base}/`)).text();
160 const path = /\/ui\/[0-9a-f]{12}\/arm-socket\.js/.exec(page)?.[0];
161 expect(path).toBeDefined();
162 // Not in the first load: it comes once the first view is up, and until then the old paths serve.
163 expect(page).toContain(`const { ArmSocket } = await import("${path}");`);
164 expect(page).not.toContain(`<link rel="modulepreload" href="${path}">`);
165 expect(page).toContain("startViews();\n openSock()");
166 // One socket on /arm, behind the relay: the pose paints the screen, frames fill the camera panes, the driver's targets go as commands.
167 expect(page).toMatch(/new ArmSocket\(\{\s*url: `\$\{location\.protocol === "https:" \? "wss" : "ws"\}:\/\/\$\{location\.host\}\/arm`/);
168 expect(page).toContain("onPose: (d) => applyStatus(d)");
169 expect(page).toContain("onFrame: (f) => showFrame(f)");
170 expect(page).toContain("sock?.watch(");
171 expect(page).toContain("sock.command(frame)");
172 expect(page).toContain("sock.paint(c, f)");
173 // While it is down: /status polled, /follow for a drive, the MJPEG stream.
174 expect(page).toContain('if (sockUp) return void setTimeout(poll, 500);');
175 expect(page).toContain('const r = await timed("/status");');
176 expect(page).toContain("/follow`);");
177 expect(page).toMatch(/else if \(sockUp\) \{[^\n]*\}[^\n]*\n\s*else if \(plan\.mode === "snapshot" && c\.snapshot\) snap\(c\);\n\s*else c\.img\.src = streamUrl\(c\);/);
178 // The page asks for JSON, so /metrics behind the relay gives it the arm's metrics rather than the Prometheus text.
179 expect(page).toContain('headers: { accept: "application/json" }');
180 const js = await (await fetch(`${base}${path}`)).text();
181 for (const t of ["watch", "cmd", "end", "seen", "report", "ping"]) expect(js).toContain(`t: "${t}"`);
182 });
183
138184 test("a first visit downloads under 50 KB gzipped: the page and the modules it imports up front; the rest comes when needed", async () => {
139185 const page = await (await fetch(`${base}/`)).text();
140186 const script = page.slice(page.indexOf('<script type="module">'));
@@ -147,11 +193,12 @@ describe("viewer page", () => {
147193 bytes += gzipSync(Buffer.from(await r.arrayBuffer()), { level: 6 }).length;
148194 }
149195 expect(bytes).toBeLessThan(50_000);
150 // three.js, the leader's mapping, the calibration panel and the Machine panel are imported where they are used, not up front.
151 for (const lazy of ["three", "/leader-map.js", "calibration.js", "reach.js", "machine-panel.js", "machine.js"]) expect([...upfront].some((p) => p === lazy || p.endsWith(`/${lazy}`))).toBe(false);
196 // three.js, the leader's mapping, the calibration, Machine and Metrics panels are imported where they are used, not up front.
197 for (const lazy of ["three", "/leader-map.js", "calibration.js", "reach.js", "machine-panel.js", "machine.js", "metrics.js"]) expect([...upfront].some((p) => p === lazy || p.endsWith(`/${lazy}`))).toBe(false);
152198 expect(script).toContain('import("/leader-map.js")');
153199 expect(script).toMatch(/import\("\/ui\/[0-9a-f]{12}\/calibration\.js"\)/);
154200 expect(script).toMatch(/import\("\/ui\/[0-9a-f]{12}\/machine-panel\.js"\)/);
201 expect(script).toMatch(/import\("\/ui\/[0-9a-f]{12}\/metrics\.js"\)/);
155202 });
156203
157204 test("names its modules by their content, serves them for good, and serves nothing else from src/ui", async () => {
@@ -476,6 +523,28 @@ describe("the safe zone, the jam watch, and the model routes", () => {
476523 expect((await fetch(`${zurl}/vendor/three/../ws/index.js`)).status).toBe(404);
477524 });
478525
526 test("the model and the vendored libraries go under versioned names, cached for a year; the meshes lightened", async () => {
527 const info = (await (await fetch(`${zurl}/model.json`)).json()) as { urdf: string };
528 const version = /^\/model\/([0-9a-f]{12})\/so101_new_calib\.urdf$/.exec(info.urdf)?.[1];
529 expect(version).toBeDefined();
530 expect((await fetch(`${zurl}${info.urdf}`)).headers.get("cache-control")).toBe("public, max-age=31536000, immutable");
531 // Under an old version (a page from before a deploy): still served, but not kept for good.
532 expect((await fetch(`${zurl}/model/000000000000/so101_new_calib.urdf`)).headers.get("cache-control")).toBe("no-cache");
533 expect((await fetch(`${zurl}/model/so101_new_calib.urdf`)).headers.get("cache-control")).toBe("max-age=86400");
534 expect((await fetch(`${zurl}/model/${version}/../package.json`)).status).toBe(404);
535 const mesh = await fetch(`${zurl}/model/${version}/assets/wrist_roll_pitch_so101_v2.stl`);
536 const original = statSync(new URL("../../../Simulation/SO101/assets/wrist_roll_pitch_so101_v2.stl", import.meta.url)).size;
537 const bytes = new Uint8Array(await mesh.arrayBuffer());
538 expect(bytes.length * 2).toBeLessThan(original);
539 expect(bytes.length).toBe(84 + new DataView(bytes.buffer).getUint32(80, true) * 50); // still a binary STL
540 const page = await (await fetch(`${zurl}/`)).text();
541 const vendor = /"three": "\/vendor\/([0-9a-f]{12})\/three\/build\/three\.module\.js"/.exec(page)?.[1];
542 expect(vendor).toBeDefined();
543 expect(page).not.toContain("__VENDOR__");
544 expect((await fetch(`${zurl}/vendor/${vendor}/three/build/three.module.js`)).headers.get("cache-control")).toBe("public, max-age=31536000, immutable");
545 expect((await fetch(`${zurl}/vendor/${vendor}/ws/index.js`)).status).toBe(404);
546 });
547
479548 test("serves the printable sheets, and nothing else from that folder", async () => {
480549 const list = (await (await fetch(`${zurl}/printables`)).json()) as Array<{ file: string; url: string; bytes: number }>;
481550 expect(list.map((p) => p.file)).toEqual(["charuco_board.png", "cube_markers.png", "gripper_marker.png"]);
Software/so101/src/server.ts +77 −7
@@ -57,6 +57,7 @@ import type { FeetechBus } from "./bus.ts";
5757 import { ArmModel, gripperTicksToRadians, ticksToRadians, type Violation } from "./kinematics.ts";
5858 import { clamp, type Frame } from "./mapping.ts";
5959 import { MachineLog, perServo } from "./machine-log.ts";
60 import { lighten, LIGHTEN_VERSION, urdfFiles } from "./mesh-lite.ts";
6061 import { Metrics } from "./metrics.ts";
6162 import { MotionError, moveToPose, playTrajectory, recordTrajectory, STALL_SPAN_TICKS } from "./motion.ts";
6263 import { assertPoseName, parsePoseFile, serializePoseFile, type PoseFile } from "./poses.ts";
@@ -72,6 +73,8 @@ export interface ServerOptions {
7273 posesFile?: string;
7374 /** Folder holding so101_new_calib.urdf and assets/; without it there is no safe zone and no 3D model. */
7475 modelDir?: string;
76 /** Work out the model's version and lightened meshes at start rather than on the first request. */
77 warmModel?: boolean;
7578 /** Folder with the printable PNG sheets of the vision kit; found beside the package or in the repo layout. */
7679 printablesDir?: string;
7780 /** Torque cap while a stream drives the arm, percent. */
@@ -145,6 +148,7 @@ const JAM_WINDOW = 6;
145148 /** Consecutive too-far frames for one joint before it is eased over to the leader instead of left behind. */
146149 const RESUME_AFTER_DROPS = 20;
147150 const VENDORED = new Set(["three", "urdf-loader"]);
151 const MODEL_FILES = [".urdf", ".stl", ".dae", ".png", ".jpg"];
148152 /** A module the page loads from src/ui: plain JavaScript, never its tests. */
149153 const isUiModule = (name: string) => /^[a-z][a-z0-9-]*\.js$/.test(name);
150154 const MIME: Record<string, string> = { ".js": "text/javascript", ".mjs": "text/javascript", ".json": "application/json", ".urdf": "application/xml", ".stl": "model/stl", ".dae": "model/vnd.collada+xml", ".png": "image/png", ".jpg": "image/jpeg" };
@@ -635,6 +639,44 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
635639 return hash.digest("hex").slice(0, 12);
636640 };
637641 const nodeModules = resolve(here, "../node_modules");
642 /**
643 * The model and the vendored libraries, like the page's modules, under names that change with their content: the
644 * page asks for /model/<version>/… and /vendor/<version>/…, cached for a year; the plain paths stay, cached a day.
645 */
646 const hashOf = (parts: Iterable<string | Uint8Array>) => {
647 const hash = createHash("sha256");
648 for (const p of parts) hash.update(p);
649 return hash.digest("hex").slice(0, 12);
650 };
651 let modelVersionMemo: string | undefined;
652 const modelVersion = () =>
653 (modelVersionMemo ??= hashOf(
654 (function* () {
655 yield `lighten ${LIGHTEN_VERSION}`;
656 const walk = function* (dir: string, rel: string): Generator<string | Uint8Array> {
657 for (const f of readdirSync(dir, { withFileTypes: true }).sort((a, b) => a.name.localeCompare(b.name))) {
658 if (f.isDirectory()) yield* walk(join(dir, f.name), `${rel}${f.name}/`);
659 else if (MODEL_FILES.includes(extname(f.name).toLowerCase())) yield `${rel}${f.name}`, yield readFileSync(join(dir, f.name));
660 }
661 };
662 if (modelDir) yield* walk(resolve(modelDir), "");
663 })(),
664 ));
665 let vendorVersionMemo: string | undefined;
666 const vendorVersion = () =>
667 (vendorVersionMemo ??= hashOf(
668 [...VENDORED].map((pkg) => {
669 try {
670 return `${pkg}@${(JSON.parse(readFileSync(join(nodeModules, pkg, "package.json"), "utf8")) as { version?: string }).version}`;
671 } catch {
672 return pkg;
673 }
674 }),
675 ));
676 /** Meshes go out lightened (src/mesh-lite.ts), worked out once per file. */
677 const lightened = new Map<string, Uint8Array>();
678 /** Asked under the current version: kept for good. Under an old one (a page from before a deploy): served, but checked next time. */
679 const versioned = (asked: string, current: string, plain: string) => (asked === current ? "public, max-age=31536000, immutable" : asked ? "no-cache" : plain);
638680 const http = createServer(async (req, res) => {
639681 try {
640682 const url = new URL(req.url ?? "/", "http://x");
@@ -647,9 +689,15 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
647689 }
648690 };
649691 if (req.method === "GET" && url.pathname.startsWith("/model/") && modelDir) {
650 const path = safeFile(resolve(modelDir), decode(url.pathname.slice("/model/".length)));
651 if (!path || ![".urdf", ".stl", ".dae", ".png", ".jpg"].includes(extname(path).toLowerCase())) return json(res, 404, { error: "no such model file" });
652 return file(res, path, "max-age=86400");
692 const [, asked = "", rel = ""] = /^\/model\/(?:([0-9a-f]{12})\/)?(.*)$/.exec(url.pathname)!;
693 const path = safeFile(resolve(modelDir), decode(rel));
694 if (!path || !MODEL_FILES.includes(extname(path).toLowerCase())) return json(res, 404, { error: "no such model file" });
695 const cache = versioned(asked, modelVersion(), "max-age=86400");
696 if (extname(path).toLowerCase() !== ".stl") return file(res, path, cache);
697 let body = lightened.get(path);
698 if (!body) lightened.set(path, (body = lighten(new Uint8Array(readFileSync(path)))));
699 res.writeHead(200, { "content-type": MIME[".stl"]!, "cache-control": cache });
700 return res.end(body);
653701 }
654702 if (req.method === "GET" && url.pathname === "/printables") {
655703 let files: string[] = [];
@@ -666,10 +714,12 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
666714 return file(res, path, "max-age=3600");
667715 }
668716 if (req.method === "GET" && url.pathname.startsWith("/vendor/")) {
669 const [pkg, ...rest] = decode(url.pathname.slice("/vendor/".length)).split("/");
717 const parts = decode(url.pathname.slice("/vendor/".length)).split("/");
718 const asked = /^[0-9a-f]{12}$/.test(parts[0] ?? "") ? parts.shift()! : "";
719 const [pkg, ...rest] = parts;
670720 const path = pkg && VENDORED.has(pkg) && rest.length ? safeFile(join(nodeModules, pkg), rest.join("/")) : null;
671721 if (!path || ![".js", ".mjs", ".json"].includes(extname(path).toLowerCase())) return json(res, 404, { error: "no such vendored file" });
672 return file(res, path, "max-age=86400");
722 return file(res, path, versioned(asked, vendorVersion(), "max-age=86400"));
673723 }
674724 const ui = /^\/ui\/[0-9a-f]{12}\/([a-z][a-z0-9-]*\.js)$/.exec(url.pathname);
675725 if (req.method === "GET" && ui) {
@@ -697,7 +747,7 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
697747 case "GET /":
698748 case "GET /index.html":
699749 res.writeHead(200, { "content-type": "text/html; charset=utf-8", "cache-control": "no-store" });
700 return res.end(readFileSync(viewerPath, "utf8").replaceAll("/ui/__VERSION__/", `/ui/${uiVersion()}/`)); // read per request: a viewer change deploys with rsync alone
750 return res.end(readFileSync(viewerPath, "utf8").replaceAll("/ui/__VERSION__/", `/ui/${uiVersion()}/`).replaceAll("/vendor/__VENDOR__/", `/vendor/${vendorVersion()}/`)); // read per request: a viewer change deploys with rsync alone
701751 case "GET /leader-map.js":
702752 return file(res, join(here, "leader-map.js"), "no-cache");
703753 case "GET /status":
@@ -727,7 +777,7 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
727777 case "GET /model.json":
728778 if (!model) return json(res, 404, { error: "no model loaded on this server" });
729779 return json(res, 200, {
730 urdf: `/model/${URDF}`,
780 urdf: `/model/${modelVersion()}/${URDF}`,
731781 floorZ: model.floorZ,
732782 baseTop: model.baseTop,
733783 joints: model.movable.map((j) => ({ name: j.name, id: Number(Object.entries(JOINTS).find(([, n]) => n === j.name)?.[0] ?? 0), lower: j.lower, upper: j.upper })),
@@ -1254,10 +1304,30 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
12541304 await new Promise<void>((resolve) => http.listen(options.port ?? 8101, options.host ?? "0.0.0.0", resolve));
12551305 const address = http.address();
12561306 const port = typeof address === "object" && address ? address.port : options.port ?? 8101;
1307 // The model's version and its lightened meshes, worked out now, a file a turn, rather than inside a first visitor's
1308 // request while the cameras and the poses wait on the same thread.
1309 let warming: ReturnType<typeof setTimeout> | undefined;
1310 if (options.warmModel && modelDir) {
1311 let urdf = "";
1312 try {
1313 urdf = readFileSync(join(modelDir, URDF), "utf8");
1314 } catch {
1315 /* no model */
1316 }
1317 const todo = urdfFiles(urdf).map((f) => safeFile(resolve(modelDir), f)).filter((p): p is string => p !== null && extname(p).toLowerCase() === ".stl");
1318 const step = () => {
1319 const path = todo.shift();
1320 if (!path) return void modelVersion();
1321 if (!lightened.has(path)) lightened.set(path, lighten(new Uint8Array(readFileSync(path))));
1322 warming = setTimeout(step, 0);
1323 };
1324 warming = setTimeout(step, 0);
1325 }
12571326 return {
12581327 port,
12591328 close: () =>
12601329 new Promise<void>((resolve) => {
1330 clearTimeout(warming);
12611331 clearInterval(watching);
12621332 machine.close();
12631333 bus.onSyncWrite = null;
Software/so101/src/telemetry.test.ts added +63
@@ -0,0 +1,63 @@
1 import { expect, test } from "bun:test";
2 import { Latest } from "./latest.ts";
3 import { earn, noCredit, parseReport, Telemetry, vouch, Window } from "./telemetry.ts";
4
5 test("a window keeps a span of samples and answers quantiles over it", () => {
6 const w = new Window(1000);
7 for (let i = 1; i <= 100; i++) w.add(i, 10_000);
8 expect(w.quantile(0.5, 10_000)).toBe(50);
9 expect(w.quantile(0.95, 10_000)).toBe(95);
10 w.add(500, 10_900);
11 expect(w.sum(200, 10_900)).toBe(500);
12 expect(w.quantile(0.5, 11_500)).toBe(500); // the first hundred are older than a second now
13 expect(w.quantile(0.5, 13_000)).toBeNull();
14 });
15
16 test("a report is checked: nonsense dropped, counts capped", () => {
17 expect(parseReport(null)).toBeNull();
18 expect(parseReport({ rtt: [10, -1, "x", 1e9, Infinity], age: "no", frames: 2.7, sent: -3, dropped: 1e12 })).toEqual({ rtt: [10], age: [], frames: 2, sent: 0, dropped: 100_000, stalls: 0, reconnects: 0 });
19 expect(parseReport({ rtt: Array(500).fill(1) })!.rtt).toHaveLength(200);
20 });
21
22 test("a report is held to what crossed its socket: no more commands or frames than the app saw", () => {
23 const claim = { rtt: Array(200).fill(0), age: Array(200).fill(0), frames: 100_000, sent: 100_000, dropped: 100_000, stalls: 100_000, reconnects: 100_000 };
24 const credit = earn(earn(earn(noCredit(), "command"), "command", 2), "frames", 2);
25 expect(vouch(claim, credit)).toEqual({ report: { rtt: [0, 0, 0], age: [0, 0], frames: 2, sent: 3, dropped: 2_000, stalls: 10, reconnects: 10 }, left: noCredit() });
26 expect(vouch(claim, noCredit()).report).toMatchObject({ rtt: [], age: [], frames: 0, sent: 0 });
27 // A round trip measured after the report that counted its command still has its command's credit.
28 const first = vouch({ ...claim, rtt: [], frames: 0, age: [] }, credit);
29 expect(first.left).toEqual({ rtt: 3, sent: 0, age: 2, frames: 2 });
30 expect(vouch({ ...claim, rtt: [140], sent: 0 }, first.left).report.rtt).toEqual([140]);
31 });
32
33 test("the numbers come out as Prometheus text, JSON with history, and one line", () => {
34 let framesIn = 0;
35 const t = new Telemetry({ viewers: () => 2, watching: () => 1, framesIn: () => framesIn });
36 t.report({ rtt: [100, 200], age: [300], frames: 50, sent: 20, dropped: 5, stalls: 0, reconnects: 2 }, 1000);
37 t.arm.add(40, 1000);
38 framesIn = 100;
39 t.sample(1000);
40 const h = t.json(2000);
41 expect(h).toMatchObject({ viewers: 2, watching: 1, commandRttMs: { p50: 100, p95: 200 }, frameAgeMs: { p50: 300 }, viewerFps: 5, cameraFpsIn: 10, commandsPerS: 2 });
42 expect(h.history).toHaveLength(1);
43 const text = t.prometheus(2000);
44 expect(text).toContain("# TYPE arm_command_rtt_ms summary");
45 expect(text).toContain("arm_commands_dropped_total 5");
46 expect(text).toContain("arm_reconnects_total 2");
47 expect(t.line(2000)).toContain("command→pose p50/p95 100/200ms");
48 });
49
50 test("latest wins: while the window is full, newer messages replace waiting ones", () => {
51 const out: string[] = [];
52 const dones: Array<() => void> = [];
53 const l = new Latest((data, done) => (out.push(String(data)), dones.push(done), true), 2);
54 for (const n of ["a", "b", "c", "d", "e"]) l.offer("cam", n);
55 expect(out).toEqual(["a", "b"]);
56 expect(l.skipped).toBe(2); // c and d were replaced while waiting
57 dones.shift()!();
58 expect(out).toEqual(["a", "b", "e"]);
59 dones.shift()!();
60 dones.shift()!();
61 l.offer("cam", "f");
62 expect(out).toEqual(["a", "b", "e", "f"]);
63 });
Software/so101/src/telemetry.ts added +291
@@ -0,0 +1,291 @@
1 /**
2 * How the link feels, in numbers: what each viewer measures (command round trip, camera frame age, frames shown,
3 * stalls) and what the app sees itself (the arm server's round trip, the frames the cameras send, bytes out, frames
4 * a slow viewer had to skip). Kept for the last minute in memory, nothing on disk: it is for iterating, not history.
5 *
6 * prometheus() the text a Prometheus scrape reads, on GET /metrics
7 * health() the same as JSON with a two-minute history for the sparklines, on GET /health
8 * line() the one-line summary logged every minute (summary(), which the terminal's /health prints too)
9 */
10
11 /** Timed samples, kept for `spanMs`. */
12 export class Window {
13 private t: number[] = [];
14 private v: number[] = [];
15 private readonly spanMs: number;
16 private readonly cap: number;
17
18 constructor(spanMs: number, cap = 10_000) {
19 this.spanMs = spanMs;
20 this.cap = cap;
21 }
22
23 add(value: number, now = Date.now()): void {
24 this.t.push(now);
25 this.v.push(value);
26 this.prune(now);
27 if (this.t.length > this.cap) this.t.splice(0, this.t.length - this.cap), this.v.splice(0, this.v.length - this.cap);
28 }
29
30 private prune(now: number) {
31 let i = 0;
32 while (i < this.t.length && now - this.t[i]! > this.spanMs) i++;
33 if (i) this.t.splice(0, i), this.v.splice(0, i);
34 }
35
36 /** The value below which a fraction `q` of the samples lie; null with no samples. */
37 quantile(q: number, now = Date.now()): number | null {
38 this.prune(now);
39 if (!this.v.length) return null;
40 const sorted = [...this.v].sort((a, b) => a - b);
41 return sorted[Math.min(sorted.length - 1, Math.max(0, Math.ceil(q * sorted.length) - 1))]!;
42 }
43
44 sum(withinMs = this.spanMs, now = Date.now()): number {
45 this.prune(now);
46 let s = 0;
47 for (let i = this.t.length - 1; i >= 0 && now - this.t[i]! <= withinMs; i--) s += this.v[i]!;
48 return s;
49 }
50
51 count(now = Date.now()): number {
52 this.prune(now);
53 return this.v.length;
54 }
55 }
56
57 /** What one viewer measured since its last report. */
58 export interface Report {
59 /** Command → pose round trips, ms. */
60 rtt: number[];
61 /** Camera frame ages when shown (the app's receipt → on screen), ms. */
62 age: number[];
63 /** Camera frames shown. */
64 frames: number;
65 /** Commands sent, and the ones merged away before sending (latest wins). */
66 sent: number;
67 dropped: number;
68 /** Times the socket went quiet past the stall limit, and reconnects. */
69 stalls: number;
70 reconnects: number;
71 }
72
73 const MAX_SAMPLES = 200;
74 const MAX_MS = 120_000;
75 const MAX_COUNT = 100_000;
76
77 /** A viewer's report, checked: null when it is not one. Samples outside 0..120 s are dropped, counts capped. */
78 export function parseReport(raw: unknown): Report | null {
79 if (!raw || typeof raw !== "object") return null;
80 const r = raw as Record<string, unknown>;
81 const samples = (x: unknown) => (Array.isArray(x) ? x.slice(0, MAX_SAMPLES).filter((n): n is number => typeof n === "number" && Number.isFinite(n) && n >= 0 && n <= MAX_MS) : []);
82 const count = (x: unknown) => (typeof x === "number" && Number.isFinite(x) && x >= 0 ? Math.min(MAX_COUNT, Math.floor(x)) : 0);
83 return { rtt: samples(r.rtt), age: samples(r.age), frames: count(r.frames), sent: count(r.sent), dropped: count(r.dropped), stalls: count(r.stalls), reconnects: count(r.reconnects) };
84 }
85
86 /** Merged-away commands, stalls and reconnects one report may claim: the app cannot see these, so they are only bounded. */
87 const MAX_DROPPED = 2_000;
88 const MAX_EVENTS = 10;
89
90 /** What a socket has earned the right to report: one round trip and one sent per command it received, one frame and one age per frame it was sent. */
91 export interface Credit {
92 rtt: number;
93 sent: number;
94 age: number;
95 frames: number;
96 }
97
98 export const noCredit = (): Credit => ({ rtt: 0, sent: 0, age: 0, frames: 0 });
99
100 /** Credit for one command, or for `n` frames sent. */
101 export const earn = (c: Credit, kind: "command" | "frames", n = 1): Credit =>
102 kind === "command" ? { ...c, rtt: c.rtt + n, sent: c.sent + n } : { ...c, age: c.age + n, frames: c.frames + n };
103
104 /**
105 * A report held to what crossed its socket: no more commands (or round trips) than it received, no more frames (or
106 * frame ages) than it was sent. What a report doesn't claim carries over, since a round trip is measured when the pose
107 * that acknowledges it comes back, often in a later report than the command. A page cannot claim work that never
108 * crossed its socket.
109 */
110 export function vouch(r: Report, credit: Credit): { report: Report; left: Credit } {
111 const report: Report = {
112 rtt: r.rtt.slice(0, credit.rtt),
113 age: r.age.slice(0, credit.age),
114 frames: Math.min(r.frames, credit.frames),
115 sent: Math.min(r.sent, credit.sent),
116 dropped: Math.min(r.dropped, MAX_DROPPED),
117 stalls: Math.min(r.stalls, MAX_EVENTS),
118 reconnects: Math.min(r.reconnects, MAX_EVENTS),
119 };
120 const left = { rtt: credit.rtt - report.rtt.length, sent: credit.sent - report.sent, age: credit.age - report.age.length, frames: credit.frames - report.frames };
121 return { report, left };
122 }
123
124 export interface Gauges {
125 /** Open viewer sockets. */
126 viewers(): number;
127 /** Viewers watching at least one camera. */
128 watching(): number;
129 /** Whole frames the cameras have sent the app, ever. */
130 framesIn(): number;
131 }
132
133 export interface Quantiles {
134 p50: number | null;
135 p95: number | null;
136 }
137
138 export interface Health {
139 at: number;
140 viewers: number;
141 watching: number;
142 commandRttMs: Quantiles;
143 frameAgeMs: Quantiles;
144 /** Frames shown per second per watching viewer, over the last 10 s. */
145 viewerFps: number;
146 armRttMs: Quantiles;
147 /** Frames per second coming in from the cameras, over the last 10 s. */
148 cameraFpsIn: number;
149 bytesOutPerS: number;
150 commandsPerS: number;
151 totals: Record<Counter, number>;
152 }
153
154 export type Counter = "commandsSent" | "commandsDropped" | "commandsMerged" | "framesShown" | "framesSent" | "framesSkipped" | "stalls" | "reconnects" | "bytesOut" | "armErrors" | "reportsDropped";
155 const COUNTERS: Counter[] = ["commandsSent", "commandsDropped", "commandsMerged", "framesShown", "framesSent", "framesSkipped", "stalls", "reconnects", "bytesOut", "armErrors", "reportsDropped"];
156 const HELP: Record<Counter, string> = {
157 commandsSent: "commands viewers sent",
158 commandsDropped: "commands viewers merged away before sending (latest wins)",
159 commandsMerged: "commands the app merged away before the arm server (latest wins)",
160 framesShown: "camera frames viewers showed",
161 framesSent: "camera frames the app sent viewers",
162 framesSkipped: "camera frames the app skipped for a viewer still busy with an earlier one",
163 stalls: "viewer sockets that went quiet past the stall limit",
164 reconnects: "viewer socket reconnects",
165 bytesOut: "bytes the app sent viewers over their sockets",
166 armErrors: "pose reads the arm server did not answer",
167 reportsDropped: "viewer reports dropped for coming sooner than one per socket every few seconds",
168 };
169 const RATE_MS = 10_000;
170
171 export class Telemetry {
172 readonly rtt: Window;
173 readonly age: Window;
174 readonly arm: Window;
175 readonly totals = Object.fromEntries(COUNTERS.map((c) => [c, 0])) as Record<Counter, number>;
176 private readonly rates = new Map<Counter | "framesIn", Window>();
177 private readonly history: Health[] = [];
178 private lastFramesIn = 0;
179
180 private readonly gauges: Gauges;
181 private readonly o: { windowMs?: number; historyLength?: number };
182
183 constructor(gauges: Gauges, options: { windowMs?: number; historyLength?: number } = {}) {
184 this.gauges = gauges;
185 this.o = options;
186 const o = options;
187 const span = o.windowMs ?? 60_000;
188 this.rtt = new Window(span);
189 this.age = new Window(span);
190 this.arm = new Window(span);
191 for (const c of [...COUNTERS, "framesIn" as const]) this.rates.set(c, new Window(RATE_MS, 100_000));
192 }
193
194 count(counter: Counter, n = 1, now = Date.now()): void {
195 if (!n) return;
196 this.totals[counter] += n;
197 this.rates.get(counter)!.add(n, now);
198 }
199
200 report(r: Report, now = Date.now()): void {
201 for (const v of r.rtt) this.rtt.add(v, now);
202 for (const v of r.age) this.age.add(v, now);
203 this.count("framesShown", r.frames, now);
204 this.count("commandsSent", r.sent, now);
205 this.count("commandsDropped", r.dropped, now);
206 this.count("stalls", r.stalls, now);
207 this.count("reconnects", r.reconnects, now);
208 }
209
210 private rate(c: Counter | "framesIn", now: number) {
211 return Math.round((this.rates.get(c)!.sum(RATE_MS, now) / (RATE_MS / 1000)) * 10) / 10;
212 }
213
214 health(now = Date.now()): Health {
215 const framesIn = this.gauges.framesIn();
216 if (framesIn > this.lastFramesIn) this.rates.get("framesIn")!.add(framesIn - this.lastFramesIn, now);
217 this.lastFramesIn = Math.max(this.lastFramesIn, framesIn);
218 const q = (w: Window): Quantiles => ({ p50: w.quantile(0.5, now), p95: w.quantile(0.95, now) });
219 const watching = this.gauges.watching();
220 return {
221 at: now,
222 viewers: this.gauges.viewers(),
223 watching,
224 commandRttMs: q(this.rtt),
225 frameAgeMs: q(this.age),
226 viewerFps: watching ? Math.round((this.rate("framesShown", now) / watching) * 10) / 10 : 0,
227 armRttMs: q(this.arm),
228 cameraFpsIn: this.rate("framesIn", now),
229 bytesOutPerS: this.rate("bytesOut", now),
230 commandsPerS: this.rate("commandsSent", now),
231 totals: { ...this.totals },
232 };
233 }
234
235 /** Take a point for the sparklines; the relay calls this every couple of seconds. */
236 sample(now = Date.now()): Health {
237 const h = this.health(now);
238 this.history.push(h);
239 const keep = this.o.historyLength ?? 60;
240 if (this.history.length > keep) this.history.splice(0, this.history.length - keep);
241 return h;
242 }
243
244 /** The latest numbers and the recent points, oldest first. */
245 json(now = Date.now()): Health & { history: Health[] } {
246 return { ...this.health(now), history: this.history.map(({ totals: _, ...h }) => h as Health) };
247 }
248
249 prometheus(now = Date.now()): string {
250 const h = this.health(now);
251 const out: string[] = [];
252 const metric = (name: string, type: string, help: string, lines: string[]) => out.push(`# HELP ${name} ${help}`, `# TYPE ${name} ${type}`, ...lines);
253 const summary = (name: string, help: string, v: Quantiles, w: Window) =>
254 metric(name, "summary", help, [
255 ...(v.p50 === null ? [] : [`${name}{quantile="0.5"} ${v.p50}`, `${name}{quantile="0.95"} ${v.p95}`]),
256 `${name}_count ${w.count(now)}`,
257 ]);
258 summary("arm_command_rtt_ms", "command sent to pose reflecting it, as viewers measure it, last minute", h.commandRttMs, this.rtt);
259 summary("arm_camera_frame_age_ms", "camera frame age when shown (app receipt to screen), last minute", h.frameAgeMs, this.age);
260 summary("arm_server_rtt_ms", "the app's pose read round trip to the arm server, last minute", h.armRttMs, this.arm);
261 metric("arm_viewers", "gauge", "open viewer sockets", [`arm_viewers ${h.viewers}`]);
262 metric("arm_viewers_watching", "gauge", "viewers watching a camera", [`arm_viewers_watching ${h.watching}`]);
263 metric("arm_viewer_fps", "gauge", "camera frames shown per second per watching viewer, last 10 s", [`arm_viewer_fps ${h.viewerFps}`]);
264 metric("arm_camera_fps_in", "gauge", "camera frames per second coming into the app, last 10 s", [`arm_camera_fps_in ${h.cameraFpsIn}`]);
265 for (const c of COUNTERS) {
266 const name = `arm_${c.replace(/[A-Z]/g, (x) => `_${x.toLowerCase()}`)}_total`;
267 metric(name, "counter", HELP[c], [`${name} ${this.totals[c]}`]);
268 }
269 return `${out.join("\n")}\n`;
270 }
271
272 line(now = Date.now()): string {
273 return summary(this.health(now));
274 }
275 }
276
277 /**
278 * Health as one line, for the log and the terminal's /health. It takes the JSON as it arrives (the terminal reads it
279 * off GET /health), so anything that is not a number shows as "-".
280 */
281 export function summary(h: unknown): string {
282 const o = (x: unknown): Record<string, unknown> => (x && typeof x === "object" ? (x as Record<string, unknown>) : {});
283 const n = (x: unknown) => (typeof x === "number" && Number.isFinite(x) ? x : null);
284 const num = (x: unknown) => String(n(x) ?? "-");
285 const ms = (x: unknown) => {
286 const p50 = n(o(x).p50), p95 = n(o(x).p95);
287 return p50 === null || p95 === null ? "-" : `${Math.round(p50)}/${Math.round(p95)}ms`;
288 };
289 const r = o(h), t = o(r.totals), bytes = n(r.bytesOutPerS);
290 return `link: ${num(r.viewers)} viewers (${num(r.watching)} watching) · command→pose p50/p95 ${ms(r.commandRttMs)} · frame age ${ms(r.frameAgeMs)} · ${num(r.viewerFps)} fps shown, ${num(r.cameraFpsIn)} in · arm ${ms(r.armRttMs)} · ${num(r.commandsPerS)} cmd/s · ${bytes === null ? "-" : Math.round(bytes / 1024)} KiB/s out · ${num(t.framesSkipped)} skipped, ${num(t.stalls)} stalls, ${num(t.reconnects)} reconnects`;
291 }
Software/so101/src/terminal.test.ts +21
@@ -295,4 +295,25 @@ describe("terminal door", () => {
295295 await owner.printedLine(/alice's turn ended .*bob has control/);
296296 expect(lobby!.admin.log.some((e) => /ended alice's turn/.test(JSON.stringify(e)))).toBe(true);
297297 });
298
299 test("/health: the owner's terminal reads the link's numbers on one line; anyone else is told it is the admin's", async () => {
300 const port = await start();
301 expect(typed(emptyRoom(), "/health")).toEqual({ health: true });
302 const carol = await terminal(port, ttyEnv("u-carol", "carol", "user"));
303 carol.door.input("/health");
304 await carol.printedLine(/^error: only an admin can see the link's health$/);
305 const owner = await terminal(port, ttyEnv("u-owner", "trevato", "admin"));
306 owner.door.input("/health");
307 // Signed afresh for each request: the forge's signature is good for a minute, a terminal stays longer.
308 let signed = 0;
309 const env = { ...ttyEnv("u-owner", "trevato", "admin"), PORT: String(port) };
310 const lines: string[] = [];
311 const fresh = new Door(liveUrl(env), () => (signed++, terminalHeaders(env)), (line) => lines.push(line));
312 await fresh.open();
313 open.push(fresh);
314 await fresh.health();
315 expect(signed).toBe(2);
316 expect(lines.at(-1)).toMatch(/^link: /);
317 await owner.printedLine(/^link: 0 viewers \(0 watching\) · command→pose p50\/p95 - · .* 0 stalls, 0 reconnects$/);
318 });
298319 });
Software/so101/src/terminal.ts +25 −7
@@ -14,6 +14,7 @@
1414 * /ready /extend confirm you are there when called (15 s), or add 30 s to your turn
1515 * /end [who] owner: end that turn (none: whoever drives), or take them out of the line
1616 * /mute <who> /unmute <who> owner
17 * /health admin: how the link feels (GET /health as one line: command round trips, frame ages, fps, stalls)
1718 * /machine [off] the Machine panel (/machine, src/machine-feed.ts): the joints, the link and the server as they are,
1819 * then every command in and out as it happens, in the page's own words (src/ui/machine.js); off stops it
1920 * /who /help /quit
@@ -23,6 +24,7 @@
2324 import WebSocket from "ws";
2425 import { UNPRINTABLE } from "./chat.ts";
2526 import { ID, NAME, sign } from "./identity.ts";
27 import { summary } from "./telemetry.ts";
2628 import { entryLine, jointRows, linkRows, stateRows } from "./ui/machine.js";
2729
2830 export type Env = Record<string, string | undefined>;
@@ -93,7 +95,7 @@ const said = (m: unknown): string => {
9395 return from ? `${plain(from.name)}: ${plain(message?.text)}` : `· ${plain(message?.text)}`;
9496 };
9597
96 export const HELP = "type to chat · /join /leave /ready /extend · /end [who] /mute <who> /unmute <who> (owner) · /machine [off] · /who · /quit";
98 export const HELP = "type to chat · /join /leave /ready /extend · /end [who] /mute <who> /unmute <who> /health (owner) · /machine [off] · /who · /quit";
9799
98100 /** The Machine feed's socket beside the lobby's: same app, same port. */
99101 export const machineUrl = (live: string): string => live.replace(/\/live$/, "/machine");
@@ -192,7 +194,7 @@ function target(room: Room, word: string): string | null {
192194 return ID.test(word) ? word : null;
193195 }
194196
195 export type Typed = { send: Frame } | { print: string } | { machine: boolean } | { quit: true } | null;
197 export type Typed = { send: Frame } | { print: string } | { health: true } | { machine: boolean } | { quit: true } | null;
196198
197199 /** What a typed line asks for. Validation the server repeats anyway; this only saves a round trip on a typo. */
198200 export function typed(room: Room, input: string): Typed {
@@ -223,6 +225,8 @@ export function typed(room: Room, input: string): Typed {
223225 return { machine: arg.toLowerCase() !== "off" };
224226 case "who":
225227 return { print: who(room) };
228 case "health":
229 return { health: true };
226230 case "help":
227231 return { print: HELP };
228232 case "quit":
@@ -240,18 +244,19 @@ export class Door {
240244 private machineWs: WebSocket | null = null;
241245 readonly machine: MachineState = { status: null, link: null, room: null, skew: 0 };
242246 private readonly url: string;
243 private readonly headers: Record<string, string>;
247 /** Signed afresh for each request: a signature is good for a minute, and a terminal stays open for longer. */
248 private readonly headers: () => Record<string, string>;
244249 private readonly print: (line: string) => void;
245250
246 constructor(url: string, headers: Record<string, string>, print: (line: string) => void) {
251 constructor(url: string, headers: Record<string, string> | (() => Record<string, string>), print: (line: string) => void) {
247252 this.url = url;
248 this.headers = headers;
253 this.headers = typeof headers === "function" ? headers : () => headers;
249254 this.print = print;
250255 }
251256
252257 /** Connect; resolves once seated, rejects when the app does not answer. `onClose` hears the app going away. */
253258 open(onClose: () => void = () => {}): Promise<void> {
254 const ws = new WebSocket(this.url, { headers: this.headers, maxPayload: 1 << 20 });
259 const ws = new WebSocket(this.url, { headers: this.headers(), maxPayload: 1 << 20 });
255260 this.ws = ws;
256261 ws.on("message", (data, binary) => {
257262 if (binary) return;
@@ -280,11 +285,24 @@ export class Door {
280285 if (!t) return true;
281286 if ("quit" in t) return false;
282287 if ("print" in t) this.print(t.print);
288 else if ("health" in t) void this.health();
283289 else if ("machine" in t) this.watchMachine(t.machine);
284290 else this.send(t.send);
285291 return true;
286292 }
287293
294 /** The link's health, as the app's GET /health tells this terminal's person (the admin only; the server says why not). */
295 async health(): Promise<void> {
296 const url = this.url.replace(/^ws/, "http").replace(/\/live$/, "/health");
297 try {
298 const r = await fetch(url, { headers: { ...this.headers(), accept: "application/json" }, signal: AbortSignal.timeout(3000) });
299 const body = (await r.json().catch(() => null)) as Record<string, unknown> | null;
300 this.print(r.ok ? summary(body) : `error: ${plain(body?.error ?? `the app answered ${r.status}`)}`);
301 } catch (err) {
302 this.print(`error: the app did not answer (${plain((err as Error).message)})`);
303 }
304 }
305
288306 close(): void {
289307 this.ws?.close();
290308 this.machineWs?.close();
@@ -299,7 +317,7 @@ export class Door {
299317 return this.print("machine feed closed");
300318 }
301319 if (this.machineWs) return void machineLines(this.machine).forEach((l) => this.print(l));
302 const ws = new WebSocket(machineUrl(this.url), { headers: this.headers, maxPayload: 1 << 22 });
320 const ws = new WebSocket(machineUrl(this.url), { headers: this.headers(), maxPayload: 1 << 22 });
303321 this.machineWs = ws;
304322 const now = () => Date.now() - this.machine.skew;
305323 let waiting = false;
Software/so101/src/tty.ts +1 −1
@@ -17,7 +17,7 @@ const print = (line: string) => {
1717 rl?.prompt(true);
1818 };
1919
20 const door = new Door(liveUrl(env), terminalHeaders(env), print);
20 const door = new Door(liveUrl(env), () => terminalHeaders(env), print);
2121 print(`\x1b[1mSO-101 arm\x1b[0m · ${where} · ${HELP}`);
2222 try {
2323 await door.open(() => {
Software/so101/src/ui/arm-socket-page.test.ts added +48
@@ -0,0 +1,48 @@
1 import { afterAll, expect, test } from "bun:test";
2
3 // In a browser: a document that can be hidden, and browser frames. The module reads them when it loads.
4 const listeners: Array<() => void> = [];
5 const frames: Array<() => void> = [];
6 const g = globalThis as any;
7 const document = { hidden: false, addEventListener: (_: string, f: () => void) => listeners.push(f), removeEventListener: () => {} };
8 g.document = document;
9 g.requestAnimationFrame = (f: () => void) => frames.push(f);
10 const fresh: string = "./arm-socket.js?in-a-page"; // its own copy of the module, loaded with the page's globals in place
11 const { ArmSocket }: typeof import("./arm-socket.js") = await import(fresh);
12 afterAll(() => { delete g.document; delete g.requestAnimationFrame; });
13
14 class FakeSocket {
15 static last: FakeSocket;
16 readyState = 1;
17 sent: any[] = [];
18 onopen = () => {};
19 onmessage = (_: { data: string }) => {};
20 onclose = () => {};
21 onerror = () => {};
22 constructor() { FakeSocket.last = this; }
23 send(data: string) { this.sent.push(JSON.parse(data)); }
24 close() {}
25 }
26
27 test("in a page: poses are drawn once per browser frame, and a hidden page watches no camera", () => {
28 const poses: number[] = [];
29 const s = new ArmSocket({ url: "ws://x/arm", WebSocket: FakeSocket as any, onPose: (status: any) => poses.push(status.n) });
30 const ws = FakeSocket.last;
31 ws.onopen();
32 s.watch([1, 2]);
33 expect(ws.sent.filter((m) => m.t === "watch").at(-1)).toEqual({ t: "watch", cameras: [1, 2], hz: 30 });
34 for (let n = 1; n <= 5; n++) ws.onmessage({ data: JSON.stringify({ t: "pose", status: { n } }) });
35 expect(poses).toEqual([]);
36 expect(frames).toHaveLength(1);
37 frames.shift()!();
38 expect(poses).toEqual([5]); // the latest, once
39 document.hidden = true;
40 for (const f of listeners) f();
41 expect(ws.sent.at(-1)).toEqual({ t: "watch", cameras: [], hz: 1 });
42 ws.onmessage({ data: JSON.stringify({ t: "pose", status: { n: 6 } }) });
43 expect(poses).toEqual([5, 6]); // hidden: no browser frame to wait for
44 document.hidden = false;
45 for (const f of listeners) f();
46 expect(ws.sent.at(-1)).toEqual({ t: "watch", cameras: [1, 2], hz: 30 });
47 s.close();
48 });
Software/so101/src/ui/arm-socket.js added +320
@@ -0,0 +1,320 @@
1 /**
2 * The page's end of WS /arm (src/viewer-socket.ts): one socket for the pose, the driver's commands and the camera
3 * frames, so nothing pays the public round trip per request. Plain ESM, served at /ui/ and imported by the tests.
4 *
5 * It keeps itself up: a socket that goes quiet for `stallMs` (the app pings every second and sends poses many times a
6 * second) is closed and opened again, and so is one that drops. A drive does not survive that: the page is told the
7 * drive closed, and the arm holds until the next press.
8 *
9 * And it measures what the owner iterates on, reporting to the app every few seconds (src/telemetry.ts): command →
10 * pose round trips, camera frame ages when shown, frames shown, commands sent and merged away, stalls, reconnects.
11 */
12
13 export const FRAME_HEADER = 20;
14 /** @typedef {"sent" | "dropped" | "stalls" | "reconnects" | "frames"} Counted */
15 /** The page's document, when there is one (the tests run without). @type {any} */
16 const doc = /** @type {any} */ (globalThis).document;
17 const MIMES = ["image/jpeg", "image/png", "application/octet-stream"];
18
19 /**
20 * A camera frame off the socket: the 20-byte header, then the picture.
21 * @param {ArrayBuffer} buffer
22 * @returns {{ camera: number, n: number, at: number, type: string, bytes: Uint8Array } | null}
23 */
24 export function readFrame(buffer) {
25 if (!(buffer instanceof ArrayBuffer) || buffer.byteLength < FRAME_HEADER) return null;
26 const v = new DataView(buffer);
27 if (v.getUint8(0) !== 1) return null;
28 return { camera: v.getUint32(4), n: v.getUint32(8), at: v.getFloat64(12), type: MIMES[v.getUint8(1)] ?? "application/octet-stream", bytes: new Uint8Array(buffer, FRAME_HEADER) };
29 }
30
31 /**
32 * The value below which a fraction `q` of `values` lie; null for none.
33 * @param {number[]} values
34 * @param {number} q
35 * @returns {number | null}
36 */
37 export function quantile(values, q) {
38 if (!values.length) return null;
39 const sorted = [...values].sort((a, b) => a - b);
40 return sorted[Math.min(sorted.length - 1, Math.max(0, Math.ceil(q * sorted.length) - 1))] ?? null;
41 }
42
43 /** The last `size` samples, for the page's own readout. */
44 class Recent {
45 constructor(size = 100) {
46 this.size = size;
47 /** @type {number[]} */
48 this.values = [];
49 }
50 /** @param {number} v */
51 add(v) { this.values.push(v); if (this.values.length > this.size) this.values.shift(); }
52 }
53
54 export class ArmSocket {
55 /**
56 * @param {{
57 * url: string,
58 * WebSocket?: typeof WebSocket,
59 * commandHz?: number, stallMs?: number, reportEveryMs?: number, pingEveryMs?: number, backoffMs?: number[],
60 * onPose?: (status: any, message: any) => void,
61 * onFrame?: (frame: { camera: number, n: number, at: number, type: string, bytes: Uint8Array }) => void,
62 * onDrive?: (message: { state: string, reason?: string | null, code?: number }) => void,
63 * onState?: (open: boolean) => void,
64 * }} o
65 */
66 constructor(o) {
67 this.o = { commandHz: 30, stallMs: 2000, reportEveryMs: 5000, pingEveryMs: 1000, backoffMs: [250, 500, 1000, 2000, 4000], ...o };
68 this.WS = o.WebSocket ?? globalThis.WebSocket;
69 this.ws = null;
70 this.open = false;
71 this.closed = false;
72 this.attempt = 0;
73 this.everOpen = false;
74 /** The app's clock minus ours, from the ping with the shortest round trip lately. */
75 this.offset = 0;
76 /** @type {Array<[number, number]>} */
77 this.offsets = [];
78 this.lastHeard = 0;
79 /** @type {{ cameras: number[], hz: number }} */
80 this.watching = { cameras: [], hz: 30 };
81 this.seq = 0;
82 /** @type {Record<string, number> | null} */
83 this.pending = null;
84 /** @type {ReturnType<typeof setTimeout> | undefined} */
85 this.pacer = undefined;
86 /** @type {ReturnType<typeof setTimeout> | undefined} */
87 this.pinger = undefined;
88 this.lastSentAt = -Infinity;
89 /** seq → when it went, for the commands not yet seen in a pose. @type {Map<number, number>} */
90 this.inFlight = new Map();
91 this.driving = false;
92 this.fresh();
93 this.rtt = new Recent();
94 this.age = new Recent();
95 this.shownAt = new Recent(60);
96 /** @type {Record<Counted, number>} */
97 this.totals = { sent: 0, dropped: 0, stalls: 0, reconnects: 0, frames: 0 };
98 this.connect();
99 this.watchdog = setInterval(() => this.tick(), Math.min(500, this.o.stallMs / 2));
100 this.reporter = setInterval(() => this.report(), this.o.reportEveryMs);
101 /** A pose waiting for the next browser frame, so many between two frames are drawn once. @type {any} */
102 this.poseWaiting = null;
103 // Hidden, the page watches no camera and few poses; back on screen, what it asked for.
104 this.onVisible = () => this.sendWatch();
105 doc?.addEventListener?.("visibilitychange", this.onVisible);
106 }
107
108 /** What has not been reported yet. */
109 fresh() {
110 /** @type {{ rtt: number[], age: number[] } & Record<Counted, number>} */
111 this.unreported = { rtt: [], age: [], frames: 0, sent: 0, dropped: 0, stalls: 0, reconnects: 0 };
112 }
113
114 connect() {
115 if (this.closed) return;
116 const ws = new this.WS(this.o.url);
117 ws.binaryType = "arraybuffer";
118 this.ws = ws;
119 this.lastHeard = Date.now();
120 ws.onopen = () => {
121 if (this.ws !== ws) return;
122 this.open = true;
123 this.attempt = 0;
124 if (this.everOpen) this.count("reconnects");
125 this.everOpen = true;
126 this.lastHeard = Date.now();
127 this.sendWatch();
128 this.ping();
129 this.o.onState?.(true);
130 };
131 ws.onmessage = (e) => { if (this.ws === ws) this.receive(e.data); };
132 ws.onclose = () => { if (this.ws === ws) this.lost(); };
133 ws.onerror = () => {};
134 }
135
136 lost() {
137 const was = this.open;
138 this.ws = null;
139 this.open = false;
140 this.inFlight.clear();
141 clearTimeout(this.pacer); this.pacer = undefined; this.pending = null;
142 clearTimeout(this.pinger);
143 if (this.driving) { this.driving = false; this.o.onDrive?.({ state: "closed", reason: "the connection to the app was lost; the arm holds", code: 1006 }); }
144 if (was) this.o.onState?.(false);
145 if (this.closed) return;
146 const waits = this.o.backoffMs;
147 setTimeout(() => this.connect(), waits[Math.min(this.attempt++, waits.length - 1)]);
148 }
149
150 tick() {
151 if (!this.ws || Date.now() - this.lastHeard <= this.o.stallMs) return;
152 // Quiet past the limit: count it, and start over rather than wait on a socket that may never speak again.
153 if (this.open) this.count("stalls");
154 const ws = this.ws;
155 try { ws.close(); } catch {}
156 this.lost();
157 }
158
159 /** @param {Counted} name */
160 count(name, n = 1) { this.totals[name] += n; this.unreported[name] += n; }
161
162 /** @param {Record<string, unknown>} m */
163 send(m) {
164 if (!this.open || this.ws?.readyState !== 1) return false;
165 this.ws.send(JSON.stringify(m));
166 return true;
167 }
168
169 ping() {
170 if (!this.send({ t: "ping", c: Date.now() })) return;
171 clearTimeout(this.pinger);
172 this.pinger = setTimeout(() => this.ping(), this.o.pingEveryMs);
173 }
174
175 /** @param {string | ArrayBuffer} data */
176 receive(data) {
177 this.lastHeard = Date.now();
178 if (typeof data !== "string") {
179 const frame = readFrame(data);
180 if (frame) this.o.onFrame?.(frame);
181 return;
182 }
183 let m;
184 try { m = JSON.parse(data); } catch { return; }
185 if (!m || typeof m !== "object") return;
186 if (m.t === "hello" && typeof m.now === "number") this.offset = m.now - Date.now();
187 else if (m.t === "pong" && typeof m.c === "number" && typeof m.now === "number") {
188 const rtt = Date.now() - m.c;
189 this.offsets.push([rtt, m.now - (m.c + rtt / 2)]);
190 if (this.offsets.length > 8) this.offsets.shift();
191 this.offset = this.offsets.reduce((best, o) => (o[0] < best[0] ? o : best))[1];
192 } else if (m.t === "pose" && m.status && typeof m.status === "object") {
193 if (typeof m.ack === "number") {
194 const at = this.inFlight.get(m.ack);
195 if (at !== undefined) { const ms = performance.now() - at; this.rtt.add(ms); this.unreported.rtt.push(ms); }
196 for (const seq of this.inFlight.keys()) if (seq <= m.ack) this.inFlight.delete(seq);
197 }
198 this.pose(m);
199 } else if (m.t === "drive" && typeof m.state === "string") {
200 if (m.state !== "open") this.driving = false;
201 this.o.onDrive?.(m);
202 }
203 }
204
205 /** Hand a pose to the page: at once while hidden (or with no browser frames), else once per browser frame, the latest. */
206 /** @param {any} m */
207 pose(m) {
208 const raf = /** @type {any} */ (globalThis).requestAnimationFrame;
209 if (doc?.hidden || typeof raf !== "function") return this.o.onPose?.(m.status, m);
210 const queued = this.poseWaiting !== null;
211 this.poseWaiting = m;
212 if (!queued) raf(() => { const p = this.poseWaiting; this.poseWaiting = null; if (p) this.o.onPose?.(p.status, p); });
213 }
214
215 /** The app's clock now, ms. */
216 now() { return Date.now() + this.offset; }
217
218 /** Which cameras to receive, and how many poses a second; while the page is hidden, none and one. */
219 /** @param {number[]} cameras */
220 watch(cameras, hz = this.watching.hz) {
221 this.watching = { cameras: [...cameras], hz };
222 this.sendWatch();
223 }
224 sendWatch() {
225 this.send(doc?.hidden ? { t: "watch", cameras: [], hz: 1 } : { t: "watch", ...this.watching });
226 }
227
228 /**
229 * The driver's joint targets, ticks: merged until the next slot (latest per joint wins, never queued) and sent at
230 * most `commandHz` times a second. Returns the command's number, or null when there is no socket to send it on.
231 */
232 /** @param {Record<string, number>} targets @returns {number | null} */
233 command(targets) {
234 if (!this.open) return null;
235 this.driving = true;
236 if (this.pending) this.count("dropped");
237 this.pending = { ...(this.pending ?? {}), ...targets };
238 const seq = ++this.seq;
239 if (!this.pacer) this.pacer = setTimeout(() => this.flush(), Math.max(0, this.lastSentAt + 1000 / this.o.commandHz - performance.now()));
240 return seq;
241 }
242
243 flush() {
244 clearTimeout(this.pacer);
245 this.pacer = undefined;
246 if (!this.pending) return;
247 const targets = this.pending; this.pending = null;
248 if (!this.send({ t: "cmd", seq: this.seq, targets })) return;
249 this.lastSentAt = performance.now();
250 this.inFlight.set(this.seq, this.lastSentAt);
251 if (this.inFlight.size > 64) this.inFlight.delete(/** @type {number} */ (this.inFlight.keys().next().value));
252 this.count("sent");
253 }
254
255 /** Let go: what is pending goes, then the app closes the drive and the arm holds. */
256 end() {
257 if (this.pending) this.flush();
258 this.send({ t: "end" });
259 }
260
261 /** The page put frame `n` on screen; `at` is when the app had it. Frames up to it count as seen. */
262 /** @param {number} n @param {number} at */
263 shown(n, at) {
264 const ms = Math.max(0, this.now() - at);
265 this.age.add(ms); this.unreported.age.push(ms);
266 this.shownAt.add(performance.now());
267 this.count("frames");
268 this.send({ t: "seen", n });
269 }
270
271 /** Frame `n` was not shown (its view is off, or a newer one took its place): the app may send the next. */
272 /** @param {number} n */
273 skip(n) { this.send({ t: "seen", n }); }
274
275 /**
276 * Put a frame in a pane's <img> as a blob URL: once decoded it counts as shown (and the URL it replaced is let go);
277 * one replaced before it could show is skipped. The pane keeps `blob` (its current URL) and `frameAt`.
278 * @param {{ img: any, blob?: string | null, frameAt?: number }} pane @param {{ n: number, at: number, type: string, bytes: Uint8Array }} f
279 */
280 paint(pane, f) {
281 const url = URL.createObjectURL(new Blob([/** @type {any} */ (f.bytes)], { type: f.type }));
282 pane.img.src = url;
283 pane.img.decode().catch(() => {}).then(() => {
284 if (pane.img.src === url) { if (pane.blob) URL.revokeObjectURL(pane.blob); pane.blob = url; pane.frameAt = performance.now(); this.shown(f.n, f.at); }
285 else { URL.revokeObjectURL(url); this.skip(f.n); } // a newer frame took its place first
286 });
287 }
288
289 report() {
290 const r = this.unreported;
291 if (!r.rtt.length && !r.age.length && !r.frames && !r.sent && !r.dropped && !r.stalls && !r.reconnects) return;
292 if (this.send({ t: "report", ...r })) this.fresh();
293 }
294
295 /** The page's own numbers, for its readout. */
296 stats() {
297 const recent = this.shownAt.values.filter((x) => performance.now() - x < 5000);
298 return {
299 open: this.open,
300 rttP50: quantile(this.rtt.values, 0.5), rttP95: quantile(this.rtt.values, 0.95),
301 ageP50: quantile(this.age.values, 0.5), ageP95: quantile(this.age.values, 0.95),
302 fps: recent.length / 5,
303 ...this.totals,
304 };
305 }
306
307 /** The same, as the page shows it beside "live": fps, frame age and command → pose round trip, medians. */
308 readout() {
309 const s = this.stats(), ms = (/** @type {number | null} */ v) => (v === null ? "—" : `${Math.round(v)} ms`);
310 return ` · ${s.fps.toFixed(0)} fps · frame ${ms(s.ageP50)}${s.rttP50 === null ? "" : ` · command ${ms(s.rttP50)}`}`;
311 }
312
313 close() {
314 this.closed = true;
315 clearInterval(this.watchdog); clearInterval(this.reporter); clearTimeout(this.pinger); clearTimeout(this.pacer);
316 doc?.removeEventListener?.("visibilitychange", this.onVisible);
317 const ws = this.ws; this.ws = null; this.open = false;
318 try { ws?.close(1000); } catch {}
319 }
320 }
Software/so101/src/ui/arm-socket.test.ts added +102
@@ -0,0 +1,102 @@
1 import { afterEach, describe, expect, test } from "bun:test";
2 import { createServer } from "node:http";
3 import { WebSocketServer } from "ws";
4 import { fakePi } from "../fake-pi.ts";
5 import { parseRemote, startRelay, type RunningRelay } from "../relay.ts";
6 import { frameHeader } from "../viewer-socket.ts";
7 import { ArmSocket, quantile, readFrame } from "./arm-socket.js";
8
9 const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
10 const until = async <T>(fn: () => T | undefined | false | null, ms = 3000): Promise<T> => {
11 const end = Date.now() + ms;
12 for (;;) {
13 const v = fn();
14 if (v) return v;
15 if (Date.now() > end) throw new Error("timed out");
16 await sleep(10);
17 }
18 };
19
20 const stops: Array<() => unknown> = [];
21 afterEach(async () => {
22 for (const stop of stops.splice(0).reverse()) await Promise.race([stop(), sleep(500)]);
23 });
24
25 async function boot(delayMs = 0): Promise<{ pi: Awaited<ReturnType<typeof fakePi>>; relay: RunningRelay }> {
26 const pi = await fakePi(delayMs);
27 stops.push(() => pi.close());
28 const remote = parseRemote({ ARM_SERVER: `ws://127.0.0.1:${pi.port}`, ARM_CAMERA: `http://127.0.0.1:${pi.port}/stream` })!;
29 const relay = await startRelay({ port: 0, simulation: "http://127.0.0.1:1", remote, logEveryMs: 0, viewer: { reportEveryMs: 50 } });
30 stops.push(() => relay.close());
31 return { pi, relay };
32 }
33 const socket = (o: ConstructorParameters<typeof ArmSocket>[0]) => {
34 const s = new ArmSocket(o);
35 stops.push(() => s.close());
36 return s;
37 };
38
39 describe("the page's arm socket", () => {
40 test("reads the app's frame header", () => {
41 const picture = { bytes: Buffer.from([0xff, 0xd8, 1, 2]), type: "image/jpeg", at: 1_700_000_000_123.5 } as never;
42 const h = frameHeader(2, picture);
43 h.writeUInt32BE(77, 8);
44 const buf = Buffer.concat([h, Buffer.from([0xff, 0xd8, 1, 2])]);
45 const frame = readFrame(buf.buffer.slice(buf.byteOffset, buf.byteOffset + buf.length) as ArrayBuffer)!;
46 expect(frame).toMatchObject({ camera: 2, n: 77, at: 1_700_000_000_123.5, type: "image/jpeg" });
47 expect([...frame.bytes]).toEqual([0xff, 0xd8, 1, 2]);
48 expect(readFrame(new ArrayBuffer(4))).toBeNull();
49 expect(quantile([5, 1, 3, 2, 4], 0.5)).toBe(3);
50 expect(quantile([], 0.95)).toBeNull();
51 });
52
53 test("end to end through the relay: poses, frames, latest-wins commands, and round trips that include the arm's delay", async () => {
54 const { pi, relay } = await boot(60);
55 const poses: any[] = [];
56 let shown = 0;
57 const s: ArmSocket = socket({
58 url: `ws://127.0.0.1:${relay.port}/arm`,
59 reportEveryMs: 200,
60 onPose: (status) => poses.push(status),
61 onFrame: (f) => (shown++, s.shown(f.n, f.at)),
62 });
63 await until(() => s.open);
64 s.watch([1]);
65 await until(() => poses.length >= 2);
66 // A burst of targets, faster than the command rate: only a few go, and the last one wins.
67 let last = 0;
68 for (let t = 1; t <= 30; t++) last = s.command({ 1: 2000 + t })!;
69 await until(() => pi.follow.at(-1) === JSON.stringify({ 1: 2030 }));
70 expect(pi.follow.length).toBeLessThanOrEqual(2);
71 expect(s.totals.dropped).toBe(29);
72 await until(() => poses.some((p) => p.joints[0].position === 2030) && s.rtt.values.length);
73 expect(last).toBe(30);
74 // The command reached the arm, then a /status read that began after it came back: at least the Pi's 60 ms.
75 expect(s.rtt.values[0]).toBeGreaterThanOrEqual(60);
76 await until(() => shown >= 10);
77 expect(s.stats().ageP50).toBeLessThan(500);
78 // The reports reach the app's numbers.
79 await until(() => relay.telemetry.health().commandRttMs.p50 !== null && relay.telemetry.totals.framesShown >= 10);
80 expect(relay.telemetry.health().commandRttMs.p50!).toBeGreaterThanOrEqual(60);
81 expect(relay.telemetry.totals.commandsDropped).toBe(29);
82 });
83
84 test("a socket that goes quiet is counted as a stall and opened again; a drive on it ends, holding", async () => {
85 // A server that accepts, says hello once, then never speaks.
86 let connections = 0;
87 const http = createServer();
88 const wss = new WebSocketServer({ server: http });
89 wss.on("connection", (ws) => (connections++, ws.send(JSON.stringify({ t: "hello", now: Date.now() }))));
90 const port = await new Promise<number>((r) => http.listen(0, "127.0.0.1", () => r((http.address() as { port: number }).port)));
91 stops.push(() => (wss.close(), http.closeAllConnections(), http.close()));
92 const drives: any[] = [];
93 const s = socket({ url: `ws://127.0.0.1:${port}/`, stallMs: 300, backoffMs: [50], onDrive: (m) => drives.push(m) });
94 await until(() => s.open);
95 s.command({ 1: 2000 });
96 await until(() => s.totals.reconnects >= 1, 3000);
97 expect(s.totals.stalls).toBeGreaterThanOrEqual(1);
98 expect(connections).toBeGreaterThanOrEqual(2);
99 expect(drives[0]).toMatchObject({ state: "closed" });
100 expect(s.driving).toBe(false);
101 });
102 });
Software/so101/src/ui/health.js added +98
@@ -0,0 +1,98 @@
1 // @ts-check
2 /**
3 * The admin's Health panel: GET /health (src/telemetry.ts) as one sparkline per number over the relay's two-minute
4 * history, a dashed line at each target, and the totals since the app started. Only an admin opens it, so it is not
5 * part of the page's first load: the page imports this module on the first click. Refreshed every 2 s while open,
6 * as the relay samples. Numbers reach the markup only through num(); the labels are this file's own.
7 */
8 import { sparkline } from "./sparkline.js";
9
10 /** [label, unit, the number in a /health answer, its target or null]. @type {Array<[string, string, (h: any) => unknown, number | null]>} */
11 export const HEALTH = [
12 ["command → pose p95", "ms", (h) => h.commandRttMs?.p95, 250],
13 ["camera frame age p95", "ms", (h) => h.frameAgeMs?.p95, 400],
14 ["camera fps per viewer", "fps", (h) => h.viewerFps, null],
15 ["camera fps into the app", "fps", (h) => h.cameraFpsIn, null],
16 ["arm server round trip p95", "ms", (h) => h.armRttMs?.p95, null],
17 ["commands per second", "/s", (h) => h.commandsPerS, null],
18 ["viewers", "", (h) => h.viewers, null],
19 ["bytes out", "KB/s", (h) => (typeof h.bytesOutPerS === "number" ? h.bytesOutPerS / 1024 : null), null],
20 ];
21
22 /** @param {unknown} v @returns {number | null} */
23 const num = (v) => (typeof v === "number" && Number.isFinite(v) ? v : null);
24
25 /** The sparklines' markup for a /health answer: label, line, current value, per number. @param {any} h @returns {string} */
26 export function sparks(h) {
27 const points = [...(Array.isArray(h.history) ? h.history : []), h];
28 return HEALTH.map(([label, unit, pick, target]) => {
29 const values = points.map((p) => num(pick(p))), now = values[values.length - 1] ?? null;
30 const { runs, max } = sparkline(values, { w: 120, h: 28, max: target ?? 0 });
31 const y = target === null ? null : Math.round((28 - (target / max) * 28) * 10) / 10;
32 return `<b>${label}</b><svg viewBox="0 0 120 28" aria-hidden="true">${y === null ? "" : `<line x1="0" x2="120" y1="${y}" y2="${y}"/>`}${runs.map((r) => `<polyline points="${r}"/>`).join("")}</svg>`
33 + `<output class="${target !== null && now !== null && now > target ? "over" : ""}">${now === null ? "—" : `${now < 10 ? now.toFixed(1) : Math.round(now)} ${unit}`}</output>`;
34 }).join("");
35 }
36
37 /** The totals' markup. @param {any} h @returns {string} */
38 export function totals(h) {
39 const t = h.totals ?? {};
40 return [
41 ["commands sent", t.commandsSent], ["commands dropped (latest wins)", t.commandsDropped], ["commands merged by the app", t.commandsMerged],
42 ["frames sent", t.framesSent], ["frames skipped for slow viewers", t.framesSkipped], ["frames shown", t.framesShown],
43 ["stalls", t.stalls], ["reconnects", t.reconnects], ["arm server errors", t.armErrors], ["reports dropped (too soon)", t.reportsDropped],
44 ["bytes out", num(t.bytesOut) === null ? null : `${(t.bytesOut / 1048576).toFixed(1)} MB`],
45 ].map(([k, v]) => `<b>${k}</b><span>${typeof v === "string" ? v : num(v) ?? "—"}</span>`).join("");
46 }
47
48 /** One row per number: its last two minutes as a line, a dashed target where there is one. */
49 const STYLE = `
50 .spark { display: grid; grid-template-columns: minmax(0, 1fr) 120px 9ch; align-items: center; gap: 4px 12px; font-size: 14px; margin: 8px 0; }
51 .spark b { color: var(--dim); font-weight: 500; } .spark output { text-align: right; font-variant-numeric: tabular-nums; } .spark output.over { color: var(--warn); }
52 .spark svg { width: 120px; height: 28px; overflow: visible; } .spark polyline { fill: none; stroke: var(--blue); stroke-width: 1.5; stroke-linejoin: round; }
53 .spark line { stroke: var(--warn); stroke-width: 1; stroke-dasharray: 3 3; }`;
54
55 const ARTICLE = `
56 <article>
57 <h2>Health: how the link feels</h2>
58 <p>What viewers measure over <code>WS /arm</code> and what the app sees itself, the last two minutes. Dashed lines are the targets. The same numbers are on <code>/metrics</code> for a scraper, and in the log every minute.</p>
59 <div class="spark" id="healthSparks"></div>
60 <h3>Since the app started</h3>
61 <div class="status" id="healthTotals"></div>
62 </article>`;
63
64 /** The page's document: this module only runs there (the type check has no DOM, as for src/ui/drive.js). @type {any} */
65 const doc = /** @type {any} */ (globalThis).document;
66
67 /** The panel, built once and kept. @returns {any} */
68 function panel() {
69 const found = doc.getElementById("healthPanel");
70 if (found) return found;
71 const close = Object.assign(doc.createElement("button"), { className: "close", textContent: "Close" });
72 const box = Object.assign(doc.createElement("div"), { id: "healthPanel", className: "panel", hidden: true });
73 box.append(close, Object.assign(doc.createElement("style"), { textContent: STYLE }));
74 box.insertAdjacentHTML("beforeend", ARTICLE); // fixed markup from this file, no data in it
75 close.onclick = () => (box.hidden = true);
76 (doc.getElementById("shell") ?? doc.body).append(box);
77 return box;
78 }
79
80 let refresher = /** @type {ReturnType<typeof setInterval> | undefined} */ (undefined);
81
82 /**
83 * Open the panel and keep it fresh while it is open.
84 * @param {{ api: (path: string) => Promise<any>, msg: (text: string, bad?: boolean) => void }} page
85 */
86 export async function openHealth({ api, msg }) {
87 const box = panel();
88 const show = async () => {
89 if (box.hidden && refresher !== undefined) return void (clearInterval(refresher), (refresher = undefined));
90 const h = await api("/health");
91 if (!h.ok) return msg(h.error ?? "no health from the app", true);
92 box.querySelector("#healthSparks").innerHTML = sparks(h);
93 box.querySelector("#healthTotals").innerHTML = totals(h);
94 };
95 box.hidden = false;
96 await show();
97 refresher ??= setInterval(show, 2000);
98 }
Software/so101/src/ui/metrics.js added +48
@@ -0,0 +1,48 @@
1 // @ts-check
2 /**
3 * The Metrics panel: the arm's own GET /metrics (jams, zone blocks, peaks, the recent events) drawn into the page's
4 * #metricsPanel. Opened from the Controls tab's button, so it is not part of the page's first load: the page imports
5 * this module on the first click, as it does src/ui/health.js. Event details reach the page only as text nodes.
6 */
7
8 /** The page's document: this module only runs there (the type check has no DOM, as for src/ui/drive.js). @type {any} */
9 const doc = /** @type {any} */ (globalThis).document;
10
11 const NAMES = /** @type {Record<string, string>} */ ({ 1: "shoulder_pan", 2: "shoulder_lift", 3: "elbow_flex", 4: "wrist_flex", 5: "wrist_roll", 6: "gripper" });
12
13 /** @param {string} tag @param {any} props @param {...any} children @returns {any} */
14 const el = (tag, props = {}, ...children) => {
15 const e = Object.assign(doc.createElement(tag), props);
16 e.append(...children);
17 return e;
18 };
19
20 /**
21 * The summary's [label, value] rows for a /metrics answer and this browser's teleop lag.
22 * @param {any} m @param {{ p50: number, p95: number, n: number } | null | undefined} lag
23 * @returns {Array<[string, unknown]>}
24 */
25 export function rows(m, lag) {
26 const c = m.counters ?? {};
27 return [
28 ["since", (m.since ?? "").replace("T", " ").slice(0, 19)],
29 ["jams", c.jam], ["zone blocks", c.block], ["stalls on moves", c.stall], ["lost servos", c.missing], ["hot joints", c.hot],
30 ["sessions", `${c.stream} streams · ${c.move} moves · ${c.play} playbacks · ${c.record} recordings`],
31 ["teleop lag, this browser", lag ? `p50 ${lag.p50} ms · p95 ${lag.p95} ms over ${lag.n} samples` : "—"],
32 ["closest to the table", m.closest ? `${(m.closest.z * 1000).toFixed(0)} mm (${m.closest.link.replace(/_link$/, "")}, ${m.closest.at.slice(11, 19)})` : "—"],
33 ];
34 }
35
36 /** Asks /metrics and fills the panel. @param {{ api: (path: string) => Promise<any>, lag: () => any }} page */
37 export async function showMetrics({ api, lag }) {
38 const $ = (/** @type {string} */ id) => doc.getElementById(id);
39 const m = await api("/metrics");
40 $("metricsSummary").innerHTML = rows(m, lag()).map(([k, v]) => `<b>${k}</b><span>${v}</span>`).join("");
41 $("peaks").innerHTML = "<tr><th>joint</th><th>peak load</th><th>peak temperature</th></tr>" + Object.entries(m.peaks ?? {}).map(([id, p]) => `<tr><td>${NAMES[id] ?? id}</td><td>${p.load.toFixed(0)} %</td><td>${p.temperature} °C</td></tr>`).join("");
42 // Built from text nodes: an event's detail carries people's names (who pressed the E-stop) and error strings.
43 const recent = [...(m.recent ?? [])].reverse();
44 $("events").replaceChildren(...(recent.length ? recent.map((e) => el("div", {},
45 `${String(e.t).slice(5, 19).replace("T", " ")} `, el("b", { textContent: String(e.kind) }),
46 ` ${Object.entries(e.detail ?? {}).map(([k, v]) => `${k}=${typeof v === "object" ? JSON.stringify(v) : v}`).join(" ")}`)) : [el("div", { textContent: "none yet" })]));
47 $("metricsPanel").hidden = false;
48 }
Software/so101/src/ui/metrics.test.ts added +25
@@ -0,0 +1,25 @@
1 import { describe, expect, test } from "bun:test";
2 import { rows } from "./metrics.js";
3
4 describe("the Metrics panel", () => {
5 test("names the counters, the sessions, this browser's teleop lag and the closest the arm came to the table", () => {
6 const m = {
7 since: "2026-10-20T09:15:30.123Z",
8 counters: { jam: 1, block: 2, stall: 3, missing: 0, hot: 4, stream: 5, move: 6, play: 7, record: 8 },
9 closest: { z: 0.0123, link: "gripper_link", at: "2026-10-20T09:20:01.000Z" },
10 };
11 const r = Object.fromEntries(rows(m, { p50: 40, p95: 90, n: 120 }));
12 expect(r.since).toBe("2026-10-20 09:15:30");
13 expect(r.jams).toBe(1);
14 expect(r["hot joints"]).toBe(4);
15 expect(r.sessions).toBe("5 streams · 6 moves · 7 playbacks · 8 recordings");
16 expect(r["teleop lag, this browser"]).toBe("p50 40 ms · p95 90 ms over 120 samples");
17 expect(r["closest to the table"]).toBe("12 mm (gripper, 09:20:01)");
18 });
19
20 test("without a leader or a closest point, says so with a dash", () => {
21 const r = Object.fromEntries(rows({ counters: {} }, null));
22 expect(r["teleop lag, this browser"]).toBe("—");
23 expect(r["closest to the table"]).toBe("—");
24 });
25 });
Software/so101/src/ui/sparkline.js added +29
@@ -0,0 +1,29 @@
1 // @ts-check
2 /**
3 * A sparkline's shape: recent values (oldest first, null where there was nothing to measure) as SVG polyline points
4 * in a `w` × `h` box, lowest at the bottom. The scale starts at zero, so a flat line near the top means "high",
5 * and a gap in the data is a gap in the line: one points string per unbroken run.
6 * @param {ReadonlyArray<number | null | undefined>} values
7 * @param {{ w?: number, h?: number, max?: number }} [options] `max` fixes the top of the scale (e.g. a target)
8 * @returns {{ runs: string[], max: number }}
9 */
10 export function sparkline(values, options = {}) {
11 const w = options.w ?? 120, h = options.h ?? 28;
12 const finite = values.filter((v) => typeof v === "number" && Number.isFinite(v)).map(Number);
13 const max = Math.max(options.max ?? 0, ...finite, 0) || 1;
14 const step = values.length > 1 ? w / (values.length - 1) : 0;
15 /** @type {string[]} */
16 const runs = [];
17 /** @type {string[]} */
18 let run = [];
19 values.forEach((v, i) => {
20 if (typeof v !== "number" || !Number.isFinite(v)) {
21 if (run.length) runs.push(run.join(" ")), (run = []);
22 return;
23 }
24 const x = Math.round(i * step * 10) / 10, y = Math.round((h - (Math.max(0, v) / max) * h) * 10) / 10;
25 run.push(`${x},${y}`);
26 });
27 if (run.length) runs.push(run.join(" "));
28 return { runs, max };
29 }
Software/so101/src/ui/sparkline.test.ts added +18
@@ -0,0 +1,18 @@
1 import { describe, expect, test } from "bun:test";
2 import { sparkline } from "./sparkline.js";
3
4 describe("a sparkline", () => {
5 test("values span the box from zero at the bottom to the largest at the top", () => {
6 expect(sparkline([0, 50, 100], { w: 100, h: 10 })).toEqual({ runs: ["0,10 50,5 100,0"], max: 100 });
7 });
8
9 test("a gap in the data breaks the line", () => {
10 expect(sparkline([10, null, 10, 10], { w: 30, h: 10 }).runs).toEqual(["0,0", "20,0 30,0"]);
11 });
12
13 test("a fixed top keeps small numbers small, and nothing measured is no line", () => {
14 expect(sparkline([100], { w: 10, h: 10, max: 400 })).toEqual({ runs: ["0,7.5"], max: 400 });
15 expect(sparkline([null, null]).runs).toEqual([]);
16 expect(sparkline([]).runs).toEqual([]);
17 });
18 });
Software/so101/src/viewer-socket.test.ts added +209
@@ -0,0 +1,209 @@
1 import { afterEach, describe, expect, test } from "bun:test";
2 import { mkdtempSync } from "node:fs";
3 import { tmpdir } from "node:os";
4 import { join } from "node:path";
5 import WebSocket from "ws";
6 import { fakePi } from "./fake-pi.ts";
7 import { sign } from "./identity.ts";
8 import { Lobby } from "./lobby.ts";
9 import { parseRemote, startRelay, type RunningRelay } from "./relay.ts";
10 import { FRAME_HEADER, parseTargets } from "./viewer-socket.ts";
11
12 const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
13 const KEY = "k".repeat(64);
14 const as = (name: string, role: "user" | "admin") => {
15 const identity = `id-${name}|${name}|${role}|${Date.now() + 600_000}`;
16 return { "x-forge-identity": identity, "x-forge-signature": sign(identity, KEY) };
17 };
18
19 interface Client {
20 ws: WebSocket;
21 json: Array<Record<string, any>>;
22 frames: Buffer[];
23 send(m: Record<string, unknown>): void;
24 until<T>(fn: () => T | undefined | false, ms?: number): Promise<T>;
25 }
26 async function client(port: number, headers: Record<string, string> = {}, onFrame?: (c: Client, frame: Buffer) => void): Promise<Client> {
27 const ws = new WebSocket(`ws://127.0.0.1:${port}/arm`, { headers });
28 const c: Client = {
29 ws,
30 json: [],
31 frames: [],
32 send: (m) => ws.send(JSON.stringify(m)),
33 async until(fn, ms = 3000) {
34 const end = Date.now() + ms;
35 for (;;) {
36 const v = fn();
37 if (v) return v;
38 if (Date.now() > end) throw new Error("timed out");
39 await sleep(10);
40 }
41 },
42 };
43 ws.on("message", (data, binary) => {
44 if (binary) {
45 const b = data as Buffer;
46 c.frames.push(b);
47 onFrame?.(c, b);
48 } else c.json.push(JSON.parse(String(data)));
49 });
50 await new Promise((r, j) => (ws.once("open", r), ws.once("error", j)));
51 return c;
52 }
53
54 let pi: Awaited<ReturnType<typeof fakePi>> | undefined;
55 let relay: RunningRelay | undefined;
56 const clients: Client[] = [];
57 let lobby: Lobby | undefined;
58 afterEach(async () => {
59 for (const c of clients.splice(0)) c.ws.terminate();
60 await Promise.race([relay?.close(), sleep(500)]);
61 lobby?.close();
62 pi?.close();
63 relay = pi = lobby = undefined;
64 });
65
66 async function boot(delayMs = 0, withLobby = false) {
67 pi = await fakePi(delayMs);
68 const remote = parseRemote({ ARM_SERVER: `ws://127.0.0.1:${pi.port}`, ARM_CAMERA: `http://127.0.0.1:${pi.port}/stream` })!;
69 if (withLobby) lobby = new Lobby({ identityKey: KEY, adminFile: join(mkdtempSync(join(tmpdir(), "so101-vs-")), "admin.json") });
70 relay = await startRelay({ port: 0, simulation: "http://127.0.0.1:1", remote, lobby, logEveryMs: 0, viewer: { seenTimeoutMs: 10_000 } });
71 return relay.port;
72 }
73 const open = async (...args: Parameters<typeof client>) => {
74 const c = await client(...args);
75 clients.push(c);
76 return c;
77 };
78
79 describe("WS /arm", () => {
80 test("the arm's pose comes in many times a second, from the one reader", async () => {
81 const port = await boot();
82 const c = await open(port);
83 await sleep(600);
84 const poses = c.json.filter((m) => m.t === "pose");
85 expect(c.json[0]).toMatchObject({ t: "hello" });
86 expect(poses.length).toBeGreaterThan(8);
87 expect(poses.at(-1)!.status).toMatchObject({ joints: [{ id: 1, position: 2048 }], source: `live 127.0.0.1:${pi!.port}` });
88 });
89
90 test("commands are latest-wins: a burst reaches the arm as a few frames ending on the last, and the pose acknowledges it", async () => {
91 const port = await boot();
92 const c = await open(port);
93 for (let seq = 1; seq <= 40; seq++) c.send({ t: "cmd", seq, targets: { 1: 2000 + seq } });
94 await c.until(() => pi!.follow.at(-1) === JSON.stringify({ 1: 2040 }));
95 expect(pi!.follow.length).toBeLessThanOrEqual(4);
96 await c.until(() => c.json.some((m) => m.t === "pose" && m.ack === 40 && m.status.joints[0].position === 2040));
97 expect(relay!.telemetry.totals.commandsMerged).toBeGreaterThan(30);
98 });
99
100 test("a slow viewer gets fewer frames without slowing the others", async () => {
101 const port = await boot();
102 // The fast one says it has shown each frame at once; the slow one never does.
103 const fast = await open(port, {}, (c, frame) => c.send({ t: "seen", n: frame.readUInt32BE(8) }));
104 const slow = await open(port);
105 fast.send({ t: "watch", cameras: [1] });
106 slow.send({ t: "watch", cameras: [1] });
107 await sleep(1200);
108 expect(fast.frames.length).toBeGreaterThan(20);
109 expect(slow.frames.length).toBe(4); // the frame window, then nothing until it catches up
110 const f = fast.frames.at(-1)!;
111 expect(f.readUInt8(0)).toBe(1);
112 expect(f.readUInt32BE(4)).toBe(1);
113 expect(Date.now() - f.readDoubleBE(12)).toBeLessThan(1000);
114 expect(f.subarray(FRAME_HEADER, FRAME_HEADER + 2)).toEqual(Buffer.from([0xff, 0xd8]));
115 expect(relay!.telemetry.totals.framesSkipped).toBeGreaterThan(10);
116 // Once it shows what it has, it gets the newest frame, not the backlog.
117 slow.send({ t: "seen", n: 4 });
118 await slow.until(() => slow.frames.length > 4);
119 expect(Date.now() - slow.frames[4]!.readDoubleBE(12)).toBeLessThan(200);
120 });
121
122 test("the metrics report the delays: the arm's round trip and what viewers measured", async () => {
123 const port = await boot(80);
124 const c = await open(port, {}, (c, frame) => c.send({ t: "seen", n: frame.readUInt32BE(8) }));
125 c.send({ t: "watch", cameras: [1] });
126 for (let seq = 1; seq <= 12; seq++) c.send({ t: "cmd", seq, targets: { 1: 2000 + seq } });
127 await c.until(() => c.frames.length >= 10 && c.json.filter((m) => m.t === "pose").length >= 3);
128 c.send({ t: "report", rtt: [120, 130, 140, 900], age: [210, 220], frames: 10, sent: 12, dropped: 3, stalls: 1, reconnects: 1 });
129 await sleep(100);
130 const health = (await (await fetch(`http://127.0.0.1:${port}/health`)).json()) as Record<string, any>;
131 expect(health.armRttMs.p50).toBeGreaterThanOrEqual(80);
132 expect(health.commandRttMs).toEqual({ p50: 130, p95: 900 });
133 expect(health.frameAgeMs.p50).toBe(210);
134 expect(health.viewers).toBe(1);
135 expect(health.totals).toMatchObject({ commandsSent: 12, commandsDropped: 3, stalls: 1, reconnects: 1, framesShown: 10 });
136 const text = await (await fetch(`http://127.0.0.1:${port}/metrics`)).text();
137 expect(text).toContain('arm_command_rtt_ms{quantile="0.5"} 130');
138 expect(text).toContain('arm_camera_frame_age_ms{quantile="0.95"} 220');
139 expect(text).toMatch(/arm_server_rtt_ms\{quantile="0.5"\} (8\d|9\d|1\d\d)/);
140 expect(text).toContain("arm_viewers 1");
141 });
142
143 test("a burst of reports moves the numbers once; each is held to what crossed that socket", async () => {
144 const port = await boot();
145 const c = await open(port);
146 c.send({ t: "cmd", seq: 1, targets: { 1: 1500 } });
147 c.send({ t: "cmd", seq: 2, targets: { 1: 1501 } });
148 await c.until(() => c.json.some((m) => m.t === "drive"));
149 const forged = { t: "report", rtt: Array(200).fill(0), age: Array(200).fill(0), frames: 100_000, sent: 100_000, dropped: 5, stalls: 0, reconnects: 0 };
150 for (let i = 0; i < 50; i++) c.send(forged);
151 await c.until(() => relay!.telemetry.totals.reportsDropped === 49);
152 // Two commands crossed the socket and no frames: that is all one report may claim.
153 expect(relay!.telemetry.totals).toMatchObject({ commandsSent: 2, commandsDropped: 5, framesShown: 0, reportsDropped: 49 });
154 expect(relay!.telemetry.rtt.count()).toBe(2);
155 expect(relay!.telemetry.age.count()).toBe(0);
156 // A second socket is its own: one more report from it is taken, not dropped.
157 const d = await open(port);
158 d.send(forged);
159 await sleep(100);
160 expect(relay!.telemetry.totals).toMatchObject({ commandsDropped: 10, reportsDropped: 49 });
161 expect(await (await fetch(`http://127.0.0.1:${port}/metrics`)).text()).toContain("arm_reports_dropped_total 49");
162 });
163
164 test("only the driver's commands reach the arm", async () => {
165 const port = await boot(0, true);
166 const c = await open(port, as("eve", "user"));
167 c.send({ t: "cmd", seq: 1, targets: { 1: 1000 } });
168 const refused = await c.until(() => c.json.find((m) => m.t === "drive"));
169 expect(refused).toMatchObject({ state: "refused" });
170 await sleep(100);
171 expect(pi!.follow).toEqual([]);
172 });
173
174 test("with no lobby (a run on one's own machine) every viewer drives, as on /follow", async () => {
175 const port = await boot();
176 const c = await open(port);
177 c.send({ t: "cmd", seq: 1, targets: { 1: 1500 } });
178 await c.until(() => pi!.follow.includes(JSON.stringify({ 1: 1500 })));
179 expect(c.json.find((m) => m.t === "drive")).toMatchObject({ state: "open" });
180 // /follow, the old path, lets the same anonymous page drive.
181 const follow = new WebSocket(`ws://127.0.0.1:${port}/follow`);
182 await new Promise((r, j) => (follow.once("open", r), follow.once("error", j)));
183 follow.send(JSON.stringify({ 1: 1600 }));
184 await c.until(() => pi!.follow.includes(JSON.stringify({ 1: 1600 })));
185 follow.terminate();
186 });
187
188 test("/health is the admin's; /metrics stays open to a scraper", async () => {
189 const port = await boot(0, true);
190 const health = (headers: Record<string, string> = {}) => fetch(`http://127.0.0.1:${port}/health`, { headers });
191 expect((await health()).status).toBe(401);
192 const refused = await health(as("eve", "user"));
193 expect(refused.status).toBe(403);
194 expect(await refused.json()).toEqual({ error: "only an admin can see the link's health" });
195 const ok = await health(as("trevato", "admin"));
196 expect(ok.status).toBe(200);
197 expect(await ok.json()).toMatchObject({ viewers: 0, history: [] });
198 const metrics = await fetch(`http://127.0.0.1:${port}/metrics`);
199 expect(metrics.status).toBe(200);
200 expect(await metrics.text()).toContain("arm_viewers 0");
201 });
202
203 test("targets are checked at the door", () => {
204 expect(parseTargets({ 1: 2048.4, 6: 3000 })).toEqual(new Map([[1, 2048], [6, 3000]]));
205 expect(parseTargets({ 0: 1, 254: 1, 2: -1, 3: 5000, x: 1, 4: "1" })).toBeNull();
206 expect(parseTargets([1, 2])).toBeNull();
207 expect(parseTargets(null)).toBeNull();
208 });
209 });
Software/so101/src/viewer-socket.ts added +400
@@ -0,0 +1,400 @@
1 /**
2 * One WebSocket per viewer, WS /arm: everything that has to feel live goes over it, so nothing pays a public round
3 * trip per request (through the forge's front door that is ~0.4 s each; a socket pays it once).
4 *
5 * The app sends:
6 * {"t":"hello","now"} once; "now" is the app's clock, for the page's frame ages
7 * {"t":"pose","now","askedAt","ack","status"} the arm's /status, up to 30 times a second (src/pose-feed.ts);
8 * "ack" is this viewer's last command the arm had when it was read
9 * {"t":"drive","state":"open"|"closed"|"refused","code"?,"reason"?}
10 * {"t":"pong","c","now"} the answer to a ping, for the clock offset
11 * binary: a camera frame, a 20-byte header then the picture (frameHeader below)
12 * The page sends:
13 * {"t":"watch","cameras":[1],"hz"?} which cameras to send, and how many poses a second (1..30)
14 * {"t":"cmd","seq","targets":{"1":2048}} the driver's joint targets, ticks; merged to the command rate,
15 * latest per joint winning, then sent on the arm's own /follow
16 * {"t":"end"} let go: the stream to the arm closes shortly after, and it holds
17 * {"t":"seen","n"} frames up to n are shown; until then at most `frameWindow` are out
18 * {"t":"report",...} what the page measured (src/telemetry.ts parseReport); one per
19 * socket every `reportEveryMs`, held to what crossed it (vouch)
20 * {"t":"ping","c"}
21 *
22 * A viewer still busy with earlier frames gets the newest one next, never a queue (src/latest.ts): a slow phone sees
23 * fewer frames, everyone else is not slowed. A viewer whose socket is lost leaves the arm parking, like /follow.
24 */
25
26 import type { IncomingMessage } from "node:http";
27 import type { Duplex } from "node:stream";
28 import WebSocket, { WebSocketServer } from "ws";
29 import type { Picture } from "./camera-feed.ts";
30 import type { CameraHub } from "./cameras.ts";
31 import type { Who } from "./identity.ts";
32 import { Latest } from "./latest.ts";
33 import type { Lobby } from "./lobby.ts";
34 import type { Pose, PoseFeed } from "./pose-feed.ts";
35 import { earn, noCredit, parseReport, vouch, type Credit, type Telemetry } from "./telemetry.ts";
36
37 export interface ViewerSocketOptions {
38 poses: PoseFeed;
39 cameras: CameraHub;
40 telemetry: Telemetry;
41 lobby: Lobby | null;
42 /** The arm's /follow stream, live or simulated, and the HTTP base it belongs to. */
43 openFollow: () => Promise<{ arm: WebSocket; base: string }>;
44 /** A fresh speed cap for one drive: the frame as it may go to the arm, or null for none. */
45 capper: (base: string) => (frame: string) => string | null;
46 /** The close code that tells the arm server its driver was lost: it parks. */
47 lostCode: number;
48 commandHz?: number;
49 /** How long after "end" (with no new command) the stream to the arm closes. */
50 endAfterMs?: number;
51 /** How often viewers are pinged; one that misses a ping is lost. */
52 heartbeatMs?: number;
53 /** Frames out to one viewer and not yet seen. */
54 frameWindow?: number;
55 /** How long an unseen frame holds its place before it counts as lost. */
56 seenTimeoutMs?: number;
57 /** One socket's reports are taken at most this often; the page sends one every 5 s, the rest are dropped and counted. */
58 reportEveryMs?: number;
59 }
60
61 const MAX_MESSAGE = 16_384;
62 const MAX_CAMERAS_WATCHED = 8;
63 /** A pose waits while this much is still unsent to a viewer: the next one is fresher anyway. */
64 const POSE_BACKLOG = 256 * 1024;
65 const MIME = ["image/jpeg", "image/png"];
66 export const FRAME_HEADER = 20;
67
68 /** kind 1 (u8), mime (u8: 0 jpeg, 1 png, 2 other), 2 bytes zero, camera id (u32), frame number (u32), the app's receipt time (f64 ms). */
69 export function frameHeader(camera: number, picture: Picture): Buffer {
70 const h = Buffer.alloc(FRAME_HEADER);
71 h.writeUInt8(1, 0);
72 const mime = MIME.indexOf(picture.type.toLowerCase());
73 h.writeUInt8(mime < 0 ? 2 : mime, 1);
74 h.writeUInt32BE(camera, 4);
75 h.writeDoubleBE(picture.at, 12);
76 return h;
77 }
78
79 /** A command's targets, checked: joint ids 1..253 and ticks 0..4095; null when there is nothing usable. */
80 export function parseTargets(raw: unknown): Map<number, number> | null {
81 if (!raw || typeof raw !== "object" || Array.isArray(raw)) return null;
82 const out = new Map<number, number>();
83 for (const [k, v] of Object.entries(raw as Record<string, unknown>)) {
84 if (!/^\d{1,3}$/.test(k) || typeof v !== "number" || !Number.isFinite(v)) continue;
85 const id = Number(k);
86 if (id < 1 || id > 253 || v < 0 || v > 4095) continue;
87 out.set(id, Math.round(v));
88 }
89 return out.size ? out : null;
90 }
91
92 interface Drive {
93 arm: WebSocket | null;
94 base: string;
95 cap: ((frame: string) => string | null) | null;
96 pending: Map<number, number> | null;
97 pendingSeq: number;
98 /** Commands sent to the arm: [seq, when], the last few. */
99 sent: Array<[number, number]>;
100 lastSentAt: number;
101 pacer: ReturnType<typeof setTimeout> | undefined;
102 ender: ReturnType<typeof setTimeout> | undefined;
103 }
104
105 interface Viewer {
106 ws: WebSocket;
107 who: Who;
108 req: IncomingMessage;
109 watching: Map<number, () => void>;
110 frames: Latest;
111 frameNo: number;
112 unseen: Map<number, { done: () => void; timer: ReturnType<typeof setTimeout> }>;
113 poseEveryMs: number;
114 lastPoseAt: number;
115 drive: Drive | null;
116 /** The last frame number skipped so far, for the telemetry's count. */
117 skippedSeen: number;
118 /** When this socket's last report was taken, and what it may still claim: a report is held to that (telemetry vouch). */
119 lastReportAt: number;
120 credit: Credit;
121 stopPoses: () => void;
122 }
123
124 export class ViewerSockets {
125 private readonly wss = new WebSocketServer({ noServer: true, maxPayload: MAX_MESSAGE });
126 private readonly viewers = new Set<Viewer>();
127 private readonly heartbeat: ReturnType<typeof setInterval>;
128 private readonly o: Required<Omit<ViewerSocketOptions, "lobby">> & { lobby: Lobby | null };
129
130 constructor(options: ViewerSocketOptions) {
131 this.o = { commandHz: 30, endAfterMs: 250, heartbeatMs: 1000, frameWindow: 4, seenTimeoutMs: 2000, reportEveryMs: 4000, ...options };
132 const answered = new WeakSet<WebSocket>();
133 this.heartbeat = setInterval(() => {
134 for (const v of this.viewers) {
135 if (!answered.has(v.ws)) {
136 v.ws.terminate();
137 continue;
138 }
139 answered.delete(v.ws);
140 if (v.ws.readyState === WebSocket.OPEN) v.ws.ping();
141 }
142 }, this.o.heartbeatMs);
143 this.heartbeat.unref?.();
144 this.wss.on("connection", (ws: WebSocket) => {
145 answered.add(ws);
146 ws.on("pong", () => answered.add(ws));
147 });
148 }
149
150 get count(): number {
151 return this.viewers.size;
152 }
153
154 /** Viewers watching at least one camera. */
155 get watching(): number {
156 let n = 0;
157 for (const v of this.viewers) if (v.watching.size) n++;
158 return n;
159 }
160
161 upgrade(req: IncomingMessage, socket: Duplex, head: Buffer): void {
162 const who = this.o.lobby ? this.o.lobby.who(req) : null;
163 this.wss.handleUpgrade(req, socket, head, (ws) => {
164 this.wss.emit("connection", ws, req);
165 this.join(ws, who, req);
166 });
167 }
168
169 /** A turn ended: that person's drive closes (the arm holds) and the page is told why. */
170 endTurn(id: string): void {
171 for (const v of this.viewers) if (v.who?.id === id && v.drive) this.stopDrive(v, 1000, "your turn ended");
172 }
173
174 close(): void {
175 clearInterval(this.heartbeat);
176 for (const v of this.viewers) v.ws.terminate();
177 this.wss.close();
178 }
179
180 // ------------------------------------------------------------------ one viewer
181 private send(v: Viewer, data: string | Buffer) {
182 if (v.ws.readyState !== WebSocket.OPEN) return false;
183 v.ws.send(data, { binary: typeof data !== "string" });
184 this.o.telemetry.count("bytesOut", typeof data === "string" ? Buffer.byteLength(data) : data.length);
185 return true;
186 }
187
188 private json(v: Viewer, message: Record<string, unknown>) {
189 this.send(v, JSON.stringify(message));
190 }
191
192 private join(ws: WebSocket, who: Who, req: IncomingMessage) {
193 const v: Viewer = {
194 ws,
195 who,
196 req,
197 watching: new Map(),
198 frameNo: 0,
199 unseen: new Map(),
200 poseEveryMs: 0,
201 lastPoseAt: 0,
202 drive: null,
203 skippedSeen: 0,
204 lastReportAt: -Infinity,
205 credit: noCredit(),
206 stopPoses: () => {},
207 frames: undefined as unknown as Latest,
208 };
209 v.frames = new Latest((data, done) => {
210 if (ws.readyState !== WebSocket.OPEN || typeof data === "string") return false;
211 const n = ++v.frameNo;
212 data.writeUInt32BE(n, 8);
213 v.unseen.set(n, { done, timer: setTimeout(() => this.seen(v, n), this.o.seenTimeoutMs) });
214 this.o.telemetry.count("framesSent");
215 v.credit = earn(v.credit, "frames");
216 return this.send(v, data);
217 }, this.o.frameWindow);
218 this.viewers.add(v);
219 this.json(v, { t: "hello", now: Date.now(), frameWindow: this.o.frameWindow });
220 v.stopPoses = this.o.poses.listen((pose) => this.pose(v, pose));
221 ws.on("message", (data, binary) => {
222 if (binary) return;
223 let m: Record<string, unknown>;
224 try {
225 m = JSON.parse(String(data));
226 } catch {
227 return;
228 }
229 if (m && typeof m === "object") this.receive(v, m);
230 });
231 ws.on("close", (code) => this.leave(v, code));
232 ws.on("error", () => ws.terminate());
233 }
234
235 private leave(v: Viewer, code: number) {
236 if (!this.viewers.delete(v)) return;
237 v.stopPoses();
238 for (const stop of v.watching.values()) stop();
239 v.watching.clear();
240 for (const { timer } of v.unseen.values()) clearTimeout(timer);
241 v.unseen.clear();
242 // A page closed on purpose leaves the arm holding; one that vanished leaves it parking.
243 if (v.drive) this.stopDrive(v, code === 1000 || code === 1001 || code === 1005 ? 1000 : this.o.lostCode, "the driver's connection was lost");
244 }
245
246 private pose(v: Viewer, pose: Pose) {
247 const now = Date.now();
248 if (now - v.lastPoseAt < v.poseEveryMs || v.ws.bufferedAmount > POSE_BACKLOG) return;
249 v.lastPoseAt = now;
250 let ack: number | null = null;
251 for (const [seq, at] of v.drive?.sent ?? []) if (at <= pose.askedAt) ack = seq;
252 this.json(v, { t: "pose", now, askedAt: pose.askedAt, ack, status: pose.status });
253 }
254
255 private seen(v: Viewer, upTo: number) {
256 for (const [n, entry] of v.unseen) {
257 if (n > upTo) break;
258 clearTimeout(entry.timer);
259 v.unseen.delete(n);
260 entry.done();
261 }
262 }
263
264 private receive(v: Viewer, m: Record<string, unknown>) {
265 switch (m.t) {
266 case "ping":
267 return this.json(v, { t: "pong", c: typeof m.c === "number" && Number.isFinite(m.c) ? m.c : null, now: Date.now() });
268 case "seen":
269 if (typeof m.n === "number" && Number.isSafeInteger(m.n)) this.seen(v, m.n);
270 return;
271 case "report":
272 return this.report(v, m);
273 case "watch":
274 return this.watch(v, m.cameras, m.hz);
275 case "cmd":
276 return this.command(v, m.seq, m.targets);
277 case "end":
278 return this.release(v);
279 }
280 }
281
282 /** Keyed on the socket, never on anything the page wrote: one report per `reportEveryMs`, held to what crossed it. */
283 private report(v: Viewer, m: Record<string, unknown>) {
284 const now = Date.now();
285 if (now - v.lastReportAt < this.o.reportEveryMs) return this.o.telemetry.count("reportsDropped");
286 const r = parseReport(m);
287 if (!r) return;
288 v.lastReportAt = now;
289 const { report, left } = vouch(r, v.credit);
290 v.credit = left;
291 this.o.telemetry.report(report, now);
292 }
293
294 private watch(v: Viewer, cameras: unknown, hz: unknown) {
295 if (typeof hz === "number" && Number.isFinite(hz)) v.poseEveryMs = 1000 / Math.max(1, Math.min(30, hz)) - 5;
296 if (!Array.isArray(cameras)) return;
297 const wanted = new Set(cameras.filter((id): id is number => Number.isSafeInteger(id) && id > 0).slice(0, MAX_CAMERAS_WATCHED));
298 for (const [id, stop] of v.watching) if (!wanted.has(id)) stop(), v.watching.delete(id);
299 for (const id of wanted) {
300 if (v.watching.has(id)) continue;
301 const feed = this.o.cameras.feed(id);
302 if (!feed) continue;
303 v.watching.set(
304 id,
305 feed.pictures((picture) => {
306 v.frames.offer(String(id), Buffer.concat([frameHeader(id, picture), picture.bytes]));
307 if (v.frames.skipped > v.skippedSeen) this.o.telemetry.count("framesSkipped", v.frames.skipped - v.skippedSeen), (v.skippedSeen = v.frames.skipped);
308 }),
309 );
310 }
311 }
312
313 // ------------------------------------------------------------------ driving
314 private refusal(v: Viewer): string | null {
315 const lobby = this.o.lobby;
316 if (!lobby) return null;
317 return lobby.refusal(v.who, v.req)?.error ?? null;
318 }
319
320 private command(v: Viewer, seq: unknown, raw: unknown) {
321 const targets = parseTargets(raw);
322 if (!targets) return;
323 v.credit = earn(v.credit, "command");
324 const why = this.refusal(v);
325 if (why) {
326 if (v.drive) this.stopDrive(v, 1000, why);
327 return this.json(v, { t: "drive", state: "refused", reason: why });
328 }
329 let d = v.drive;
330 if (!d) {
331 d = v.drive = { arm: null, base: "", cap: null, pending: null, pendingSeq: 0, sent: [], lastSentAt: 0, pacer: undefined, ender: undefined };
332 const drive = d;
333 this.o.openFollow().then(
334 ({ arm, base }) => {
335 if (v.drive !== drive) return void arm.close(1000);
336 drive.arm = arm;
337 drive.base = base;
338 drive.cap = this.o.capper(base);
339 arm.on("close", (code, why) => {
340 if (v.drive !== drive) return;
341 this.stopDrive(v, null, why.length ? why.toString() : "the arm server closed the stream", code);
342 });
343 arm.on("error", () => arm.terminate());
344 this.json(v, { t: "drive", state: "open" });
345 this.pace(v, drive);
346 },
347 (err: unknown) => {
348 if (v.drive !== drive) return;
349 this.stopDrive(v, null, `the arm did not answer (${(err as Error)?.message ?? String(err)})`);
350 },
351 );
352 }
353 clearTimeout(d.ender);
354 d.ender = undefined;
355 if (d.pending) this.o.telemetry.count("commandsMerged");
356 d.pending = new Map([...(d.pending ?? []), ...targets]);
357 if (typeof seq === "number" && Number.isSafeInteger(seq)) d.pendingSeq = seq;
358 this.pace(v, d);
359 }
360
361 /** Send what is pending once the command rate allows: one frame per slot, the latest per joint. */
362 private pace(v: Viewer, d: Drive) {
363 if (d.pacer || !d.pending || !d.arm) return;
364 const wait = Math.max(0, d.lastSentAt + 1000 / this.o.commandHz - Date.now());
365 d.pacer = setTimeout(() => {
366 d.pacer = undefined;
367 if (v.drive !== d || !d.pending || !d.arm || d.arm.readyState !== WebSocket.OPEN) return;
368 const frame = JSON.stringify(Object.fromEntries(d.pending));
369 d.pending = null;
370 d.lastSentAt = Date.now();
371 const capped = d.cap ? d.cap(frame) : frame;
372 if (capped !== null) d.arm.send(capped);
373 d.sent.push([d.pendingSeq, d.lastSentAt]);
374 if (d.sent.length > 64) d.sent.shift();
375 }, wait);
376 }
377
378 private release(v: Viewer) {
379 const d = v.drive;
380 if (!d) return;
381 clearTimeout(d.ender);
382 d.ender = setTimeout(() => {
383 if (v.drive === d && !d.pending) this.stopDrive(v, 1000, null);
384 else if (v.drive === d) this.release(v); // the last frame has yet to go
385 }, this.o.endAfterMs);
386 }
387
388 /** Close the drive: toward the arm with `armCode` (null when the arm closed it), and tell the page. */
389 private stopDrive(v: Viewer, armCode: number | null, reason: string | null, code = 1000) {
390 const d = v.drive;
391 if (!d) return;
392 v.drive = null;
393 clearTimeout(d.pacer);
394 clearTimeout(d.ender);
395 if (armCode !== null) {
396 if (d.arm) d.arm.close(armCode, armCode === 1000 ? undefined : reason ?? undefined);
397 }
398 this.json(v, { t: "drive", state: "closed", code, reason });
399 }
400 }
Software/so101/src/viewer.html +70 −36
@@ -12,9 +12,9 @@
1212 <title>SO-101</title>
1313 <script type="importmap">
1414 { "imports": {
15 "three": "/vendor/three/build/three.module.js",
16 "three/examples/jsm/": "/vendor/three/examples/jsm/",
17 "urdf-loader/manipulator": "/vendor/urdf-loader/src/urdf-manipulator-element.js"
15 "three": "/vendor/__VENDOR__/three/build/three.module.js",
16 "three/examples/jsm/": "/vendor/__VENDOR__/three/examples/jsm/",
17 "urdf-loader/manipulator": "/vendor/__VENDOR__/urdf-loader/src/urdf-manipulator-element.js"
1818 } }
1919 </script>
2020 <link rel="modulepreload" href="/ui/__VERSION__/camera-plan.js">
@@ -441,7 +441,7 @@
441441 <h3>Joint limits, degrees (empty: the calibrated range)</h3>
442442 <div class="trim" id="jointLimits"></div>
443443 </div>
444 <div class="row"><button id="metricsBtn">Metrics</button><button id="calib">3D calibration</button></div>
444 <div class="row"><button id="metricsBtn">Metrics</button><button id="healthBtn" hidden>Health</button><button id="calib">3D calibration</button></div>
445445 </div>
446446 <div class="tab" id="tab-machine" role="tabpanel" aria-labelledby="t-machine" hidden>
447447 <p class="note">The arm as its servos report it, and every command on its way in and out, pushed as it happens. Read-only.</p>
@@ -526,7 +526,7 @@
526526 clearTimeout(msgTimer); if (text) msgTimer = setTimeout(() => ($("msg").textContent = ""), bad ? 9000 : 5000);
527527 };
528528 const api = async (path, body, method = body === undefined ? "GET" : "POST") => {
529 const r = await fetch(path, { method, body: body === undefined ? undefined : JSON.stringify(body), cache: "no-store" });
529 const r = await fetch(path, { method, body: body === undefined ? undefined : JSON.stringify(body), cache: "no-store", headers: { accept: "application/json" } });
530530 const data = await r.json().catch(() => ({}));
531531 return Object.defineProperties(data, { ok: { value: r.ok }, status: { value: r.status } });
532532 };
@@ -596,12 +596,39 @@
596596 // Another arm has other limits, poses and joints: start the page over on it, once nothing on the page is driving.
597597 // So does a page that opened while reconnecting and never got the arm's limits.
598598 const other = linkMode !== null && realArm(l.mode) !== realArm(linkMode);
599 if ((other || (l.mode === "live" && !limits?.ok)) && linkMode !== null && !stream.ws && !leader) return location.reload();
599 if ((other || (l.mode === "live" && !limits?.ok)) && linkMode !== null && !stream.ws && !sock?.driving && !leader) return location.reload();
600600 if (linkMode === null) linkMode = l.mode;
601601 }
602602 setTimeout(pollLink, l?.mode === "live" ? 3000 : 1000);
603603 }
604604
605 // ---------------------------------------------------------------- the link: WS /arm (src/ui/arm-socket.js)
606 // Behind the relay one socket carries the pose, the driver's commands and the camera frames. Until it is up (its module
607 // comes after the first view) and while it is down: /status polled, /follow for a drive, the cameras as MJPEG.
608 let sockUp = false, sock = null;
609 const watchCams = () => sock?.watch(cams.filter(paneOn).map((c) => c.id));
610 async function openSock() {
611 if (!relayed || !("WebSocket" in window)) return;
612 const { ArmSocket } = await import("/ui/__VERSION__/arm-socket.js");
613 sock = new ArmSocket({
614 url: `${location.protocol === "https:" ? "wss" : "ws"}://${location.host}/arm`,
615 onState(open) {
616 sockUp = open;
617 for (const c of cams) if (paneOn(c)) showCam(c, true);
618 },
619 onPose: (d) => applyStatus(d), // at most once a browser frame
620 onFrame: (f) => showFrame(f),
621 onDrive(m) {
622 if (m.state === "refused" || (m.state === "closed" && m.reason)) { stream.halted = true; msg(m.reason || "the arm is busy", true); }
623 },
624 });
625 }
626 function showFrame(f) {
627 const c = cams.find((x) => x.id === f.camera);
628 if (!c || !paneOn(c) || !sockUp) return void sock.skip(f.n);
629 sock.paint(c, f);
630 }
631
605632 // ---------------------------------------------------------------- cameras
606633 // The owner lists the cameras (/admin/camera-list); this page polls the list, so a camera added or removed shows within seconds.
607634 // Which views are on is this viewer's own choice, kept in this browser: one at a time on a phone, several side by side on a wider screen.
@@ -651,8 +678,10 @@
651678 c.pane.classList.toggle("on", on); c.btn.classList.toggle("on", on); c.btn.setAttribute("aria-pressed", String(on));
652679 clearTimeout(c.timer);
653680 if (!on || document.hidden) c.img.src = "";
681 else if (sockUp) { if (!c.img.src.startsWith("blob:")) c.img.src = c.blob || ""; } // frames come over the socket
654682 else if (plan.mode === "snapshot" && c.snapshot) snap(c);
655683 else c.img.src = streamUrl(c);
684 watchCams();
656685 };
657686 function setPane(key, on) {
658687 if (key === "pane3") {
@@ -692,7 +721,7 @@
692721 c.tag = el("span", { className: "tag" });
693722 c.pane = el("div", { className: "pane" }, c.img, c.tag); c.pane.dataset.key = `cam:${info.id}`;
694723 c.btn = el("button", { type: "button" }); c.btn.setAttribute("aria-pressed", "false");
695 c.img.onerror = () => { if (paneOn(c) && plan.mode === "stream" && c.img.getAttribute("src")) setTimeout(() => { if (paneOn(c) && plan.mode === "stream" && !document.hidden) c.img.src = streamUrl(c); }, 3000); };
724 c.img.onerror = () => { if (paneOn(c) && plan.mode === "stream" && !sockUp && c.img.getAttribute("src")) setTimeout(() => { if (paneOn(c) && plan.mode === "stream" && !sockUp && !document.hidden) c.img.src = streamUrl(c); }, 3000); };
696725 c.btn.onclick = () => pick(c.pane.dataset.key);
697726 $("stage").insertBefore(c.pane, $("pane3"));
698727 $("top").querySelector(".seg").insertBefore(c.btn, $("v3"));
@@ -760,6 +789,7 @@
760789 document.addEventListener("visibilitychange", () => { for (const c of cams) if (paneOn(c)) showCam(c, true); });
761790 setInterval(() => {
762791 for (const c of cams) {
792 if (paneOn(c) && sockUp) { c.live = performance.now() - (c.frameAt ?? -Infinity) < 2000; continue; }
763793 if (!paneOn(c) || (plan.mode === "snapshot" && c.snapshot)) continue;
764794 let changed = false;
765795 try {
@@ -772,7 +802,7 @@
772802 if (c.unchanged >= Math.max(15, plan.fps ? 30 / plan.fps : 0) && !document.hidden) { c.img.src = streamUrl(c); c.unchanged = 0; }
773803 }
774804 const why = quality === "saver" ? "data saver" : "slow link";
775 const slow = plan.mode === "snapshot" ? ` · ${(1000 / plan.everyMs).toFixed(plan.everyMs > 1000 ? 1 : 0)} fps, ${why}` : plan.fps ? ` · ${plan.fps} fps, ${why}` : "";
805 const slow = sockUp ? sock.readout() : plan.mode === "snapshot" ? ` · ${(1000 / plan.everyMs).toFixed(plan.everyMs > 1000 ? 1 : 0)} fps, ${why}` : plan.fps ? ` · ${plan.fps} fps, ${why}` : "";
776806 $("live").textContent = cams.filter(paneOn).map((c) => (c.live ? `live${slow}` : c.online === false ? "offline" : "waiting")).join(" · ");
777807 }, 1000);
778808 if (boot) { reconcile(boot.cameras); setTimeout(pollCameras, 3000); }
@@ -786,6 +816,7 @@
786816
787817 // ---------------------------------------------------------------- the stream: the page as a leader
788818 // Opened while a slider, a jog button, the joystick or a model joint is held; frames go at most every 33 ms; closed shortly after letting go.
819 // With the socket up, the frames go over WS /arm instead (latest per joint wins there too); a /follow already open finishes the hold.
789820 const stream = {
790821 ws: null, closeTimer: null, halted: false,
791822 // one message per slot to the socket that is open, or the one about to be
@@ -797,6 +828,7 @@
797828 open() {
798829 if (!mayDrive()) return; // the server would refuse it: only the person whose turn it is streams
799830 clearTimeout(this.closeTimer);
831 if (sockUp && !this.ws) return;
800832 if (this.ws && this.ws.readyState <= 1) return;
801833 if (this.halted) return; // the server ended this hold (Stop, or the arm is busy): nothing more until the next press
802834 const ws = new WebSocket(`${location.protocol === "https:" ? "wss" : "ws"}://${location.host}/follow`);
@@ -810,15 +842,20 @@
810842 send(frame) {
811843 if (this.halted) return;
812844 this.open();
845 if (sockUp && !this.ws) { if (mayDrive()) sock.command(frame); return; }
813846 if (this.ws) this.frames.send(frame);
814847 },
815848 /** Now, not at the next slot, with anything pending: a finger lifted and this is where its joints stop. */
816849 flush(frame) {
817850 if (this.halted) return;
818851 this.open();
852 if (sockUp && !this.ws) { if (mayDrive()) { sock.command(frame); sock.flush(); } return; }
819853 if (this.ws) this.frames.flush(frame);
820854 },
821 end() { clearTimeout(this.closeTimer); this.closeTimer = setTimeout(() => { const ws = this.ws; this.ws = null; this.halted = false; ws?.close(); }, 250); },
855 end() {
856 if (sock?.driving) sock.end(); // the app closes the drive itself shortly after, and the arm holds
857 clearTimeout(this.closeTimer); this.closeTimer = setTimeout(() => { const ws = this.ws; this.ws = null; this.halted = false; ws?.close(); }, 250);
858 },
822859 };
823860
824861 // ---------------------------------------------------------------- the arm
@@ -878,6 +915,7 @@
878915 const r = await api("/estop", {});
879916 msg(r.ok ? "EMERGENCY STOP: the arm holds; an admin must clear it" : r.error, true);
880917 if (stream.ws) stream.ws.close();
918 if (sock?.driving) sock.end();
881919 stopLeaderDrive("emergency stop");
882920 }
883921 async function clearEStop() {
@@ -1520,28 +1558,21 @@
15201558 $("floorMargin").onchange = () => saveSettings({ floorMarginMm: Number($("floorMargin").value) });
15211559 $("baseMargin").onchange = () => saveSettings({ baseMarginMm: Number($("baseMargin").value) });
15221560
1523 // ---------------------------------------------------------------- metrics
1561 // ---------------------------------------------------------------- metrics: the arm's GET /metrics; src/ui/metrics.js comes when opened
15241562 async function showMetrics() {
1525 const m = await api("/metrics");
1526 const c = m.counters;
1527 $("metricsSummary").innerHTML = [
1528 ["since", (m.since ?? "").replace("T", " ").slice(0, 19)],
1529 ["jams", c.jam], ["zone blocks", c.block], ["stalls on moves", c.stall], ["lost servos", c.missing], ["hot joints", c.hot],
1530 ["sessions", `${c.stream} streams · ${c.move} moves · ${c.play} playbacks · ${c.record} recordings`],
1531 ["teleop lag, this browser", (() => { const lag = teleop?.lag(); return lag ? `p50 ${lag.p50} ms · p95 ${lag.p95} ms over ${lag.n} samples` : "—"; })()],
1532 ["closest to the table", m.closest ? `${(m.closest.z * 1000).toFixed(0)} mm (${m.closest.link.replace(/_link$/, "")}, ${m.closest.at.slice(11, 19)})` : "—"],
1533 ].map(([k, v]) => `<b>${k}</b><span>${v}</span>`).join("");
1534 $("peaks").innerHTML = "<tr><th>joint</th><th>peak load</th><th>peak temperature</th></tr>" + Object.entries(m.peaks).map(([id, p]) => `<tr><td>${NAMES[id] ?? id}</td><td>${p.load.toFixed(0)} %</td><td>${p.temperature} °C</td></tr>`).join("");
1535 // Built from text nodes: an event's detail carries people's names (who pressed the E-stop) and error strings.
1536 const recent = [...(m.recent ?? [])].reverse();
1537 $("events").replaceChildren(...(recent.length ? recent.map((e) => el("div", {},
1538 `${String(e.t).slice(5, 19).replace("T", " ")} `, el("b", { textContent: String(e.kind) }),
1539 ` ${Object.entries(e.detail ?? {}).map(([k, v]) => `${k}=${typeof v === "object" ? JSON.stringify(v) : v}`).join(" ")}`)) : [el("div", { textContent: "none yet" })]));
1540 $("metricsPanel").hidden = false;
1563 try { await (await import("/ui/__VERSION__/metrics.js")).showMetrics({ api, lag: () => teleop?.lag() }); }
1564 catch (err) { console.error(err); msg("the metrics did not load", true); }
15411565 }
15421566 $("metricsBtn").onclick = showMetrics;
15431567 setInterval(() => { if (!$("metricsPanel").hidden) showMetrics(); }, 5000);
15441568
1569 // ---------------------------------------------------------------- health: GET /health, behind the relay; src/ui/health.js comes when opened
1570 $("healthBtn").hidden = !relayed;
1571 $("healthBtn").onclick = async () => {
1572 try { const { openHealth } = await import("/ui/__VERSION__/health.js"); await openHealth({ api, msg }); }
1573 catch (err) { console.error(err); msg("the health panel did not load", true); }
1574 };
1575
15451576 // ---------------------------------------------------------------- the room: the turn, the queue and the chat over /live
15461577 // Behind the hosted app's relay only the person whose turn it is may drive, and the server refuses everyone else.
15471578 // The page follows suit: the Controls tab appears on your turn. Everything here arrives on one socket.
@@ -1780,7 +1811,7 @@
17801811 ws.onmessage = (e) => {
17811812 const m = JSON.parse(e.data);
17821813 if (m.type === "ping") return say({ type: "pong" });
1783 if (m.type === "status") { lobby.statusSeen = true; return m.status ? applyStatus(m.status) : noStatus(); }
1814 if (m.type === "status") { lobby.statusSeen = true; if (sockUp) return; return m.status ? applyStatus(m.status) : noStatus(); }
17841815 if (m.type === "hello") lobby.hello = m;
17851816 else if (m.type === "state") { lobby.offset = m.now - Date.now(); lobby.state = m; }
17861817 else if (m.type === "chat") return addChat(m.messages, true), showEarlier(m.more);
@@ -1905,11 +1936,11 @@
19051936 else if (d.violation) { b.className = "warn"; b.textContent = `the arm is ${say(d.violation)} (zone: ${d.zone})`; }
19061937 else b.className = "";
19071938 }
1908 /** The arm's status, from /live (pushed when it changes) or, without it, from polling /status. */
1939 /** The arm's status, from WS /arm, /live (pushed when it changes) or polling /status. */
19091940 // Until the limits, the settings and the model are in, a status is kept aside, not drawn; then it is applied whole,
1910 // so an E-stop it carries still locks the controls.
1941 // so an E-stop it carries still locks the controls. `at` is the best guess of when the arm read its pose.
19111942 let started = false, early = null;
1912 function applyStatus(d) {
1943 function applyStatus(d, at = performance.now()) {
19131944 if (!started) return void (early = d);
19141945 {
19151946 const wasStopped = Boolean(status?.estop);
@@ -1918,14 +1949,13 @@
19181949 if (Boolean(d.estop) !== wasStopped) applyLock();
19191950 // Stopped, reconnecting or no longer this person's turn: a held thumb lets go, and its zero goes out now.
19201951 if (!canDrive({ mayDrive: mayDrive(), estop: d.estop, reconnecting: d.reconnecting, limits })) driver.releaseAll();
1921 // The round trip to the arm, which every command also makes: shown once it is long enough to feel.
1952 // The polled round trip to the arm, which every command also makes: shown once it is long enough to feel.
19221953 const lag = [...rtts].sort((a, b) => a - b)[rtts.length >> 1] ?? 0;
1923 $("lag").hidden = rtts.length < 3 || lag <= 250; $("lag").textContent = `· ${Math.round(lag)} ms lag`;
1954 $("lag").hidden = sockUp || rtts.length < 3 || lag <= 250; $("lag").textContent = `· ${Math.round(lag)} ms lag`;
19241955 $("dot").className = d.reconnecting ? "dot warn" : "dot on";
19251956 $("state").textContent = d.reconnecting ? "reconnecting…" : d.driving === null ? "idle" : d.driving === "record" ? `recording ${d.recording}` : leaderStreaming() ? `teleop · ${teleop.frames} frames` : d.driving;
19261957 if (leaderStreaming()) {
1927 // The follower's pose was read somewhere between asking and the answer: the middle is the best guess of when.
1928 teleop.observe(Object.fromEntries(d.joints.filter((j) => typeof j.position === "number").map((j) => [j.id, j.position])), (started + performance.now()) / 2);
1958 teleop.observe(Object.fromEntries(d.joints.filter((j) => typeof j.position === "number").map((j) => [j.id, j.position])), at);
19291959 renderTracks(); renderLag();
19301960 }
19311961 $("estop").classList.toggle("on", Boolean(d.estop)); $("estop").textContent = d.estop ? "Stopped" : "Stop";
@@ -1942,13 +1972,16 @@
19421972 $("dot").className = reconnecting ? "dot warn" : "dot"; $("state").textContent = reconnecting ? "reconnecting…" : "no arm";
19431973 }
19441974 async function poll() {
1945 // The room's socket carries the status: nothing to poll while it is open and has said where the arm is.
1975 // Nothing to poll while WS /arm or the room's socket carries the status.
1976 if (sockUp) return void setTimeout(poll, 500);
19461977 if (lobby.statusSeen && lobby.ws?.readyState === 1) return void setTimeout(poll, 1000);
1978 const asked = performance.now();
19471979 try {
19481980 const r = await timed("/status");
19491981 const d = await r.json();
19501982 if (!r.ok) throw new Error();
1951 applyStatus(d);
1983 // The pose was read somewhere between asking and the answer: the middle is the best guess of when.
1984 applyStatus(d, (asked + performance.now()) / 2);
19521985 } catch { noStatus(); }
19531986 // Hidden, the page keeps the arm's state fresh at a walk: nobody is looking, and a phone's battery and data matter.
19541987 setTimeout(poll, document.hidden ? 2000 : 200);
@@ -1967,6 +2000,7 @@
19672000 limits = l; modelInfo = mi; started = true;
19682001 cameraFirst = !relayed || link?.mode === "live" || q.has("stream");
19692002 startViews();
2003 openSock().catch((err) => console.error(err));
19702004 if (early) applyStatus(early);
19712005 poll();
19722006 if (relayed) pollLink();
Software/so101/src/weigh.ts added +53
@@ -0,0 +1,53 @@
1 /**
2 * What a visit to the page costs, file by file: what the first visit downloads, and what a second one still has to
3 * ask for given each answer's cache-control. Pure parsing here; bench/first-load.ts does the fetching and prints it.
4 */
5
6 export { urdfFiles } from "./mesh-lite.ts";
7
8 /** The page's shell: modules preloaded or imported up front, stylesheets, the import map's targets kept apart. */
9 export function pageLinks(html: string): { shell: string[]; mapped: string[] } {
10 const shell = new Set<string>();
11 for (const m of html.matchAll(/<link\b[^>]*\brel="(?:modulepreload|stylesheet|preload|manifest)"[^>]*\bhref="([^"]+)"/g)) shell.add(m[1]!);
12 for (const m of html.matchAll(/<script\b[^>]*\bsrc="([^"]+)"/g)) shell.add(m[1]!);
13 for (const block of html.matchAll(/<script type="module">([\s\S]*?)<\/script>/g)) for (const s of staticImports(block[1]!)) if (s.startsWith("/")) shell.add(s);
14 const mapped: string[] = [];
15 const map = /<script type="importmap">([\s\S]*?)<\/script>/.exec(html);
16 if (map) {
17 try {
18 const imports = (JSON.parse(map[1]!) as { imports?: Record<string, string> }).imports ?? {};
19 for (const [name, target] of Object.entries(imports)) if (!name.endsWith("/")) mapped.push(target);
20 } catch {
21 /* no map */
22 }
23 }
24 return { shell: [...shell], mapped };
25 }
26
27 /** `import … from "x"` and `export … from "x"` at the top level, and `import "x"`. */
28 export function staticImports(js: string): string[] {
29 const out = new Set<string>();
30 for (const m of js.matchAll(/(?:^|[;\n}])\s*(?:import|export)\s+(?:[\w*{}\s,$]+?\s+from\s+)?["']([^"']+)["']/g)) out.add(m[1]!);
31 return [...out];
32 }
33
34 /** Every module a file pulls in, statically or with import("…"), as written. */
35 export function allImports(js: string): string[] {
36 const out = new Set(staticImports(js));
37 for (const m of js.matchAll(/\bimport\(\s*["']([^"']+)["']\s*\)/g)) out.add(m[1]!);
38 return [...out];
39 }
40
41 export type Repeat = "free" | "revalidate" | "refetch";
42
43 /**
44 * What a second visit within a day pays for one answer: nothing (fresh in the cache), a round trip answered 304
45 * (no-cache with a validator), or the whole thing again (no-store, or nothing to validate with).
46 */
47 export function repeatCost(cacheControl: string | null, etag: string | null): Repeat {
48 const cc = (cacheControl ?? "").toLowerCase();
49 if (cc.includes("no-store")) return "refetch";
50 const maxAge = Number(/max-age=(\d+)/.exec(cc)?.[1] ?? 0);
51 if (!cc.includes("no-cache") && maxAge >= 86_400) return "free";
52 return etag ? "revalidate" : "refetch";
53 }

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs