Public: anyone can read this; the crew can change it.
Migrated from github.com/Trevato/SO-ARM100
workbenchthis machine
TOP: smooth driving and a camera that never freezes, through the public URL - one WebSocket, measured end to end #80
quill/arm:quill/54-top-smooth-driving-and-a-camera-that-nev → main 17 commits fork
30 files
changed +2607
−101
Software/README.md +80
| @@ -229,6 +229,80 @@ and shared by everyone watching, shows a grey offline frame while its source | ||
| 229 | 229 | is down (the other cameras keep playing) and reconnects by itself. Viewers |
| 230 | 230 | get a tab per camera and can turn several on for a grid; that choice is |
| 231 | 231 | their own browser's. A camera added or removed shows up within a few seconds. |
| 232 | ||
| 233 | **One socket per viewer: `WS /arm`.** Through the forge's front door every | |
| 234 | request pays a round trip of about 0.4 s, so behind the app the page doesn't | |
| 235 | poll. It opens one WebSocket, `/arm` (`src/viewer-socket.ts` on the app, | |
| 236 | `src/ui/arm-socket.js` on the page), and everything that has to feel live | |
| 237 | goes over it. The module isn't part of the first load: it comes once the | |
| 238 | first view is up, and until then the page uses the old paths below. Over | |
| 239 | the socket go: | |
| 240 | ||
| 241 | - the arm's pose, up to 30 times a second. The app reads the arm server once | |
| 242 | for everyone (`src/pose-feed.ts`), and the page paints at most once a frame | |
| 243 | (none at all for cameras while the tab is hidden); | |
| 244 | - the driver's joint targets (`{"t":"cmd","seq","targets":{"1":2048}}`). | |
| 245 | Each joint keeps only its latest target, and the targets go to the arm's | |
| 246 | `/follow` at most 30 times a second, so a burst never queues. Every | |
| 247 | command is checked against the turn as it arrives, as on `/follow`. | |
| 248 | `{"t":"end"}` lets go, and the arm holds; | |
| 249 | - the camera frames as binary messages: a 20-byte header (camera id, frame | |
| 250 | number, the app's receipt time), then the JPEG. The page acknowledges | |
| 251 | each frame it has shown (`{"t":"seen","n"}`), and at most 4 are | |
| 252 | unacknowledged at once. A viewer still busy with earlier frames gets the | |
| 253 | newest frame next, never a backlog. A slow phone sees fewer frames, and | |
| 254 | nobody else is slowed; | |
| 255 | - every 5 s, what the page measured (`{"t":"report",…}`): command → pose | |
| 256 | round trips, frame ages when shown, frames shown, stalls and reconnects. | |
| 257 | The app takes at most one report per socket every 4 s; the rest are | |
| 258 | dropped and counted (`arm_reports_dropped_total`). Each report is held to | |
| 259 | what crossed that socket: over the socket's life, no more commands or | |
| 260 | round trips than commands it received, and no more frames or frame ages | |
| 261 | than frames it was sent. What one report doesn't use carries over to the | |
| 262 | next, because a round trip often lands a report after its command. A page | |
| 263 | can't fill the numbers with work that never happened. | |
| 264 | ||
| 265 | The page reconnects by itself (after 0.25, 0.5, 1, 2, then every 4 s), and | |
| 266 | it also reconnects when the socket goes quiet for 2 s. A drive doesn't | |
| 267 | survive a reconnect: the arm holds until the next press. While the socket is | |
| 268 | down the page uses the old paths: `/status` polled 5 times a second, | |
| 269 | `/follow` for a drive, and the MJPEG `<img>` for the cameras. When the | |
| 270 | socket is back the page moves over to it again. The top bar's camera | |
| 271 | readout then shows the page's own medians, e.g. | |
| 272 | `live · 24 fps · frame 90 ms · command 140 ms`. The leader arm still | |
| 273 | streams on its own `/follow`. With no lobby (a run on your own machine) | |
| 274 | every viewer may drive on `/arm`, as on `/follow`. | |
| 275 | ||
| 276 | **How the link feels: `/metrics` and `/health`.** The app combines what every | |
| 277 | page reports with what it measures itself: the arm server's round trip, | |
| 278 | camera frames in, frames skipped for a slow viewer, bytes out, and commands | |
| 279 | merged away. | |
| 280 | ||
| 281 | - `GET /metrics` is Prometheus text, open to anyone, like a scrape target: | |
| 282 | current numbers and totals only, no names. Scrape it with | |
| 283 | `curl https://<app>/metrics` or a Prometheus `scrape_configs` job with | |
| 284 | `metrics_path: /metrics`. With `accept: application/json` the same path | |
| 285 | answers the arm server's own metrics (jams, stalls, peak loads), which the | |
| 286 | page's Metrics panel shows. | |
| 287 | It is open on purpose, so a scraper needs no account. What it shows is | |
| 288 | already visible to anyone who opens the page: how many people are watching | |
| 289 | and how quick the arm feels. It has no names, addresses, history or | |
| 290 | settings, and it changes nothing. The one thing a stranger could do is skew | |
| 291 | the viewer-reported numbers, and the per-socket limit above bounds that to | |
| 292 | their own share of real traffic. | |
| 293 | - `GET /health` is the same as JSON (`commandRttMs`, `frameAgeMs`, | |
| 294 | `armRttMs` as `{p50, p95}`, `viewerFps`, `cameraFpsIn`, `bytesOutPerS`, | |
| 295 | `commandsPerS`, `totals`) plus two minutes of history, one point every 2 s. | |
| 296 | Only the admin may read it (401 or 403 for anyone else). Run on your own | |
| 297 | machine, every visitor is the admin. On the page, **Admin → Health** draws | |
| 298 | it (`src/ui/health.js`, fetched the first time it is opened): one | |
| 299 | sparkline per number over those two minutes (`src/ui/sparkline.js`), | |
| 300 | a dashed line at the targets (command→pose p95 250 ms, frame age p95 | |
| 301 | 400 ms), the current value turning amber above them, and the totals since | |
| 302 | the app started. It refreshes every 2 s while it is open. | |
| 303 | - The app logs the same numbers as one line every minute: | |
| 304 | `link: 2 viewers (1 watching) · command→pose p50/p95 140/310ms · frame age 90/180ms · …`. | |
| 305 | ||
| 232 | 306 | **Egress in `forge.json`.** `forge.json` lists `100.89.95.68:8101` (the arm) |
| 233 | 307 | and `100.89.95.68:8080` (the camera) under `egress`: the destinations the app |
| 234 | 308 | may reach on the Pi. The declaration does nothing until a site admin grants |
| @@ -421,6 +495,8 @@ watches. A line you type is said in the chat; commands start with `/`: | ||
| 421 | 495 | - `/end [name]`, owner: end that turn (no name: whoever drives), or take |
| 422 | 496 | them out of the line. |
| 423 | 497 | - `/mute <name>`, `/unmute <name>`, owner. |
| 498 | - `/health`, owner: the link's numbers from `GET /health` on one line | |
| 499 | (command → pose round trips, frame ages, fps, stalls, reconnects). | |
| 424 | 500 | - `/machine`: the Machine panel, the same rows the page's Machine tab shows |
| 425 | 501 | (one line per joint, then the link and the server), followed by every |
| 426 | 502 | command in and out and every clamp, line by line as it happens. `/machine` |
| @@ -428,6 +504,10 @@ watches. A line you type is said in the chat; commands start with `/`: | ||
| 428 | 504 | guests too: it is read-only. |
| 429 | 505 | - `/who` (who drives, who waits), `/help`, `/quit`. |
| 430 | 506 | |
| 507 | The terminal doesn't drive. Driving means joint targets many times a | |
| 508 | second from a slider, the 3D model or a leader arm, and a typed line can't | |
| 509 | hold a joint. A Stop from the page (or the owner's `/end`) stops the arm. | |
| 510 | ||
| 431 | 511 | The terminal is a client of the running server, like the page: it opens |
| 432 | 512 | the same `/live` socket on `127.0.0.1:$PORT`, as the forge's identity for |
| 433 | 513 | the person at the keyboard (`FORGE_APP_USER*`, signed with |
Software/so101/bench/fake-serve.ts +1 −1
| @@ -36,7 +36,7 @@ const dir = appData ? join(appData, "so101") : mkdtempSync(join(tmpdir(), "so101 | ||
| 36 | 36 | mkdirSync(dir, { recursive: true }); |
| 37 | 37 | const port = Number(process.env.PORT ?? process.argv[2] ?? 8102); |
| 38 | 38 | const host = process.env.HOST ?? "127.0.0.1"; |
| 39 | const simulation = await startServer(new FeetechBus(fake), { host: "127.0.0.1", port: 0, posesFile: join(dir, "poses.json"), modelDir: new URL("../../../Simulation/SO101", import.meta.url).pathname }); | |
| 39 | const simulation = await startServer(new FeetechBus(fake), { host: "127.0.0.1", port: 0, posesFile: join(dir, "poses.json"), modelDir: new URL("../../../Simulation/SO101", import.meta.url).pathname, warmModel: true }); | |
| 40 | 40 | let remote: Remote | null = null; |
| 41 | 41 | let problem: string | null = null; |
| 42 | 42 | try { |
Software/so101/bench/first-load.ts added +58
| @@ -0,0 +1,58 @@ | ||
| 1 | /** | |
| 2 | * Weigh the page: what a first visit downloads (bytes on the wire, compressed as the browser would get them) and what | |
| 3 | * a second visit still asks for, file by file, with each answer's cache-control. | |
| 4 | * bun run bench/first-load.ts [base] base defaults to http://127.0.0.1:8102; the preview's URL works too | |
| 5 | * "shell" is what the first paint waits for (the HTML and the modules it preloads); "3d" is three.js, the loader, | |
| 6 | * the URDF and its meshes, which come after. | |
| 7 | */ | |
| 8 | import { allImports, pageLinks, repeatCost, urdfFiles, type Repeat } from "../src/weigh.ts"; | |
| 9 | ||
| 10 | const base = (process.argv[2] ?? "http://127.0.0.1:8102").replace(/\/$/, ""); | |
| 11 | interface Row { part: "shell" | "3d"; path: string; bytes: number; ms: number; cache: string; repeat: Repeat } | |
| 12 | const rows: Row[] = []; | |
| 13 | const seen = new Set<string>(); | |
| 14 | ||
| 15 | async function get(path: string, part: Row["part"]): Promise<string> { | |
| 16 | const started = performance.now(); | |
| 17 | const r = await fetch(base + path, { headers: { "accept-encoding": "br, gzip" }, decompress: false } as RequestInit); | |
| 18 | const raw = new Uint8Array(await r.arrayBuffer()); | |
| 19 | const ms = performance.now() - started; | |
| 20 | if (!r.ok) throw new Error(`${path}: ${r.status}`); | |
| 21 | const cache = r.headers.get("cache-control") ?? ""; | |
| 22 | rows.push({ part, path, bytes: raw.length, ms, cache, repeat: repeatCost(cache, r.headers.get("etag")) }); | |
| 23 | const enc = r.headers.get("content-encoding"); | |
| 24 | const plain = enc === "gzip" ? Bun.gunzipSync(raw) : enc === "br" ? (await import("node:zlib")).brotliDecompressSync(raw) : raw; | |
| 25 | return new TextDecoder().decode(plain); | |
| 26 | } | |
| 27 | ||
| 28 | /** A module and everything it imports, relative or absolute (bare names are the import map's, already listed). */ | |
| 29 | async function crawl(path: string, part: Row["part"]): Promise<void> { | |
| 30 | if (seen.has(path)) return; | |
| 31 | seen.add(path); | |
| 32 | const js = await get(path, part); | |
| 33 | for (const spec of allImports(js)) { | |
| 34 | if (spec.startsWith("./") || spec.startsWith("../")) await crawl(new URL(spec, `http://x${path}`).pathname, part); | |
| 35 | else if (spec.startsWith("/") && part === "3d") await crawl(spec, part); | |
| 36 | } | |
| 37 | } | |
| 38 | ||
| 39 | const html = await get("/", "shell"); | |
| 40 | seen.add("/"); | |
| 41 | const { shell, mapped } = pageLinks(html); | |
| 42 | for (const path of shell) await crawl(path, "shell"); | |
| 43 | for (const path of mapped) await crawl(path, "3d"); | |
| 44 | for (const m of html.matchAll(/import\("(\/[^"]+\.js)"\)/g)) await crawl(m[1]!, "3d"); | |
| 45 | const info = JSON.parse(await get("/model.json", "3d")) as { urdf?: string }; | |
| 46 | if (info.urdf) { | |
| 47 | const urdf = await get(info.urdf, "3d"); | |
| 48 | for (const f of urdfFiles(urdf)) await get(new URL(f, `http://x${info.urdf}`).pathname, "3d"); | |
| 49 | } | |
| 50 | ||
| 51 | const kb = (n: number) => `${(n / 1024).toFixed(1)} KB`; | |
| 52 | for (const r of rows) console.log(`${r.part.padEnd(5)} ${kb(r.bytes).padStart(10)} ${`${r.ms.toFixed(0)} ms`.padStart(7)} ${r.repeat.padEnd(10)} ${r.cache.padEnd(36)} ${r.path}`); | |
| 53 | for (const part of ["shell", "3d"] as const) { | |
| 54 | const mine = rows.filter((r) => r.part === part); | |
| 55 | const again = mine.filter((r) => r.repeat !== "free"); | |
| 56 | console.log(`${part}: first visit ${kb(mine.reduce((s, r) => s + r.bytes, 0))} in ${mine.length} requests; ` + | |
| 57 | `a second visit asks ${again.length} again (${kb(again.filter((r) => r.repeat === "refetch").reduce((s, r) => s + r.bytes, 0))} refetched)`); | |
| 58 | } |
Software/so101/src/camera-feed.ts +55 −5
| @@ -30,7 +30,18 @@ export interface FeedOptions { | ||
| 30 | 30 | |
| 31 | 31 | export type FeedState = "idle" | "connecting" | "live" | "offline"; |
| 32 | 32 | |
| 33 | /** One whole picture: what a WebSocket viewer gets (src/viewer-socket.ts), with the moment the app had all of it. */ | |
| 34 | export interface Picture { | |
| 35 | type: string; | |
| 36 | bytes: Buffer; | |
| 37 | /** Date.now() when its last byte came in from the camera. */ | |
| 38 | at: number; | |
| 39 | } | |
| 40 | export type PictureListener = (picture: Picture) => void; | |
| 41 | ||
| 33 | 42 | const BOUNDARY = "armframe"; |
| 43 | /** A picture larger than this is not gathered whole (a 1080p JPEG is well under 1 MB). */ | |
| 44 | const MAX_PICTURE = 4 << 20; | |
| 34 | 45 | const NEXT = Buffer.from(`\r\n--${BOUNDARY}\r\n`); |
| 35 | 46 | const head = (type: string) => Buffer.from(`Content-Type: ${type}\r\n\r\n`); |
| 36 | 47 | |
| @@ -56,6 +67,11 @@ export function parseFps(text: string | null): number | null { | ||
| 56 | 67 | |
| 57 | 68 | export class CameraFeed { |
| 58 | 69 | private readonly viewers = new Set<Viewer>(); |
| 70 | private readonly listeners = new Set<PictureListener>(); | |
| 71 | /** The part being gathered whole, only while someone wants whole pictures. */ | |
| 72 | private gathering: { type: string; chunks: Buffer[]; size: number } | null = null; | |
| 73 | /** Whole pictures the source sent, since the start. */ | |
| 74 | framesIn = 0; | |
| 59 | 75 | private _state: FeedState; |
| 60 | 76 | private attempt = 0; |
| 61 | 77 | private controller: AbortController | null = null; |
| @@ -81,9 +97,26 @@ export class CameraFeed { | ||
| 81 | 97 | return this._state; |
| 82 | 98 | } |
| 83 | 99 | |
| 84 | /** How many viewers are watching right now. */ | |
| 100 | /** How many viewers are watching right now, by either way. */ | |
| 85 | 101 | get watching(): number { |
| 86 | return this.viewers.size; | |
| 102 | return this.viewers.size + this.listeners.size; | |
| 103 | } | |
| 104 | ||
| 105 | /** | |
| 106 | * Whole pictures, one call each, for as long as the returned function is not called: the offline picture at once | |
| 107 | * when the source is down, then every live frame. Counts as a viewer, so the upstream stays open meanwhile. | |
| 108 | */ | |
| 109 | pictures(listener: PictureListener): () => void { | |
| 110 | if (this.closed) return () => {}; | |
| 111 | this.listeners.add(listener); | |
| 112 | clearTimeout(this.linger); | |
| 113 | if (this._state === "offline") { | |
| 114 | listener({ type: "image/png", bytes: OFFLINE_FRAME, at: Date.now() }); | |
| 115 | this.offline(); | |
| 116 | } else if (this._state === "idle") this.connect(); | |
| 117 | return () => { | |
| 118 | if (this.listeners.delete(listener)) this.left(); | |
| 119 | }; | |
| 87 | 120 | } |
| 88 | 121 | |
| 89 | 122 | /** |
| @@ -115,14 +148,19 @@ export class CameraFeed { | ||
| 115 | 148 | this.stop(); |
| 116 | 149 | for (const v of this.viewers) v.res.end(); |
| 117 | 150 | this.viewers.clear(); |
| 151 | this.listeners.clear(); | |
| 118 | 152 | } |
| 119 | 153 | |
| 120 | 154 | // ------------------------------------------------------------------ viewers |
| 121 | 155 | private leave(viewer: Viewer) { |
| 122 | if (!this.viewers.delete(viewer) || this.viewers.size) return; | |
| 156 | if (this.viewers.delete(viewer)) this.left(); | |
| 157 | } | |
| 158 | ||
| 159 | private left() { | |
| 160 | if (this.watching) return; | |
| 123 | 161 | clearTimeout(this.linger); |
| 124 | 162 | this.linger = setTimeout(() => { |
| 125 | if (!this.viewers.size) this.stop(); | |
| 163 | if (!this.watching) this.stop(); | |
| 126 | 164 | }, this.o.lingerMs); |
| 127 | 165 | this.linger.unref?.(); |
| 128 | 166 | } |
| @@ -145,6 +183,11 @@ export class CameraFeed { | ||
| 145 | 183 | v.inPart = false; |
| 146 | 184 | this.whole(v, "image/png", OFFLINE_FRAME); |
| 147 | 185 | } |
| 186 | this.tell({ type: "image/png", bytes: OFFLINE_FRAME, at: Date.now() }); | |
| 187 | } | |
| 188 | ||
| 189 | private tell(picture: Picture) { | |
| 190 | for (const l of this.listeners) l(picture); | |
| 148 | 191 | } |
| 149 | 192 | |
| 150 | 193 | // ------------------------------------------------------------------ the source |
| @@ -167,7 +210,7 @@ export class CameraFeed { | ||
| 167 | 210 | if (this.source && !this.retry && !this.closed) { |
| 168 | 211 | this.retry = setTimeout(() => { |
| 169 | 212 | this.retry = undefined; |
| 170 | if (this.viewers.size && this._state === "offline" && !this.controller) this.connect(); | |
| 213 | if (this.watching && this._state === "offline" && !this.controller) this.connect(); | |
| 171 | 214 | }, this.o.retryMs); |
| 172 | 215 | this.retry.unref?.(); |
| 173 | 216 | } |
| @@ -210,6 +253,7 @@ export class CameraFeed { | ||
| 210 | 253 | } |
| 211 | 254 | const parser = new MjpegParser(boundary, { |
| 212 | 255 | start: (type) => { |
| 256 | this.gathering = this.listeners.size ? { type, chunks: [], size: 0 } : null; | |
| 213 | 257 | const now = performance.now(); |
| 214 | 258 | for (const v of this.viewers) { |
| 215 | 259 | // A frame a little early still counts as on time: a 10 fps source held to 5 sends every other frame, not every third. |
| @@ -221,6 +265,8 @@ export class CameraFeed { | ||
| 221 | 265 | }, |
| 222 | 266 | data: (bytes) => { |
| 223 | 267 | for (const v of this.viewers) if (v.inPart) this.write(v, bytes); |
| 268 | const g = this.gathering; | |
| 269 | if (g && (g.size += bytes.length) <= MAX_PICTURE) g.chunks.push(Buffer.from(bytes)); | |
| 224 | 270 | }, |
| 225 | 271 | end: () => { |
| 226 | 272 | if (this._state !== "live") { |
| @@ -230,6 +276,10 @@ export class CameraFeed { | ||
| 230 | 276 | this.repeat = undefined; |
| 231 | 277 | } |
| 232 | 278 | for (const v of this.viewers) if (v.inPart) this.write(v, NEXT), (v.inPart = false); |
| 279 | this.framesIn++; | |
| 280 | const g = this.gathering; | |
| 281 | this.gathering = null; | |
| 282 | if (g && g.size <= MAX_PICTURE) this.tell({ type: g.type, bytes: Buffer.concat(g.chunks), at: Date.now() }); | |
| 233 | 283 | }, |
| 234 | 284 | }); |
| 235 | 285 | this.lastByte = Date.now(); |
Software/so101/src/cameras.ts +7
| @@ -128,6 +128,13 @@ export class CameraHub { | ||
| 128 | 128 | return this.specs.find((c) => c.id === id)?.source ?? null; |
| 129 | 129 | } |
| 130 | 130 | |
| 131 | /** Whole pictures the cameras have sent the app, all feeds together, since the start. */ | |
| 132 | get framesIn(): number { | |
| 133 | let n = 0; | |
| 134 | for (const feed of this.feeds.values()) n += feed.framesIn; | |
| 135 | return n; | |
| 136 | } | |
| 137 | ||
| 131 | 138 | close(): void { |
| 132 | 139 | for (const feed of this.feeds.values()) feed.close(); |
| 133 | 140 | this.feeds.clear(); |
Software/so101/src/fake-pi.ts added +47
| @@ -0,0 +1,47 @@ | ||
| 1 | /** | |
| 2 | * A Pi stand-in for the tests: an arm server's /status (answered after `delayMs`), an MJPEG /stream with a 20 kB frame | |
| 3 | * every 30 ms, and a /follow that keeps every frame it is sent and moves joint 1 to it. | |
| 4 | */ | |
| 5 | ||
| 6 | import { createServer, type ServerResponse } from "node:http"; | |
| 7 | import { WebSocketServer } from "ws"; | |
| 8 | ||
| 9 | const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms)); | |
| 10 | ||
| 11 | export async function fakePi(delayMs = 0) { | |
| 12 | const follow: string[] = []; | |
| 13 | const cams = new Set<ServerResponse>(); | |
| 14 | let pan = 2048; | |
| 15 | const http = createServer(async (req, res) => { | |
| 16 | if (req.url?.startsWith("/status")) { | |
| 17 | await sleep(delayMs); | |
| 18 | res.writeHead(200, { "content-type": "application/json" }); | |
| 19 | return res.end(JSON.stringify({ joints: [{ id: 1, joint: "shoulder_pan", position: pan }], driving: null })); | |
| 20 | } | |
| 21 | if (req.url === "/stream") { | |
| 22 | res.writeHead(200, { "content-type": "multipart/x-mixed-replace;boundary=b" }); | |
| 23 | cams.add(res); | |
| 24 | res.socket?.once("close", () => cams.delete(res)); | |
| 25 | return; | |
| 26 | } | |
| 27 | res.writeHead(404).end(); | |
| 28 | }); | |
| 29 | const body = Buffer.concat([Buffer.from([0xff, 0xd8]), Buffer.alloc(20_000, 7), Buffer.from([0xff, 0xd9])]); | |
| 30 | const camera = setInterval(() => { | |
| 31 | for (const res of cams) res.write(Buffer.concat([Buffer.from(`--b\r\nContent-Type: image/jpeg\r\nContent-Length: ${body.length}\r\n\r\n`), body, Buffer.from("\r\n")])); | |
| 32 | }, 30); | |
| 33 | const wss = new WebSocketServer({ server: http, path: "/follow" }); | |
| 34 | wss.on("connection", (ws) => | |
| 35 | ws.on("message", (data) => { | |
| 36 | follow.push(String(data)); | |
| 37 | const p = (JSON.parse(String(data)) as Record<string, number>)["1"]; | |
| 38 | if (p !== undefined) pan = p; | |
| 39 | }), | |
| 40 | ); | |
| 41 | const port = await new Promise<number>((r) => http.listen(0, "127.0.0.1", () => r((http.address() as { port: number }).port))); | |
| 42 | return { | |
| 43 | port, | |
| 44 | follow, | |
| 45 | close: () => (clearInterval(camera), wss.close(), http.closeAllConnections(), http.close()), | |
| 46 | }; | |
| 47 | } |
Software/so101/src/latest.ts added +60
| @@ -0,0 +1,60 @@ | ||
| 1 | /** | |
| 2 | * Latest wins: at most `capacity` messages in flight to a viewer at a time; whatever comes meanwhile replaces what was waiting under | |
| 3 | * the same key, so a slow viewer is handed fewer, fresher messages and never a queue of stale ones. Nothing here waits | |
| 4 | * on one viewer for another's sake. | |
| 5 | */ | |
| 6 | ||
| 7 | /** Sends `data`, calling `done` once it has left (or failed); returns false when it could not be sent at all. */ | |
| 8 | export type Send = (data: Buffer | string, done: () => void) => boolean; | |
| 9 | ||
| 10 | export class Latest { | |
| 11 | private busy = 0; | |
| 12 | private readonly waiting = new Map<string, Buffer | string>(); | |
| 13 | /** Messages replaced before they could be sent. */ | |
| 14 | skipped = 0; | |
| 15 | sent = 0; | |
| 16 | bytes = 0; | |
| 17 | ||
| 18 | /** `capacity` is how many may be in flight at once: one viewer's round trip times it is the frame rate it can get. */ | |
| 19 | private readonly send: Send; | |
| 20 | private readonly capacity: number; | |
| 21 | ||
| 22 | constructor(send: Send, capacity = 1) { | |
| 23 | this.send = send; | |
| 24 | this.capacity = capacity; | |
| 25 | } | |
| 26 | ||
| 27 | offer(key: string, data: Buffer | string): void { | |
| 28 | if (this.waiting.has(key)) this.skipped++; | |
| 29 | this.waiting.set(key, data); | |
| 30 | if (this.busy < this.capacity) this.next(); | |
| 31 | } | |
| 32 | ||
| 33 | /** Messages sent and not yet done. */ | |
| 34 | get inFlight(): number { | |
| 35 | return this.busy; | |
| 36 | } | |
| 37 | ||
| 38 | private next(): void { | |
| 39 | const first = this.waiting.entries().next(); | |
| 40 | if (first.done) return; | |
| 41 | const [key, data] = first.value; | |
| 42 | this.waiting.delete(key); | |
| 43 | this.busy++; | |
| 44 | let finished = false; | |
| 45 | const done = () => { | |
| 46 | if (finished) return; | |
| 47 | finished = true; | |
| 48 | this.busy--; | |
| 49 | this.next(); | |
| 50 | }; | |
| 51 | if (!this.send(data, done)) { | |
| 52 | finished = true; | |
| 53 | this.busy--; | |
| 54 | this.waiting.clear(); | |
| 55 | return; | |
| 56 | } | |
| 57 | this.sent++; | |
| 58 | this.bytes += typeof data === "string" ? Buffer.byteLength(data) : data.length; | |
| 59 | } | |
| 60 | } |
Software/so101/src/mesh-lite.test.ts added +61
| @@ -0,0 +1,61 @@ | ||
| 1 | import { describe, expect, test } from "bun:test"; | |
| 2 | import { readFileSync } from "node:fs"; | |
| 3 | import { gzipSync } from "node:zlib"; | |
| 4 | import { isBinaryStl, lighten, urdfFiles } from "./mesh-lite.ts"; | |
| 5 | ||
| 6 | /** A binary STL from triangles given as nine numbers each. */ | |
| 7 | function stl(triangles: number[][]): Uint8Array { | |
| 8 | const out = new Uint8Array(84 + triangles.length * 50); | |
| 9 | const v = new DataView(out.buffer); | |
| 10 | v.setUint32(80, triangles.length, true); | |
| 11 | triangles.forEach((t, i) => t.forEach((x, j) => v.setFloat32(84 + i * 50 + 12 + j * 4, x, true))); | |
| 12 | return out; | |
| 13 | } | |
| 14 | const read = (bytes: Uint8Array) => { | |
| 15 | const v = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength); | |
| 16 | const n = v.getUint32(80, true); | |
| 17 | return Array.from({ length: n }, (_, i) => ({ | |
| 18 | normal: [0, 1, 2].map((j) => Math.round(v.getFloat32(84 + i * 50 + j * 4, true) * 1000) / 1000 + 0), | |
| 19 | corners: Array.from({ length: 9 }, (_, j) => Math.round(v.getFloat32(84 + i * 50 + 12 + j * 4, true) * 1e6) / 1e6 + 0), | |
| 20 | })); | |
| 21 | }; | |
| 22 | ||
| 23 | describe("lightened meshes", () => { | |
| 24 | test("corners snap to the grid; a sliver that collapses is dropped, a repeat too, and the faces get their normals", () => { | |
| 25 | const out = read(lighten(stl([ | |
| 26 | [0, 0, 0, 0.01, 0, 0, 0, 0.01, 0.00004], // stays, its last corner snaps to z = 0 | |
| 27 | [0, 0, 0, 0.00005, 0, 0, 0, 0.01, 0], // two corners within the grid: gone | |
| 28 | [0.01, 0, 0, 0, 0.01, 0, 0, 0, 0], // the first face again, turned the same way round: gone | |
| 29 | [0, 0, 0, 0, 0.01, 0, 0.01, 0, 0], // the same corners turned the other way: the back face, kept | |
| 30 | ]))); | |
| 31 | expect(out).toEqual([ | |
| 32 | { normal: [0, 0, 1], corners: [0, 0, 0, 0.01, 0, 0, 0, 0.01, 0] }, | |
| 33 | { normal: [0, 0, -1], corners: [0, 0, 0, 0, 0.01, 0, 0.01, 0, 0] }, | |
| 34 | ]); | |
| 35 | }); | |
| 36 | ||
| 37 | test("anything but a well-formed binary STL passes through untouched", () => { | |
| 38 | const ascii = new TextEncoder().encode("solid x\nendsolid x\n".padEnd(120, " ")); | |
| 39 | expect(isBinaryStl(ascii)).toBe(false); | |
| 40 | expect(lighten(ascii)).toBe(ascii); | |
| 41 | }); | |
| 42 | ||
| 43 | test("the arm's heaviest mesh gets several times smaller on the wire and keeps its size", () => { | |
| 44 | const original = new Uint8Array(readFileSync(new URL("../../../Simulation/SO101/assets/wrist_roll_pitch_so101_v2.stl", import.meta.url))); | |
| 45 | const light = lighten(original); | |
| 46 | expect(gzipSync(light).length * 4).toBeLessThan(gzipSync(original).length); | |
| 47 | const extent = (b: Uint8Array) => { | |
| 48 | const v = new DataView(b.buffer, b.byteOffset, b.byteLength), n = v.getUint32(80, true); | |
| 49 | let lo = Infinity, hi = -Infinity; | |
| 50 | for (let i = 0; i < n; i++) for (let k = 0; k < 3; k++) { const x = v.getFloat32(84 + i * 50 + 12 + k * 12, true); lo = Math.min(lo, x); hi = Math.max(hi, x); } | |
| 51 | return [lo, hi]; | |
| 52 | }; | |
| 53 | const [a, b] = [extent(original), extent(light)]; | |
| 54 | expect(Math.abs(a[0]! - b[0]!)).toBeLessThanOrEqual(0.0001 + 1e-7); | |
| 55 | expect(Math.abs(a[1]! - b[1]!)).toBeLessThanOrEqual(0.0001 + 1e-7); | |
| 56 | }); | |
| 57 | ||
| 58 | test("a URDF's mesh files, once each, package:// prefixes taken off", () => { | |
| 59 | expect(urdfFiles('<mesh filename="assets/a.stl"/><mesh filename="package://so101/assets/b.stl"/><mesh filename="assets/a.stl"/>')).toEqual(["assets/a.stl", "assets/b.stl"]); | |
| 60 | }); | |
| 61 | }); |
Software/so101/src/mesh-lite.ts added +73
| @@ -0,0 +1,73 @@ | ||
| 1 | /** | |
| 2 | * The arm's meshes, lightened for the browser: a binary STL with its vertices snapped to a grid (0.2 mm by default, in | |
| 3 | * the URDF's metres), the triangles that collapse or repeat dropped, and each face's normal worked out again. The | |
| 4 | * vendor's meshes are tessellated far finer than a phone can show; this is 6-8 times smaller on the wire and looks | |
| 5 | * the same. Only what the page downloads changes: the server's own geometry (src/kinematics.ts) reads the originals. | |
| 6 | */ | |
| 7 | ||
| 8 | const HEADER = 84; | |
| 9 | const TRIANGLE = 50; | |
| 10 | /** Bump when the output changes, so the model's cached URLs change with it. */ | |
| 11 | export const LIGHTEN_VERSION = 1; | |
| 12 | ||
| 13 | /** The mesh files a URDF names, relative to it. */ | |
| 14 | export function urdfFiles(urdf: string): string[] { | |
| 15 | return [...new Set([...urdf.matchAll(/filename="([^"]+)"/g)].map((m) => m[1]!.replace(/^package:\/\/[^/]+\//, "")))]; | |
| 16 | } | |
| 17 | ||
| 18 | /** Is this a binary STL whose length matches its triangle count? (ASCII STL and anything else pass through.) */ | |
| 19 | export function isBinaryStl(bytes: Uint8Array): boolean { | |
| 20 | if (bytes.length < HEADER) return false; | |
| 21 | const n = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength).getUint32(80, true); | |
| 22 | return bytes.length === HEADER + n * TRIANGLE; | |
| 23 | } | |
| 24 | ||
| 25 | export function lighten(bytes: Uint8Array, grid = 0.0002): Uint8Array { | |
| 26 | if (!isBinaryStl(bytes) || !(grid > 0)) return bytes; | |
| 27 | const src = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength); | |
| 28 | const n = src.getUint32(80, true); | |
| 29 | const ids = new Map<string, number>(); | |
| 30 | const points: number[] = []; | |
| 31 | const vertex = (o: number) => { | |
| 32 | const x = Math.round(src.getFloat32(o, true) / grid), y = Math.round(src.getFloat32(o + 4, true) / grid), z = Math.round(src.getFloat32(o + 8, true) / grid); | |
| 33 | const key = `${x},${y},${z}`; | |
| 34 | let id = ids.get(key); | |
| 35 | if (id === undefined) { | |
| 36 | id = points.length / 3; | |
| 37 | ids.set(key, id); | |
| 38 | points.push(x * grid, y * grid, z * grid); | |
| 39 | } | |
| 40 | return id; | |
| 41 | }; | |
| 42 | const kept: number[] = []; | |
| 43 | const seen = new Set<string>(); | |
| 44 | for (let i = 0; i < n; i++) { | |
| 45 | const o = HEADER + i * TRIANGLE + 12; | |
| 46 | const a = vertex(o), b = vertex(o + 12), c = vertex(o + 24); | |
| 47 | if (a === b || b === c || a === c) continue; | |
| 48 | // The same three corners in the same turn are the same face; the opposite turn is the other side, and stays. | |
| 49 | const r = Math.min(a, b, c); | |
| 50 | const key = r === a ? `${a},${b},${c}` : r === b ? `${b},${c},${a}` : `${c},${a},${b}`; | |
| 51 | if (seen.has(key)) continue; | |
| 52 | seen.add(key); | |
| 53 | kept.push(a, b, c); | |
| 54 | } | |
| 55 | const m = kept.length / 3; | |
| 56 | const out = new Uint8Array(HEADER + m * TRIANGLE); | |
| 57 | out.set(bytes.subarray(0, 80)); | |
| 58 | const dst = new DataView(out.buffer); | |
| 59 | dst.setUint32(80, m, true); | |
| 60 | for (let i = 0; i < m; i++) { | |
| 61 | const o = HEADER + i * TRIANGLE; | |
| 62 | const [a, b, c] = [kept[3 * i]! * 3, kept[3 * i + 1]! * 3, kept[3 * i + 2]! * 3]; | |
| 63 | const ux = points[b]! - points[a]!, uy = points[b + 1]! - points[a + 1]!, uz = points[b + 2]! - points[a + 2]!; | |
| 64 | const vx = points[c]! - points[a]!, vy = points[c + 1]! - points[a + 1]!, vz = points[c + 2]! - points[a + 2]!; | |
| 65 | const nx = uy * vz - uz * vy, ny = uz * vx - ux * vz, nz = ux * vy - uy * vx; | |
| 66 | const len = Math.hypot(nx, ny, nz) || 1; | |
| 67 | dst.setFloat32(o, nx / len, true); | |
| 68 | dst.setFloat32(o + 4, ny / len, true); | |
| 69 | dst.setFloat32(o + 8, nz / len, true); | |
| 70 | for (const [k, p] of [a, b, c].entries()) for (let j = 0; j < 3; j++) dst.setFloat32(o + 12 + k * 12 + j * 4, points[p + j]!, true); | |
| 71 | } | |
| 72 | return out; | |
| 73 | } |
Software/so101/src/pose-feed.ts added +87
| @@ -0,0 +1,87 @@ | ||
| 1 | /** | |
| 2 | * The arm's pose for everyone, read once: one reader asks the arm server's /status as fast as it answers, up to `hz` | |
| 3 | * times a second, and hands every answer to every listener. Nobody listening, nobody asks. A reader that fails waits | |
| 4 | * a little and tries again; listeners keep the last pose they had meanwhile. | |
| 5 | */ | |
| 6 | ||
| 7 | export interface Pose { | |
| 8 | status: Record<string, unknown>; | |
| 9 | /** Date.now() when the read that produced it was started, and when its answer came. */ | |
| 10 | askedAt: number; | |
| 11 | at: number; | |
| 12 | } | |
| 13 | export type PoseListener = (pose: Pose) => void; | |
| 14 | ||
| 15 | export interface PoseFeedOptions { | |
| 16 | /** Reads the arm's status once; throws when the arm server does not answer. */ | |
| 17 | read: () => Promise<Record<string, unknown>>; | |
| 18 | /** Most reads a second. */ | |
| 19 | hz?: number; | |
| 20 | /** How long to wait after a failed read. */ | |
| 21 | retryMs?: number; | |
| 22 | /** Told each round trip, ms, and each failure. */ | |
| 23 | onRead?: (ms: number) => void; | |
| 24 | onError?: (err: unknown) => void; | |
| 25 | } | |
| 26 | ||
| 27 | export class PoseFeed { | |
| 28 | private readonly listeners = new Set<PoseListener>(); | |
| 29 | private running = false; | |
| 30 | private closed = false; | |
| 31 | private wake: (() => void) | null = null; | |
| 32 | last: Pose | null = null; | |
| 33 | ||
| 34 | private readonly o: PoseFeedOptions; | |
| 35 | ||
| 36 | constructor(options: PoseFeedOptions) { | |
| 37 | this.o = options; | |
| 38 | } | |
| 39 | ||
| 40 | listen(listener: PoseListener): () => void { | |
| 41 | this.listeners.add(listener); | |
| 42 | if (this.last) listener(this.last); | |
| 43 | if (!this.running && !this.closed) void this.loop(); | |
| 44 | return () => void this.listeners.delete(listener); | |
| 45 | } | |
| 46 | ||
| 47 | get listening(): number { | |
| 48 | return this.listeners.size; | |
| 49 | } | |
| 50 | ||
| 51 | close(): void { | |
| 52 | this.closed = true; | |
| 53 | this.listeners.clear(); | |
| 54 | this.wake?.(); | |
| 55 | } | |
| 56 | ||
| 57 | private sleep(ms: number) { | |
| 58 | return new Promise<void>((resolve) => { | |
| 59 | const timer = setTimeout(() => ((this.wake = null), resolve()), ms); | |
| 60 | this.wake = () => (clearTimeout(timer), (this.wake = null), resolve()); | |
| 61 | }); | |
| 62 | } | |
| 63 | ||
| 64 | private async loop() { | |
| 65 | this.running = true; | |
| 66 | const period = 1000 / (this.o.hz ?? 30); | |
| 67 | try { | |
| 68 | while (this.listeners.size && !this.closed) { | |
| 69 | const askedAt = Date.now(); | |
| 70 | const started = performance.now(); | |
| 71 | try { | |
| 72 | const status = await this.o.read(); | |
| 73 | const ms = performance.now() - started; | |
| 74 | this.o.onRead?.(ms); | |
| 75 | this.last = { status, askedAt, at: Date.now() }; | |
| 76 | for (const l of this.listeners) l(this.last); | |
| 77 | await this.sleep(Math.max(0, period - ms)); | |
| 78 | } catch (err) { | |
| 79 | this.o.onError?.(err); | |
| 80 | await this.sleep(this.o.retryMs ?? 500); | |
| 81 | } | |
| 82 | } | |
| 83 | } finally { | |
| 84 | this.running = false; | |
| 85 | } | |
| 86 | } | |
| 87 | } |
Software/so101/src/relay.ts +107 −40
| @@ -22,8 +22,11 @@ | ||
| 22 | 22 | * The owner's speed cap (src/speed-cap.ts) is applied here too, to moves, poses, grips, playbacks and every stream frame. |
| 23 | 23 | * |
| 24 | 24 | * WS /live the lobby: the turn, the queue, the chat, and the arm's /status pushed when it changes |
| 25 | * WS /arm the viewer's live link: the pose at 30 Hz, the driver's commands, camera frames (src/viewer-socket.ts) | |
| 25 | 26 | * WS /machine read-only, for anyone: the arm's raw status, the link's round trip and frame rate, and every |
| 26 | 27 | * command in and out with every safety clamp, pushed as it happens (src/machine-feed.ts) |
| 28 | * GET /metrics how the link feels, as Prometheus text (src/telemetry.ts); with accept: application/json, the arm's own metrics | |
| 29 | * GET /health the same numbers as JSON, with two minutes of history for the admin's sparklines; the admin's only | |
| 27 | 30 | * POST /join {"name": "…"}: become someone here, no forge account needed (src/accounts.ts); the answer |
| 28 | 31 | * sets the HttpOnly session cookie and names you: {"you": {"id", "name", "owner"}} |
| 29 | 32 | * GET /admin, POST /admin the owner's page: pause, end a turn, take control, the settings, the log |
| @@ -61,7 +64,10 @@ import { LeaderCalibrations } from "./leader-calibrations.ts"; | ||
| 61 | 64 | import { proxyFor, whyNot } from "./egress.ts"; |
| 62 | 65 | import { CalibrationError } from "./leader-map.js"; |
| 63 | 66 | import type { Lobby } from "./lobby.ts"; |
| 67 | import { PoseFeed } from "./pose-feed.ts"; | |
| 64 | 68 | import { capBody, FrameCap } from "./speed-cap.ts"; |
| 69 | import { Telemetry } from "./telemetry.ts"; | |
| 70 | import { ViewerSockets, type ViewerSocketOptions } from "./viewer-socket.ts"; | |
| 65 | 71 | |
| 66 | 72 | export interface Remote { |
| 67 | 73 | /** The arm server's HTTP base, no trailing slash. */ |
| @@ -111,10 +117,17 @@ export interface RelayOptions { | ||
| 111 | 117 | statusEveryMs?: number; |
| 112 | 118 | /** Where HTTP_PROXY and NO_PROXY are read from; process.env by default. */ |
| 113 | 119 | env?: Record<string, string | undefined>; |
| 120 | /** Most pose reads a second from the arm server, shared by every viewer. */ | |
| 121 | poseHz?: number; | |
| 122 | /** Timings of the viewer sockets on /arm. */ | |
| 123 | viewer?: Pick<ViewerSocketOptions, "commandHz" | "endAfterMs" | "heartbeatMs" | "frameWindow" | "seenTimeoutMs" | "reportEveryMs">; | |
| 124 | /** How often the one-line link summary is logged; 0 never. */ | |
| 125 | logEveryMs?: number; | |
| 114 | 126 | } |
| 115 | 127 | |
| 116 | 128 | export interface RunningRelay { |
| 117 | 129 | readonly port: number; |
| 130 | readonly telemetry: Telemetry; | |
| 118 | 131 | link(): Link; |
| 119 | 132 | close(): Promise<void>; |
| 120 | 133 | } |
| @@ -599,6 +612,16 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> { | ||
| 599 | 612 | by = who!.name; |
| 600 | 613 | } |
| 601 | 614 | if (req.method === "GET" && url.pathname === "/link") return json(res, 200, link()); |
| 615 | if (req.method === "GET" && url.pathname === "/health") { | |
| 616 | // The history behind the admin's sparklines is the admin's; /metrics stays open for a scraper, current numbers only. | |
| 617 | const who = lobby?.who(req); | |
| 618 | if (lobby && !who?.owner) return json(res, who ? 403 : 401, { error: "only an admin can see the link's health" }); | |
| 619 | return json(res, 200, telemetry.json()); | |
| 620 | } | |
| 621 | if (req.method === "GET" && url.pathname === "/metrics" && !String(req.headers.accept ?? "").includes("application/json")) { | |
| 622 | res.writeHead(200, { "content-type": "text/plain; version=0.0.4; charset=utf-8", "cache-control": "no-store" }); | |
| 623 | return res.end(telemetry.prometheus()); | |
| 624 | } | |
| 602 | 625 | if (req.method === "GET" && url.pathname === "/status") return await status(res, path); |
| 603 | 626 | |
| 604 | 627 | const camera = CAMERA.exec(url.pathname); |
| @@ -672,24 +695,15 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> { | ||
| 672 | 695 | }); |
| 673 | 696 | /** Close codes a server may send on; the rest (1005, 1006, 1015) only describe a connection that died. */ |
| 674 | 697 | const sendable = (code: number) => (code >= 1000 && code <= 1003) || (code >= 1007 && code <= 1014) || (code >= 3000 && code <= 4999); |
| 675 | /** Each open /follow stream and whose it is, so a turn that ends can close it. */ | |
| 676 | const streams = new Map<WebSocket, string>(); | |
| 677 | 698 | /** |
| 678 | * `base` is the arm server's HTTP side, asked where the joints are so the speed cap starts from there. On the real arm, | |
| 679 | * frames are not passed on while the link reconnects: the stream goes quiet, and the arm server's dead-man holds the arm. | |
| 699 | * The owner's speed cap for one stream to the arm at `base`: each frame as it may go on, or null for none. | |
| 700 | * It asks where the joints are once at the start, and again (at most once a second) while a cap holds a joint back for want of it. | |
| 680 | 701 | */ |
| 681 | const bridge = (browser: WebSocket, arm: WebSocket, base: string, driver: string | null) => { | |
| 682 | const real = base !== simulation; | |
| 683 | if (driver !== null) { | |
| 684 | streams.set(browser, driver); | |
| 685 | browser.on("close", () => streams.delete(browser)); | |
| 686 | } | |
| 687 | arm.on("message", (data, binary) => browser.readyState === WebSocket.OPEN && browser.send(data, { binary })); | |
| 702 | const capper = (base: string) => { | |
| 688 | 703 | const cap = new FrameCap(); |
| 689 | 704 | let seeded = false; |
| 690 | 705 | let asking = false; |
| 691 | 706 | let askedAt = 0; |
| 692 | /** Where the arm is, once at the start and again (at most once a second) while a cap holds a joint back for want of it. */ | |
| 693 | 707 | const seed = () => { |
| 694 | 708 | if (!lobby || seeded || asking || Date.now() - askedAt < 1000) return; |
| 695 | 709 | asking = true; |
| @@ -705,30 +719,78 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> { | ||
| 705 | 719 | .finally(() => (asking = false)); |
| 706 | 720 | }; |
| 707 | 721 | seed(); |
| 708 | /** A frame as the panel shows it: servo id → ticks. */ | |
| 722 | const real = base !== simulation; | |
| 723 | /** A frame as the Machine panel shows it: servo id → ticks. */ | |
| 709 | 724 | const values = (text: string) => perServo(parseFrame(text) ?? []); |
| 710 | browser.on("message", (data, binary) => { | |
| 725 | return (frame: string): string | null => { | |
| 726 | // On the real arm, frames are not passed on while the link reconnects: the stream goes quiet, and the arm server's dead-man holds the arm. | |
| 727 | if (real && mode() !== "live") return void machine.note("clamp", "not passed on", { why: RECONNECTING }), null; | |
| 728 | const percent = lobby?.maxSpeedPercent ?? 100; | |
| 729 | if (percent >= 100) { | |
| 730 | if (lobby) cap.cap(frame, percent); // keeps each joint's last target, so a cap set mid-stream starts from there | |
| 731 | machine.note("sent", "follow frame", { values: values(frame) }); | |
| 732 | return frame; | |
| 733 | } | |
| 734 | if (!seeded) seed(); | |
| 735 | const capped = cap.cap(frame, percent); | |
| 736 | if (capped !== frame) machine.note("clamp", "speed cap", { percent, asked: values(frame), sent: capped === null ? {} : values(capped) }); | |
| 737 | if (capped !== null) machine.note("sent", "follow frame", { values: values(capped) }); | |
| 738 | return capped; | |
| 739 | }; | |
| 740 | }; | |
| 741 | /** | |
| 742 | * The arm's /follow: the live arm server's while it is live, the simulation's when there is none or it has been gone 60 s. | |
| 743 | * Reconnecting, or a live arm server that does not take the stream, is refused: driving never jumps to the simulation. | |
| 744 | */ | |
| 745 | const openFollow = async (): Promise<{ arm: WebSocket; base: string }> => { | |
| 746 | if (remote && mode() === "reconnecting") throw new Error(`${RECONNECTING}…`); | |
| 747 | if (remote && mode() === "live") { | |
| 748 | try { | |
| 749 | return { arm: await open(`${remote.ws}/follow`, readTimeoutMs), base: remote.http }; | |
| 750 | } catch (err) { | |
| 751 | throw new Error(`the arm server at ${remote.host} did not take the stream (${reason(err)})`); | |
| 752 | } | |
| 753 | } | |
| 754 | return { arm: await open(`${simulationWs}/follow`, readTimeoutMs), base: simulation }; | |
| 755 | }; | |
| 756 | ||
| 757 | // ------------------------------------------------------------------ WS /arm: the pose, the commands and the cameras, one socket per viewer | |
| 758 | let viewers: ViewerSockets | null = null; | |
| 759 | const telemetry = new Telemetry({ viewers: () => viewers?.count ?? 0, watching: () => viewers?.watching ?? 0, framesIn: () => cameras.framesIn }); | |
| 760 | const poses = new PoseFeed({ | |
| 761 | hz: options.poseHz ?? 30, | |
| 762 | read: async () => { | |
| 763 | // Reconnecting, the last known pose stands; the link is retrying, so nothing more is asked of the arm server meanwhile. | |
| 764 | if (remote && mode() === "reconnecting" && armLink?.last) return { ...armLink.last, source: source(), reconnecting: true }; | |
| 765 | const base = remote && mode() === "live" ? remote.http : simulation; | |
| 766 | const r = await fetch(`${base}/status`, { signal: AbortSignal.timeout(readTimeoutMs), cache: "no-store" }); | |
| 767 | if (!r.ok) throw new Error(`/status answered ${r.status}`); | |
| 768 | return { ...((await r.json()) as Record<string, unknown>), source: source() }; | |
| 769 | }, | |
| 770 | onRead: (ms) => telemetry.arm.add(ms), | |
| 771 | onError: () => telemetry.count("armErrors"), | |
| 772 | }); | |
| 773 | viewers = new ViewerSockets({ ...options.viewer, poses, cameras, telemetry, lobby, openFollow, capper, lostCode: LOST_CLOSE }); | |
| 774 | ||
| 775 | /** Each open /follow stream and whose it is, so a turn that ends can close it. */ | |
| 776 | const streams = new Map<WebSocket, string>(); | |
| 777 | /** `base` is the arm server's HTTP side, asked where the joints are so the speed cap starts from there. */ | |
| 778 | const bridge = (browser: WebSocket, arm: WebSocket, base: string, driver: string | null) => { | |
| 779 | if (driver !== null) { | |
| 780 | streams.set(browser, driver); | |
| 781 | browser.on("close", () => streams.delete(browser)); | |
| 782 | } | |
| 783 | arm.on("message", (data, binary) => browser.readyState === WebSocket.OPEN && browser.send(data, { binary })); | |
| 784 | const cap = capper(base); | |
| 785 | browser.on("message", (data) => { | |
| 711 | 786 | if (driver !== null) { |
| 712 | 787 | // The turn may have ended while this frame was on its way, before the close reaches the page: it is not passed on. |
| 713 | 788 | if (lobby && !lobby.turns.isDriver(driver) && !lobby.local) return void machine.note("clamp", "not passed on", { why: "the turn ended" }); |
| 714 | 789 | lobby?.activity(driver); |
| 715 | 790 | } |
| 716 | 791 | if (arm.readyState !== WebSocket.OPEN) return; |
| 717 | if (real && mode() !== "live") return void machine.note("clamp", "not passed on", { why: RECONNECTING }); | |
| 718 | const percent = lobby?.maxSpeedPercent ?? 100; | |
| 719 | const text = String(data); | |
| 720 | if (percent >= 100) { | |
| 721 | if (lobby) cap.cap(text, percent); // keeps each joint's last target, so a cap set mid-stream starts from there | |
| 722 | machine.note("sent", "follow frame", { values: values(text) }); | |
| 723 | arm.send(data, { binary }); | |
| 724 | return; | |
| 725 | } | |
| 726 | if (!seeded) seed(); | |
| 727 | const capped = cap.cap(text, percent); | |
| 728 | if (capped !== text) machine.note("clamp", "speed cap", { percent, asked: values(text), sent: capped === null ? {} : values(capped) }); | |
| 729 | if (capped === null) return; | |
| 730 | machine.note("sent", "follow frame", { values: values(capped) }); | |
| 731 | arm.send(capped); | |
| 792 | const capped = cap(String(data)); | |
| 793 | if (capped !== null) arm.send(capped); | |
| 732 | 794 | }); |
| 733 | 795 | arm.on("close", (code, why) => browser.close(sendable(code) ? code : 1011, why.length ? why.toString() : "the arm server went away")); |
| 734 | 796 | // A browser that closed the stream on purpose leaves the arm holding; one that vanished leaves it parking. |
| @@ -749,10 +811,11 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> { | ||
| 749 | 811 | }; |
| 750 | 812 | http.on("upgrade", (req: IncomingMessage, socket: Duplex, head: Buffer) => { |
| 751 | 813 | const url = new URL(req.url ?? "/", "http://x"); |
| 752 | if (url.pathname !== "/follow" && url.pathname !== "/machine" && !(url.pathname === "/live" && lobby)) return refuse(socket, "404 Not Found"); | |
| 814 | if (url.pathname !== "/follow" && url.pathname !== "/arm" && url.pathname !== "/machine" && !(url.pathname === "/live" && lobby)) return refuse(socket, "404 Not Found"); | |
| 753 | 815 | if (!sameOrigin(req)) return refuse(socket, "403 Forbidden"); |
| 754 | 816 | socket.on("error", () => socket.destroy()); |
| 755 | 817 | if (url.pathname === "/live") return lobby!.upgrade(req, socket, head); |
| 818 | if (url.pathname === "/arm") return viewers!.upgrade(req, socket, head); | |
| 756 | 819 | if (url.pathname === "/machine") return machine.upgrade(req, socket, head); |
| 757 | 820 | let driver: string | null = null; |
| 758 | 821 | if (lobby) { |
| @@ -769,17 +832,10 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> { | ||
| 769 | 832 | return wss.handleUpgrade(req, socket, head, (browser) => browser.close(1013, `${RECONNECTING}…`)); |
| 770 | 833 | } |
| 771 | 834 | try { |
| 772 | if (!remote || mode() !== "live") throw null; | |
| 773 | arm = await open(`${remote.ws}/follow`, readTimeoutMs); | |
| 774 | base = remote.http; | |
| 835 | ({ arm, base } = await openFollow()); | |
| 775 | 836 | } catch (err) { |
| 776 | if (err) return wss.handleUpgrade(req, socket, head, (browser) => browser.close(1013, `the arm server at ${remote!.host} did not take the stream (${reason(err)})`)); | |
| 777 | try { | |
| 778 | arm = await open(`${simulationWs}/follow`, readTimeoutMs); | |
| 779 | base = simulation; | |
| 780 | } catch { | |
| 781 | return refuse(socket, "502 Bad Gateway"); | |
| 782 | } | |
| 837 | if (remote && mode() !== "offline") return wss.handleUpgrade(req, socket, head, (browser) => browser.close(1013, (err as Error).message)); | |
| 838 | return refuse(socket, "502 Bad Gateway"); | |
| 783 | 839 | } |
| 784 | 840 | if (socket.destroyed) return arm.close(); |
| 785 | 841 | wss.handleUpgrade(req, socket, head, (browser) => bridge(browser, arm, base, driver)); |
| @@ -792,6 +848,7 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> { | ||
| 792 | 848 | const arrived = lobby?.onArrive((tell) => (pushed ? tell(pushed) : void push())); |
| 793 | 849 | const turnEnded = lobby?.onTurnEnd((who) => { |
| 794 | 850 | for (const [browser, id] of streams) if (id === who.id) browser.close(1000, "your turn ended"); |
| 851 | viewers!.endTurn(who.id); | |
| 795 | 852 | const base = remote && onArm() ? remote.http : simulation; |
| 796 | 853 | fetch(`${base}/stop`, { method: "POST", signal: AbortSignal.timeout(readTimeoutMs) }).then( |
| 797 | 854 | (r) => r.body?.cancel(), |
| @@ -805,16 +862,26 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> { | ||
| 805 | 862 | await (armLink as ArmLink | null)?.ready; // bounded by the first attempt's timeout: the first page already knows which arm it has |
| 806 | 863 | const timer = lobby ? setInterval(follow, followEveryMs) : undefined; |
| 807 | 864 | timer?.unref?.(); |
| 865 | const sampler = setInterval(() => telemetry.sample(), 2000); | |
| 866 | sampler.unref?.(); | |
| 867 | const logEveryMs = options.logEveryMs ?? 60_000; | |
| 868 | const logger = logEveryMs > 0 ? setInterval(() => console.log(telemetry.line()), logEveryMs) : undefined; | |
| 869 | logger?.unref?.(); | |
| 808 | 870 | const pusher = lobby ? setInterval(() => void push(), statusEveryMs) : undefined; |
| 809 | 871 | pusher?.unref?.(); |
| 810 | 872 | await new Promise<void>((resolve) => http.listen(options.port ?? 8102, options.host ?? "127.0.0.1", resolve)); |
| 811 | 873 | const address = http.address(); |
| 812 | 874 | return { |
| 813 | 875 | port: typeof address === "object" && address ? address.port : options.port ?? 8102, |
| 876 | telemetry, | |
| 814 | 877 | link, |
| 815 | 878 | close: () => |
| 816 | 879 | new Promise<void>((resolve) => { |
| 817 | 880 | clearInterval(timer); |
| 881 | clearInterval(sampler); | |
| 882 | clearInterval(logger); | |
| 883 | viewers!.close(); | |
| 884 | poses.close(); | |
| 818 | 885 | clearInterval(pusher); |
| 819 | 886 | settingsChanged?.(); |
| 820 | 887 | turnEnded?.(); |
Software/so101/src/server.test.ts +73 −4
| @@ -1,5 +1,5 @@ | ||
| 1 | 1 | import { afterAll, beforeAll, describe, expect, test } from "bun:test"; |
| 2 | import { existsSync, mkdtempSync, readFileSync, writeFileSync } from "node:fs"; | |
| 2 | import { existsSync, mkdtempSync, readFileSync, statSync, writeFileSync } from "node:fs"; | |
| 3 | 3 | import { connect } from "node:net"; |
| 4 | 4 | import { tmpdir } from "node:os"; |
| 5 | 5 | import { join } from "node:path"; |
| @@ -128,13 +128,59 @@ describe("viewer page", () => { | ||
| 128 | 128 | }); |
| 129 | 129 | |
| 130 | 130 | test("never builds HTML from people's text: event details and E-stop names go in as text nodes", async () => { |
| 131 | const html = await (await fetch(`${base}/`)).text(); | |
| 131 | // The page, and the Metrics panel it loads when opened (where the events are drawn). | |
| 132 | const page = await (await fetch(`${base}/`)).text(); | |
| 133 | const metrics = /\/ui\/[0-9a-f]{12}\/metrics\.js/.exec(page)?.[0]; | |
| 134 | expect(metrics).toBeDefined(); | |
| 135 | const html = page + "\n" + (await (await fetch(`${base}${metrics}`)).text()); | |
| 132 | 136 | const sinks = html.split("\n").filter((line) => /\.innerHTML\s*=/.test(line)); |
| 133 | 137 | expect(sinks.length).toBeGreaterThan(0); |
| 134 | 138 | for (const line of sinks) expect(line).not.toMatch(/detail|\.by\b|estop|chat|\bname\b/); |
| 135 | 139 | expect(html).toContain('$("events").replaceChildren('); |
| 136 | 140 | }); |
| 137 | 141 | |
| 142 | test("the admin's Health panel draws GET /health as sparklines, with the targets, only behind the relay, and loads when opened", async () => { | |
| 143 | const page = await (await fetch(`${base}/`)).text(); | |
| 144 | const path = /\/ui\/[0-9a-f]{12}\/health\.js/.exec(page)?.[0]; | |
| 145 | expect(path).toBeDefined(); | |
| 146 | expect(page).toContain(`const { openHealth } = await import("${path}")`); | |
| 147 | expect(page).not.toContain("sparkline.js"); | |
| 148 | expect(page).toContain('<button id="healthBtn" hidden>Health</button>'); | |
| 149 | expect(page).toContain('$("healthBtn").hidden = !relayed;'); | |
| 150 | const js = await (await fetch(`${base}${path}`)).text(); | |
| 151 | expect(js).toContain('import { sparkline } from "./sparkline.js";'); | |
| 152 | expect(js).toContain('const h = await api("/health");'); | |
| 153 | expect(js).toContain('["command → pose p95", "ms", (h) => h.commandRttMs?.p95, 250]'); | |
| 154 | expect(js).toContain('["camera frame age p95", "ms", (h) => h.frameAgeMs?.p95, 400]'); | |
| 155 | expect((await fetch(`${base}${path!.replace("health.js", "sparkline.js")}`)).status).toBe(200); | |
| 156 | }); | |
| 157 | ||
| 158 | test("the page drives, watches and shows the cameras over WS /arm, and falls back to the old paths while it is down", async () => { | |
| 159 | const page = await (await fetch(`${base}/`)).text(); | |
| 160 | const path = /\/ui\/[0-9a-f]{12}\/arm-socket\.js/.exec(page)?.[0]; | |
| 161 | expect(path).toBeDefined(); | |
| 162 | // Not in the first load: it comes once the first view is up, and until then the old paths serve. | |
| 163 | expect(page).toContain(`const { ArmSocket } = await import("${path}");`); | |
| 164 | expect(page).not.toContain(`<link rel="modulepreload" href="${path}">`); | |
| 165 | expect(page).toContain("startViews();\n openSock()"); | |
| 166 | // One socket on /arm, behind the relay: the pose paints the screen, frames fill the camera panes, the driver's targets go as commands. | |
| 167 | expect(page).toMatch(/new ArmSocket\(\{\s*url: `\$\{location\.protocol === "https:" \? "wss" : "ws"\}:\/\/\$\{location\.host\}\/arm`/); | |
| 168 | expect(page).toContain("onPose: (d) => applyStatus(d)"); | |
| 169 | expect(page).toContain("onFrame: (f) => showFrame(f)"); | |
| 170 | expect(page).toContain("sock?.watch("); | |
| 171 | expect(page).toContain("sock.command(frame)"); | |
| 172 | expect(page).toContain("sock.paint(c, f)"); | |
| 173 | // While it is down: /status polled, /follow for a drive, the MJPEG stream. | |
| 174 | expect(page).toContain('if (sockUp) return void setTimeout(poll, 500);'); | |
| 175 | expect(page).toContain('const r = await timed("/status");'); | |
| 176 | expect(page).toContain("/follow`);"); | |
| 177 | expect(page).toMatch(/else if \(sockUp\) \{[^\n]*\}[^\n]*\n\s*else if \(plan\.mode === "snapshot" && c\.snapshot\) snap\(c\);\n\s*else c\.img\.src = streamUrl\(c\);/); | |
| 178 | // The page asks for JSON, so /metrics behind the relay gives it the arm's metrics rather than the Prometheus text. | |
| 179 | expect(page).toContain('headers: { accept: "application/json" }'); | |
| 180 | const js = await (await fetch(`${base}${path}`)).text(); | |
| 181 | for (const t of ["watch", "cmd", "end", "seen", "report", "ping"]) expect(js).toContain(`t: "${t}"`); | |
| 182 | }); | |
| 183 | ||
| 138 | 184 | test("a first visit downloads under 50 KB gzipped: the page and the modules it imports up front; the rest comes when needed", async () => { |
| 139 | 185 | const page = await (await fetch(`${base}/`)).text(); |
| 140 | 186 | const script = page.slice(page.indexOf('<script type="module">')); |
| @@ -147,11 +193,12 @@ describe("viewer page", () => { | ||
| 147 | 193 | bytes += gzipSync(Buffer.from(await r.arrayBuffer()), { level: 6 }).length; |
| 148 | 194 | } |
| 149 | 195 | expect(bytes).toBeLessThan(50_000); |
| 150 | // three.js, the leader's mapping, the calibration panel and the Machine panel are imported where they are used, not up front. | |
| 151 | for (const lazy of ["three", "/leader-map.js", "calibration.js", "reach.js", "machine-panel.js", "machine.js"]) expect([...upfront].some((p) => p === lazy || p.endsWith(`/${lazy}`))).toBe(false); | |
| 196 | // three.js, the leader's mapping, the calibration, Machine and Metrics panels are imported where they are used, not up front. | |
| 197 | for (const lazy of ["three", "/leader-map.js", "calibration.js", "reach.js", "machine-panel.js", "machine.js", "metrics.js"]) expect([...upfront].some((p) => p === lazy || p.endsWith(`/${lazy}`))).toBe(false); | |
| 152 | 198 | expect(script).toContain('import("/leader-map.js")'); |
| 153 | 199 | expect(script).toMatch(/import\("\/ui\/[0-9a-f]{12}\/calibration\.js"\)/); |
| 154 | 200 | expect(script).toMatch(/import\("\/ui\/[0-9a-f]{12}\/machine-panel\.js"\)/); |
| 201 | expect(script).toMatch(/import\("\/ui\/[0-9a-f]{12}\/metrics\.js"\)/); | |
| 155 | 202 | }); |
| 156 | 203 | |
| 157 | 204 | test("names its modules by their content, serves them for good, and serves nothing else from src/ui", async () => { |
| @@ -476,6 +523,28 @@ describe("the safe zone, the jam watch, and the model routes", () => { | ||
| 476 | 523 | expect((await fetch(`${zurl}/vendor/three/../ws/index.js`)).status).toBe(404); |
| 477 | 524 | }); |
| 478 | 525 | |
| 526 | test("the model and the vendored libraries go under versioned names, cached for a year; the meshes lightened", async () => { | |
| 527 | const info = (await (await fetch(`${zurl}/model.json`)).json()) as { urdf: string }; | |
| 528 | const version = /^\/model\/([0-9a-f]{12})\/so101_new_calib\.urdf$/.exec(info.urdf)?.[1]; | |
| 529 | expect(version).toBeDefined(); | |
| 530 | expect((await fetch(`${zurl}${info.urdf}`)).headers.get("cache-control")).toBe("public, max-age=31536000, immutable"); | |
| 531 | // Under an old version (a page from before a deploy): still served, but not kept for good. | |
| 532 | expect((await fetch(`${zurl}/model/000000000000/so101_new_calib.urdf`)).headers.get("cache-control")).toBe("no-cache"); | |
| 533 | expect((await fetch(`${zurl}/model/so101_new_calib.urdf`)).headers.get("cache-control")).toBe("max-age=86400"); | |
| 534 | expect((await fetch(`${zurl}/model/${version}/../package.json`)).status).toBe(404); | |
| 535 | const mesh = await fetch(`${zurl}/model/${version}/assets/wrist_roll_pitch_so101_v2.stl`); | |
| 536 | const original = statSync(new URL("../../../Simulation/SO101/assets/wrist_roll_pitch_so101_v2.stl", import.meta.url)).size; | |
| 537 | const bytes = new Uint8Array(await mesh.arrayBuffer()); | |
| 538 | expect(bytes.length * 2).toBeLessThan(original); | |
| 539 | expect(bytes.length).toBe(84 + new DataView(bytes.buffer).getUint32(80, true) * 50); // still a binary STL | |
| 540 | const page = await (await fetch(`${zurl}/`)).text(); | |
| 541 | const vendor = /"three": "\/vendor\/([0-9a-f]{12})\/three\/build\/three\.module\.js"/.exec(page)?.[1]; | |
| 542 | expect(vendor).toBeDefined(); | |
| 543 | expect(page).not.toContain("__VENDOR__"); | |
| 544 | expect((await fetch(`${zurl}/vendor/${vendor}/three/build/three.module.js`)).headers.get("cache-control")).toBe("public, max-age=31536000, immutable"); | |
| 545 | expect((await fetch(`${zurl}/vendor/${vendor}/ws/index.js`)).status).toBe(404); | |
| 546 | }); | |
| 547 | ||
| 479 | 548 | test("serves the printable sheets, and nothing else from that folder", async () => { |
| 480 | 549 | const list = (await (await fetch(`${zurl}/printables`)).json()) as Array<{ file: string; url: string; bytes: number }>; |
| 481 | 550 | expect(list.map((p) => p.file)).toEqual(["charuco_board.png", "cube_markers.png", "gripper_marker.png"]); |
Software/so101/src/server.ts +77 −7
| @@ -57,6 +57,7 @@ import type { FeetechBus } from "./bus.ts"; | ||
| 57 | 57 | import { ArmModel, gripperTicksToRadians, ticksToRadians, type Violation } from "./kinematics.ts"; |
| 58 | 58 | import { clamp, type Frame } from "./mapping.ts"; |
| 59 | 59 | import { MachineLog, perServo } from "./machine-log.ts"; |
| 60 | import { lighten, LIGHTEN_VERSION, urdfFiles } from "./mesh-lite.ts"; | |
| 60 | 61 | import { Metrics } from "./metrics.ts"; |
| 61 | 62 | import { MotionError, moveToPose, playTrajectory, recordTrajectory, STALL_SPAN_TICKS } from "./motion.ts"; |
| 62 | 63 | import { assertPoseName, parsePoseFile, serializePoseFile, type PoseFile } from "./poses.ts"; |
| @@ -72,6 +73,8 @@ export interface ServerOptions { | ||
| 72 | 73 | posesFile?: string; |
| 73 | 74 | /** Folder holding so101_new_calib.urdf and assets/; without it there is no safe zone and no 3D model. */ |
| 74 | 75 | modelDir?: string; |
| 76 | /** Work out the model's version and lightened meshes at start rather than on the first request. */ | |
| 77 | warmModel?: boolean; | |
| 75 | 78 | /** Folder with the printable PNG sheets of the vision kit; found beside the package or in the repo layout. */ |
| 76 | 79 | printablesDir?: string; |
| 77 | 80 | /** Torque cap while a stream drives the arm, percent. */ |
| @@ -145,6 +148,7 @@ const JAM_WINDOW = 6; | ||
| 145 | 148 | /** Consecutive too-far frames for one joint before it is eased over to the leader instead of left behind. */ |
| 146 | 149 | const RESUME_AFTER_DROPS = 20; |
| 147 | 150 | const VENDORED = new Set(["three", "urdf-loader"]); |
| 151 | const MODEL_FILES = [".urdf", ".stl", ".dae", ".png", ".jpg"]; | |
| 148 | 152 | /** A module the page loads from src/ui: plain JavaScript, never its tests. */ |
| 149 | 153 | const isUiModule = (name: string) => /^[a-z][a-z0-9-]*\.js$/.test(name); |
| 150 | 154 | const MIME: Record<string, string> = { ".js": "text/javascript", ".mjs": "text/javascript", ".json": "application/json", ".urdf": "application/xml", ".stl": "model/stl", ".dae": "model/vnd.collada+xml", ".png": "image/png", ".jpg": "image/jpeg" }; |
| @@ -635,6 +639,44 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}): | ||
| 635 | 639 | return hash.digest("hex").slice(0, 12); |
| 636 | 640 | }; |
| 637 | 641 | const nodeModules = resolve(here, "../node_modules"); |
| 642 | /** | |
| 643 | * The model and the vendored libraries, like the page's modules, under names that change with their content: the | |
| 644 | * page asks for /model/<version>/… and /vendor/<version>/…, cached for a year; the plain paths stay, cached a day. | |
| 645 | */ | |
| 646 | const hashOf = (parts: Iterable<string | Uint8Array>) => { | |
| 647 | const hash = createHash("sha256"); | |
| 648 | for (const p of parts) hash.update(p); | |
| 649 | return hash.digest("hex").slice(0, 12); | |
| 650 | }; | |
| 651 | let modelVersionMemo: string | undefined; | |
| 652 | const modelVersion = () => | |
| 653 | (modelVersionMemo ??= hashOf( | |
| 654 | (function* () { | |
| 655 | yield `lighten ${LIGHTEN_VERSION}`; | |
| 656 | const walk = function* (dir: string, rel: string): Generator<string | Uint8Array> { | |
| 657 | for (const f of readdirSync(dir, { withFileTypes: true }).sort((a, b) => a.name.localeCompare(b.name))) { | |
| 658 | if (f.isDirectory()) yield* walk(join(dir, f.name), `${rel}${f.name}/`); | |
| 659 | else if (MODEL_FILES.includes(extname(f.name).toLowerCase())) yield `${rel}${f.name}`, yield readFileSync(join(dir, f.name)); | |
| 660 | } | |
| 661 | }; | |
| 662 | if (modelDir) yield* walk(resolve(modelDir), ""); | |
| 663 | })(), | |
| 664 | )); | |
| 665 | let vendorVersionMemo: string | undefined; | |
| 666 | const vendorVersion = () => | |
| 667 | (vendorVersionMemo ??= hashOf( | |
| 668 | [...VENDORED].map((pkg) => { | |
| 669 | try { | |
| 670 | return `${pkg}@${(JSON.parse(readFileSync(join(nodeModules, pkg, "package.json"), "utf8")) as { version?: string }).version}`; | |
| 671 | } catch { | |
| 672 | return pkg; | |
| 673 | } | |
| 674 | }), | |
| 675 | )); | |
| 676 | /** Meshes go out lightened (src/mesh-lite.ts), worked out once per file. */ | |
| 677 | const lightened = new Map<string, Uint8Array>(); | |
| 678 | /** Asked under the current version: kept for good. Under an old one (a page from before a deploy): served, but checked next time. */ | |
| 679 | const versioned = (asked: string, current: string, plain: string) => (asked === current ? "public, max-age=31536000, immutable" : asked ? "no-cache" : plain); | |
| 638 | 680 | const http = createServer(async (req, res) => { |
| 639 | 681 | try { |
| 640 | 682 | const url = new URL(req.url ?? "/", "http://x"); |
| @@ -647,9 +689,15 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}): | ||
| 647 | 689 | } |
| 648 | 690 | }; |
| 649 | 691 | if (req.method === "GET" && url.pathname.startsWith("/model/") && modelDir) { |
| 650 | const path = safeFile(resolve(modelDir), decode(url.pathname.slice("/model/".length))); | |
| 651 | if (!path || ![".urdf", ".stl", ".dae", ".png", ".jpg"].includes(extname(path).toLowerCase())) return json(res, 404, { error: "no such model file" }); | |
| 652 | return file(res, path, "max-age=86400"); | |
| 692 | const [, asked = "", rel = ""] = /^\/model\/(?:([0-9a-f]{12})\/)?(.*)$/.exec(url.pathname)!; | |
| 693 | const path = safeFile(resolve(modelDir), decode(rel)); | |
| 694 | if (!path || !MODEL_FILES.includes(extname(path).toLowerCase())) return json(res, 404, { error: "no such model file" }); | |
| 695 | const cache = versioned(asked, modelVersion(), "max-age=86400"); | |
| 696 | if (extname(path).toLowerCase() !== ".stl") return file(res, path, cache); | |
| 697 | let body = lightened.get(path); | |
| 698 | if (!body) lightened.set(path, (body = lighten(new Uint8Array(readFileSync(path))))); | |
| 699 | res.writeHead(200, { "content-type": MIME[".stl"]!, "cache-control": cache }); | |
| 700 | return res.end(body); | |
| 653 | 701 | } |
| 654 | 702 | if (req.method === "GET" && url.pathname === "/printables") { |
| 655 | 703 | let files: string[] = []; |
| @@ -666,10 +714,12 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}): | ||
| 666 | 714 | return file(res, path, "max-age=3600"); |
| 667 | 715 | } |
| 668 | 716 | if (req.method === "GET" && url.pathname.startsWith("/vendor/")) { |
| 669 | const [pkg, ...rest] = decode(url.pathname.slice("/vendor/".length)).split("/"); | |
| 717 | const parts = decode(url.pathname.slice("/vendor/".length)).split("/"); | |
| 718 | const asked = /^[0-9a-f]{12}$/.test(parts[0] ?? "") ? parts.shift()! : ""; | |
| 719 | const [pkg, ...rest] = parts; | |
| 670 | 720 | const path = pkg && VENDORED.has(pkg) && rest.length ? safeFile(join(nodeModules, pkg), rest.join("/")) : null; |
| 671 | 721 | if (!path || ![".js", ".mjs", ".json"].includes(extname(path).toLowerCase())) return json(res, 404, { error: "no such vendored file" }); |
| 672 | return file(res, path, "max-age=86400"); | |
| 722 | return file(res, path, versioned(asked, vendorVersion(), "max-age=86400")); | |
| 673 | 723 | } |
| 674 | 724 | const ui = /^\/ui\/[0-9a-f]{12}\/([a-z][a-z0-9-]*\.js)$/.exec(url.pathname); |
| 675 | 725 | if (req.method === "GET" && ui) { |
| @@ -697,7 +747,7 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}): | ||
| 697 | 747 | case "GET /": |
| 698 | 748 | case "GET /index.html": |
| 699 | 749 | res.writeHead(200, { "content-type": "text/html; charset=utf-8", "cache-control": "no-store" }); |
| 700 | return res.end(readFileSync(viewerPath, "utf8").replaceAll("/ui/__VERSION__/", `/ui/${uiVersion()}/`)); // read per request: a viewer change deploys with rsync alone | |
| 750 | return res.end(readFileSync(viewerPath, "utf8").replaceAll("/ui/__VERSION__/", `/ui/${uiVersion()}/`).replaceAll("/vendor/__VENDOR__/", `/vendor/${vendorVersion()}/`)); // read per request: a viewer change deploys with rsync alone | |
| 701 | 751 | case "GET /leader-map.js": |
| 702 | 752 | return file(res, join(here, "leader-map.js"), "no-cache"); |
| 703 | 753 | case "GET /status": |
| @@ -727,7 +777,7 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}): | ||
| 727 | 777 | case "GET /model.json": |
| 728 | 778 | if (!model) return json(res, 404, { error: "no model loaded on this server" }); |
| 729 | 779 | return json(res, 200, { |
| 730 | urdf: `/model/${URDF}`, | |
| 780 | urdf: `/model/${modelVersion()}/${URDF}`, | |
| 731 | 781 | floorZ: model.floorZ, |
| 732 | 782 | baseTop: model.baseTop, |
| 733 | 783 | joints: model.movable.map((j) => ({ name: j.name, id: Number(Object.entries(JOINTS).find(([, n]) => n === j.name)?.[0] ?? 0), lower: j.lower, upper: j.upper })), |
| @@ -1254,10 +1304,30 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}): | ||
| 1254 | 1304 | await new Promise<void>((resolve) => http.listen(options.port ?? 8101, options.host ?? "0.0.0.0", resolve)); |
| 1255 | 1305 | const address = http.address(); |
| 1256 | 1306 | const port = typeof address === "object" && address ? address.port : options.port ?? 8101; |
| 1307 | // The model's version and its lightened meshes, worked out now, a file a turn, rather than inside a first visitor's | |
| 1308 | // request while the cameras and the poses wait on the same thread. | |
| 1309 | let warming: ReturnType<typeof setTimeout> | undefined; | |
| 1310 | if (options.warmModel && modelDir) { | |
| 1311 | let urdf = ""; | |
| 1312 | try { | |
| 1313 | urdf = readFileSync(join(modelDir, URDF), "utf8"); | |
| 1314 | } catch { | |
| 1315 | /* no model */ | |
| 1316 | } | |
| 1317 | const todo = urdfFiles(urdf).map((f) => safeFile(resolve(modelDir), f)).filter((p): p is string => p !== null && extname(p).toLowerCase() === ".stl"); | |
| 1318 | const step = () => { | |
| 1319 | const path = todo.shift(); | |
| 1320 | if (!path) return void modelVersion(); | |
| 1321 | if (!lightened.has(path)) lightened.set(path, lighten(new Uint8Array(readFileSync(path)))); | |
| 1322 | warming = setTimeout(step, 0); | |
| 1323 | }; | |
| 1324 | warming = setTimeout(step, 0); | |
| 1325 | } | |
| 1257 | 1326 | return { |
| 1258 | 1327 | port, |
| 1259 | 1328 | close: () => |
| 1260 | 1329 | new Promise<void>((resolve) => { |
| 1330 | clearTimeout(warming); | |
| 1261 | 1331 | clearInterval(watching); |
| 1262 | 1332 | machine.close(); |
| 1263 | 1333 | bus.onSyncWrite = null; |
Software/so101/src/telemetry.test.ts added +63
| @@ -0,0 +1,63 @@ | ||
| 1 | import { expect, test } from "bun:test"; | |
| 2 | import { Latest } from "./latest.ts"; | |
| 3 | import { earn, noCredit, parseReport, Telemetry, vouch, Window } from "./telemetry.ts"; | |
| 4 | ||
| 5 | test("a window keeps a span of samples and answers quantiles over it", () => { | |
| 6 | const w = new Window(1000); | |
| 7 | for (let i = 1; i <= 100; i++) w.add(i, 10_000); | |
| 8 | expect(w.quantile(0.5, 10_000)).toBe(50); | |
| 9 | expect(w.quantile(0.95, 10_000)).toBe(95); | |
| 10 | w.add(500, 10_900); | |
| 11 | expect(w.sum(200, 10_900)).toBe(500); | |
| 12 | expect(w.quantile(0.5, 11_500)).toBe(500); // the first hundred are older than a second now | |
| 13 | expect(w.quantile(0.5, 13_000)).toBeNull(); | |
| 14 | }); | |
| 15 | ||
| 16 | test("a report is checked: nonsense dropped, counts capped", () => { | |
| 17 | expect(parseReport(null)).toBeNull(); | |
| 18 | expect(parseReport({ rtt: [10, -1, "x", 1e9, Infinity], age: "no", frames: 2.7, sent: -3, dropped: 1e12 })).toEqual({ rtt: [10], age: [], frames: 2, sent: 0, dropped: 100_000, stalls: 0, reconnects: 0 }); | |
| 19 | expect(parseReport({ rtt: Array(500).fill(1) })!.rtt).toHaveLength(200); | |
| 20 | }); | |
| 21 | ||
| 22 | test("a report is held to what crossed its socket: no more commands or frames than the app saw", () => { | |
| 23 | const claim = { rtt: Array(200).fill(0), age: Array(200).fill(0), frames: 100_000, sent: 100_000, dropped: 100_000, stalls: 100_000, reconnects: 100_000 }; | |
| 24 | const credit = earn(earn(earn(noCredit(), "command"), "command", 2), "frames", 2); | |
| 25 | expect(vouch(claim, credit)).toEqual({ report: { rtt: [0, 0, 0], age: [0, 0], frames: 2, sent: 3, dropped: 2_000, stalls: 10, reconnects: 10 }, left: noCredit() }); | |
| 26 | expect(vouch(claim, noCredit()).report).toMatchObject({ rtt: [], age: [], frames: 0, sent: 0 }); | |
| 27 | // A round trip measured after the report that counted its command still has its command's credit. | |
| 28 | const first = vouch({ ...claim, rtt: [], frames: 0, age: [] }, credit); | |
| 29 | expect(first.left).toEqual({ rtt: 3, sent: 0, age: 2, frames: 2 }); | |
| 30 | expect(vouch({ ...claim, rtt: [140], sent: 0 }, first.left).report.rtt).toEqual([140]); | |
| 31 | }); | |
| 32 | ||
| 33 | test("the numbers come out as Prometheus text, JSON with history, and one line", () => { | |
| 34 | let framesIn = 0; | |
| 35 | const t = new Telemetry({ viewers: () => 2, watching: () => 1, framesIn: () => framesIn }); | |
| 36 | t.report({ rtt: [100, 200], age: [300], frames: 50, sent: 20, dropped: 5, stalls: 0, reconnects: 2 }, 1000); | |
| 37 | t.arm.add(40, 1000); | |
| 38 | framesIn = 100; | |
| 39 | t.sample(1000); | |
| 40 | const h = t.json(2000); | |
| 41 | expect(h).toMatchObject({ viewers: 2, watching: 1, commandRttMs: { p50: 100, p95: 200 }, frameAgeMs: { p50: 300 }, viewerFps: 5, cameraFpsIn: 10, commandsPerS: 2 }); | |
| 42 | expect(h.history).toHaveLength(1); | |
| 43 | const text = t.prometheus(2000); | |
| 44 | expect(text).toContain("# TYPE arm_command_rtt_ms summary"); | |
| 45 | expect(text).toContain("arm_commands_dropped_total 5"); | |
| 46 | expect(text).toContain("arm_reconnects_total 2"); | |
| 47 | expect(t.line(2000)).toContain("command→pose p50/p95 100/200ms"); | |
| 48 | }); | |
| 49 | ||
| 50 | test("latest wins: while the window is full, newer messages replace waiting ones", () => { | |
| 51 | const out: string[] = []; | |
| 52 | const dones: Array<() => void> = []; | |
| 53 | const l = new Latest((data, done) => (out.push(String(data)), dones.push(done), true), 2); | |
| 54 | for (const n of ["a", "b", "c", "d", "e"]) l.offer("cam", n); | |
| 55 | expect(out).toEqual(["a", "b"]); | |
| 56 | expect(l.skipped).toBe(2); // c and d were replaced while waiting | |
| 57 | dones.shift()!(); | |
| 58 | expect(out).toEqual(["a", "b", "e"]); | |
| 59 | dones.shift()!(); | |
| 60 | dones.shift()!(); | |
| 61 | l.offer("cam", "f"); | |
| 62 | expect(out).toEqual(["a", "b", "e", "f"]); | |
| 63 | }); |
Software/so101/src/telemetry.ts added +291
| @@ -0,0 +1,291 @@ | ||
| 1 | /** | |
| 2 | * How the link feels, in numbers: what each viewer measures (command round trip, camera frame age, frames shown, | |
| 3 | * stalls) and what the app sees itself (the arm server's round trip, the frames the cameras send, bytes out, frames | |
| 4 | * a slow viewer had to skip). Kept for the last minute in memory, nothing on disk: it is for iterating, not history. | |
| 5 | * | |
| 6 | * prometheus() the text a Prometheus scrape reads, on GET /metrics | |
| 7 | * health() the same as JSON with a two-minute history for the sparklines, on GET /health | |
| 8 | * line() the one-line summary logged every minute (summary(), which the terminal's /health prints too) | |
| 9 | */ | |
| 10 | ||
| 11 | /** Timed samples, kept for `spanMs`. */ | |
| 12 | export class Window { | |
| 13 | private t: number[] = []; | |
| 14 | private v: number[] = []; | |
| 15 | private readonly spanMs: number; | |
| 16 | private readonly cap: number; | |
| 17 | ||
| 18 | constructor(spanMs: number, cap = 10_000) { | |
| 19 | this.spanMs = spanMs; | |
| 20 | this.cap = cap; | |
| 21 | } | |
| 22 | ||
| 23 | add(value: number, now = Date.now()): void { | |
| 24 | this.t.push(now); | |
| 25 | this.v.push(value); | |
| 26 | this.prune(now); | |
| 27 | if (this.t.length > this.cap) this.t.splice(0, this.t.length - this.cap), this.v.splice(0, this.v.length - this.cap); | |
| 28 | } | |
| 29 | ||
| 30 | private prune(now: number) { | |
| 31 | let i = 0; | |
| 32 | while (i < this.t.length && now - this.t[i]! > this.spanMs) i++; | |
| 33 | if (i) this.t.splice(0, i), this.v.splice(0, i); | |
| 34 | } | |
| 35 | ||
| 36 | /** The value below which a fraction `q` of the samples lie; null with no samples. */ | |
| 37 | quantile(q: number, now = Date.now()): number | null { | |
| 38 | this.prune(now); | |
| 39 | if (!this.v.length) return null; | |
| 40 | const sorted = [...this.v].sort((a, b) => a - b); | |
| 41 | return sorted[Math.min(sorted.length - 1, Math.max(0, Math.ceil(q * sorted.length) - 1))]!; | |
| 42 | } | |
| 43 | ||
| 44 | sum(withinMs = this.spanMs, now = Date.now()): number { | |
| 45 | this.prune(now); | |
| 46 | let s = 0; | |
| 47 | for (let i = this.t.length - 1; i >= 0 && now - this.t[i]! <= withinMs; i--) s += this.v[i]!; | |
| 48 | return s; | |
| 49 | } | |
| 50 | ||
| 51 | count(now = Date.now()): number { | |
| 52 | this.prune(now); | |
| 53 | return this.v.length; | |
| 54 | } | |
| 55 | } | |
| 56 | ||
| 57 | /** What one viewer measured since its last report. */ | |
| 58 | export interface Report { | |
| 59 | /** Command → pose round trips, ms. */ | |
| 60 | rtt: number[]; | |
| 61 | /** Camera frame ages when shown (the app's receipt → on screen), ms. */ | |
| 62 | age: number[]; | |
| 63 | /** Camera frames shown. */ | |
| 64 | frames: number; | |
| 65 | /** Commands sent, and the ones merged away before sending (latest wins). */ | |
| 66 | sent: number; | |
| 67 | dropped: number; | |
| 68 | /** Times the socket went quiet past the stall limit, and reconnects. */ | |
| 69 | stalls: number; | |
| 70 | reconnects: number; | |
| 71 | } | |
| 72 | ||
| 73 | const MAX_SAMPLES = 200; | |
| 74 | const MAX_MS = 120_000; | |
| 75 | const MAX_COUNT = 100_000; | |
| 76 | ||
| 77 | /** A viewer's report, checked: null when it is not one. Samples outside 0..120 s are dropped, counts capped. */ | |
| 78 | export function parseReport(raw: unknown): Report | null { | |
| 79 | if (!raw || typeof raw !== "object") return null; | |
| 80 | const r = raw as Record<string, unknown>; | |
| 81 | const samples = (x: unknown) => (Array.isArray(x) ? x.slice(0, MAX_SAMPLES).filter((n): n is number => typeof n === "number" && Number.isFinite(n) && n >= 0 && n <= MAX_MS) : []); | |
| 82 | const count = (x: unknown) => (typeof x === "number" && Number.isFinite(x) && x >= 0 ? Math.min(MAX_COUNT, Math.floor(x)) : 0); | |
| 83 | return { rtt: samples(r.rtt), age: samples(r.age), frames: count(r.frames), sent: count(r.sent), dropped: count(r.dropped), stalls: count(r.stalls), reconnects: count(r.reconnects) }; | |
| 84 | } | |
| 85 | ||
| 86 | /** Merged-away commands, stalls and reconnects one report may claim: the app cannot see these, so they are only bounded. */ | |
| 87 | const MAX_DROPPED = 2_000; | |
| 88 | const MAX_EVENTS = 10; | |
| 89 | ||
| 90 | /** What a socket has earned the right to report: one round trip and one sent per command it received, one frame and one age per frame it was sent. */ | |
| 91 | export interface Credit { | |
| 92 | rtt: number; | |
| 93 | sent: number; | |
| 94 | age: number; | |
| 95 | frames: number; | |
| 96 | } | |
| 97 | ||
| 98 | export const noCredit = (): Credit => ({ rtt: 0, sent: 0, age: 0, frames: 0 }); | |
| 99 | ||
| 100 | /** Credit for one command, or for `n` frames sent. */ | |
| 101 | export const earn = (c: Credit, kind: "command" | "frames", n = 1): Credit => | |
| 102 | kind === "command" ? { ...c, rtt: c.rtt + n, sent: c.sent + n } : { ...c, age: c.age + n, frames: c.frames + n }; | |
| 103 | ||
| 104 | /** | |
| 105 | * A report held to what crossed its socket: no more commands (or round trips) than it received, no more frames (or | |
| 106 | * frame ages) than it was sent. What a report doesn't claim carries over, since a round trip is measured when the pose | |
| 107 | * that acknowledges it comes back, often in a later report than the command. A page cannot claim work that never | |
| 108 | * crossed its socket. | |
| 109 | */ | |
| 110 | export function vouch(r: Report, credit: Credit): { report: Report; left: Credit } { | |
| 111 | const report: Report = { | |
| 112 | rtt: r.rtt.slice(0, credit.rtt), | |
| 113 | age: r.age.slice(0, credit.age), | |
| 114 | frames: Math.min(r.frames, credit.frames), | |
| 115 | sent: Math.min(r.sent, credit.sent), | |
| 116 | dropped: Math.min(r.dropped, MAX_DROPPED), | |
| 117 | stalls: Math.min(r.stalls, MAX_EVENTS), | |
| 118 | reconnects: Math.min(r.reconnects, MAX_EVENTS), | |
| 119 | }; | |
| 120 | const left = { rtt: credit.rtt - report.rtt.length, sent: credit.sent - report.sent, age: credit.age - report.age.length, frames: credit.frames - report.frames }; | |
| 121 | return { report, left }; | |
| 122 | } | |
| 123 | ||
| 124 | export interface Gauges { | |
| 125 | /** Open viewer sockets. */ | |
| 126 | viewers(): number; | |
| 127 | /** Viewers watching at least one camera. */ | |
| 128 | watching(): number; | |
| 129 | /** Whole frames the cameras have sent the app, ever. */ | |
| 130 | framesIn(): number; | |
| 131 | } | |
| 132 | ||
| 133 | export interface Quantiles { | |
| 134 | p50: number | null; | |
| 135 | p95: number | null; | |
| 136 | } | |
| 137 | ||
| 138 | export interface Health { | |
| 139 | at: number; | |
| 140 | viewers: number; | |
| 141 | watching: number; | |
| 142 | commandRttMs: Quantiles; | |
| 143 | frameAgeMs: Quantiles; | |
| 144 | /** Frames shown per second per watching viewer, over the last 10 s. */ | |
| 145 | viewerFps: number; | |
| 146 | armRttMs: Quantiles; | |
| 147 | /** Frames per second coming in from the cameras, over the last 10 s. */ | |
| 148 | cameraFpsIn: number; | |
| 149 | bytesOutPerS: number; | |
| 150 | commandsPerS: number; | |
| 151 | totals: Record<Counter, number>; | |
| 152 | } | |
| 153 | ||
| 154 | export type Counter = "commandsSent" | "commandsDropped" | "commandsMerged" | "framesShown" | "framesSent" | "framesSkipped" | "stalls" | "reconnects" | "bytesOut" | "armErrors" | "reportsDropped"; | |
| 155 | const COUNTERS: Counter[] = ["commandsSent", "commandsDropped", "commandsMerged", "framesShown", "framesSent", "framesSkipped", "stalls", "reconnects", "bytesOut", "armErrors", "reportsDropped"]; | |
| 156 | const HELP: Record<Counter, string> = { | |
| 157 | commandsSent: "commands viewers sent", | |
| 158 | commandsDropped: "commands viewers merged away before sending (latest wins)", | |
| 159 | commandsMerged: "commands the app merged away before the arm server (latest wins)", | |
| 160 | framesShown: "camera frames viewers showed", | |
| 161 | framesSent: "camera frames the app sent viewers", | |
| 162 | framesSkipped: "camera frames the app skipped for a viewer still busy with an earlier one", | |
| 163 | stalls: "viewer sockets that went quiet past the stall limit", | |
| 164 | reconnects: "viewer socket reconnects", | |
| 165 | bytesOut: "bytes the app sent viewers over their sockets", | |
| 166 | armErrors: "pose reads the arm server did not answer", | |
| 167 | reportsDropped: "viewer reports dropped for coming sooner than one per socket every few seconds", | |
| 168 | }; | |
| 169 | const RATE_MS = 10_000; | |
| 170 | ||
| 171 | export class Telemetry { | |
| 172 | readonly rtt: Window; | |
| 173 | readonly age: Window; | |
| 174 | readonly arm: Window; | |
| 175 | readonly totals = Object.fromEntries(COUNTERS.map((c) => [c, 0])) as Record<Counter, number>; | |
| 176 | private readonly rates = new Map<Counter | "framesIn", Window>(); | |
| 177 | private readonly history: Health[] = []; | |
| 178 | private lastFramesIn = 0; | |
| 179 | ||
| 180 | private readonly gauges: Gauges; | |
| 181 | private readonly o: { windowMs?: number; historyLength?: number }; | |
| 182 | ||
| 183 | constructor(gauges: Gauges, options: { windowMs?: number; historyLength?: number } = {}) { | |
| 184 | this.gauges = gauges; | |
| 185 | this.o = options; | |
| 186 | const o = options; | |
| 187 | const span = o.windowMs ?? 60_000; | |
| 188 | this.rtt = new Window(span); | |
| 189 | this.age = new Window(span); | |
| 190 | this.arm = new Window(span); | |
| 191 | for (const c of [...COUNTERS, "framesIn" as const]) this.rates.set(c, new Window(RATE_MS, 100_000)); | |
| 192 | } | |
| 193 | ||
| 194 | count(counter: Counter, n = 1, now = Date.now()): void { | |
| 195 | if (!n) return; | |
| 196 | this.totals[counter] += n; | |
| 197 | this.rates.get(counter)!.add(n, now); | |
| 198 | } | |
| 199 | ||
| 200 | report(r: Report, now = Date.now()): void { | |
| 201 | for (const v of r.rtt) this.rtt.add(v, now); | |
| 202 | for (const v of r.age) this.age.add(v, now); | |
| 203 | this.count("framesShown", r.frames, now); | |
| 204 | this.count("commandsSent", r.sent, now); | |
| 205 | this.count("commandsDropped", r.dropped, now); | |
| 206 | this.count("stalls", r.stalls, now); | |
| 207 | this.count("reconnects", r.reconnects, now); | |
| 208 | } | |
| 209 | ||
| 210 | private rate(c: Counter | "framesIn", now: number) { | |
| 211 | return Math.round((this.rates.get(c)!.sum(RATE_MS, now) / (RATE_MS / 1000)) * 10) / 10; | |
| 212 | } | |
| 213 | ||
| 214 | health(now = Date.now()): Health { | |
| 215 | const framesIn = this.gauges.framesIn(); | |
| 216 | if (framesIn > this.lastFramesIn) this.rates.get("framesIn")!.add(framesIn - this.lastFramesIn, now); | |
| 217 | this.lastFramesIn = Math.max(this.lastFramesIn, framesIn); | |
| 218 | const q = (w: Window): Quantiles => ({ p50: w.quantile(0.5, now), p95: w.quantile(0.95, now) }); | |
| 219 | const watching = this.gauges.watching(); | |
| 220 | return { | |
| 221 | at: now, | |
| 222 | viewers: this.gauges.viewers(), | |
| 223 | watching, | |
| 224 | commandRttMs: q(this.rtt), | |
| 225 | frameAgeMs: q(this.age), | |
| 226 | viewerFps: watching ? Math.round((this.rate("framesShown", now) / watching) * 10) / 10 : 0, | |
| 227 | armRttMs: q(this.arm), | |
| 228 | cameraFpsIn: this.rate("framesIn", now), | |
| 229 | bytesOutPerS: this.rate("bytesOut", now), | |
| 230 | commandsPerS: this.rate("commandsSent", now), | |
| 231 | totals: { ...this.totals }, | |
| 232 | }; | |
| 233 | } | |
| 234 | ||
| 235 | /** Take a point for the sparklines; the relay calls this every couple of seconds. */ | |
| 236 | sample(now = Date.now()): Health { | |
| 237 | const h = this.health(now); | |
| 238 | this.history.push(h); | |
| 239 | const keep = this.o.historyLength ?? 60; | |
| 240 | if (this.history.length > keep) this.history.splice(0, this.history.length - keep); | |
| 241 | return h; | |
| 242 | } | |
| 243 | ||
| 244 | /** The latest numbers and the recent points, oldest first. */ | |
| 245 | json(now = Date.now()): Health & { history: Health[] } { | |
| 246 | return { ...this.health(now), history: this.history.map(({ totals: _, ...h }) => h as Health) }; | |
| 247 | } | |
| 248 | ||
| 249 | prometheus(now = Date.now()): string { | |
| 250 | const h = this.health(now); | |
| 251 | const out: string[] = []; | |
| 252 | const metric = (name: string, type: string, help: string, lines: string[]) => out.push(`# HELP ${name} ${help}`, `# TYPE ${name} ${type}`, ...lines); | |
| 253 | const summary = (name: string, help: string, v: Quantiles, w: Window) => | |
| 254 | metric(name, "summary", help, [ | |
| 255 | ...(v.p50 === null ? [] : [`${name}{quantile="0.5"} ${v.p50}`, `${name}{quantile="0.95"} ${v.p95}`]), | |
| 256 | `${name}_count ${w.count(now)}`, | |
| 257 | ]); | |
| 258 | summary("arm_command_rtt_ms", "command sent to pose reflecting it, as viewers measure it, last minute", h.commandRttMs, this.rtt); | |
| 259 | summary("arm_camera_frame_age_ms", "camera frame age when shown (app receipt to screen), last minute", h.frameAgeMs, this.age); | |
| 260 | summary("arm_server_rtt_ms", "the app's pose read round trip to the arm server, last minute", h.armRttMs, this.arm); | |
| 261 | metric("arm_viewers", "gauge", "open viewer sockets", [`arm_viewers ${h.viewers}`]); | |
| 262 | metric("arm_viewers_watching", "gauge", "viewers watching a camera", [`arm_viewers_watching ${h.watching}`]); | |
| 263 | metric("arm_viewer_fps", "gauge", "camera frames shown per second per watching viewer, last 10 s", [`arm_viewer_fps ${h.viewerFps}`]); | |
| 264 | metric("arm_camera_fps_in", "gauge", "camera frames per second coming into the app, last 10 s", [`arm_camera_fps_in ${h.cameraFpsIn}`]); | |
| 265 | for (const c of COUNTERS) { | |
| 266 | const name = `arm_${c.replace(/[A-Z]/g, (x) => `_${x.toLowerCase()}`)}_total`; | |
| 267 | metric(name, "counter", HELP[c], [`${name} ${this.totals[c]}`]); | |
| 268 | } | |
| 269 | return `${out.join("\n")}\n`; | |
| 270 | } | |
| 271 | ||
| 272 | line(now = Date.now()): string { | |
| 273 | return summary(this.health(now)); | |
| 274 | } | |
| 275 | } | |
| 276 | ||
| 277 | /** | |
| 278 | * Health as one line, for the log and the terminal's /health. It takes the JSON as it arrives (the terminal reads it | |
| 279 | * off GET /health), so anything that is not a number shows as "-". | |
| 280 | */ | |
| 281 | export function summary(h: unknown): string { | |
| 282 | const o = (x: unknown): Record<string, unknown> => (x && typeof x === "object" ? (x as Record<string, unknown>) : {}); | |
| 283 | const n = (x: unknown) => (typeof x === "number" && Number.isFinite(x) ? x : null); | |
| 284 | const num = (x: unknown) => String(n(x) ?? "-"); | |
| 285 | const ms = (x: unknown) => { | |
| 286 | const p50 = n(o(x).p50), p95 = n(o(x).p95); | |
| 287 | return p50 === null || p95 === null ? "-" : `${Math.round(p50)}/${Math.round(p95)}ms`; | |
| 288 | }; | |
| 289 | const r = o(h), t = o(r.totals), bytes = n(r.bytesOutPerS); | |
| 290 | return `link: ${num(r.viewers)} viewers (${num(r.watching)} watching) · command→pose p50/p95 ${ms(r.commandRttMs)} · frame age ${ms(r.frameAgeMs)} · ${num(r.viewerFps)} fps shown, ${num(r.cameraFpsIn)} in · arm ${ms(r.armRttMs)} · ${num(r.commandsPerS)} cmd/s · ${bytes === null ? "-" : Math.round(bytes / 1024)} KiB/s out · ${num(t.framesSkipped)} skipped, ${num(t.stalls)} stalls, ${num(t.reconnects)} reconnects`; | |
| 291 | } |
Software/so101/src/terminal.test.ts +21
| @@ -295,4 +295,25 @@ describe("terminal door", () => { | ||
| 295 | 295 | await owner.printedLine(/alice's turn ended .*bob has control/); |
| 296 | 296 | expect(lobby!.admin.log.some((e) => /ended alice's turn/.test(JSON.stringify(e)))).toBe(true); |
| 297 | 297 | }); |
| 298 | ||
| 299 | test("/health: the owner's terminal reads the link's numbers on one line; anyone else is told it is the admin's", async () => { | |
| 300 | const port = await start(); | |
| 301 | expect(typed(emptyRoom(), "/health")).toEqual({ health: true }); | |
| 302 | const carol = await terminal(port, ttyEnv("u-carol", "carol", "user")); | |
| 303 | carol.door.input("/health"); | |
| 304 | await carol.printedLine(/^error: only an admin can see the link's health$/); | |
| 305 | const owner = await terminal(port, ttyEnv("u-owner", "trevato", "admin")); | |
| 306 | owner.door.input("/health"); | |
| 307 | // Signed afresh for each request: the forge's signature is good for a minute, a terminal stays longer. | |
| 308 | let signed = 0; | |
| 309 | const env = { ...ttyEnv("u-owner", "trevato", "admin"), PORT: String(port) }; | |
| 310 | const lines: string[] = []; | |
| 311 | const fresh = new Door(liveUrl(env), () => (signed++, terminalHeaders(env)), (line) => lines.push(line)); | |
| 312 | await fresh.open(); | |
| 313 | open.push(fresh); | |
| 314 | await fresh.health(); | |
| 315 | expect(signed).toBe(2); | |
| 316 | expect(lines.at(-1)).toMatch(/^link: /); | |
| 317 | await owner.printedLine(/^link: 0 viewers \(0 watching\) · command→pose p50\/p95 - · .* 0 stalls, 0 reconnects$/); | |
| 318 | }); | |
| 298 | 319 | }); |
Software/so101/src/terminal.ts +25 −7
| @@ -14,6 +14,7 @@ | ||
| 14 | 14 | * /ready /extend confirm you are there when called (15 s), or add 30 s to your turn |
| 15 | 15 | * /end [who] owner: end that turn (none: whoever drives), or take them out of the line |
| 16 | 16 | * /mute <who> /unmute <who> owner |
| 17 | * /health admin: how the link feels (GET /health as one line: command round trips, frame ages, fps, stalls) | |
| 17 | 18 | * /machine [off] the Machine panel (/machine, src/machine-feed.ts): the joints, the link and the server as they are, |
| 18 | 19 | * then every command in and out as it happens, in the page's own words (src/ui/machine.js); off stops it |
| 19 | 20 | * /who /help /quit |
| @@ -23,6 +24,7 @@ | ||
| 23 | 24 | import WebSocket from "ws"; |
| 24 | 25 | import { UNPRINTABLE } from "./chat.ts"; |
| 25 | 26 | import { ID, NAME, sign } from "./identity.ts"; |
| 27 | import { summary } from "./telemetry.ts"; | |
| 26 | 28 | import { entryLine, jointRows, linkRows, stateRows } from "./ui/machine.js"; |
| 27 | 29 | |
| 28 | 30 | export type Env = Record<string, string | undefined>; |
| @@ -93,7 +95,7 @@ const said = (m: unknown): string => { | ||
| 93 | 95 | return from ? `${plain(from.name)}: ${plain(message?.text)}` : `· ${plain(message?.text)}`; |
| 94 | 96 | }; |
| 95 | 97 | |
| 96 | export const HELP = "type to chat · /join /leave /ready /extend · /end [who] /mute <who> /unmute <who> (owner) · /machine [off] · /who · /quit"; | |
| 98 | export const HELP = "type to chat · /join /leave /ready /extend · /end [who] /mute <who> /unmute <who> /health (owner) · /machine [off] · /who · /quit"; | |
| 97 | 99 | |
| 98 | 100 | /** The Machine feed's socket beside the lobby's: same app, same port. */ |
| 99 | 101 | export const machineUrl = (live: string): string => live.replace(/\/live$/, "/machine"); |
| @@ -192,7 +194,7 @@ function target(room: Room, word: string): string | null { | ||
| 192 | 194 | return ID.test(word) ? word : null; |
| 193 | 195 | } |
| 194 | 196 | |
| 195 | export type Typed = { send: Frame } | { print: string } | { machine: boolean } | { quit: true } | null; | |
| 197 | export type Typed = { send: Frame } | { print: string } | { health: true } | { machine: boolean } | { quit: true } | null; | |
| 196 | 198 | |
| 197 | 199 | /** What a typed line asks for. Validation the server repeats anyway; this only saves a round trip on a typo. */ |
| 198 | 200 | export function typed(room: Room, input: string): Typed { |
| @@ -223,6 +225,8 @@ export function typed(room: Room, input: string): Typed { | ||
| 223 | 225 | return { machine: arg.toLowerCase() !== "off" }; |
| 224 | 226 | case "who": |
| 225 | 227 | return { print: who(room) }; |
| 228 | case "health": | |
| 229 | return { health: true }; | |
| 226 | 230 | case "help": |
| 227 | 231 | return { print: HELP }; |
| 228 | 232 | case "quit": |
| @@ -240,18 +244,19 @@ export class Door { | ||
| 240 | 244 | private machineWs: WebSocket | null = null; |
| 241 | 245 | readonly machine: MachineState = { status: null, link: null, room: null, skew: 0 }; |
| 242 | 246 | private readonly url: string; |
| 243 | private readonly headers: Record<string, string>; | |
| 247 | /** Signed afresh for each request: a signature is good for a minute, and a terminal stays open for longer. */ | |
| 248 | private readonly headers: () => Record<string, string>; | |
| 244 | 249 | private readonly print: (line: string) => void; |
| 245 | 250 | |
| 246 | constructor(url: string, headers: Record<string, string>, print: (line: string) => void) { | |
| 251 | constructor(url: string, headers: Record<string, string> | (() => Record<string, string>), print: (line: string) => void) { | |
| 247 | 252 | this.url = url; |
| 248 | this.headers = headers; | |
| 253 | this.headers = typeof headers === "function" ? headers : () => headers; | |
| 249 | 254 | this.print = print; |
| 250 | 255 | } |
| 251 | 256 | |
| 252 | 257 | /** Connect; resolves once seated, rejects when the app does not answer. `onClose` hears the app going away. */ |
| 253 | 258 | open(onClose: () => void = () => {}): Promise<void> { |
| 254 | const ws = new WebSocket(this.url, { headers: this.headers, maxPayload: 1 << 20 }); | |
| 259 | const ws = new WebSocket(this.url, { headers: this.headers(), maxPayload: 1 << 20 }); | |
| 255 | 260 | this.ws = ws; |
| 256 | 261 | ws.on("message", (data, binary) => { |
| 257 | 262 | if (binary) return; |
| @@ -280,11 +285,24 @@ export class Door { | ||
| 280 | 285 | if (!t) return true; |
| 281 | 286 | if ("quit" in t) return false; |
| 282 | 287 | if ("print" in t) this.print(t.print); |
| 288 | else if ("health" in t) void this.health(); | |
| 283 | 289 | else if ("machine" in t) this.watchMachine(t.machine); |
| 284 | 290 | else this.send(t.send); |
| 285 | 291 | return true; |
| 286 | 292 | } |
| 287 | 293 | |
| 294 | /** The link's health, as the app's GET /health tells this terminal's person (the admin only; the server says why not). */ | |
| 295 | async health(): Promise<void> { | |
| 296 | const url = this.url.replace(/^ws/, "http").replace(/\/live$/, "/health"); | |
| 297 | try { | |
| 298 | const r = await fetch(url, { headers: { ...this.headers(), accept: "application/json" }, signal: AbortSignal.timeout(3000) }); | |
| 299 | const body = (await r.json().catch(() => null)) as Record<string, unknown> | null; | |
| 300 | this.print(r.ok ? summary(body) : `error: ${plain(body?.error ?? `the app answered ${r.status}`)}`); | |
| 301 | } catch (err) { | |
| 302 | this.print(`error: the app did not answer (${plain((err as Error).message)})`); | |
| 303 | } | |
| 304 | } | |
| 305 | ||
| 288 | 306 | close(): void { |
| 289 | 307 | this.ws?.close(); |
| 290 | 308 | this.machineWs?.close(); |
| @@ -299,7 +317,7 @@ export class Door { | ||
| 299 | 317 | return this.print("machine feed closed"); |
| 300 | 318 | } |
| 301 | 319 | if (this.machineWs) return void machineLines(this.machine).forEach((l) => this.print(l)); |
| 302 | const ws = new WebSocket(machineUrl(this.url), { headers: this.headers, maxPayload: 1 << 22 }); | |
| 320 | const ws = new WebSocket(machineUrl(this.url), { headers: this.headers(), maxPayload: 1 << 22 }); | |
| 303 | 321 | this.machineWs = ws; |
| 304 | 322 | const now = () => Date.now() - this.machine.skew; |
| 305 | 323 | let waiting = false; |
Software/so101/src/tty.ts +1 −1
| @@ -17,7 +17,7 @@ const print = (line: string) => { | ||
| 17 | 17 | rl?.prompt(true); |
| 18 | 18 | }; |
| 19 | 19 | |
| 20 | const door = new Door(liveUrl(env), terminalHeaders(env), print); | |
| 20 | const door = new Door(liveUrl(env), () => terminalHeaders(env), print); | |
| 21 | 21 | print(`\x1b[1mSO-101 arm\x1b[0m · ${where} · ${HELP}`); |
| 22 | 22 | try { |
| 23 | 23 | await door.open(() => { |
Software/so101/src/ui/arm-socket-page.test.ts added +48
| @@ -0,0 +1,48 @@ | ||
| 1 | import { afterAll, expect, test } from "bun:test"; | |
| 2 | ||
| 3 | // In a browser: a document that can be hidden, and browser frames. The module reads them when it loads. | |
| 4 | const listeners: Array<() => void> = []; | |
| 5 | const frames: Array<() => void> = []; | |
| 6 | const g = globalThis as any; | |
| 7 | const document = { hidden: false, addEventListener: (_: string, f: () => void) => listeners.push(f), removeEventListener: () => {} }; | |
| 8 | g.document = document; | |
| 9 | g.requestAnimationFrame = (f: () => void) => frames.push(f); | |
| 10 | const fresh: string = "./arm-socket.js?in-a-page"; // its own copy of the module, loaded with the page's globals in place | |
| 11 | const { ArmSocket }: typeof import("./arm-socket.js") = await import(fresh); | |
| 12 | afterAll(() => { delete g.document; delete g.requestAnimationFrame; }); | |
| 13 | ||
| 14 | class FakeSocket { | |
| 15 | static last: FakeSocket; | |
| 16 | readyState = 1; | |
| 17 | sent: any[] = []; | |
| 18 | onopen = () => {}; | |
| 19 | onmessage = (_: { data: string }) => {}; | |
| 20 | onclose = () => {}; | |
| 21 | onerror = () => {}; | |
| 22 | constructor() { FakeSocket.last = this; } | |
| 23 | send(data: string) { this.sent.push(JSON.parse(data)); } | |
| 24 | close() {} | |
| 25 | } | |
| 26 | ||
| 27 | test("in a page: poses are drawn once per browser frame, and a hidden page watches no camera", () => { | |
| 28 | const poses: number[] = []; | |
| 29 | const s = new ArmSocket({ url: "ws://x/arm", WebSocket: FakeSocket as any, onPose: (status: any) => poses.push(status.n) }); | |
| 30 | const ws = FakeSocket.last; | |
| 31 | ws.onopen(); | |
| 32 | s.watch([1, 2]); | |
| 33 | expect(ws.sent.filter((m) => m.t === "watch").at(-1)).toEqual({ t: "watch", cameras: [1, 2], hz: 30 }); | |
| 34 | for (let n = 1; n <= 5; n++) ws.onmessage({ data: JSON.stringify({ t: "pose", status: { n } }) }); | |
| 35 | expect(poses).toEqual([]); | |
| 36 | expect(frames).toHaveLength(1); | |
| 37 | frames.shift()!(); | |
| 38 | expect(poses).toEqual([5]); // the latest, once | |
| 39 | document.hidden = true; | |
| 40 | for (const f of listeners) f(); | |
| 41 | expect(ws.sent.at(-1)).toEqual({ t: "watch", cameras: [], hz: 1 }); | |
| 42 | ws.onmessage({ data: JSON.stringify({ t: "pose", status: { n: 6 } }) }); | |
| 43 | expect(poses).toEqual([5, 6]); // hidden: no browser frame to wait for | |
| 44 | document.hidden = false; | |
| 45 | for (const f of listeners) f(); | |
| 46 | expect(ws.sent.at(-1)).toEqual({ t: "watch", cameras: [1, 2], hz: 30 }); | |
| 47 | s.close(); | |
| 48 | }); |
Software/so101/src/ui/arm-socket.js added +320
| @@ -0,0 +1,320 @@ | ||
| 1 | /** | |
| 2 | * The page's end of WS /arm (src/viewer-socket.ts): one socket for the pose, the driver's commands and the camera | |
| 3 | * frames, so nothing pays the public round trip per request. Plain ESM, served at /ui/ and imported by the tests. | |
| 4 | * | |
| 5 | * It keeps itself up: a socket that goes quiet for `stallMs` (the app pings every second and sends poses many times a | |
| 6 | * second) is closed and opened again, and so is one that drops. A drive does not survive that: the page is told the | |
| 7 | * drive closed, and the arm holds until the next press. | |
| 8 | * | |
| 9 | * And it measures what the owner iterates on, reporting to the app every few seconds (src/telemetry.ts): command → | |
| 10 | * pose round trips, camera frame ages when shown, frames shown, commands sent and merged away, stalls, reconnects. | |
| 11 | */ | |
| 12 | ||
| 13 | export const FRAME_HEADER = 20; | |
| 14 | /** @typedef {"sent" | "dropped" | "stalls" | "reconnects" | "frames"} Counted */ | |
| 15 | /** The page's document, when there is one (the tests run without). @type {any} */ | |
| 16 | const doc = /** @type {any} */ (globalThis).document; | |
| 17 | const MIMES = ["image/jpeg", "image/png", "application/octet-stream"]; | |
| 18 | ||
| 19 | /** | |
| 20 | * A camera frame off the socket: the 20-byte header, then the picture. | |
| 21 | * @param {ArrayBuffer} buffer | |
| 22 | * @returns {{ camera: number, n: number, at: number, type: string, bytes: Uint8Array } | null} | |
| 23 | */ | |
| 24 | export function readFrame(buffer) { | |
| 25 | if (!(buffer instanceof ArrayBuffer) || buffer.byteLength < FRAME_HEADER) return null; | |
| 26 | const v = new DataView(buffer); | |
| 27 | if (v.getUint8(0) !== 1) return null; | |
| 28 | return { camera: v.getUint32(4), n: v.getUint32(8), at: v.getFloat64(12), type: MIMES[v.getUint8(1)] ?? "application/octet-stream", bytes: new Uint8Array(buffer, FRAME_HEADER) }; | |
| 29 | } | |
| 30 | ||
| 31 | /** | |
| 32 | * The value below which a fraction `q` of `values` lie; null for none. | |
| 33 | * @param {number[]} values | |
| 34 | * @param {number} q | |
| 35 | * @returns {number | null} | |
| 36 | */ | |
| 37 | export function quantile(values, q) { | |
| 38 | if (!values.length) return null; | |
| 39 | const sorted = [...values].sort((a, b) => a - b); | |
| 40 | return sorted[Math.min(sorted.length - 1, Math.max(0, Math.ceil(q * sorted.length) - 1))] ?? null; | |
| 41 | } | |
| 42 | ||
| 43 | /** The last `size` samples, for the page's own readout. */ | |
| 44 | class Recent { | |
| 45 | constructor(size = 100) { | |
| 46 | this.size = size; | |
| 47 | /** @type {number[]} */ | |
| 48 | this.values = []; | |
| 49 | } | |
| 50 | /** @param {number} v */ | |
| 51 | add(v) { this.values.push(v); if (this.values.length > this.size) this.values.shift(); } | |
| 52 | } | |
| 53 | ||
| 54 | export class ArmSocket { | |
| 55 | /** | |
| 56 | * @param {{ | |
| 57 | * url: string, | |
| 58 | * WebSocket?: typeof WebSocket, | |
| 59 | * commandHz?: number, stallMs?: number, reportEveryMs?: number, pingEveryMs?: number, backoffMs?: number[], | |
| 60 | * onPose?: (status: any, message: any) => void, | |
| 61 | * onFrame?: (frame: { camera: number, n: number, at: number, type: string, bytes: Uint8Array }) => void, | |
| 62 | * onDrive?: (message: { state: string, reason?: string | null, code?: number }) => void, | |
| 63 | * onState?: (open: boolean) => void, | |
| 64 | * }} o | |
| 65 | */ | |
| 66 | constructor(o) { | |
| 67 | this.o = { commandHz: 30, stallMs: 2000, reportEveryMs: 5000, pingEveryMs: 1000, backoffMs: [250, 500, 1000, 2000, 4000], ...o }; | |
| 68 | this.WS = o.WebSocket ?? globalThis.WebSocket; | |
| 69 | this.ws = null; | |
| 70 | this.open = false; | |
| 71 | this.closed = false; | |
| 72 | this.attempt = 0; | |
| 73 | this.everOpen = false; | |
| 74 | /** The app's clock minus ours, from the ping with the shortest round trip lately. */ | |
| 75 | this.offset = 0; | |
| 76 | /** @type {Array<[number, number]>} */ | |
| 77 | this.offsets = []; | |
| 78 | this.lastHeard = 0; | |
| 79 | /** @type {{ cameras: number[], hz: number }} */ | |
| 80 | this.watching = { cameras: [], hz: 30 }; | |
| 81 | this.seq = 0; | |
| 82 | /** @type {Record<string, number> | null} */ | |
| 83 | this.pending = null; | |
| 84 | /** @type {ReturnType<typeof setTimeout> | undefined} */ | |
| 85 | this.pacer = undefined; | |
| 86 | /** @type {ReturnType<typeof setTimeout> | undefined} */ | |
| 87 | this.pinger = undefined; | |
| 88 | this.lastSentAt = -Infinity; | |
| 89 | /** seq → when it went, for the commands not yet seen in a pose. @type {Map<number, number>} */ | |
| 90 | this.inFlight = new Map(); | |
| 91 | this.driving = false; | |
| 92 | this.fresh(); | |
| 93 | this.rtt = new Recent(); | |
| 94 | this.age = new Recent(); | |
| 95 | this.shownAt = new Recent(60); | |
| 96 | /** @type {Record<Counted, number>} */ | |
| 97 | this.totals = { sent: 0, dropped: 0, stalls: 0, reconnects: 0, frames: 0 }; | |
| 98 | this.connect(); | |
| 99 | this.watchdog = setInterval(() => this.tick(), Math.min(500, this.o.stallMs / 2)); | |
| 100 | this.reporter = setInterval(() => this.report(), this.o.reportEveryMs); | |
| 101 | /** A pose waiting for the next browser frame, so many between two frames are drawn once. @type {any} */ | |
| 102 | this.poseWaiting = null; | |
| 103 | // Hidden, the page watches no camera and few poses; back on screen, what it asked for. | |
| 104 | this.onVisible = () => this.sendWatch(); | |
| 105 | doc?.addEventListener?.("visibilitychange", this.onVisible); | |
| 106 | } | |
| 107 | ||
| 108 | /** What has not been reported yet. */ | |
| 109 | fresh() { | |
| 110 | /** @type {{ rtt: number[], age: number[] } & Record<Counted, number>} */ | |
| 111 | this.unreported = { rtt: [], age: [], frames: 0, sent: 0, dropped: 0, stalls: 0, reconnects: 0 }; | |
| 112 | } | |
| 113 | ||
| 114 | connect() { | |
| 115 | if (this.closed) return; | |
| 116 | const ws = new this.WS(this.o.url); | |
| 117 | ws.binaryType = "arraybuffer"; | |
| 118 | this.ws = ws; | |
| 119 | this.lastHeard = Date.now(); | |
| 120 | ws.onopen = () => { | |
| 121 | if (this.ws !== ws) return; | |
| 122 | this.open = true; | |
| 123 | this.attempt = 0; | |
| 124 | if (this.everOpen) this.count("reconnects"); | |
| 125 | this.everOpen = true; | |
| 126 | this.lastHeard = Date.now(); | |
| 127 | this.sendWatch(); | |
| 128 | this.ping(); | |
| 129 | this.o.onState?.(true); | |
| 130 | }; | |
| 131 | ws.onmessage = (e) => { if (this.ws === ws) this.receive(e.data); }; | |
| 132 | ws.onclose = () => { if (this.ws === ws) this.lost(); }; | |
| 133 | ws.onerror = () => {}; | |
| 134 | } | |
| 135 | ||
| 136 | lost() { | |
| 137 | const was = this.open; | |
| 138 | this.ws = null; | |
| 139 | this.open = false; | |
| 140 | this.inFlight.clear(); | |
| 141 | clearTimeout(this.pacer); this.pacer = undefined; this.pending = null; | |
| 142 | clearTimeout(this.pinger); | |
| 143 | if (this.driving) { this.driving = false; this.o.onDrive?.({ state: "closed", reason: "the connection to the app was lost; the arm holds", code: 1006 }); } | |
| 144 | if (was) this.o.onState?.(false); | |
| 145 | if (this.closed) return; | |
| 146 | const waits = this.o.backoffMs; | |
| 147 | setTimeout(() => this.connect(), waits[Math.min(this.attempt++, waits.length - 1)]); | |
| 148 | } | |
| 149 | ||
| 150 | tick() { | |
| 151 | if (!this.ws || Date.now() - this.lastHeard <= this.o.stallMs) return; | |
| 152 | // Quiet past the limit: count it, and start over rather than wait on a socket that may never speak again. | |
| 153 | if (this.open) this.count("stalls"); | |
| 154 | const ws = this.ws; | |
| 155 | try { ws.close(); } catch {} | |
| 156 | this.lost(); | |
| 157 | } | |
| 158 | ||
| 159 | /** @param {Counted} name */ | |
| 160 | count(name, n = 1) { this.totals[name] += n; this.unreported[name] += n; } | |
| 161 | ||
| 162 | /** @param {Record<string, unknown>} m */ | |
| 163 | send(m) { | |
| 164 | if (!this.open || this.ws?.readyState !== 1) return false; | |
| 165 | this.ws.send(JSON.stringify(m)); | |
| 166 | return true; | |
| 167 | } | |
| 168 | ||
| 169 | ping() { | |
| 170 | if (!this.send({ t: "ping", c: Date.now() })) return; | |
| 171 | clearTimeout(this.pinger); | |
| 172 | this.pinger = setTimeout(() => this.ping(), this.o.pingEveryMs); | |
| 173 | } | |
| 174 | ||
| 175 | /** @param {string | ArrayBuffer} data */ | |
| 176 | receive(data) { | |
| 177 | this.lastHeard = Date.now(); | |
| 178 | if (typeof data !== "string") { | |
| 179 | const frame = readFrame(data); | |
| 180 | if (frame) this.o.onFrame?.(frame); | |
| 181 | return; | |
| 182 | } | |
| 183 | let m; | |
| 184 | try { m = JSON.parse(data); } catch { return; } | |
| 185 | if (!m || typeof m !== "object") return; | |
| 186 | if (m.t === "hello" && typeof m.now === "number") this.offset = m.now - Date.now(); | |
| 187 | else if (m.t === "pong" && typeof m.c === "number" && typeof m.now === "number") { | |
| 188 | const rtt = Date.now() - m.c; | |
| 189 | this.offsets.push([rtt, m.now - (m.c + rtt / 2)]); | |
| 190 | if (this.offsets.length > 8) this.offsets.shift(); | |
| 191 | this.offset = this.offsets.reduce((best, o) => (o[0] < best[0] ? o : best))[1]; | |
| 192 | } else if (m.t === "pose" && m.status && typeof m.status === "object") { | |
| 193 | if (typeof m.ack === "number") { | |
| 194 | const at = this.inFlight.get(m.ack); | |
| 195 | if (at !== undefined) { const ms = performance.now() - at; this.rtt.add(ms); this.unreported.rtt.push(ms); } | |
| 196 | for (const seq of this.inFlight.keys()) if (seq <= m.ack) this.inFlight.delete(seq); | |
| 197 | } | |
| 198 | this.pose(m); | |
| 199 | } else if (m.t === "drive" && typeof m.state === "string") { | |
| 200 | if (m.state !== "open") this.driving = false; | |
| 201 | this.o.onDrive?.(m); | |
| 202 | } | |
| 203 | } | |
| 204 | ||
| 205 | /** Hand a pose to the page: at once while hidden (or with no browser frames), else once per browser frame, the latest. */ | |
| 206 | /** @param {any} m */ | |
| 207 | pose(m) { | |
| 208 | const raf = /** @type {any} */ (globalThis).requestAnimationFrame; | |
| 209 | if (doc?.hidden || typeof raf !== "function") return this.o.onPose?.(m.status, m); | |
| 210 | const queued = this.poseWaiting !== null; | |
| 211 | this.poseWaiting = m; | |
| 212 | if (!queued) raf(() => { const p = this.poseWaiting; this.poseWaiting = null; if (p) this.o.onPose?.(p.status, p); }); | |
| 213 | } | |
| 214 | ||
| 215 | /** The app's clock now, ms. */ | |
| 216 | now() { return Date.now() + this.offset; } | |
| 217 | ||
| 218 | /** Which cameras to receive, and how many poses a second; while the page is hidden, none and one. */ | |
| 219 | /** @param {number[]} cameras */ | |
| 220 | watch(cameras, hz = this.watching.hz) { | |
| 221 | this.watching = { cameras: [...cameras], hz }; | |
| 222 | this.sendWatch(); | |
| 223 | } | |
| 224 | sendWatch() { | |
| 225 | this.send(doc?.hidden ? { t: "watch", cameras: [], hz: 1 } : { t: "watch", ...this.watching }); | |
| 226 | } | |
| 227 | ||
| 228 | /** | |
| 229 | * The driver's joint targets, ticks: merged until the next slot (latest per joint wins, never queued) and sent at | |
| 230 | * most `commandHz` times a second. Returns the command's number, or null when there is no socket to send it on. | |
| 231 | */ | |
| 232 | /** @param {Record<string, number>} targets @returns {number | null} */ | |
| 233 | command(targets) { | |
| 234 | if (!this.open) return null; | |
| 235 | this.driving = true; | |
| 236 | if (this.pending) this.count("dropped"); | |
| 237 | this.pending = { ...(this.pending ?? {}), ...targets }; | |
| 238 | const seq = ++this.seq; | |
| 239 | if (!this.pacer) this.pacer = setTimeout(() => this.flush(), Math.max(0, this.lastSentAt + 1000 / this.o.commandHz - performance.now())); | |
| 240 | return seq; | |
| 241 | } | |
| 242 | ||
| 243 | flush() { | |
| 244 | clearTimeout(this.pacer); | |
| 245 | this.pacer = undefined; | |
| 246 | if (!this.pending) return; | |
| 247 | const targets = this.pending; this.pending = null; | |
| 248 | if (!this.send({ t: "cmd", seq: this.seq, targets })) return; | |
| 249 | this.lastSentAt = performance.now(); | |
| 250 | this.inFlight.set(this.seq, this.lastSentAt); | |
| 251 | if (this.inFlight.size > 64) this.inFlight.delete(/** @type {number} */ (this.inFlight.keys().next().value)); | |
| 252 | this.count("sent"); | |
| 253 | } | |
| 254 | ||
| 255 | /** Let go: what is pending goes, then the app closes the drive and the arm holds. */ | |
| 256 | end() { | |
| 257 | if (this.pending) this.flush(); | |
| 258 | this.send({ t: "end" }); | |
| 259 | } | |
| 260 | ||
| 261 | /** The page put frame `n` on screen; `at` is when the app had it. Frames up to it count as seen. */ | |
| 262 | /** @param {number} n @param {number} at */ | |
| 263 | shown(n, at) { | |
| 264 | const ms = Math.max(0, this.now() - at); | |
| 265 | this.age.add(ms); this.unreported.age.push(ms); | |
| 266 | this.shownAt.add(performance.now()); | |
| 267 | this.count("frames"); | |
| 268 | this.send({ t: "seen", n }); | |
| 269 | } | |
| 270 | ||
| 271 | /** Frame `n` was not shown (its view is off, or a newer one took its place): the app may send the next. */ | |
| 272 | /** @param {number} n */ | |
| 273 | skip(n) { this.send({ t: "seen", n }); } | |
| 274 | ||
| 275 | /** | |
| 276 | * Put a frame in a pane's <img> as a blob URL: once decoded it counts as shown (and the URL it replaced is let go); | |
| 277 | * one replaced before it could show is skipped. The pane keeps `blob` (its current URL) and `frameAt`. | |
| 278 | * @param {{ img: any, blob?: string | null, frameAt?: number }} pane @param {{ n: number, at: number, type: string, bytes: Uint8Array }} f | |
| 279 | */ | |
| 280 | paint(pane, f) { | |
| 281 | const url = URL.createObjectURL(new Blob([/** @type {any} */ (f.bytes)], { type: f.type })); | |
| 282 | pane.img.src = url; | |
| 283 | pane.img.decode().catch(() => {}).then(() => { | |
| 284 | if (pane.img.src === url) { if (pane.blob) URL.revokeObjectURL(pane.blob); pane.blob = url; pane.frameAt = performance.now(); this.shown(f.n, f.at); } | |
| 285 | else { URL.revokeObjectURL(url); this.skip(f.n); } // a newer frame took its place first | |
| 286 | }); | |
| 287 | } | |
| 288 | ||
| 289 | report() { | |
| 290 | const r = this.unreported; | |
| 291 | if (!r.rtt.length && !r.age.length && !r.frames && !r.sent && !r.dropped && !r.stalls && !r.reconnects) return; | |
| 292 | if (this.send({ t: "report", ...r })) this.fresh(); | |
| 293 | } | |
| 294 | ||
| 295 | /** The page's own numbers, for its readout. */ | |
| 296 | stats() { | |
| 297 | const recent = this.shownAt.values.filter((x) => performance.now() - x < 5000); | |
| 298 | return { | |
| 299 | open: this.open, | |
| 300 | rttP50: quantile(this.rtt.values, 0.5), rttP95: quantile(this.rtt.values, 0.95), | |
| 301 | ageP50: quantile(this.age.values, 0.5), ageP95: quantile(this.age.values, 0.95), | |
| 302 | fps: recent.length / 5, | |
| 303 | ...this.totals, | |
| 304 | }; | |
| 305 | } | |
| 306 | ||
| 307 | /** The same, as the page shows it beside "live": fps, frame age and command → pose round trip, medians. */ | |
| 308 | readout() { | |
| 309 | const s = this.stats(), ms = (/** @type {number | null} */ v) => (v === null ? "—" : `${Math.round(v)} ms`); | |
| 310 | return ` · ${s.fps.toFixed(0)} fps · frame ${ms(s.ageP50)}${s.rttP50 === null ? "" : ` · command ${ms(s.rttP50)}`}`; | |
| 311 | } | |
| 312 | ||
| 313 | close() { | |
| 314 | this.closed = true; | |
| 315 | clearInterval(this.watchdog); clearInterval(this.reporter); clearTimeout(this.pinger); clearTimeout(this.pacer); | |
| 316 | doc?.removeEventListener?.("visibilitychange", this.onVisible); | |
| 317 | const ws = this.ws; this.ws = null; this.open = false; | |
| 318 | try { ws?.close(1000); } catch {} | |
| 319 | } | |
| 320 | } |
Software/so101/src/ui/arm-socket.test.ts added +102
| @@ -0,0 +1,102 @@ | ||
| 1 | import { afterEach, describe, expect, test } from "bun:test"; | |
| 2 | import { createServer } from "node:http"; | |
| 3 | import { WebSocketServer } from "ws"; | |
| 4 | import { fakePi } from "../fake-pi.ts"; | |
| 5 | import { parseRemote, startRelay, type RunningRelay } from "../relay.ts"; | |
| 6 | import { frameHeader } from "../viewer-socket.ts"; | |
| 7 | import { ArmSocket, quantile, readFrame } from "./arm-socket.js"; | |
| 8 | ||
| 9 | const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms)); | |
| 10 | const until = async <T>(fn: () => T | undefined | false | null, ms = 3000): Promise<T> => { | |
| 11 | const end = Date.now() + ms; | |
| 12 | for (;;) { | |
| 13 | const v = fn(); | |
| 14 | if (v) return v; | |
| 15 | if (Date.now() > end) throw new Error("timed out"); | |
| 16 | await sleep(10); | |
| 17 | } | |
| 18 | }; | |
| 19 | ||
| 20 | const stops: Array<() => unknown> = []; | |
| 21 | afterEach(async () => { | |
| 22 | for (const stop of stops.splice(0).reverse()) await Promise.race([stop(), sleep(500)]); | |
| 23 | }); | |
| 24 | ||
| 25 | async function boot(delayMs = 0): Promise<{ pi: Awaited<ReturnType<typeof fakePi>>; relay: RunningRelay }> { | |
| 26 | const pi = await fakePi(delayMs); | |
| 27 | stops.push(() => pi.close()); | |
| 28 | const remote = parseRemote({ ARM_SERVER: `ws://127.0.0.1:${pi.port}`, ARM_CAMERA: `http://127.0.0.1:${pi.port}/stream` })!; | |
| 29 | const relay = await startRelay({ port: 0, simulation: "http://127.0.0.1:1", remote, logEveryMs: 0, viewer: { reportEveryMs: 50 } }); | |
| 30 | stops.push(() => relay.close()); | |
| 31 | return { pi, relay }; | |
| 32 | } | |
| 33 | const socket = (o: ConstructorParameters<typeof ArmSocket>[0]) => { | |
| 34 | const s = new ArmSocket(o); | |
| 35 | stops.push(() => s.close()); | |
| 36 | return s; | |
| 37 | }; | |
| 38 | ||
| 39 | describe("the page's arm socket", () => { | |
| 40 | test("reads the app's frame header", () => { | |
| 41 | const picture = { bytes: Buffer.from([0xff, 0xd8, 1, 2]), type: "image/jpeg", at: 1_700_000_000_123.5 } as never; | |
| 42 | const h = frameHeader(2, picture); | |
| 43 | h.writeUInt32BE(77, 8); | |
| 44 | const buf = Buffer.concat([h, Buffer.from([0xff, 0xd8, 1, 2])]); | |
| 45 | const frame = readFrame(buf.buffer.slice(buf.byteOffset, buf.byteOffset + buf.length) as ArrayBuffer)!; | |
| 46 | expect(frame).toMatchObject({ camera: 2, n: 77, at: 1_700_000_000_123.5, type: "image/jpeg" }); | |
| 47 | expect([...frame.bytes]).toEqual([0xff, 0xd8, 1, 2]); | |
| 48 | expect(readFrame(new ArrayBuffer(4))).toBeNull(); | |
| 49 | expect(quantile([5, 1, 3, 2, 4], 0.5)).toBe(3); | |
| 50 | expect(quantile([], 0.95)).toBeNull(); | |
| 51 | }); | |
| 52 | ||
| 53 | test("end to end through the relay: poses, frames, latest-wins commands, and round trips that include the arm's delay", async () => { | |
| 54 | const { pi, relay } = await boot(60); | |
| 55 | const poses: any[] = []; | |
| 56 | let shown = 0; | |
| 57 | const s: ArmSocket = socket({ | |
| 58 | url: `ws://127.0.0.1:${relay.port}/arm`, | |
| 59 | reportEveryMs: 200, | |
| 60 | onPose: (status) => poses.push(status), | |
| 61 | onFrame: (f) => (shown++, s.shown(f.n, f.at)), | |
| 62 | }); | |
| 63 | await until(() => s.open); | |
| 64 | s.watch([1]); | |
| 65 | await until(() => poses.length >= 2); | |
| 66 | // A burst of targets, faster than the command rate: only a few go, and the last one wins. | |
| 67 | let last = 0; | |
| 68 | for (let t = 1; t <= 30; t++) last = s.command({ 1: 2000 + t })!; | |
| 69 | await until(() => pi.follow.at(-1) === JSON.stringify({ 1: 2030 })); | |
| 70 | expect(pi.follow.length).toBeLessThanOrEqual(2); | |
| 71 | expect(s.totals.dropped).toBe(29); | |
| 72 | await until(() => poses.some((p) => p.joints[0].position === 2030) && s.rtt.values.length); | |
| 73 | expect(last).toBe(30); | |
| 74 | // The command reached the arm, then a /status read that began after it came back: at least the Pi's 60 ms. | |
| 75 | expect(s.rtt.values[0]).toBeGreaterThanOrEqual(60); | |
| 76 | await until(() => shown >= 10); | |
| 77 | expect(s.stats().ageP50).toBeLessThan(500); | |
| 78 | // The reports reach the app's numbers. | |
| 79 | await until(() => relay.telemetry.health().commandRttMs.p50 !== null && relay.telemetry.totals.framesShown >= 10); | |
| 80 | expect(relay.telemetry.health().commandRttMs.p50!).toBeGreaterThanOrEqual(60); | |
| 81 | expect(relay.telemetry.totals.commandsDropped).toBe(29); | |
| 82 | }); | |
| 83 | ||
| 84 | test("a socket that goes quiet is counted as a stall and opened again; a drive on it ends, holding", async () => { | |
| 85 | // A server that accepts, says hello once, then never speaks. | |
| 86 | let connections = 0; | |
| 87 | const http = createServer(); | |
| 88 | const wss = new WebSocketServer({ server: http }); | |
| 89 | wss.on("connection", (ws) => (connections++, ws.send(JSON.stringify({ t: "hello", now: Date.now() })))); | |
| 90 | const port = await new Promise<number>((r) => http.listen(0, "127.0.0.1", () => r((http.address() as { port: number }).port))); | |
| 91 | stops.push(() => (wss.close(), http.closeAllConnections(), http.close())); | |
| 92 | const drives: any[] = []; | |
| 93 | const s = socket({ url: `ws://127.0.0.1:${port}/`, stallMs: 300, backoffMs: [50], onDrive: (m) => drives.push(m) }); | |
| 94 | await until(() => s.open); | |
| 95 | s.command({ 1: 2000 }); | |
| 96 | await until(() => s.totals.reconnects >= 1, 3000); | |
| 97 | expect(s.totals.stalls).toBeGreaterThanOrEqual(1); | |
| 98 | expect(connections).toBeGreaterThanOrEqual(2); | |
| 99 | expect(drives[0]).toMatchObject({ state: "closed" }); | |
| 100 | expect(s.driving).toBe(false); | |
| 101 | }); | |
| 102 | }); |
Software/so101/src/ui/health.js added +98
| @@ -0,0 +1,98 @@ | ||
| 1 | // @ts-check | |
| 2 | /** | |
| 3 | * The admin's Health panel: GET /health (src/telemetry.ts) as one sparkline per number over the relay's two-minute | |
| 4 | * history, a dashed line at each target, and the totals since the app started. Only an admin opens it, so it is not | |
| 5 | * part of the page's first load: the page imports this module on the first click. Refreshed every 2 s while open, | |
| 6 | * as the relay samples. Numbers reach the markup only through num(); the labels are this file's own. | |
| 7 | */ | |
| 8 | import { sparkline } from "./sparkline.js"; | |
| 9 | ||
| 10 | /** [label, unit, the number in a /health answer, its target or null]. @type {Array<[string, string, (h: any) => unknown, number | null]>} */ | |
| 11 | export const HEALTH = [ | |
| 12 | ["command → pose p95", "ms", (h) => h.commandRttMs?.p95, 250], | |
| 13 | ["camera frame age p95", "ms", (h) => h.frameAgeMs?.p95, 400], | |
| 14 | ["camera fps per viewer", "fps", (h) => h.viewerFps, null], | |
| 15 | ["camera fps into the app", "fps", (h) => h.cameraFpsIn, null], | |
| 16 | ["arm server round trip p95", "ms", (h) => h.armRttMs?.p95, null], | |
| 17 | ["commands per second", "/s", (h) => h.commandsPerS, null], | |
| 18 | ["viewers", "", (h) => h.viewers, null], | |
| 19 | ["bytes out", "KB/s", (h) => (typeof h.bytesOutPerS === "number" ? h.bytesOutPerS / 1024 : null), null], | |
| 20 | ]; | |
| 21 | ||
| 22 | /** @param {unknown} v @returns {number | null} */ | |
| 23 | const num = (v) => (typeof v === "number" && Number.isFinite(v) ? v : null); | |
| 24 | ||
| 25 | /** The sparklines' markup for a /health answer: label, line, current value, per number. @param {any} h @returns {string} */ | |
| 26 | export function sparks(h) { | |
| 27 | const points = [...(Array.isArray(h.history) ? h.history : []), h]; | |
| 28 | return HEALTH.map(([label, unit, pick, target]) => { | |
| 29 | const values = points.map((p) => num(pick(p))), now = values[values.length - 1] ?? null; | |
| 30 | const { runs, max } = sparkline(values, { w: 120, h: 28, max: target ?? 0 }); | |
| 31 | const y = target === null ? null : Math.round((28 - (target / max) * 28) * 10) / 10; | |
| 32 | return `<b>${label}</b><svg viewBox="0 0 120 28" aria-hidden="true">${y === null ? "" : `<line x1="0" x2="120" y1="${y}" y2="${y}"/>`}${runs.map((r) => `<polyline points="${r}"/>`).join("")}</svg>` | |
| 33 | + `<output class="${target !== null && now !== null && now > target ? "over" : ""}">${now === null ? "—" : `${now < 10 ? now.toFixed(1) : Math.round(now)} ${unit}`}</output>`; | |
| 34 | }).join(""); | |
| 35 | } | |
| 36 | ||
| 37 | /** The totals' markup. @param {any} h @returns {string} */ | |
| 38 | export function totals(h) { | |
| 39 | const t = h.totals ?? {}; | |
| 40 | return [ | |
| 41 | ["commands sent", t.commandsSent], ["commands dropped (latest wins)", t.commandsDropped], ["commands merged by the app", t.commandsMerged], | |
| 42 | ["frames sent", t.framesSent], ["frames skipped for slow viewers", t.framesSkipped], ["frames shown", t.framesShown], | |
| 43 | ["stalls", t.stalls], ["reconnects", t.reconnects], ["arm server errors", t.armErrors], ["reports dropped (too soon)", t.reportsDropped], | |
| 44 | ["bytes out", num(t.bytesOut) === null ? null : `${(t.bytesOut / 1048576).toFixed(1)} MB`], | |
| 45 | ].map(([k, v]) => `<b>${k}</b><span>${typeof v === "string" ? v : num(v) ?? "—"}</span>`).join(""); | |
| 46 | } | |
| 47 | ||
| 48 | /** One row per number: its last two minutes as a line, a dashed target where there is one. */ | |
| 49 | const STYLE = ` | |
| 50 | .spark { display: grid; grid-template-columns: minmax(0, 1fr) 120px 9ch; align-items: center; gap: 4px 12px; font-size: 14px; margin: 8px 0; } | |
| 51 | .spark b { color: var(--dim); font-weight: 500; } .spark output { text-align: right; font-variant-numeric: tabular-nums; } .spark output.over { color: var(--warn); } | |
| 52 | .spark svg { width: 120px; height: 28px; overflow: visible; } .spark polyline { fill: none; stroke: var(--blue); stroke-width: 1.5; stroke-linejoin: round; } | |
| 53 | .spark line { stroke: var(--warn); stroke-width: 1; stroke-dasharray: 3 3; }`; | |
| 54 | ||
| 55 | const ARTICLE = ` | |
| 56 | <article> | |
| 57 | <h2>Health: how the link feels</h2> | |
| 58 | <p>What viewers measure over <code>WS /arm</code> and what the app sees itself, the last two minutes. Dashed lines are the targets. The same numbers are on <code>/metrics</code> for a scraper, and in the log every minute.</p> | |
| 59 | <div class="spark" id="healthSparks"></div> | |
| 60 | <h3>Since the app started</h3> | |
| 61 | <div class="status" id="healthTotals"></div> | |
| 62 | </article>`; | |
| 63 | ||
| 64 | /** The page's document: this module only runs there (the type check has no DOM, as for src/ui/drive.js). @type {any} */ | |
| 65 | const doc = /** @type {any} */ (globalThis).document; | |
| 66 | ||
| 67 | /** The panel, built once and kept. @returns {any} */ | |
| 68 | function panel() { | |
| 69 | const found = doc.getElementById("healthPanel"); | |
| 70 | if (found) return found; | |
| 71 | const close = Object.assign(doc.createElement("button"), { className: "close", textContent: "Close" }); | |
| 72 | const box = Object.assign(doc.createElement("div"), { id: "healthPanel", className: "panel", hidden: true }); | |
| 73 | box.append(close, Object.assign(doc.createElement("style"), { textContent: STYLE })); | |
| 74 | box.insertAdjacentHTML("beforeend", ARTICLE); // fixed markup from this file, no data in it | |
| 75 | close.onclick = () => (box.hidden = true); | |
| 76 | (doc.getElementById("shell") ?? doc.body).append(box); | |
| 77 | return box; | |
| 78 | } | |
| 79 | ||
| 80 | let refresher = /** @type {ReturnType<typeof setInterval> | undefined} */ (undefined); | |
| 81 | ||
| 82 | /** | |
| 83 | * Open the panel and keep it fresh while it is open. | |
| 84 | * @param {{ api: (path: string) => Promise<any>, msg: (text: string, bad?: boolean) => void }} page | |
| 85 | */ | |
| 86 | export async function openHealth({ api, msg }) { | |
| 87 | const box = panel(); | |
| 88 | const show = async () => { | |
| 89 | if (box.hidden && refresher !== undefined) return void (clearInterval(refresher), (refresher = undefined)); | |
| 90 | const h = await api("/health"); | |
| 91 | if (!h.ok) return msg(h.error ?? "no health from the app", true); | |
| 92 | box.querySelector("#healthSparks").innerHTML = sparks(h); | |
| 93 | box.querySelector("#healthTotals").innerHTML = totals(h); | |
| 94 | }; | |
| 95 | box.hidden = false; | |
| 96 | await show(); | |
| 97 | refresher ??= setInterval(show, 2000); | |
| 98 | } |
Software/so101/src/ui/metrics.js added +48
| @@ -0,0 +1,48 @@ | ||
| 1 | // @ts-check | |
| 2 | /** | |
| 3 | * The Metrics panel: the arm's own GET /metrics (jams, zone blocks, peaks, the recent events) drawn into the page's | |
| 4 | * #metricsPanel. Opened from the Controls tab's button, so it is not part of the page's first load: the page imports | |
| 5 | * this module on the first click, as it does src/ui/health.js. Event details reach the page only as text nodes. | |
| 6 | */ | |
| 7 | ||
| 8 | /** The page's document: this module only runs there (the type check has no DOM, as for src/ui/drive.js). @type {any} */ | |
| 9 | const doc = /** @type {any} */ (globalThis).document; | |
| 10 | ||
| 11 | const NAMES = /** @type {Record<string, string>} */ ({ 1: "shoulder_pan", 2: "shoulder_lift", 3: "elbow_flex", 4: "wrist_flex", 5: "wrist_roll", 6: "gripper" }); | |
| 12 | ||
| 13 | /** @param {string} tag @param {any} props @param {...any} children @returns {any} */ | |
| 14 | const el = (tag, props = {}, ...children) => { | |
| 15 | const e = Object.assign(doc.createElement(tag), props); | |
| 16 | e.append(...children); | |
| 17 | return e; | |
| 18 | }; | |
| 19 | ||
| 20 | /** | |
| 21 | * The summary's [label, value] rows for a /metrics answer and this browser's teleop lag. | |
| 22 | * @param {any} m @param {{ p50: number, p95: number, n: number } | null | undefined} lag | |
| 23 | * @returns {Array<[string, unknown]>} | |
| 24 | */ | |
| 25 | export function rows(m, lag) { | |
| 26 | const c = m.counters ?? {}; | |
| 27 | return [ | |
| 28 | ["since", (m.since ?? "").replace("T", " ").slice(0, 19)], | |
| 29 | ["jams", c.jam], ["zone blocks", c.block], ["stalls on moves", c.stall], ["lost servos", c.missing], ["hot joints", c.hot], | |
| 30 | ["sessions", `${c.stream} streams · ${c.move} moves · ${c.play} playbacks · ${c.record} recordings`], | |
| 31 | ["teleop lag, this browser", lag ? `p50 ${lag.p50} ms · p95 ${lag.p95} ms over ${lag.n} samples` : "—"], | |
| 32 | ["closest to the table", m.closest ? `${(m.closest.z * 1000).toFixed(0)} mm (${m.closest.link.replace(/_link$/, "")}, ${m.closest.at.slice(11, 19)})` : "—"], | |
| 33 | ]; | |
| 34 | } | |
| 35 | ||
| 36 | /** Asks /metrics and fills the panel. @param {{ api: (path: string) => Promise<any>, lag: () => any }} page */ | |
| 37 | export async function showMetrics({ api, lag }) { | |
| 38 | const $ = (/** @type {string} */ id) => doc.getElementById(id); | |
| 39 | const m = await api("/metrics"); | |
| 40 | $("metricsSummary").innerHTML = rows(m, lag()).map(([k, v]) => `<b>${k}</b><span>${v}</span>`).join(""); | |
| 41 | $("peaks").innerHTML = "<tr><th>joint</th><th>peak load</th><th>peak temperature</th></tr>" + Object.entries(m.peaks ?? {}).map(([id, p]) => `<tr><td>${NAMES[id] ?? id}</td><td>${p.load.toFixed(0)} %</td><td>${p.temperature} °C</td></tr>`).join(""); | |
| 42 | // Built from text nodes: an event's detail carries people's names (who pressed the E-stop) and error strings. | |
| 43 | const recent = [...(m.recent ?? [])].reverse(); | |
| 44 | $("events").replaceChildren(...(recent.length ? recent.map((e) => el("div", {}, | |
| 45 | `${String(e.t).slice(5, 19).replace("T", " ")} `, el("b", { textContent: String(e.kind) }), | |
| 46 | ` ${Object.entries(e.detail ?? {}).map(([k, v]) => `${k}=${typeof v === "object" ? JSON.stringify(v) : v}`).join(" ")}`)) : [el("div", { textContent: "none yet" })])); | |
| 47 | $("metricsPanel").hidden = false; | |
| 48 | } |
Software/so101/src/ui/metrics.test.ts added +25
| @@ -0,0 +1,25 @@ | ||
| 1 | import { describe, expect, test } from "bun:test"; | |
| 2 | import { rows } from "./metrics.js"; | |
| 3 | ||
| 4 | describe("the Metrics panel", () => { | |
| 5 | test("names the counters, the sessions, this browser's teleop lag and the closest the arm came to the table", () => { | |
| 6 | const m = { | |
| 7 | since: "2026-10-20T09:15:30.123Z", | |
| 8 | counters: { jam: 1, block: 2, stall: 3, missing: 0, hot: 4, stream: 5, move: 6, play: 7, record: 8 }, | |
| 9 | closest: { z: 0.0123, link: "gripper_link", at: "2026-10-20T09:20:01.000Z" }, | |
| 10 | }; | |
| 11 | const r = Object.fromEntries(rows(m, { p50: 40, p95: 90, n: 120 })); | |
| 12 | expect(r.since).toBe("2026-10-20 09:15:30"); | |
| 13 | expect(r.jams).toBe(1); | |
| 14 | expect(r["hot joints"]).toBe(4); | |
| 15 | expect(r.sessions).toBe("5 streams · 6 moves · 7 playbacks · 8 recordings"); | |
| 16 | expect(r["teleop lag, this browser"]).toBe("p50 40 ms · p95 90 ms over 120 samples"); | |
| 17 | expect(r["closest to the table"]).toBe("12 mm (gripper, 09:20:01)"); | |
| 18 | }); | |
| 19 | ||
| 20 | test("without a leader or a closest point, says so with a dash", () => { | |
| 21 | const r = Object.fromEntries(rows({ counters: {} }, null)); | |
| 22 | expect(r["teleop lag, this browser"]).toBe("—"); | |
| 23 | expect(r["closest to the table"]).toBe("—"); | |
| 24 | }); | |
| 25 | }); |
Software/so101/src/ui/sparkline.js added +29
| @@ -0,0 +1,29 @@ | ||
| 1 | // @ts-check | |
| 2 | /** | |
| 3 | * A sparkline's shape: recent values (oldest first, null where there was nothing to measure) as SVG polyline points | |
| 4 | * in a `w` × `h` box, lowest at the bottom. The scale starts at zero, so a flat line near the top means "high", | |
| 5 | * and a gap in the data is a gap in the line: one points string per unbroken run. | |
| 6 | * @param {ReadonlyArray<number | null | undefined>} values | |
| 7 | * @param {{ w?: number, h?: number, max?: number }} [options] `max` fixes the top of the scale (e.g. a target) | |
| 8 | * @returns {{ runs: string[], max: number }} | |
| 9 | */ | |
| 10 | export function sparkline(values, options = {}) { | |
| 11 | const w = options.w ?? 120, h = options.h ?? 28; | |
| 12 | const finite = values.filter((v) => typeof v === "number" && Number.isFinite(v)).map(Number); | |
| 13 | const max = Math.max(options.max ?? 0, ...finite, 0) || 1; | |
| 14 | const step = values.length > 1 ? w / (values.length - 1) : 0; | |
| 15 | /** @type {string[]} */ | |
| 16 | const runs = []; | |
| 17 | /** @type {string[]} */ | |
| 18 | let run = []; | |
| 19 | values.forEach((v, i) => { | |
| 20 | if (typeof v !== "number" || !Number.isFinite(v)) { | |
| 21 | if (run.length) runs.push(run.join(" ")), (run = []); | |
| 22 | return; | |
| 23 | } | |
| 24 | const x = Math.round(i * step * 10) / 10, y = Math.round((h - (Math.max(0, v) / max) * h) * 10) / 10; | |
| 25 | run.push(`${x},${y}`); | |
| 26 | }); | |
| 27 | if (run.length) runs.push(run.join(" ")); | |
| 28 | return { runs, max }; | |
| 29 | } |
Software/so101/src/ui/sparkline.test.ts added +18
| @@ -0,0 +1,18 @@ | ||
| 1 | import { describe, expect, test } from "bun:test"; | |
| 2 | import { sparkline } from "./sparkline.js"; | |
| 3 | ||
| 4 | describe("a sparkline", () => { | |
| 5 | test("values span the box from zero at the bottom to the largest at the top", () => { | |
| 6 | expect(sparkline([0, 50, 100], { w: 100, h: 10 })).toEqual({ runs: ["0,10 50,5 100,0"], max: 100 }); | |
| 7 | }); | |
| 8 | ||
| 9 | test("a gap in the data breaks the line", () => { | |
| 10 | expect(sparkline([10, null, 10, 10], { w: 30, h: 10 }).runs).toEqual(["0,0", "20,0 30,0"]); | |
| 11 | }); | |
| 12 | ||
| 13 | test("a fixed top keeps small numbers small, and nothing measured is no line", () => { | |
| 14 | expect(sparkline([100], { w: 10, h: 10, max: 400 })).toEqual({ runs: ["0,7.5"], max: 400 }); | |
| 15 | expect(sparkline([null, null]).runs).toEqual([]); | |
| 16 | expect(sparkline([]).runs).toEqual([]); | |
| 17 | }); | |
| 18 | }); |
Software/so101/src/viewer-socket.test.ts added +209
| @@ -0,0 +1,209 @@ | ||
| 1 | import { afterEach, describe, expect, test } from "bun:test"; | |
| 2 | import { mkdtempSync } from "node:fs"; | |
| 3 | import { tmpdir } from "node:os"; | |
| 4 | import { join } from "node:path"; | |
| 5 | import WebSocket from "ws"; | |
| 6 | import { fakePi } from "./fake-pi.ts"; | |
| 7 | import { sign } from "./identity.ts"; | |
| 8 | import { Lobby } from "./lobby.ts"; | |
| 9 | import { parseRemote, startRelay, type RunningRelay } from "./relay.ts"; | |
| 10 | import { FRAME_HEADER, parseTargets } from "./viewer-socket.ts"; | |
| 11 | ||
| 12 | const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms)); | |
| 13 | const KEY = "k".repeat(64); | |
| 14 | const as = (name: string, role: "user" | "admin") => { | |
| 15 | const identity = `id-${name}|${name}|${role}|${Date.now() + 600_000}`; | |
| 16 | return { "x-forge-identity": identity, "x-forge-signature": sign(identity, KEY) }; | |
| 17 | }; | |
| 18 | ||
| 19 | interface Client { | |
| 20 | ws: WebSocket; | |
| 21 | json: Array<Record<string, any>>; | |
| 22 | frames: Buffer[]; | |
| 23 | send(m: Record<string, unknown>): void; | |
| 24 | until<T>(fn: () => T | undefined | false, ms?: number): Promise<T>; | |
| 25 | } | |
| 26 | async function client(port: number, headers: Record<string, string> = {}, onFrame?: (c: Client, frame: Buffer) => void): Promise<Client> { | |
| 27 | const ws = new WebSocket(`ws://127.0.0.1:${port}/arm`, { headers }); | |
| 28 | const c: Client = { | |
| 29 | ws, | |
| 30 | json: [], | |
| 31 | frames: [], | |
| 32 | send: (m) => ws.send(JSON.stringify(m)), | |
| 33 | async until(fn, ms = 3000) { | |
| 34 | const end = Date.now() + ms; | |
| 35 | for (;;) { | |
| 36 | const v = fn(); | |
| 37 | if (v) return v; | |
| 38 | if (Date.now() > end) throw new Error("timed out"); | |
| 39 | await sleep(10); | |
| 40 | } | |
| 41 | }, | |
| 42 | }; | |
| 43 | ws.on("message", (data, binary) => { | |
| 44 | if (binary) { | |
| 45 | const b = data as Buffer; | |
| 46 | c.frames.push(b); | |
| 47 | onFrame?.(c, b); | |
| 48 | } else c.json.push(JSON.parse(String(data))); | |
| 49 | }); | |
| 50 | await new Promise((r, j) => (ws.once("open", r), ws.once("error", j))); | |
| 51 | return c; | |
| 52 | } | |
| 53 | ||
| 54 | let pi: Awaited<ReturnType<typeof fakePi>> | undefined; | |
| 55 | let relay: RunningRelay | undefined; | |
| 56 | const clients: Client[] = []; | |
| 57 | let lobby: Lobby | undefined; | |
| 58 | afterEach(async () => { | |
| 59 | for (const c of clients.splice(0)) c.ws.terminate(); | |
| 60 | await Promise.race([relay?.close(), sleep(500)]); | |
| 61 | lobby?.close(); | |
| 62 | pi?.close(); | |
| 63 | relay = pi = lobby = undefined; | |
| 64 | }); | |
| 65 | ||
| 66 | async function boot(delayMs = 0, withLobby = false) { | |
| 67 | pi = await fakePi(delayMs); | |
| 68 | const remote = parseRemote({ ARM_SERVER: `ws://127.0.0.1:${pi.port}`, ARM_CAMERA: `http://127.0.0.1:${pi.port}/stream` })!; | |
| 69 | if (withLobby) lobby = new Lobby({ identityKey: KEY, adminFile: join(mkdtempSync(join(tmpdir(), "so101-vs-")), "admin.json") }); | |
| 70 | relay = await startRelay({ port: 0, simulation: "http://127.0.0.1:1", remote, lobby, logEveryMs: 0, viewer: { seenTimeoutMs: 10_000 } }); | |
| 71 | return relay.port; | |
| 72 | } | |
| 73 | const open = async (...args: Parameters<typeof client>) => { | |
| 74 | const c = await client(...args); | |
| 75 | clients.push(c); | |
| 76 | return c; | |
| 77 | }; | |
| 78 | ||
| 79 | describe("WS /arm", () => { | |
| 80 | test("the arm's pose comes in many times a second, from the one reader", async () => { | |
| 81 | const port = await boot(); | |
| 82 | const c = await open(port); | |
| 83 | await sleep(600); | |
| 84 | const poses = c.json.filter((m) => m.t === "pose"); | |
| 85 | expect(c.json[0]).toMatchObject({ t: "hello" }); | |
| 86 | expect(poses.length).toBeGreaterThan(8); | |
| 87 | expect(poses.at(-1)!.status).toMatchObject({ joints: [{ id: 1, position: 2048 }], source: `live 127.0.0.1:${pi!.port}` }); | |
| 88 | }); | |
| 89 | ||
| 90 | test("commands are latest-wins: a burst reaches the arm as a few frames ending on the last, and the pose acknowledges it", async () => { | |
| 91 | const port = await boot(); | |
| 92 | const c = await open(port); | |
| 93 | for (let seq = 1; seq <= 40; seq++) c.send({ t: "cmd", seq, targets: { 1: 2000 + seq } }); | |
| 94 | await c.until(() => pi!.follow.at(-1) === JSON.stringify({ 1: 2040 })); | |
| 95 | expect(pi!.follow.length).toBeLessThanOrEqual(4); | |
| 96 | await c.until(() => c.json.some((m) => m.t === "pose" && m.ack === 40 && m.status.joints[0].position === 2040)); | |
| 97 | expect(relay!.telemetry.totals.commandsMerged).toBeGreaterThan(30); | |
| 98 | }); | |
| 99 | ||
| 100 | test("a slow viewer gets fewer frames without slowing the others", async () => { | |
| 101 | const port = await boot(); | |
| 102 | // The fast one says it has shown each frame at once; the slow one never does. | |
| 103 | const fast = await open(port, {}, (c, frame) => c.send({ t: "seen", n: frame.readUInt32BE(8) })); | |
| 104 | const slow = await open(port); | |
| 105 | fast.send({ t: "watch", cameras: [1] }); | |
| 106 | slow.send({ t: "watch", cameras: [1] }); | |
| 107 | await sleep(1200); | |
| 108 | expect(fast.frames.length).toBeGreaterThan(20); | |
| 109 | expect(slow.frames.length).toBe(4); // the frame window, then nothing until it catches up | |
| 110 | const f = fast.frames.at(-1)!; | |
| 111 | expect(f.readUInt8(0)).toBe(1); | |
| 112 | expect(f.readUInt32BE(4)).toBe(1); | |
| 113 | expect(Date.now() - f.readDoubleBE(12)).toBeLessThan(1000); | |
| 114 | expect(f.subarray(FRAME_HEADER, FRAME_HEADER + 2)).toEqual(Buffer.from([0xff, 0xd8])); | |
| 115 | expect(relay!.telemetry.totals.framesSkipped).toBeGreaterThan(10); | |
| 116 | // Once it shows what it has, it gets the newest frame, not the backlog. | |
| 117 | slow.send({ t: "seen", n: 4 }); | |
| 118 | await slow.until(() => slow.frames.length > 4); | |
| 119 | expect(Date.now() - slow.frames[4]!.readDoubleBE(12)).toBeLessThan(200); | |
| 120 | }); | |
| 121 | ||
| 122 | test("the metrics report the delays: the arm's round trip and what viewers measured", async () => { | |
| 123 | const port = await boot(80); | |
| 124 | const c = await open(port, {}, (c, frame) => c.send({ t: "seen", n: frame.readUInt32BE(8) })); | |
| 125 | c.send({ t: "watch", cameras: [1] }); | |
| 126 | for (let seq = 1; seq <= 12; seq++) c.send({ t: "cmd", seq, targets: { 1: 2000 + seq } }); | |
| 127 | await c.until(() => c.frames.length >= 10 && c.json.filter((m) => m.t === "pose").length >= 3); | |
| 128 | c.send({ t: "report", rtt: [120, 130, 140, 900], age: [210, 220], frames: 10, sent: 12, dropped: 3, stalls: 1, reconnects: 1 }); | |
| 129 | await sleep(100); | |
| 130 | const health = (await (await fetch(`http://127.0.0.1:${port}/health`)).json()) as Record<string, any>; | |
| 131 | expect(health.armRttMs.p50).toBeGreaterThanOrEqual(80); | |
| 132 | expect(health.commandRttMs).toEqual({ p50: 130, p95: 900 }); | |
| 133 | expect(health.frameAgeMs.p50).toBe(210); | |
| 134 | expect(health.viewers).toBe(1); | |
| 135 | expect(health.totals).toMatchObject({ commandsSent: 12, commandsDropped: 3, stalls: 1, reconnects: 1, framesShown: 10 }); | |
| 136 | const text = await (await fetch(`http://127.0.0.1:${port}/metrics`)).text(); | |
| 137 | expect(text).toContain('arm_command_rtt_ms{quantile="0.5"} 130'); | |
| 138 | expect(text).toContain('arm_camera_frame_age_ms{quantile="0.95"} 220'); | |
| 139 | expect(text).toMatch(/arm_server_rtt_ms\{quantile="0.5"\} (8\d|9\d|1\d\d)/); | |
| 140 | expect(text).toContain("arm_viewers 1"); | |
| 141 | }); | |
| 142 | ||
| 143 | test("a burst of reports moves the numbers once; each is held to what crossed that socket", async () => { | |
| 144 | const port = await boot(); | |
| 145 | const c = await open(port); | |
| 146 | c.send({ t: "cmd", seq: 1, targets: { 1: 1500 } }); | |
| 147 | c.send({ t: "cmd", seq: 2, targets: { 1: 1501 } }); | |
| 148 | await c.until(() => c.json.some((m) => m.t === "drive")); | |
| 149 | const forged = { t: "report", rtt: Array(200).fill(0), age: Array(200).fill(0), frames: 100_000, sent: 100_000, dropped: 5, stalls: 0, reconnects: 0 }; | |
| 150 | for (let i = 0; i < 50; i++) c.send(forged); | |
| 151 | await c.until(() => relay!.telemetry.totals.reportsDropped === 49); | |
| 152 | // Two commands crossed the socket and no frames: that is all one report may claim. | |
| 153 | expect(relay!.telemetry.totals).toMatchObject({ commandsSent: 2, commandsDropped: 5, framesShown: 0, reportsDropped: 49 }); | |
| 154 | expect(relay!.telemetry.rtt.count()).toBe(2); | |
| 155 | expect(relay!.telemetry.age.count()).toBe(0); | |
| 156 | // A second socket is its own: one more report from it is taken, not dropped. | |
| 157 | const d = await open(port); | |
| 158 | d.send(forged); | |
| 159 | await sleep(100); | |
| 160 | expect(relay!.telemetry.totals).toMatchObject({ commandsDropped: 10, reportsDropped: 49 }); | |
| 161 | expect(await (await fetch(`http://127.0.0.1:${port}/metrics`)).text()).toContain("arm_reports_dropped_total 49"); | |
| 162 | }); | |
| 163 | ||
| 164 | test("only the driver's commands reach the arm", async () => { | |
| 165 | const port = await boot(0, true); | |
| 166 | const c = await open(port, as("eve", "user")); | |
| 167 | c.send({ t: "cmd", seq: 1, targets: { 1: 1000 } }); | |
| 168 | const refused = await c.until(() => c.json.find((m) => m.t === "drive")); | |
| 169 | expect(refused).toMatchObject({ state: "refused" }); | |
| 170 | await sleep(100); | |
| 171 | expect(pi!.follow).toEqual([]); | |
| 172 | }); | |
| 173 | ||
| 174 | test("with no lobby (a run on one's own machine) every viewer drives, as on /follow", async () => { | |
| 175 | const port = await boot(); | |
| 176 | const c = await open(port); | |
| 177 | c.send({ t: "cmd", seq: 1, targets: { 1: 1500 } }); | |
| 178 | await c.until(() => pi!.follow.includes(JSON.stringify({ 1: 1500 }))); | |
| 179 | expect(c.json.find((m) => m.t === "drive")).toMatchObject({ state: "open" }); | |
| 180 | // /follow, the old path, lets the same anonymous page drive. | |
| 181 | const follow = new WebSocket(`ws://127.0.0.1:${port}/follow`); | |
| 182 | await new Promise((r, j) => (follow.once("open", r), follow.once("error", j))); | |
| 183 | follow.send(JSON.stringify({ 1: 1600 })); | |
| 184 | await c.until(() => pi!.follow.includes(JSON.stringify({ 1: 1600 }))); | |
| 185 | follow.terminate(); | |
| 186 | }); | |
| 187 | ||
| 188 | test("/health is the admin's; /metrics stays open to a scraper", async () => { | |
| 189 | const port = await boot(0, true); | |
| 190 | const health = (headers: Record<string, string> = {}) => fetch(`http://127.0.0.1:${port}/health`, { headers }); | |
| 191 | expect((await health()).status).toBe(401); | |
| 192 | const refused = await health(as("eve", "user")); | |
| 193 | expect(refused.status).toBe(403); | |
| 194 | expect(await refused.json()).toEqual({ error: "only an admin can see the link's health" }); | |
| 195 | const ok = await health(as("trevato", "admin")); | |
| 196 | expect(ok.status).toBe(200); | |
| 197 | expect(await ok.json()).toMatchObject({ viewers: 0, history: [] }); | |
| 198 | const metrics = await fetch(`http://127.0.0.1:${port}/metrics`); | |
| 199 | expect(metrics.status).toBe(200); | |
| 200 | expect(await metrics.text()).toContain("arm_viewers 0"); | |
| 201 | }); | |
| 202 | ||
| 203 | test("targets are checked at the door", () => { | |
| 204 | expect(parseTargets({ 1: 2048.4, 6: 3000 })).toEqual(new Map([[1, 2048], [6, 3000]])); | |
| 205 | expect(parseTargets({ 0: 1, 254: 1, 2: -1, 3: 5000, x: 1, 4: "1" })).toBeNull(); | |
| 206 | expect(parseTargets([1, 2])).toBeNull(); | |
| 207 | expect(parseTargets(null)).toBeNull(); | |
| 208 | }); | |
| 209 | }); |
Software/so101/src/viewer-socket.ts added +400
| @@ -0,0 +1,400 @@ | ||
| 1 | /** | |
| 2 | * One WebSocket per viewer, WS /arm: everything that has to feel live goes over it, so nothing pays a public round | |
| 3 | * trip per request (through the forge's front door that is ~0.4 s each; a socket pays it once). | |
| 4 | * | |
| 5 | * The app sends: | |
| 6 | * {"t":"hello","now"} once; "now" is the app's clock, for the page's frame ages | |
| 7 | * {"t":"pose","now","askedAt","ack","status"} the arm's /status, up to 30 times a second (src/pose-feed.ts); | |
| 8 | * "ack" is this viewer's last command the arm had when it was read | |
| 9 | * {"t":"drive","state":"open"|"closed"|"refused","code"?,"reason"?} | |
| 10 | * {"t":"pong","c","now"} the answer to a ping, for the clock offset | |
| 11 | * binary: a camera frame, a 20-byte header then the picture (frameHeader below) | |
| 12 | * The page sends: | |
| 13 | * {"t":"watch","cameras":[1],"hz"?} which cameras to send, and how many poses a second (1..30) | |
| 14 | * {"t":"cmd","seq","targets":{"1":2048}} the driver's joint targets, ticks; merged to the command rate, | |
| 15 | * latest per joint winning, then sent on the arm's own /follow | |
| 16 | * {"t":"end"} let go: the stream to the arm closes shortly after, and it holds | |
| 17 | * {"t":"seen","n"} frames up to n are shown; until then at most `frameWindow` are out | |
| 18 | * {"t":"report",...} what the page measured (src/telemetry.ts parseReport); one per | |
| 19 | * socket every `reportEveryMs`, held to what crossed it (vouch) | |
| 20 | * {"t":"ping","c"} | |
| 21 | * | |
| 22 | * A viewer still busy with earlier frames gets the newest one next, never a queue (src/latest.ts): a slow phone sees | |
| 23 | * fewer frames, everyone else is not slowed. A viewer whose socket is lost leaves the arm parking, like /follow. | |
| 24 | */ | |
| 25 | ||
| 26 | import type { IncomingMessage } from "node:http"; | |
| 27 | import type { Duplex } from "node:stream"; | |
| 28 | import WebSocket, { WebSocketServer } from "ws"; | |
| 29 | import type { Picture } from "./camera-feed.ts"; | |
| 30 | import type { CameraHub } from "./cameras.ts"; | |
| 31 | import type { Who } from "./identity.ts"; | |
| 32 | import { Latest } from "./latest.ts"; | |
| 33 | import type { Lobby } from "./lobby.ts"; | |
| 34 | import type { Pose, PoseFeed } from "./pose-feed.ts"; | |
| 35 | import { earn, noCredit, parseReport, vouch, type Credit, type Telemetry } from "./telemetry.ts"; | |
| 36 | ||
| 37 | export interface ViewerSocketOptions { | |
| 38 | poses: PoseFeed; | |
| 39 | cameras: CameraHub; | |
| 40 | telemetry: Telemetry; | |
| 41 | lobby: Lobby | null; | |
| 42 | /** The arm's /follow stream, live or simulated, and the HTTP base it belongs to. */ | |
| 43 | openFollow: () => Promise<{ arm: WebSocket; base: string }>; | |
| 44 | /** A fresh speed cap for one drive: the frame as it may go to the arm, or null for none. */ | |
| 45 | capper: (base: string) => (frame: string) => string | null; | |
| 46 | /** The close code that tells the arm server its driver was lost: it parks. */ | |
| 47 | lostCode: number; | |
| 48 | commandHz?: number; | |
| 49 | /** How long after "end" (with no new command) the stream to the arm closes. */ | |
| 50 | endAfterMs?: number; | |
| 51 | /** How often viewers are pinged; one that misses a ping is lost. */ | |
| 52 | heartbeatMs?: number; | |
| 53 | /** Frames out to one viewer and not yet seen. */ | |
| 54 | frameWindow?: number; | |
| 55 | /** How long an unseen frame holds its place before it counts as lost. */ | |
| 56 | seenTimeoutMs?: number; | |
| 57 | /** One socket's reports are taken at most this often; the page sends one every 5 s, the rest are dropped and counted. */ | |
| 58 | reportEveryMs?: number; | |
| 59 | } | |
| 60 | ||
| 61 | const MAX_MESSAGE = 16_384; | |
| 62 | const MAX_CAMERAS_WATCHED = 8; | |
| 63 | /** A pose waits while this much is still unsent to a viewer: the next one is fresher anyway. */ | |
| 64 | const POSE_BACKLOG = 256 * 1024; | |
| 65 | const MIME = ["image/jpeg", "image/png"]; | |
| 66 | export const FRAME_HEADER = 20; | |
| 67 | ||
| 68 | /** kind 1 (u8), mime (u8: 0 jpeg, 1 png, 2 other), 2 bytes zero, camera id (u32), frame number (u32), the app's receipt time (f64 ms). */ | |
| 69 | export function frameHeader(camera: number, picture: Picture): Buffer { | |
| 70 | const h = Buffer.alloc(FRAME_HEADER); | |
| 71 | h.writeUInt8(1, 0); | |
| 72 | const mime = MIME.indexOf(picture.type.toLowerCase()); | |
| 73 | h.writeUInt8(mime < 0 ? 2 : mime, 1); | |
| 74 | h.writeUInt32BE(camera, 4); | |
| 75 | h.writeDoubleBE(picture.at, 12); | |
| 76 | return h; | |
| 77 | } | |
| 78 | ||
| 79 | /** A command's targets, checked: joint ids 1..253 and ticks 0..4095; null when there is nothing usable. */ | |
| 80 | export function parseTargets(raw: unknown): Map<number, number> | null { | |
| 81 | if (!raw || typeof raw !== "object" || Array.isArray(raw)) return null; | |
| 82 | const out = new Map<number, number>(); | |
| 83 | for (const [k, v] of Object.entries(raw as Record<string, unknown>)) { | |
| 84 | if (!/^\d{1,3}$/.test(k) || typeof v !== "number" || !Number.isFinite(v)) continue; | |
| 85 | const id = Number(k); | |
| 86 | if (id < 1 || id > 253 || v < 0 || v > 4095) continue; | |
| 87 | out.set(id, Math.round(v)); | |
| 88 | } | |
| 89 | return out.size ? out : null; | |
| 90 | } | |
| 91 | ||
| 92 | interface Drive { | |
| 93 | arm: WebSocket | null; | |
| 94 | base: string; | |
| 95 | cap: ((frame: string) => string | null) | null; | |
| 96 | pending: Map<number, number> | null; | |
| 97 | pendingSeq: number; | |
| 98 | /** Commands sent to the arm: [seq, when], the last few. */ | |
| 99 | sent: Array<[number, number]>; | |
| 100 | lastSentAt: number; | |
| 101 | pacer: ReturnType<typeof setTimeout> | undefined; | |
| 102 | ender: ReturnType<typeof setTimeout> | undefined; | |
| 103 | } | |
| 104 | ||
| 105 | interface Viewer { | |
| 106 | ws: WebSocket; | |
| 107 | who: Who; | |
| 108 | req: IncomingMessage; | |
| 109 | watching: Map<number, () => void>; | |
| 110 | frames: Latest; | |
| 111 | frameNo: number; | |
| 112 | unseen: Map<number, { done: () => void; timer: ReturnType<typeof setTimeout> }>; | |
| 113 | poseEveryMs: number; | |
| 114 | lastPoseAt: number; | |
| 115 | drive: Drive | null; | |
| 116 | /** The last frame number skipped so far, for the telemetry's count. */ | |
| 117 | skippedSeen: number; | |
| 118 | /** When this socket's last report was taken, and what it may still claim: a report is held to that (telemetry vouch). */ | |
| 119 | lastReportAt: number; | |
| 120 | credit: Credit; | |
| 121 | stopPoses: () => void; | |
| 122 | } | |
| 123 | ||
| 124 | export class ViewerSockets { | |
| 125 | private readonly wss = new WebSocketServer({ noServer: true, maxPayload: MAX_MESSAGE }); | |
| 126 | private readonly viewers = new Set<Viewer>(); | |
| 127 | private readonly heartbeat: ReturnType<typeof setInterval>; | |
| 128 | private readonly o: Required<Omit<ViewerSocketOptions, "lobby">> & { lobby: Lobby | null }; | |
| 129 | ||
| 130 | constructor(options: ViewerSocketOptions) { | |
| 131 | this.o = { commandHz: 30, endAfterMs: 250, heartbeatMs: 1000, frameWindow: 4, seenTimeoutMs: 2000, reportEveryMs: 4000, ...options }; | |
| 132 | const answered = new WeakSet<WebSocket>(); | |
| 133 | this.heartbeat = setInterval(() => { | |
| 134 | for (const v of this.viewers) { | |
| 135 | if (!answered.has(v.ws)) { | |
| 136 | v.ws.terminate(); | |
| 137 | continue; | |
| 138 | } | |
| 139 | answered.delete(v.ws); | |
| 140 | if (v.ws.readyState === WebSocket.OPEN) v.ws.ping(); | |
| 141 | } | |
| 142 | }, this.o.heartbeatMs); | |
| 143 | this.heartbeat.unref?.(); | |
| 144 | this.wss.on("connection", (ws: WebSocket) => { | |
| 145 | answered.add(ws); | |
| 146 | ws.on("pong", () => answered.add(ws)); | |
| 147 | }); | |
| 148 | } | |
| 149 | ||
| 150 | get count(): number { | |
| 151 | return this.viewers.size; | |
| 152 | } | |
| 153 | ||
| 154 | /** Viewers watching at least one camera. */ | |
| 155 | get watching(): number { | |
| 156 | let n = 0; | |
| 157 | for (const v of this.viewers) if (v.watching.size) n++; | |
| 158 | return n; | |
| 159 | } | |
| 160 | ||
| 161 | upgrade(req: IncomingMessage, socket: Duplex, head: Buffer): void { | |
| 162 | const who = this.o.lobby ? this.o.lobby.who(req) : null; | |
| 163 | this.wss.handleUpgrade(req, socket, head, (ws) => { | |
| 164 | this.wss.emit("connection", ws, req); | |
| 165 | this.join(ws, who, req); | |
| 166 | }); | |
| 167 | } | |
| 168 | ||
| 169 | /** A turn ended: that person's drive closes (the arm holds) and the page is told why. */ | |
| 170 | endTurn(id: string): void { | |
| 171 | for (const v of this.viewers) if (v.who?.id === id && v.drive) this.stopDrive(v, 1000, "your turn ended"); | |
| 172 | } | |
| 173 | ||
| 174 | close(): void { | |
| 175 | clearInterval(this.heartbeat); | |
| 176 | for (const v of this.viewers) v.ws.terminate(); | |
| 177 | this.wss.close(); | |
| 178 | } | |
| 179 | ||
| 180 | // ------------------------------------------------------------------ one viewer | |
| 181 | private send(v: Viewer, data: string | Buffer) { | |
| 182 | if (v.ws.readyState !== WebSocket.OPEN) return false; | |
| 183 | v.ws.send(data, { binary: typeof data !== "string" }); | |
| 184 | this.o.telemetry.count("bytesOut", typeof data === "string" ? Buffer.byteLength(data) : data.length); | |
| 185 | return true; | |
| 186 | } | |
| 187 | ||
| 188 | private json(v: Viewer, message: Record<string, unknown>) { | |
| 189 | this.send(v, JSON.stringify(message)); | |
| 190 | } | |
| 191 | ||
| 192 | private join(ws: WebSocket, who: Who, req: IncomingMessage) { | |
| 193 | const v: Viewer = { | |
| 194 | ws, | |
| 195 | who, | |
| 196 | req, | |
| 197 | watching: new Map(), | |
| 198 | frameNo: 0, | |
| 199 | unseen: new Map(), | |
| 200 | poseEveryMs: 0, | |
| 201 | lastPoseAt: 0, | |
| 202 | drive: null, | |
| 203 | skippedSeen: 0, | |
| 204 | lastReportAt: -Infinity, | |
| 205 | credit: noCredit(), | |
| 206 | stopPoses: () => {}, | |
| 207 | frames: undefined as unknown as Latest, | |
| 208 | }; | |
| 209 | v.frames = new Latest((data, done) => { | |
| 210 | if (ws.readyState !== WebSocket.OPEN || typeof data === "string") return false; | |
| 211 | const n = ++v.frameNo; | |
| 212 | data.writeUInt32BE(n, 8); | |
| 213 | v.unseen.set(n, { done, timer: setTimeout(() => this.seen(v, n), this.o.seenTimeoutMs) }); | |
| 214 | this.o.telemetry.count("framesSent"); | |
| 215 | v.credit = earn(v.credit, "frames"); | |
| 216 | return this.send(v, data); | |
| 217 | }, this.o.frameWindow); | |
| 218 | this.viewers.add(v); | |
| 219 | this.json(v, { t: "hello", now: Date.now(), frameWindow: this.o.frameWindow }); | |
| 220 | v.stopPoses = this.o.poses.listen((pose) => this.pose(v, pose)); | |
| 221 | ws.on("message", (data, binary) => { | |
| 222 | if (binary) return; | |
| 223 | let m: Record<string, unknown>; | |
| 224 | try { | |
| 225 | m = JSON.parse(String(data)); | |
| 226 | } catch { | |
| 227 | return; | |
| 228 | } | |
| 229 | if (m && typeof m === "object") this.receive(v, m); | |
| 230 | }); | |
| 231 | ws.on("close", (code) => this.leave(v, code)); | |
| 232 | ws.on("error", () => ws.terminate()); | |
| 233 | } | |
| 234 | ||
| 235 | private leave(v: Viewer, code: number) { | |
| 236 | if (!this.viewers.delete(v)) return; | |
| 237 | v.stopPoses(); | |
| 238 | for (const stop of v.watching.values()) stop(); | |
| 239 | v.watching.clear(); | |
| 240 | for (const { timer } of v.unseen.values()) clearTimeout(timer); | |
| 241 | v.unseen.clear(); | |
| 242 | // A page closed on purpose leaves the arm holding; one that vanished leaves it parking. | |
| 243 | if (v.drive) this.stopDrive(v, code === 1000 || code === 1001 || code === 1005 ? 1000 : this.o.lostCode, "the driver's connection was lost"); | |
| 244 | } | |
| 245 | ||
| 246 | private pose(v: Viewer, pose: Pose) { | |
| 247 | const now = Date.now(); | |
| 248 | if (now - v.lastPoseAt < v.poseEveryMs || v.ws.bufferedAmount > POSE_BACKLOG) return; | |
| 249 | v.lastPoseAt = now; | |
| 250 | let ack: number | null = null; | |
| 251 | for (const [seq, at] of v.drive?.sent ?? []) if (at <= pose.askedAt) ack = seq; | |
| 252 | this.json(v, { t: "pose", now, askedAt: pose.askedAt, ack, status: pose.status }); | |
| 253 | } | |
| 254 | ||
| 255 | private seen(v: Viewer, upTo: number) { | |
| 256 | for (const [n, entry] of v.unseen) { | |
| 257 | if (n > upTo) break; | |
| 258 | clearTimeout(entry.timer); | |
| 259 | v.unseen.delete(n); | |
| 260 | entry.done(); | |
| 261 | } | |
| 262 | } | |
| 263 | ||
| 264 | private receive(v: Viewer, m: Record<string, unknown>) { | |
| 265 | switch (m.t) { | |
| 266 | case "ping": | |
| 267 | return this.json(v, { t: "pong", c: typeof m.c === "number" && Number.isFinite(m.c) ? m.c : null, now: Date.now() }); | |
| 268 | case "seen": | |
| 269 | if (typeof m.n === "number" && Number.isSafeInteger(m.n)) this.seen(v, m.n); | |
| 270 | return; | |
| 271 | case "report": | |
| 272 | return this.report(v, m); | |
| 273 | case "watch": | |
| 274 | return this.watch(v, m.cameras, m.hz); | |
| 275 | case "cmd": | |
| 276 | return this.command(v, m.seq, m.targets); | |
| 277 | case "end": | |
| 278 | return this.release(v); | |
| 279 | } | |
| 280 | } | |
| 281 | ||
| 282 | /** Keyed on the socket, never on anything the page wrote: one report per `reportEveryMs`, held to what crossed it. */ | |
| 283 | private report(v: Viewer, m: Record<string, unknown>) { | |
| 284 | const now = Date.now(); | |
| 285 | if (now - v.lastReportAt < this.o.reportEveryMs) return this.o.telemetry.count("reportsDropped"); | |
| 286 | const r = parseReport(m); | |
| 287 | if (!r) return; | |
| 288 | v.lastReportAt = now; | |
| 289 | const { report, left } = vouch(r, v.credit); | |
| 290 | v.credit = left; | |
| 291 | this.o.telemetry.report(report, now); | |
| 292 | } | |
| 293 | ||
| 294 | private watch(v: Viewer, cameras: unknown, hz: unknown) { | |
| 295 | if (typeof hz === "number" && Number.isFinite(hz)) v.poseEveryMs = 1000 / Math.max(1, Math.min(30, hz)) - 5; | |
| 296 | if (!Array.isArray(cameras)) return; | |
| 297 | const wanted = new Set(cameras.filter((id): id is number => Number.isSafeInteger(id) && id > 0).slice(0, MAX_CAMERAS_WATCHED)); | |
| 298 | for (const [id, stop] of v.watching) if (!wanted.has(id)) stop(), v.watching.delete(id); | |
| 299 | for (const id of wanted) { | |
| 300 | if (v.watching.has(id)) continue; | |
| 301 | const feed = this.o.cameras.feed(id); | |
| 302 | if (!feed) continue; | |
| 303 | v.watching.set( | |
| 304 | id, | |
| 305 | feed.pictures((picture) => { | |
| 306 | v.frames.offer(String(id), Buffer.concat([frameHeader(id, picture), picture.bytes])); | |
| 307 | if (v.frames.skipped > v.skippedSeen) this.o.telemetry.count("framesSkipped", v.frames.skipped - v.skippedSeen), (v.skippedSeen = v.frames.skipped); | |
| 308 | }), | |
| 309 | ); | |
| 310 | } | |
| 311 | } | |
| 312 | ||
| 313 | // ------------------------------------------------------------------ driving | |
| 314 | private refusal(v: Viewer): string | null { | |
| 315 | const lobby = this.o.lobby; | |
| 316 | if (!lobby) return null; | |
| 317 | return lobby.refusal(v.who, v.req)?.error ?? null; | |
| 318 | } | |
| 319 | ||
| 320 | private command(v: Viewer, seq: unknown, raw: unknown) { | |
| 321 | const targets = parseTargets(raw); | |
| 322 | if (!targets) return; | |
| 323 | v.credit = earn(v.credit, "command"); | |
| 324 | const why = this.refusal(v); | |
| 325 | if (why) { | |
| 326 | if (v.drive) this.stopDrive(v, 1000, why); | |
| 327 | return this.json(v, { t: "drive", state: "refused", reason: why }); | |
| 328 | } | |
| 329 | let d = v.drive; | |
| 330 | if (!d) { | |
| 331 | d = v.drive = { arm: null, base: "", cap: null, pending: null, pendingSeq: 0, sent: [], lastSentAt: 0, pacer: undefined, ender: undefined }; | |
| 332 | const drive = d; | |
| 333 | this.o.openFollow().then( | |
| 334 | ({ arm, base }) => { | |
| 335 | if (v.drive !== drive) return void arm.close(1000); | |
| 336 | drive.arm = arm; | |
| 337 | drive.base = base; | |
| 338 | drive.cap = this.o.capper(base); | |
| 339 | arm.on("close", (code, why) => { | |
| 340 | if (v.drive !== drive) return; | |
| 341 | this.stopDrive(v, null, why.length ? why.toString() : "the arm server closed the stream", code); | |
| 342 | }); | |
| 343 | arm.on("error", () => arm.terminate()); | |
| 344 | this.json(v, { t: "drive", state: "open" }); | |
| 345 | this.pace(v, drive); | |
| 346 | }, | |
| 347 | (err: unknown) => { | |
| 348 | if (v.drive !== drive) return; | |
| 349 | this.stopDrive(v, null, `the arm did not answer (${(err as Error)?.message ?? String(err)})`); | |
| 350 | }, | |
| 351 | ); | |
| 352 | } | |
| 353 | clearTimeout(d.ender); | |
| 354 | d.ender = undefined; | |
| 355 | if (d.pending) this.o.telemetry.count("commandsMerged"); | |
| 356 | d.pending = new Map([...(d.pending ?? []), ...targets]); | |
| 357 | if (typeof seq === "number" && Number.isSafeInteger(seq)) d.pendingSeq = seq; | |
| 358 | this.pace(v, d); | |
| 359 | } | |
| 360 | ||
| 361 | /** Send what is pending once the command rate allows: one frame per slot, the latest per joint. */ | |
| 362 | private pace(v: Viewer, d: Drive) { | |
| 363 | if (d.pacer || !d.pending || !d.arm) return; | |
| 364 | const wait = Math.max(0, d.lastSentAt + 1000 / this.o.commandHz - Date.now()); | |
| 365 | d.pacer = setTimeout(() => { | |
| 366 | d.pacer = undefined; | |
| 367 | if (v.drive !== d || !d.pending || !d.arm || d.arm.readyState !== WebSocket.OPEN) return; | |
| 368 | const frame = JSON.stringify(Object.fromEntries(d.pending)); | |
| 369 | d.pending = null; | |
| 370 | d.lastSentAt = Date.now(); | |
| 371 | const capped = d.cap ? d.cap(frame) : frame; | |
| 372 | if (capped !== null) d.arm.send(capped); | |
| 373 | d.sent.push([d.pendingSeq, d.lastSentAt]); | |
| 374 | if (d.sent.length > 64) d.sent.shift(); | |
| 375 | }, wait); | |
| 376 | } | |
| 377 | ||
| 378 | private release(v: Viewer) { | |
| 379 | const d = v.drive; | |
| 380 | if (!d) return; | |
| 381 | clearTimeout(d.ender); | |
| 382 | d.ender = setTimeout(() => { | |
| 383 | if (v.drive === d && !d.pending) this.stopDrive(v, 1000, null); | |
| 384 | else if (v.drive === d) this.release(v); // the last frame has yet to go | |
| 385 | }, this.o.endAfterMs); | |
| 386 | } | |
| 387 | ||
| 388 | /** Close the drive: toward the arm with `armCode` (null when the arm closed it), and tell the page. */ | |
| 389 | private stopDrive(v: Viewer, armCode: number | null, reason: string | null, code = 1000) { | |
| 390 | const d = v.drive; | |
| 391 | if (!d) return; | |
| 392 | v.drive = null; | |
| 393 | clearTimeout(d.pacer); | |
| 394 | clearTimeout(d.ender); | |
| 395 | if (armCode !== null) { | |
| 396 | if (d.arm) d.arm.close(armCode, armCode === 1000 ? undefined : reason ?? undefined); | |
| 397 | } | |
| 398 | this.json(v, { t: "drive", state: "closed", code, reason }); | |
| 399 | } | |
| 400 | } |
Software/so101/src/viewer.html +70 −36
| @@ -12,9 +12,9 @@ | ||
| 12 | 12 | <title>SO-101</title> |
| 13 | 13 | <script type="importmap"> |
| 14 | 14 | { "imports": { |
| 15 | "three": "/vendor/three/build/three.module.js", | |
| 16 | "three/examples/jsm/": "/vendor/three/examples/jsm/", | |
| 17 | "urdf-loader/manipulator": "/vendor/urdf-loader/src/urdf-manipulator-element.js" | |
| 15 | "three": "/vendor/__VENDOR__/three/build/three.module.js", | |
| 16 | "three/examples/jsm/": "/vendor/__VENDOR__/three/examples/jsm/", | |
| 17 | "urdf-loader/manipulator": "/vendor/__VENDOR__/urdf-loader/src/urdf-manipulator-element.js" | |
| 18 | 18 | } } |
| 19 | 19 | </script> |
| 20 | 20 | <link rel="modulepreload" href="/ui/__VERSION__/camera-plan.js"> |
| @@ -441,7 +441,7 @@ | ||
| 441 | 441 | <h3>Joint limits, degrees (empty: the calibrated range)</h3> |
| 442 | 442 | <div class="trim" id="jointLimits"></div> |
| 443 | 443 | </div> |
| 444 | <div class="row"><button id="metricsBtn">Metrics</button><button id="calib">3D calibration</button></div> | |
| 444 | <div class="row"><button id="metricsBtn">Metrics</button><button id="healthBtn" hidden>Health</button><button id="calib">3D calibration</button></div> | |
| 445 | 445 | </div> |
| 446 | 446 | <div class="tab" id="tab-machine" role="tabpanel" aria-labelledby="t-machine" hidden> |
| 447 | 447 | <p class="note">The arm as its servos report it, and every command on its way in and out, pushed as it happens. Read-only.</p> |
| @@ -526,7 +526,7 @@ | ||
| 526 | 526 | clearTimeout(msgTimer); if (text) msgTimer = setTimeout(() => ($("msg").textContent = ""), bad ? 9000 : 5000); |
| 527 | 527 | }; |
| 528 | 528 | const api = async (path, body, method = body === undefined ? "GET" : "POST") => { |
| 529 | const r = await fetch(path, { method, body: body === undefined ? undefined : JSON.stringify(body), cache: "no-store" }); | |
| 529 | const r = await fetch(path, { method, body: body === undefined ? undefined : JSON.stringify(body), cache: "no-store", headers: { accept: "application/json" } }); | |
| 530 | 530 | const data = await r.json().catch(() => ({})); |
| 531 | 531 | return Object.defineProperties(data, { ok: { value: r.ok }, status: { value: r.status } }); |
| 532 | 532 | }; |
| @@ -596,12 +596,39 @@ | ||
| 596 | 596 | // Another arm has other limits, poses and joints: start the page over on it, once nothing on the page is driving. |
| 597 | 597 | // So does a page that opened while reconnecting and never got the arm's limits. |
| 598 | 598 | const other = linkMode !== null && realArm(l.mode) !== realArm(linkMode); |
| 599 | if ((other || (l.mode === "live" && !limits?.ok)) && linkMode !== null && !stream.ws && !leader) return location.reload(); | |
| 599 | if ((other || (l.mode === "live" && !limits?.ok)) && linkMode !== null && !stream.ws && !sock?.driving && !leader) return location.reload(); | |
| 600 | 600 | if (linkMode === null) linkMode = l.mode; |
| 601 | 601 | } |
| 602 | 602 | setTimeout(pollLink, l?.mode === "live" ? 3000 : 1000); |
| 603 | 603 | } |
| 604 | 604 | |
| 605 | // ---------------------------------------------------------------- the link: WS /arm (src/ui/arm-socket.js) | |
| 606 | // Behind the relay one socket carries the pose, the driver's commands and the camera frames. Until it is up (its module | |
| 607 | // comes after the first view) and while it is down: /status polled, /follow for a drive, the cameras as MJPEG. | |
| 608 | let sockUp = false, sock = null; | |
| 609 | const watchCams = () => sock?.watch(cams.filter(paneOn).map((c) => c.id)); | |
| 610 | async function openSock() { | |
| 611 | if (!relayed || !("WebSocket" in window)) return; | |
| 612 | const { ArmSocket } = await import("/ui/__VERSION__/arm-socket.js"); | |
| 613 | sock = new ArmSocket({ | |
| 614 | url: `${location.protocol === "https:" ? "wss" : "ws"}://${location.host}/arm`, | |
| 615 | onState(open) { | |
| 616 | sockUp = open; | |
| 617 | for (const c of cams) if (paneOn(c)) showCam(c, true); | |
| 618 | }, | |
| 619 | onPose: (d) => applyStatus(d), // at most once a browser frame | |
| 620 | onFrame: (f) => showFrame(f), | |
| 621 | onDrive(m) { | |
| 622 | if (m.state === "refused" || (m.state === "closed" && m.reason)) { stream.halted = true; msg(m.reason || "the arm is busy", true); } | |
| 623 | }, | |
| 624 | }); | |
| 625 | } | |
| 626 | function showFrame(f) { | |
| 627 | const c = cams.find((x) => x.id === f.camera); | |
| 628 | if (!c || !paneOn(c) || !sockUp) return void sock.skip(f.n); | |
| 629 | sock.paint(c, f); | |
| 630 | } | |
| 631 | ||
| 605 | 632 | // ---------------------------------------------------------------- cameras |
| 606 | 633 | // The owner lists the cameras (/admin/camera-list); this page polls the list, so a camera added or removed shows within seconds. |
| 607 | 634 | // Which views are on is this viewer's own choice, kept in this browser: one at a time on a phone, several side by side on a wider screen. |
| @@ -651,8 +678,10 @@ | ||
| 651 | 678 | c.pane.classList.toggle("on", on); c.btn.classList.toggle("on", on); c.btn.setAttribute("aria-pressed", String(on)); |
| 652 | 679 | clearTimeout(c.timer); |
| 653 | 680 | if (!on || document.hidden) c.img.src = ""; |
| 681 | else if (sockUp) { if (!c.img.src.startsWith("blob:")) c.img.src = c.blob || ""; } // frames come over the socket | |
| 654 | 682 | else if (plan.mode === "snapshot" && c.snapshot) snap(c); |
| 655 | 683 | else c.img.src = streamUrl(c); |
| 684 | watchCams(); | |
| 656 | 685 | }; |
| 657 | 686 | function setPane(key, on) { |
| 658 | 687 | if (key === "pane3") { |
| @@ -692,7 +721,7 @@ | ||
| 692 | 721 | c.tag = el("span", { className: "tag" }); |
| 693 | 722 | c.pane = el("div", { className: "pane" }, c.img, c.tag); c.pane.dataset.key = `cam:${info.id}`; |
| 694 | 723 | c.btn = el("button", { type: "button" }); c.btn.setAttribute("aria-pressed", "false"); |
| 695 | c.img.onerror = () => { if (paneOn(c) && plan.mode === "stream" && c.img.getAttribute("src")) setTimeout(() => { if (paneOn(c) && plan.mode === "stream" && !document.hidden) c.img.src = streamUrl(c); }, 3000); }; | |
| 724 | c.img.onerror = () => { if (paneOn(c) && plan.mode === "stream" && !sockUp && c.img.getAttribute("src")) setTimeout(() => { if (paneOn(c) && plan.mode === "stream" && !sockUp && !document.hidden) c.img.src = streamUrl(c); }, 3000); }; | |
| 696 | 725 | c.btn.onclick = () => pick(c.pane.dataset.key); |
| 697 | 726 | $("stage").insertBefore(c.pane, $("pane3")); |
| 698 | 727 | $("top").querySelector(".seg").insertBefore(c.btn, $("v3")); |
| @@ -760,6 +789,7 @@ | ||
| 760 | 789 | document.addEventListener("visibilitychange", () => { for (const c of cams) if (paneOn(c)) showCam(c, true); }); |
| 761 | 790 | setInterval(() => { |
| 762 | 791 | for (const c of cams) { |
| 792 | if (paneOn(c) && sockUp) { c.live = performance.now() - (c.frameAt ?? -Infinity) < 2000; continue; } | |
| 763 | 793 | if (!paneOn(c) || (plan.mode === "snapshot" && c.snapshot)) continue; |
| 764 | 794 | let changed = false; |
| 765 | 795 | try { |
| @@ -772,7 +802,7 @@ | ||
| 772 | 802 | if (c.unchanged >= Math.max(15, plan.fps ? 30 / plan.fps : 0) && !document.hidden) { c.img.src = streamUrl(c); c.unchanged = 0; } |
| 773 | 803 | } |
| 774 | 804 | const why = quality === "saver" ? "data saver" : "slow link"; |
| 775 | const slow = plan.mode === "snapshot" ? ` · ${(1000 / plan.everyMs).toFixed(plan.everyMs > 1000 ? 1 : 0)} fps, ${why}` : plan.fps ? ` · ${plan.fps} fps, ${why}` : ""; | |
| 805 | const slow = sockUp ? sock.readout() : plan.mode === "snapshot" ? ` · ${(1000 / plan.everyMs).toFixed(plan.everyMs > 1000 ? 1 : 0)} fps, ${why}` : plan.fps ? ` · ${plan.fps} fps, ${why}` : ""; | |
| 776 | 806 | $("live").textContent = cams.filter(paneOn).map((c) => (c.live ? `live${slow}` : c.online === false ? "offline" : "waiting")).join(" · "); |
| 777 | 807 | }, 1000); |
| 778 | 808 | if (boot) { reconcile(boot.cameras); setTimeout(pollCameras, 3000); } |
| @@ -786,6 +816,7 @@ | ||
| 786 | 816 | |
| 787 | 817 | // ---------------------------------------------------------------- the stream: the page as a leader |
| 788 | 818 | // Opened while a slider, a jog button, the joystick or a model joint is held; frames go at most every 33 ms; closed shortly after letting go. |
| 819 | // With the socket up, the frames go over WS /arm instead (latest per joint wins there too); a /follow already open finishes the hold. | |
| 789 | 820 | const stream = { |
| 790 | 821 | ws: null, closeTimer: null, halted: false, |
| 791 | 822 | // one message per slot to the socket that is open, or the one about to be |
| @@ -797,6 +828,7 @@ | ||
| 797 | 828 | open() { |
| 798 | 829 | if (!mayDrive()) return; // the server would refuse it: only the person whose turn it is streams |
| 799 | 830 | clearTimeout(this.closeTimer); |
| 831 | if (sockUp && !this.ws) return; | |
| 800 | 832 | if (this.ws && this.ws.readyState <= 1) return; |
| 801 | 833 | if (this.halted) return; // the server ended this hold (Stop, or the arm is busy): nothing more until the next press |
| 802 | 834 | const ws = new WebSocket(`${location.protocol === "https:" ? "wss" : "ws"}://${location.host}/follow`); |
| @@ -810,15 +842,20 @@ | ||
| 810 | 842 | send(frame) { |
| 811 | 843 | if (this.halted) return; |
| 812 | 844 | this.open(); |
| 845 | if (sockUp && !this.ws) { if (mayDrive()) sock.command(frame); return; } | |
| 813 | 846 | if (this.ws) this.frames.send(frame); |
| 814 | 847 | }, |
| 815 | 848 | /** Now, not at the next slot, with anything pending: a finger lifted and this is where its joints stop. */ |
| 816 | 849 | flush(frame) { |
| 817 | 850 | if (this.halted) return; |
| 818 | 851 | this.open(); |
| 852 | if (sockUp && !this.ws) { if (mayDrive()) { sock.command(frame); sock.flush(); } return; } | |
| 819 | 853 | if (this.ws) this.frames.flush(frame); |
| 820 | 854 | }, |
| 821 | end() { clearTimeout(this.closeTimer); this.closeTimer = setTimeout(() => { const ws = this.ws; this.ws = null; this.halted = false; ws?.close(); }, 250); }, | |
| 855 | end() { | |
| 856 | if (sock?.driving) sock.end(); // the app closes the drive itself shortly after, and the arm holds | |
| 857 | clearTimeout(this.closeTimer); this.closeTimer = setTimeout(() => { const ws = this.ws; this.ws = null; this.halted = false; ws?.close(); }, 250); | |
| 858 | }, | |
| 822 | 859 | }; |
| 823 | 860 | |
| 824 | 861 | // ---------------------------------------------------------------- the arm |
| @@ -878,6 +915,7 @@ | ||
| 878 | 915 | const r = await api("/estop", {}); |
| 879 | 916 | msg(r.ok ? "EMERGENCY STOP: the arm holds; an admin must clear it" : r.error, true); |
| 880 | 917 | if (stream.ws) stream.ws.close(); |
| 918 | if (sock?.driving) sock.end(); | |
| 881 | 919 | stopLeaderDrive("emergency stop"); |
| 882 | 920 | } |
| 883 | 921 | async function clearEStop() { |
| @@ -1520,28 +1558,21 @@ | ||
| 1520 | 1558 | $("floorMargin").onchange = () => saveSettings({ floorMarginMm: Number($("floorMargin").value) }); |
| 1521 | 1559 | $("baseMargin").onchange = () => saveSettings({ baseMarginMm: Number($("baseMargin").value) }); |
| 1522 | 1560 | |
| 1523 | // ---------------------------------------------------------------- metrics | |
| 1561 | // ---------------------------------------------------------------- metrics: the arm's GET /metrics; src/ui/metrics.js comes when opened | |
| 1524 | 1562 | async function showMetrics() { |
| 1525 | const m = await api("/metrics"); | |
| 1526 | const c = m.counters; | |
| 1527 | $("metricsSummary").innerHTML = [ | |
| 1528 | ["since", (m.since ?? "").replace("T", " ").slice(0, 19)], | |
| 1529 | ["jams", c.jam], ["zone blocks", c.block], ["stalls on moves", c.stall], ["lost servos", c.missing], ["hot joints", c.hot], | |
| 1530 | ["sessions", `${c.stream} streams · ${c.move} moves · ${c.play} playbacks · ${c.record} recordings`], | |
| 1531 | ["teleop lag, this browser", (() => { const lag = teleop?.lag(); return lag ? `p50 ${lag.p50} ms · p95 ${lag.p95} ms over ${lag.n} samples` : "—"; })()], | |
| 1532 | ["closest to the table", m.closest ? `${(m.closest.z * 1000).toFixed(0)} mm (${m.closest.link.replace(/_link$/, "")}, ${m.closest.at.slice(11, 19)})` : "—"], | |
| 1533 | ].map(([k, v]) => `<b>${k}</b><span>${v}</span>`).join(""); | |
| 1534 | $("peaks").innerHTML = "<tr><th>joint</th><th>peak load</th><th>peak temperature</th></tr>" + Object.entries(m.peaks).map(([id, p]) => `<tr><td>${NAMES[id] ?? id}</td><td>${p.load.toFixed(0)} %</td><td>${p.temperature} °C</td></tr>`).join(""); | |
| 1535 | // Built from text nodes: an event's detail carries people's names (who pressed the E-stop) and error strings. | |
| 1536 | const recent = [...(m.recent ?? [])].reverse(); | |
| 1537 | $("events").replaceChildren(...(recent.length ? recent.map((e) => el("div", {}, | |
| 1538 | `${String(e.t).slice(5, 19).replace("T", " ")} `, el("b", { textContent: String(e.kind) }), | |
| 1539 | ` ${Object.entries(e.detail ?? {}).map(([k, v]) => `${k}=${typeof v === "object" ? JSON.stringify(v) : v}`).join(" ")}`)) : [el("div", { textContent: "none yet" })])); | |
| 1540 | $("metricsPanel").hidden = false; | |
| 1563 | try { await (await import("/ui/__VERSION__/metrics.js")).showMetrics({ api, lag: () => teleop?.lag() }); } | |
| 1564 | catch (err) { console.error(err); msg("the metrics did not load", true); } | |
| 1541 | 1565 | } |
| 1542 | 1566 | $("metricsBtn").onclick = showMetrics; |
| 1543 | 1567 | setInterval(() => { if (!$("metricsPanel").hidden) showMetrics(); }, 5000); |
| 1544 | 1568 | |
| 1569 | // ---------------------------------------------------------------- health: GET /health, behind the relay; src/ui/health.js comes when opened | |
| 1570 | $("healthBtn").hidden = !relayed; | |
| 1571 | $("healthBtn").onclick = async () => { | |
| 1572 | try { const { openHealth } = await import("/ui/__VERSION__/health.js"); await openHealth({ api, msg }); } | |
| 1573 | catch (err) { console.error(err); msg("the health panel did not load", true); } | |
| 1574 | }; | |
| 1575 | ||
| 1545 | 1576 | // ---------------------------------------------------------------- the room: the turn, the queue and the chat over /live |
| 1546 | 1577 | // Behind the hosted app's relay only the person whose turn it is may drive, and the server refuses everyone else. |
| 1547 | 1578 | // The page follows suit: the Controls tab appears on your turn. Everything here arrives on one socket. |
| @@ -1780,7 +1811,7 @@ | ||
| 1780 | 1811 | ws.onmessage = (e) => { |
| 1781 | 1812 | const m = JSON.parse(e.data); |
| 1782 | 1813 | if (m.type === "ping") return say({ type: "pong" }); |
| 1783 | if (m.type === "status") { lobby.statusSeen = true; return m.status ? applyStatus(m.status) : noStatus(); } | |
| 1814 | if (m.type === "status") { lobby.statusSeen = true; if (sockUp) return; return m.status ? applyStatus(m.status) : noStatus(); } | |
| 1784 | 1815 | if (m.type === "hello") lobby.hello = m; |
| 1785 | 1816 | else if (m.type === "state") { lobby.offset = m.now - Date.now(); lobby.state = m; } |
| 1786 | 1817 | else if (m.type === "chat") return addChat(m.messages, true), showEarlier(m.more); |
| @@ -1905,11 +1936,11 @@ | ||
| 1905 | 1936 | else if (d.violation) { b.className = "warn"; b.textContent = `the arm is ${say(d.violation)} (zone: ${d.zone})`; } |
| 1906 | 1937 | else b.className = ""; |
| 1907 | 1938 | } |
| 1908 | /** The arm's status, from /live (pushed when it changes) or, without it, from polling /status. */ | |
| 1939 | /** The arm's status, from WS /arm, /live (pushed when it changes) or polling /status. */ | |
| 1909 | 1940 | // Until the limits, the settings and the model are in, a status is kept aside, not drawn; then it is applied whole, |
| 1910 | // so an E-stop it carries still locks the controls. | |
| 1941 | // so an E-stop it carries still locks the controls. `at` is the best guess of when the arm read its pose. | |
| 1911 | 1942 | let started = false, early = null; |
| 1912 | function applyStatus(d) { | |
| 1943 | function applyStatus(d, at = performance.now()) { | |
| 1913 | 1944 | if (!started) return void (early = d); |
| 1914 | 1945 | { |
| 1915 | 1946 | const wasStopped = Boolean(status?.estop); |
| @@ -1918,14 +1949,13 @@ | ||
| 1918 | 1949 | if (Boolean(d.estop) !== wasStopped) applyLock(); |
| 1919 | 1950 | // Stopped, reconnecting or no longer this person's turn: a held thumb lets go, and its zero goes out now. |
| 1920 | 1951 | if (!canDrive({ mayDrive: mayDrive(), estop: d.estop, reconnecting: d.reconnecting, limits })) driver.releaseAll(); |
| 1921 | // The round trip to the arm, which every command also makes: shown once it is long enough to feel. | |
| 1952 | // The polled round trip to the arm, which every command also makes: shown once it is long enough to feel. | |
| 1922 | 1953 | const lag = [...rtts].sort((a, b) => a - b)[rtts.length >> 1] ?? 0; |
| 1923 | $("lag").hidden = rtts.length < 3 || lag <= 250; $("lag").textContent = `· ${Math.round(lag)} ms lag`; | |
| 1954 | $("lag").hidden = sockUp || rtts.length < 3 || lag <= 250; $("lag").textContent = `· ${Math.round(lag)} ms lag`; | |
| 1924 | 1955 | $("dot").className = d.reconnecting ? "dot warn" : "dot on"; |
| 1925 | 1956 | $("state").textContent = d.reconnecting ? "reconnecting…" : d.driving === null ? "idle" : d.driving === "record" ? `recording ${d.recording}` : leaderStreaming() ? `teleop · ${teleop.frames} frames` : d.driving; |
| 1926 | 1957 | if (leaderStreaming()) { |
| 1927 | // The follower's pose was read somewhere between asking and the answer: the middle is the best guess of when. | |
| 1928 | teleop.observe(Object.fromEntries(d.joints.filter((j) => typeof j.position === "number").map((j) => [j.id, j.position])), (started + performance.now()) / 2); | |
| 1958 | teleop.observe(Object.fromEntries(d.joints.filter((j) => typeof j.position === "number").map((j) => [j.id, j.position])), at); | |
| 1929 | 1959 | renderTracks(); renderLag(); |
| 1930 | 1960 | } |
| 1931 | 1961 | $("estop").classList.toggle("on", Boolean(d.estop)); $("estop").textContent = d.estop ? "Stopped" : "Stop"; |
| @@ -1942,13 +1972,16 @@ | ||
| 1942 | 1972 | $("dot").className = reconnecting ? "dot warn" : "dot"; $("state").textContent = reconnecting ? "reconnecting…" : "no arm"; |
| 1943 | 1973 | } |
| 1944 | 1974 | async function poll() { |
| 1945 | // The room's socket carries the status: nothing to poll while it is open and has said where the arm is. | |
| 1975 | // Nothing to poll while WS /arm or the room's socket carries the status. | |
| 1976 | if (sockUp) return void setTimeout(poll, 500); | |
| 1946 | 1977 | if (lobby.statusSeen && lobby.ws?.readyState === 1) return void setTimeout(poll, 1000); |
| 1978 | const asked = performance.now(); | |
| 1947 | 1979 | try { |
| 1948 | 1980 | const r = await timed("/status"); |
| 1949 | 1981 | const d = await r.json(); |
| 1950 | 1982 | if (!r.ok) throw new Error(); |
| 1951 | applyStatus(d); | |
| 1983 | // The pose was read somewhere between asking and the answer: the middle is the best guess of when. | |
| 1984 | applyStatus(d, (asked + performance.now()) / 2); | |
| 1952 | 1985 | } catch { noStatus(); } |
| 1953 | 1986 | // Hidden, the page keeps the arm's state fresh at a walk: nobody is looking, and a phone's battery and data matter. |
| 1954 | 1987 | setTimeout(poll, document.hidden ? 2000 : 200); |
| @@ -1967,6 +2000,7 @@ | ||
| 1967 | 2000 | limits = l; modelInfo = mi; started = true; |
| 1968 | 2001 | cameraFirst = !relayed || link?.mode === "live" || q.has("stream"); |
| 1969 | 2002 | startViews(); |
| 2003 | openSock().catch((err) => console.error(err)); | |
| 1970 | 2004 | if (early) applyStatus(early); |
| 1971 | 2005 | poll(); |
| 1972 | 2006 | if (relayed) pollLink(); |
Software/so101/src/weigh.ts added +53
| @@ -0,0 +1,53 @@ | ||
| 1 | /** | |
| 2 | * What a visit to the page costs, file by file: what the first visit downloads, and what a second one still has to | |
| 3 | * ask for given each answer's cache-control. Pure parsing here; bench/first-load.ts does the fetching and prints it. | |
| 4 | */ | |
| 5 | ||
| 6 | export { urdfFiles } from "./mesh-lite.ts"; | |
| 7 | ||
| 8 | /** The page's shell: modules preloaded or imported up front, stylesheets, the import map's targets kept apart. */ | |
| 9 | export function pageLinks(html: string): { shell: string[]; mapped: string[] } { | |
| 10 | const shell = new Set<string>(); | |
| 11 | for (const m of html.matchAll(/<link\b[^>]*\brel="(?:modulepreload|stylesheet|preload|manifest)"[^>]*\bhref="([^"]+)"/g)) shell.add(m[1]!); | |
| 12 | for (const m of html.matchAll(/<script\b[^>]*\bsrc="([^"]+)"/g)) shell.add(m[1]!); | |
| 13 | for (const block of html.matchAll(/<script type="module">([\s\S]*?)<\/script>/g)) for (const s of staticImports(block[1]!)) if (s.startsWith("/")) shell.add(s); | |
| 14 | const mapped: string[] = []; | |
| 15 | const map = /<script type="importmap">([\s\S]*?)<\/script>/.exec(html); | |
| 16 | if (map) { | |
| 17 | try { | |
| 18 | const imports = (JSON.parse(map[1]!) as { imports?: Record<string, string> }).imports ?? {}; | |
| 19 | for (const [name, target] of Object.entries(imports)) if (!name.endsWith("/")) mapped.push(target); | |
| 20 | } catch { | |
| 21 | /* no map */ | |
| 22 | } | |
| 23 | } | |
| 24 | return { shell: [...shell], mapped }; | |
| 25 | } | |
| 26 | ||
| 27 | /** `import … from "x"` and `export … from "x"` at the top level, and `import "x"`. */ | |
| 28 | export function staticImports(js: string): string[] { | |
| 29 | const out = new Set<string>(); | |
| 30 | for (const m of js.matchAll(/(?:^|[;\n}])\s*(?:import|export)\s+(?:[\w*{}\s,$]+?\s+from\s+)?["']([^"']+)["']/g)) out.add(m[1]!); | |
| 31 | return [...out]; | |
| 32 | } | |
| 33 | ||
| 34 | /** Every module a file pulls in, statically or with import("…"), as written. */ | |
| 35 | export function allImports(js: string): string[] { | |
| 36 | const out = new Set(staticImports(js)); | |
| 37 | for (const m of js.matchAll(/\bimport\(\s*["']([^"']+)["']\s*\)/g)) out.add(m[1]!); | |
| 38 | return [...out]; | |
| 39 | } | |
| 40 | ||
| 41 | export type Repeat = "free" | "revalidate" | "refetch"; | |
| 42 | ||
| 43 | /** | |
| 44 | * What a second visit within a day pays for one answer: nothing (fresh in the cache), a round trip answered 304 | |
| 45 | * (no-cache with a validator), or the whole thing again (no-store, or nothing to validate with). | |
| 46 | */ | |
| 47 | export function repeatCost(cacheControl: string | null, etag: string | null): Repeat { | |
| 48 | const cc = (cacheControl ?? "").toLowerCase(); | |
| 49 | if (cc.includes("no-store")) return "refetch"; | |
| 50 | const maxAge = Number(/max-age=(\d+)/.exec(cc)?.[1] ?? 0); | |
| 51 | if (!cc.includes("no-cache") && maxAge >= 86_400) return "free"; | |
| 52 | return etag ? "revalidate" : "refetch"; | |
| 53 | } |
new above
offline