Public: anyone can read this; the crew can change it.
Migrated from github.com/Trevato/SO-ARM100
workbenchthis machine
Cameras stream with the arm unplugged: serve starts with no serial port, and resting stops driving, not cameras #624
Check · Review · Land done · Live — Landed; nothing deploys from this branch, so this is the end of its path.
quill/arm:quill/623-cameras-stream-with-the-arm-unplugged-se → main 1 commit fork 2 threads · 2 unresolved
9 files
changed +190
−33
Software/so101/README.md +2 −2
| @@ -32,7 +32,7 @@ Signed out, the Queue tab offers **Sign in with the forge** first and picking a | ||
| 32 | 32 | |
| 33 | 33 | The owner can close the whole app when the shop is dark: on `/admin`, **Open / Closed** has **Close the arm** with an optional note for visitors ("Back Saturday 10am CT", at most 140 characters) and an optional "back around" time (the owner's local time); **Open the arm** undoes it. In the terminal: `/close [note]` and `/open`. Over `/live`, the owner sends `{"type": "open", "open": false, "note": "…", "reopenAt": "2026-10-31T10:00-05:00"}`. Every change is in the admin log; the switch is kept in `so101/open.json` under `FORGE_APP_DATA` (src/opening.ts). |
| 34 | 34 | |
| 35 | Closed means: every turn ends (the owner's too) and the line is emptied, nobody can join, the cameras are refused and every open camera and `/follow` stream is closed, and nothing reaches the arm but a STOP (anything else is 503). `/` is then a calm card, **The arm is resting**, with the note, when it should be back and when it was last open, a "Get notified when it's back" button (the browser's own notification, while that tab stays open), and links to the story (`/story`) and the code (src/resting-page.ts). A page that was open when it closed reloads into the card, and the card reloads into the arm when it opens. The reopen time is only shown: the arm never reopens by itself. | |
| 35 | Closed means: every turn ends (the owner's too) and the line is emptied, nobody can join, every open `/follow` stream is closed, and nothing reaches the arm but a STOP (anything else is 503). `/` is then a calm card, **The arm is resting**, with the note, when it should be back and when it was last open, a "Get notified when it's back" button (the browser's own notification, while that tab stays open), and links to the story (`/story`) and the code (src/resting-page.ts). Resting stops driving, never the cameras: each streams (`/camera/<id>/snapshot`, `stream`, `ws`) whenever its own source answers, and the home still shows the arm's camera on its resting card. A page that was open when it closed reloads into the card, and the card reloads into the arm when it opens. The reopen time is only shown: the arm never reopens by itself. | |
| 36 | 36 | |
| 37 | 37 | Closing never locks the owner out. Sign-in (the forge's `/__forge/signin`), `/admin` and `/story` keep working while closed; only turns, cameras, `/follow` and commands are refused. The card has a quiet **Owner? Sign in to open** link (sign-in, then `/admin`), and a signed-in owner sees **Open now** right on the card, which opens it and goes back to the arm (a same-origin form post to `/admin`; anyone else gets no button and is refused). |
| 38 | 38 | |
| @@ -127,7 +127,7 @@ ignored and said once on the arm server's stderr. The arm server reads it at sta | ||
| 127 | 127 | |
| 128 | 128 | The app is a workshop: one Pi, several **stations**, each with its cameras and controls. Today there are two, **the arm** and **the grow box** (spinach under a UV light). `/` is the workshop home, a card per station with its live camera snapshot first and its controls after, and the arm's own page is at `/arm` (its URL is unchanged for anyone who has it). The page and its title say "Workshop"; it shows who is looking ("Signed in as …", or "Watching as a guest · Sign in"), has a skip link, and keeps working while the arm is closed. |
| 129 | 129 | |
| 130 | - **Any subset present.** The grow box has nothing to enumerate (the light is USB power only), so it is always there. The arm's card is there only while the link to the Pi's `so101 serve` is live; unplugged, closed by the owner or unreachable it says why ("The arm: not connected", "The arm is resting") and offers no controls, and the owner sees **Open now** on it. Plugging the arm back in brings its card back by itself, with no restart. `so101 serve` runs with the arm unplugged: when the port is missing or cannot be opened it logs `arm not connected (<path>: <reason>)` and keeps serving, so the cameras, `/version` and the grow box stay up and the Pi can still update itself. `GET /status` answers 200 with `arm: "not connected"`; every route that needs the servos (moves, poses, torque, calibration, the drive stream) answers 503 `{"error":"arm not connected"}`, and STOP answers 200. Plugging the arm in needs a restart of the service for now. | |
| 130 | - **Any subset present.** The grow box has nothing to enumerate (the light is USB power only), so it is always there. The arm's card is there only while the link to the Pi's `so101 serve` is live; unplugged, closed by the owner or unreachable it says why ("The arm: not connected", "The arm is resting") and offers no controls (its cameras still show), and the owner sees **Open now** on it. Plugging the arm back in brings its card back by itself, with no restart. `so101 serve` runs with the arm unplugged: when the port is missing or cannot be opened it logs `arm not connected (<path>: <reason>); trying again every 5 s` and keeps serving, so the cameras, `/version` and the grow box stay up and the Pi can still update itself. `GET /status` answers 200 with `arm: "not connected"`; every route that needs the servos (moves, poses, torque, calibration, the drive stream) answers 503 `{"error":"arm not connected"}`, and STOP answers 200. The port is tried again every 5 s (src/arm-plug.ts): when the arm answers, the server starts over with it and logs `arm connected: serving <path>`, with no restart of the service. | |
| 131 | 131 | - **"Camera can't see the plants."** On the grow box's card, each still is checked in the browser once it has loaded: the part inside the camera's box (set on Tune, `/admin/camera-list`) is counted for blown-out and black pixels (`visibility` in src/ui/still-frame.js), and when the plants cannot be seen the card says so over the foot of the still, with a link to Tune (src/ui/grow-hint.js). The relay never decodes a JPEG, so the terminal and the JSON doors cannot show this hint; they have no pixels to count. |
| 132 | 132 | - **The config** is `so101/stations.json` under `FORGE_APP_DATA`: a list of `{id, name, kind: "arm"|"grow", cameras: [camera ids], controls: [{label, href}]}` (src/stations.ts). Without a file it is the arm (camera 1) and the grow box (camera 2, from the camera list): edit it to match the cameras. With only an arm in it, `/` is the arm's page as before. |
| 133 | 133 | - **Live.** `GET /care/events` is a server-sent stream that says `care` after every change to the book, from any door; the page listens with a few lines of script and refreshes itself (not while a field has focus). Without the script the page works the same, on reload. A refused form returns to `/care?error=…` with the message in an alert line. |
Software/so101/src/arm-plug.test.ts added +58
| @@ -0,0 +1,58 @@ | ||
| 1 | import { afterAll, expect, test } from "bun:test"; | |
| 2 | import { mkdtempSync, rmSync } from "node:fs"; | |
| 3 | import { tmpdir } from "node:os"; | |
| 4 | import { createServer } from "node:net"; | |
| 5 | import { join } from "node:path"; | |
| 6 | import { retryArm } from "./arm-plug.ts"; | |
| 7 | ||
| 8 | const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms)); | |
| 9 | ||
| 10 | test("retryArm tries again until the arm answers, hands it over once, and stops trying", async () => { | |
| 11 | let tries = 0; | |
| 12 | const got: string[] = []; | |
| 13 | retryArm(async () => (++tries < 3 ? null : "bus"), (bus) => void got.push(bus), 10); | |
| 14 | await sleep(100); | |
| 15 | expect(got).toEqual(["bus"]); | |
| 16 | expect(tries).toBe(3); | |
| 17 | }); | |
| 18 | ||
| 19 | test("retryArm counts a thrown open as not there, and its stop stops it", async () => { | |
| 20 | let tries = 0; | |
| 21 | const stop = retryArm(async () => (++tries, Promise.reject(new Error("ENOENT"))), () => {}, 10); | |
| 22 | await sleep(45); | |
| 23 | stop(); | |
| 24 | const seen = tries; | |
| 25 | expect(seen).toBeGreaterThan(1); | |
| 26 | await sleep(40); | |
| 27 | expect(tries).toBe(seen); | |
| 28 | }); | |
| 29 | ||
| 30 | const dir = mkdtempSync(join(tmpdir(), "so101-plug-")); | |
| 31 | afterAll(() => rmSync(dir, { recursive: true, force: true })); | |
| 32 | ||
| 33 | test("`so101 serve` with SO101_PORT at a missing path serves /status 200, arm: not connected, and says it will try again", async () => { | |
| 34 | const free = createServer(); | |
| 35 | await new Promise<void>((r) => free.listen(0, "127.0.0.1", r)); | |
| 36 | const listen = (free.address() as { port: number }).port; | |
| 37 | await new Promise((r) => free.close(r)); | |
| 38 | const script = `import { main } from ${JSON.stringify(join(import.meta.dir, "cli.ts"))}; await main(["serve", "--listen", "127.0.0.1:${listen}", "--file", ${JSON.stringify(join(dir, "poses.json"))}]);`; | |
| 39 | const child = Bun.spawn(["bun", "-e", script], { env: { ...process.env, SO101_PORT: "/dev/does-not-exist-ttyACM9" }, stdout: "pipe", stderr: "pipe" }); | |
| 40 | try { | |
| 41 | const reader = child.stdout.getReader(); | |
| 42 | let out = ""; | |
| 43 | while (!/on http:\/\/127\.0\.0\.1:(\d+)/.test(out)) { | |
| 44 | const { value, done } = await reader.read(); | |
| 45 | if (done) throw new Error(`serve exited: ${out}${await new Response(child.stderr).text()}`); | |
| 46 | out += new TextDecoder().decode(value); | |
| 47 | } | |
| 48 | expect(out).toContain("arm not connected (/dev/does-not-exist-ttyACM9"); | |
| 49 | expect(out).toContain("trying again every 5 s"); | |
| 50 | const port = /on http:\/\/127\.0\.0\.1:(\d+)/.exec(out)![1]; | |
| 51 | const res = await fetch(`http://127.0.0.1:${port}/status`); | |
| 52 | expect(res.status).toBe(200); | |
| 53 | expect(((await res.json()) as { arm: string }).arm).toBe("not connected"); | |
| 54 | } finally { | |
| 55 | child.kill(); | |
| 56 | await child.exited; | |
| 57 | } | |
| 58 | }, 20_000); |
Software/so101/src/arm-plug.ts added +25
| @@ -0,0 +1,25 @@ | ||
| 1 | /** `so101 serve` with the arm unplugged serves on without it and tries the port again every few seconds (hot-plug). */ | |
| 2 | ||
| 3 | /** How often an unplugged arm's port is tried again, ms. */ | |
| 4 | export const REPLUG_MS = 5000; | |
| 5 | ||
| 6 | /** | |
| 7 | * Tries `open` every `everyMs` until it gives a bus, then hands it to `plugged` once and stops. | |
| 8 | * `open` answers null while the arm is not there; tries never overlap. Returns the stop. | |
| 9 | */ | |
| 10 | export function retryArm<B>(open: () => Promise<B | null>, plugged: (bus: B) => void | Promise<void>, everyMs = REPLUG_MS): () => void { | |
| 11 | let stopped = false; | |
| 12 | let timer: ReturnType<typeof setTimeout> | undefined; | |
| 13 | const attempt = async () => { | |
| 14 | const bus = await open().catch(() => null); | |
| 15 | if (stopped) return; | |
| 16 | if (bus === null) return void (timer = setTimeout(attempt, everyMs)); | |
| 17 | stopped = true; | |
| 18 | await plugged(bus); | |
| 19 | }; | |
| 20 | timer = setTimeout(attempt, everyMs); | |
| 21 | return () => { | |
| 22 | stopped = true; | |
| 23 | clearTimeout(timer); | |
| 24 | }; | |
| 25 | } |
Software/so101/src/cli.ts +20 −5
| @@ -9,6 +9,7 @@ import { FeetechBus, range } from "./bus.ts"; | ||
| 9 | 9 | import { describeError, hex } from "./protocol.ts"; |
| 10 | 10 | import { MAX_FRAME_STEP, MotionError, moveToPose, name as jointName, playTrajectory, recordTrajectory, sleep } from "./motion.ts"; |
| 11 | 11 | import { streamLeader } from "./leader.ts"; |
| 12 | import { REPLUG_MS, retryArm } from "./arm-plug.ts"; | |
| 12 | 13 | import { assertPoseName, parsePoseFile, serializePoseFile, type PoseFile } from "./poses.ts"; |
| 13 | 14 | import { durationSeconds, largestStep, parseTrajectory, serializeTrajectory } from "./trajectory.ts"; |
| 14 | 15 | import { |
| @@ -530,15 +531,29 @@ const COMMANDS: Record<string, Command> = { | ||
| 530 | 531 | const [host, portText] = (values.listen ?? "0.0.0.0:8101").split(":"); |
| 531 | 532 | const port = bounded(integer(portText ?? "8101", "--listen port"), 1, 65535, "--listen port"); |
| 532 | 533 | const { growKeyFile, startServer } = await import("./server.ts"); // lazily: it reaches bun:sqlite, which Node lacks |
| 533 | const bus = await openArm(values); | |
| 534 | let bus = await openArm(values, true); | |
| 534 | 535 | const cameraControl = (unit: number, profile: Parameters<typeof applyProfile>[1]) => applyProfile(cameraDevice(unit), profile); |
| 535 | 536 | const cameraStill = (unit: number) => grabStill(unit, cameraDevice(unit)); |
| 537 | const options = { host, port, posesFile: values.file ?? "poses.json", modelDir: values.model, gainSyncMs: 2000, cameraControl, cameraStill, lightMs: 15_000, sensorsMs: 60_000 }; | |
| 538 | let server = await startServer(bus, options); | |
| 539 | // Unplugged, the port is tried again every 5 s; when the arm answers, the server starts over with it (the cameras blink once). | |
| 540 | const stopRetry = bus | |
| 541 | ? () => {} | |
| 542 | : retryArm( | |
| 543 | () => openArm(values, false), | |
| 544 | async (found) => { | |
| 545 | await server.close(); | |
| 546 | bus = found; | |
| 547 | server = await startServer(found, options); | |
| 548 | console.log(`arm connected: serving ${found.path}`); | |
| 549 | }, | |
| 550 | ); | |
| 536 | 551 | try { |
If startServer throws here after server.close(), nothing catches it: no server, no retry. Wrap in try/catch, log, and try again or exit so the service manager restarts it. |
||
| 537 | const server = await startServer(bus, { host, port, posesFile: values.file ?? "poses.json", modelDir: values.model, gainSyncMs: 2000, cameraControl, cameraStill, lightMs: 15_000, sensorsMs: 60_000 }); | |
| 538 | 552 | console.log(`serving ${bus?.path ?? "no arm"} on http://${host}:${server.port} — open it in a browser; the API is listed in src/server.ts`); |
| 539 | 553 | console.log(`the grow routes need the key in ${growKeyFile(dirname(resolve(values.file ?? "poses.json")))}: give the hosted app the same in ARM_GROW_KEY`); |
| 540 | 554 | console.log("ctrl-c to stop"); |
| 541 | 555 | await new Promise<void>((resolve) => process.once("SIGINT", () => resolve())); |
| 556 | stopRetry(); | |
| 542 | 557 | await server.close(); |
| 543 | 558 | if (bus) { |
| 544 | 559 | await bus.syncWrite(STS3215.Torque_Enable, [1, 2, 3, 4, 5, 6].map((id) => [id, 0] as const)); |
| @@ -634,14 +649,14 @@ async function withBus<T>(values: Values, fn: (bus: FeetechBus) => Promise<T>): | ||
| 634 | 649 | } |
| 635 | 650 | } |
| 636 | 651 | |
| 637 | /** The arm's bus for `serve`, or null (one logged line) when the port is missing or cannot be opened: the cameras and the grow box serve on. */ | |
| 638 | async function openArm(values: Values): Promise<FeetechBus | null> { | |
| 652 | /** The arm's bus for `serve`, or null (one logged line when `say`) when the port is missing or cannot be opened: the cameras and the grow box serve on. */ | |
| 653 | async function openArm(values: Values, say: boolean): Promise<FeetechBus | null> { | |
| 639 | 654 | let path = values.port ?? process.env.SO101_PORT; |
| 640 | 655 | try { |
| 641 | 656 | path ??= await detectPort(); |
| 642 | 657 | return await FeetechBus.open(path, { baudRate: values.baud ? integer(values.baud, "--baud") : 1_000_000 }); |
| 643 | 658 | } catch (err) { |
| 644 | console.log(`arm not connected (${path ?? "no port"}: ${message(err)})`); | |
| 659 | if (say) console.log(`arm not connected (${path ?? "no port"}: ${message(err)}); trying again every ${REPLUG_MS / 1000} s`); | |
| 645 | 660 | return null; |
| 646 | 661 | } |
| 647 | 662 | } |
Software/so101/src/opening.ts +1 −1
| @@ -1,7 +1,7 @@ | ||
| 1 | 1 | /** |
| 2 | 2 | * Open or closed: the owner's switch for the whole room, and the rule for when the arm is resting. |
| 3 | 3 | * |
| 4 | * Closed (the owner's choice) means no turns and no line, the cameras stop and nothing but a STOP goes to the arm; the | |
| 4 | * Closed (the owner's choice) means no turns and no line and nothing but a STOP goes to the arm (the cameras watch on); the | |
| 5 | 5 | * page is the resting card (src/resting-page.ts). The owner may leave a note ("Back Saturday 10am CT") and a time it |
| 6 | 6 | * should be back; the time is shown, it never reopens by itself. |
| 7 | 7 | * |
Software/so101/src/relay.ts +13 −12
| @@ -61,8 +61,8 @@ | ||
| 61 | 61 | * (the arm server's own /camera-profile/ is never passed through: only these two reach it, for the owner) |
| 62 | 62 | * Resting (the owner closed it, or a real arm unheard for two minutes; src/opening.ts): / is the resting card |
| 63 | 63 | * (src/resting-page.ts), /status answers {"open": false, "reason", "note", "reopenAt", "lastOpenAt"} without asking the |
| 64 | * arm, every turn has ended, the cameras and /follow are refused (and those open are closed), and nothing goes to the arm | |
| 65 | * but a STOP: anything else is 503. Sign-in (/__forge/…), /admin and /story keep working, so the owner can always open | |
| 64 | * arm, every turn has ended, /follow is refused (and those open are closed), and nothing goes to the arm but a STOP: | |
| 65 | * anything else is 503. The cameras never rest: each streams whenever its own source answers (#623). Sign-in (/__forge/…), /admin and /story keep working, so the owner can always open | |
| 66 | 66 | * it again (#578): the card links an anonymous visitor to sign-in and shows the owner an Open now button. Open, /status |
| 67 | 67 | * carries "open": true. |
| 68 | 68 | * |
| @@ -835,7 +835,8 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> { | ||
| 835 | 835 | station, |
| 836 | 836 | present, |
| 837 | 837 | ...(station.kind === "arm" && rest ? { resting: restingWords(rest) } : {}), |
| 838 | cameras: present ? station.cameras.filter((id) => listed.has(id)).map((id) => ({ id, name: listed.get(id)!, region: cameras.look(id)?.region ?? null })) : [], | |
| 838 | // A camera does not depend on the arm: its station lists it while the arm rests or is gone, and it streams if its source answers. | |
| 839 | cameras: station.cameras.filter((id) => listed.has(id)).map((id) => ({ id, name: listed.get(id)!, region: cameras.look(id)?.region ?? null })), | |
| 839 | 840 | }; |
| 840 | 841 | }); |
| 841 | 842 | }; |
| @@ -1168,7 +1169,7 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> { | ||
| 1168 | 1169 | if (CAMERA_PROFILE.test(url.pathname)) return json(res, 404, { error: "no such route" }); |
| 1169 | 1170 | if (lobby && url.pathname === "/admin/lift") return await handleLift(lobby, req, res, readBody); |
| 1170 | 1171 | if (lobby && (url.pathname === "/admin" || url.pathname === "/admin/health")) return await handleAdmin(lobby, req, res, url, readBody, askPrecision, askServos, videoCards, options.piSync ? () => options.piSync!.versions() : undefined, tuneP, lightHooks); |
| 1171 | if (serveStory(req, res, url.pathname, { cameras: () => (restingNow() ? [] : cameras.list()), numbers: storyNumbers, signedIn: lobby && !lobby.local ? (r) => lobby.signedIn(r) : undefined })) return; | |
| 1172 | if (serveStory(req, res, url.pathname, { cameras: () => cameras.list(), numbers: storyNumbers, signedIn: lobby && !lobby.local ? (r) => lobby.signedIn(r) : undefined })) return; | |
| 1172 | 1173 | if (url.pathname === "/admin/versions") return versionsRead(req, res); |
| 1173 | 1174 | if (lobby && url.pathname === "/admin/data") return serveDataPage(lobby, req, res); |
| 1174 | 1175 | if (url.pathname === "/data") return serveData(facts ?? undefined, lobby ?? undefined, req, res, url); |
| @@ -1234,7 +1235,6 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> { | ||
| 1234 | 1235 | const camera = CAMERA.exec(url.pathname); |
| 1235 | 1236 | if (camera) { |
| 1236 | 1237 | if (!reading) return json(res, 405, { error: "cameras are read-only" }); |
| 1237 | if (rest) return json(res, 503, { error: `${restingWords(rest)}: the cameras are off`, open: false }); | |
| 1238 | 1238 | const id = Number(camera[1]); |
| 1239 | 1239 | const source = cameras.source(id); |
| 1240 | 1240 | if (source === null) return json(res, 404, { error: "no such camera" }); |
| @@ -1456,6 +1456,9 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> { | ||
| 1456 | 1456 | const refuse = (socket: Duplex, status: string) => { |
| 1457 | 1457 | socket.end(`HTTP/1.1 ${status}\r\nconnection: close\r\ncontent-length: 0\r\n\r\n`); |
| 1458 | 1458 | }; |
| 1459 | /** The camera sockets (JPEG and H.264) among wss's clients: resting never closes them. */ | |
| 1460 | const viewers = new WeakSet<WebSocket>(); | |
| 1461 | const watching = (browser: WebSocket) => viewers.add(browser); | |
| 1459 | 1462 | /** Per camera, while anyone watches over WebSocket: the fan-out and the one tap on the shared feed. */ |
| 1460 | 1463 | const watched = new Map<number, { fan: CameraSocket; viewers: number; detach: () => void }>(); |
| 1461 | 1464 | const watchCamera = (id: number, browser: WebSocket) => { |
| @@ -1481,15 +1484,14 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> { | ||
| 1481 | 1484 | const url = new URL(req.url ?? "/", "http://x"); |
| 1482 | 1485 | const video = VIDEO.exec(url.pathname); |
| 1483 | 1486 | const cameraWs = CAMERA_WS.exec(url.pathname); |
| 1484 | // Resting, the cameras are off; a /follow is closed with the lobby's reason below (4503), which a page can read. | |
| 1485 | if ((cameraWs || video) && restingNow()) return refuse(socket, "503 Service Unavailable"); | |
| 1487 | // Resting stops driving, never the cameras: a /follow is closed with the lobby's reason below (4503), which a page can read. | |
| 1486 | 1488 | if (cameraWs) { |
| 1487 | 1489 | if (!sameOrigin(req)) return refuse(socket, "403 Forbidden"); |
| 1488 | 1490 | const id = Number(cameraWs[1]); |
| 1489 | 1491 | if (!cameras.has(id)) return refuse(socket, "404 Not Found"); |
| 1490 | 1492 | socket.on("error", () => socket.destroy()); |
| 1491 | 1493 | track("camera", socket); |
| 1492 | return wss.handleUpgrade(req, socket, head, (browser) => watchCamera(id, browser)); | |
| 1494 | return wss.handleUpgrade(req, socket, head, (browser) => (watching(browser), watchCamera(id, browser))); | |
| 1493 | 1495 | } |
| 1494 | 1496 | if (url.pathname !== "/follow" && url.pathname !== "/machine" && !video && !(url.pathname === "/live" && lobby)) return refuse(socket, "404 Not Found"); |
| 1495 | 1497 | if (!sameOrigin(req)) return refuse(socket, "403 Forbidden"); |
| @@ -1499,7 +1501,7 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> { | ||
| 1499 | 1501 | if (!feed?.available) return refuse(socket, "404 Not Found"); |
| 1500 | 1502 | const who = lobby?.who(req); // watching is open, like /stream; the driver's picture gets the best rendition |
| 1501 | 1503 | const driver = !!who && !!lobby?.turns.isDriver(who.id); |
| 1502 | return wss.handleUpgrade(req, socket, head, (browser) => feed.serve(browser, driver)); | |
| 1504 | return wss.handleUpgrade(req, socket, head, (browser) => (watching(browser), feed.serve(browser, driver))); | |
| 1503 | 1505 | } |
| 1504 | 1506 | if (!video) track(url.pathname === "/live" ? "live" : url.pathname === "/machine" ? "machine" : "follow", socket); |
| 1505 | 1507 | if (url.pathname === "/live") return lobby!.upgrade(req, socket, head); |
| @@ -1540,12 +1542,11 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> { | ||
| 1540 | 1542 | // ------------------------------------------------------------------ a turn that ends |
| 1541 | 1543 | // Its streams close (the arm server holds the arm when a stream ends) and a move, playback or recording stops. |
| 1542 | 1544 | const settingsChanged = lobby?.onSettingsChange(follow); |
| 1543 | // Resting: every stream through here (/follow, the cameras over WebSocket and H.264) and every MJPEG viewer is ended. | |
| 1545 | // Resting: every drive stream through here (/follow) is ended; the cameras (MJPEG, WebSocket, H.264) watch on. | |
| 1544 | 1546 | const restingChanged = lobby?.onResting((rest) => { |
| 1545 | 1547 | pushed = ""; |
| 1546 | 1548 | if (!rest) return; |
| 1547 | for (const client of wss.clients) client.close(1000, restingWords(rest)); | |
| 1548 | cameras.endViewers(); | |
| 1549 | for (const client of wss.clients) if (!viewers.has(client)) client.close(1000, restingWords(rest)); | |
| 1549 | 1550 | }); |
| 1550 | 1551 | /** What the lobby is told of the arm, for the offline rule: since when a real arm is gone, and when it last spoke. */ |
| 1551 | 1552 | const armSeen = () => lobby?.armSeen({ downSince: remote && armLink && armLink.state !== "live" ? armLink.downSince : null, lastSeenAt: armLink?.lastAt ?? 0 }); |
Software/so101/src/resting.test.ts +64 −9
| @@ -1,11 +1,13 @@ | ||
| 1 | 1 | import { afterEach, describe, expect, test } from "bun:test"; |
| 2 | 2 | import { mkdtempSync, rmSync, writeFileSync } from "node:fs"; |
| 3 | 3 | import { tmpdir } from "node:os"; |
| 4 | import { createServer, type Server } from "node:http"; | |
| 4 | 5 | import { join } from "node:path"; |
| 5 | 6 | import WebSocket from "ws"; |
| 6 | 7 | import { FeetechBus } from "./bus.ts"; |
| 7 | 8 | import { FakeBus } from "./fake-bus.ts"; |
| 8 | 9 | import { sign } from "./identity.ts"; |
| 10 | import { CameraHub } from "./cameras.ts"; | |
| 9 | 11 | import { Lobby } from "./lobby.ts"; |
| 10 | 12 | import { parseRemote, startRelay, type RunningRelay } from "./relay.ts"; |
| 11 | 13 | import { OWNER_SIGNIN, REPO_URL, RESTING_CSP, RESTING_STYLE, STORY_PATH, renderResting } from "./resting-page.ts"; |
| @@ -26,26 +28,32 @@ let dir = ""; | ||
| 26 | 28 | let relay: RunningRelay | undefined; |
| 27 | 29 | let lobby: Lobby | undefined; |
| 28 | 30 | let sim: RunningServer | undefined; |
| 31 | let camera: Server | undefined; | |
| 32 | let hub: CameraHub | undefined; | |
| 29 | 33 | afterEach(async () => { |
| 30 | 34 | await shut(relay); |
| 31 | 35 | await shut(sim); |
| 32 | 36 | lobby?.close(); |
| 37 | hub?.close(); | |
| 38 | camera?.closeAllConnections(); | |
| 39 | camera?.close(); | |
| 40 | camera = hub = undefined; | |
| 33 | 41 | if (dir) rmSync(dir, { recursive: true, force: true }); |
| 34 | 42 | dir = ""; |
| 35 | 43 | relay = lobby = sim = undefined; |
| 36 | 44 | }); |
| 37 | 45 | |
| 38 | async function boot(options: { offlineAfterMs?: number; deadArm?: boolean } = {}) { | |
| 46 | async function boot(options: { offlineAfterMs?: number; deadArm?: boolean; cameras?: CameraHub } = {}) { | |
| 39 | 47 | dir = mkdtempSync(join(tmpdir(), "so101-resting-")); |
| 40 | 48 | sim = await startServer(new FeetechBus(new FakeBus([1, 2, 3, 4, 5, 6])), { host: "127.0.0.1", port: 0, posesFile: join(dir, "poses.json") }); |
| 41 | 49 | lobby = new Lobby({ identityKey: KEY, adminFile: join(dir, "admin.json"), openFile: join(dir, "open.json"), offlineAfterMs: options.offlineAfterMs }); |
| 42 | 50 | const remote = options.deadArm ? parseRemote({ ARM_SERVER: "ws://127.0.0.1:1" }) : null; |
| 43 | relay = await startRelay({ port: 0, simulation: `http://127.0.0.1:${sim.port}`, remote, lobby, link: { backoffMs: [20], connectTimeoutMs: 50, offlineAfterMs: 60_000 } }); | |
| 51 | relay = await startRelay({ port: 0, simulation: `http://127.0.0.1:${sim.port}`, remote, lobby, cameras: options.cameras, link: { backoffMs: [20], connectTimeoutMs: 50, offlineAfterMs: 60_000 } }); | |
| 44 | 52 | return `http://127.0.0.1:${relay.port}`; |
| 45 | 53 | } |
| 46 | 54 | |
| 47 | 55 | describe("closed by the owner", () => { |
| 48 | test("the owner closes it on /admin: /status says so for everyone, / is the resting card, a command and the cameras are refused", async () => { | |
| 56 | test("the owner closes it on /admin: /status says so for everyone, / is the resting card, a command is refused", async () => { | |
| 49 | 57 | const base = await boot(); |
| 50 | 58 | const open = await (await fetch(`${base}/status`)).json() as any; |
| 51 | 59 | expect(open.open).toBe(true); |
| @@ -80,12 +88,7 @@ describe("closed by the owner", () => { | ||
| 80 | 88 | expect((await refused.json() as any).error).toMatch(/The arm is resting/); |
| 81 | 89 | } |
| 82 | 90 | expect((await fetch(`${base}/stop`, { method: "POST", headers: owner })).status).toBe(200); // a STOP is never refused |
| 83 | expect((await fetch(`${base}/camera/1/stream`)).status).toBe(503); | |
| 84 | expect((await fetch(`${base}/camera/1/snapshot`)).status).toBe(503); | |
| 85 | ||
| 86 | const story = await fetch(`${base}/story`); | |
| 87 | expect(story.status).toBe(200); // the card links to it: it stays up, with no camera to break | |
| 88 | expect(await story.text()).toContain("The camera is not connected right now"); | |
| 91 | expect((await fetch(`${base}/story`)).status).toBe(200); // the card links to it: it stays up | |
| 89 | 92 | |
| 90 | 93 | const follow = new WebSocket(`ws://127.0.0.1:${relay!.port}/follow`, { headers: owner }); |
| 91 | 94 | const refused = await new Promise<string>((resolve) => follow.on("close", (code, why) => resolve(`${code} ${why}`))); |
| @@ -309,3 +312,55 @@ describe("the resting card", () => { | ||
| 309 | 312 | expect(prop(rules, "h1", "font-size")).toBe("2.25rem"); |
| 310 | 313 | }); |
| 311 | 314 | }); |
| 315 | ||
| 316 | // A fake camera source: /snapshot is one JPEG, anything else an MJPEG stream of it every 20 ms. | |
| 317 | const JPEG = Buffer.from([0xff, 0xd8, 0xff, 0xe0, 0, 0x10, 0x4a, 0x46, 0x49, 0x46, 0, 1, 0xff, 0xd9]); | |
| 318 | async function fakeCamera(): Promise<number> { | |
| 319 | camera = createServer((req, res) => { | |
| 320 | if (req.url?.startsWith("/snapshot")) return res.writeHead(200, { "content-type": "image/jpeg" }).end(JPEG); | |
| 321 | res.writeHead(200, { "content-type": "multipart/x-mixed-replace;boundary=frame" }); | |
| 322 | const timer = setInterval(() => res.write(Buffer.concat([Buffer.from(`--frame\r\ncontent-type: image/jpeg\r\ncontent-length: ${JPEG.length}\r\n\r\n`), JPEG, Buffer.from("\r\n")])), 20); | |
| 323 | res.on("close", () => clearInterval(timer)); | |
| 324 | }); | |
| 325 | await new Promise<void>((r) => camera!.listen(0, "127.0.0.1", r)); | |
| 326 | return (camera.address() as { port: number }).port; | |
| 327 | } | |
| 328 | ||
| 329 | describe("the cameras do not rest (#623)", () => { | |
| 330 | test("with the arm offline and closed, a camera's snapshot is 200 from its own source and its WebSocket opens and streams", async () => { | |
| 331 | const port = await fakeCamera(); | |
| 332 | hub = new CameraHub({ seed: [{ name: "Camera 1", source: "http://127.0.0.1:1/stream" }, { name: "Above", source: `http://127.0.0.1:${port}/stream` }] }); | |
| 333 | const base = await boot({ deadArm: true, offlineAfterMs: 300, cameras: hub }); | |
| 334 | // A viewer who was watching before it rested keeps watching. | |
| 335 | const before = new WebSocket(`ws://127.0.0.1:${relay!.port}/camera/2/ws`); | |
| 336 | await new Promise((r, j) => (before.once("message", r), before.once("error", j))); | |
| 337 | let closedBefore = false; | |
| 338 | before.on("close", () => (closedBefore = true)); | |
| 339 | ||
| 340 | let status: Record<string, unknown> = {}; | |
| 341 | for (let i = 0; i < 40 && status.open !== false; i++) { | |
| 342 | await sleep(100); | |
| 343 | status = await (await fetch(`${base}/status`)).json() as any; | |
| 344 | } | |
| 345 | expect(status).toMatchObject({ open: false, reason: "offline" }); | |
| 346 | expect((await fetch(`${base}/admin`, form({ action: "close", note: "away" }))).status).toBe(303); | |
| 347 | expect((await (await fetch(`${base}/status`)).json() as any).reason).toBe("closed"); | |
| 348 | ||
| 349 | const snapshot = await fetch(`${base}/camera/2/snapshot`); | |
| 350 | expect(snapshot.status).toBe(200); | |
| 351 | expect(Buffer.compare(Buffer.from(await snapshot.arrayBuffer()), JPEG)).toBe(0); | |
| 352 | expect(((await (await fetch(`${base}/cameras`)).json()) as { name: string }[]).map((c) => c.name)).toEqual(["Camera 1", "Above"]); | |
| 353 | ||
| 354 | const ws = new WebSocket(`ws://127.0.0.1:${relay!.port}/camera/2/ws`); | |
| 355 | const frame = await new Promise<Buffer>((r, j) => (ws.once("message", (d) => r(d as Buffer)), ws.once("error", j))); | |
| 356 | expect(Buffer.compare(frame, JPEG)).toBe(0); | |
| 357 | expect(closedBefore).toBe(false); | |
| 358 | ws.close(); | |
| 359 | before.close(); | |
| 360 | ||
| 361 | // Driving is what rests. | |
| 362 | const follow = new WebSocket(`ws://127.0.0.1:${relay!.port}/follow`, { headers: owner }); | |
| 363 | const refused = await new Promise<number>((resolve) => follow.on("close", (code) => resolve(code))); | |
| 364 | expect(refused).toBe(4503); | |
| 365 | }); | |
| 366 | }); |
Software/so101/src/stations.test.ts +4 −3
| @@ -180,14 +180,15 @@ test("/admin/stations: a guest gets 401, a visitor 403, bad JSON 400, a huge bod | ||
| 180 | 180 | expect(((await (await fetch(`${base}/admin/stations`, { headers: owner })).json()) as { id: string }[]).map((s) => s.id)).toEqual(["grow"]); |
| 181 | 181 | }); |
| 182 | 182 | |
| 183 | test("a closed arm is not present on /stations or the home, which still offers sign-in or Open now", async () => { | |
| 183 | test("a closed arm is not present on /stations or the home, which still offers sign-in or Open now, and still shows its camera", async () => { | |
| 184 | 184 | const { base } = await bootWithArm(true); |
| 185 | const list = (await (await fetch(`${base}/stations`)).json()) as { id: string; present: boolean; resting?: string; cameras: unknown[] }[]; | |
| 186 | expect(list.find((s) => s.id === "arm")).toMatchObject({ present: false, cameras: [] }); | |
| 185 | const list = (await (await fetch(`${base}/stations`)).json()) as { id: string; present: boolean; resting?: string; cameras: { id: number }[] }[]; | |
| 186 | expect(list.find((s) => s.id === "arm")).toMatchObject({ present: false, cameras: [{ id: 1 }] }); | |
| 187 | 187 | expect(list.find((s) => s.id === "arm")!.resting).toBeTruthy(); |
| 188 | 188 | expect(list.find((s) => s.id === "grow")).toMatchObject({ present: true }); |
| 189 | 189 | const guest = await (await fetch(`${base}/`)).text(); |
| 190 | 190 | expect(guest).toContain('data-station="arm" data-present="false"'); |
| 191 | expect(guest).toContain("/camera/1/snapshot"); // the cameras do not rest with the arm (#623) | |
| 191 | 192 | expect(guest).toContain("Watching as a guest"); |
| 192 | 193 | expect(guest).toContain("/__forge/signin"); |
| 193 | 194 | expect(guest).toContain('href="#main"'); |
Software/so101/src/workshop-page.ts +3 −1
| @@ -58,7 +58,9 @@ function card({ station, present, resting, cameras }: StationView, looker: Looke | ||
| 58 | 58 | const head = `<h2>${esc(station.name)}</h2>`; |
| 59 | 59 | if (!present) { |
| 60 | 60 | const reopen = resting && looker.owner ? `<form method="post" action="/admin"><input type="hidden" name="action" value="open"><input type="hidden" name="then" value="home"><button>Open now</button></form>` : ""; |
| 61 | return `<section class="card" data-station="${esc(station.id)}" data-present="false">${head}<p>${esc(resting ?? `${station.name}: not connected`)}</p>${reopen}</section>`; | |
| 61 | // The arm is away; its cameras are not: they stream whenever their own source answers. | |
| 62 | const stills = cameras.map((c, i) => `<img src="/camera/${c.id}/snapshot" alt="${esc(c.name)}"${i ? ' loading="lazy"' : ""} width="640" height="480">`).join(""); | |
| 63 | return `<section class="card" data-station="${esc(station.id)}" data-present="false">${head}<p>${esc(resting ?? `${station.name}: not connected`)}</p>${stills}${reopen}</section>`; | |
| 62 | 64 | } |
| 63 | 65 | if (!cameras.length) { |
| 64 | 66 | const setup = looker.owner ? ` <a href="/admin/camera-list">Cameras</a> · <a href="/admin/stations/edit">Stations</a>` : ""; |
await plugged(bus)can reject andattemptis called from setTimeout, so the rejection is unhandled. Catch it and reschedule, or hand the error to the caller.