Public: anyone can read this; the crew can change it.
Migrated from github.com/Trevato/SO-ARM100
workbenchthis machine
Point the app at my real arm: a setting for the server with the camera and the SO-101, simulation when unset #4
Check · Review · Land done · Live — Landed; nothing deploys from this branch, so this is the end of its path.
quill/arm:quill/3-point-the-app-at-my-real-arm-a-setting-f → main 1 commit fork
6 files
changed +569
−6
Software/README.md +33 −1
| @@ -148,7 +148,8 @@ play, a stop button (space bar), fullscreen (F), a metrics panel, a settings | ||
| 148 | 148 | panel for the safe zone, and a 3D-calibration panel that explains the vision |
| 149 | 149 | quest and shows its status. `bench/fake-serve.ts` runs the same server on a |
| 150 | 150 | simulated bus for working on the page. The page expects MJPEG streams on |
| 151 | ports 8080 and 8081 of the same host (`?stream=<url>` overrides the first). | |
| 151 | ports 8080 and 8081 of the same host (`?stream=<url>` overrides the first), | |
| 152 | or, behind the hosted app's relay, at `/camera/1/stream` and `/camera/2/stream`. | |
| 152 | 153 | |
| 153 | 154 | On the forge (`forge.json` at the repository root), the app is that same |
| 154 | 155 | simulated arm: no servo board is attached, so every move arrives at once. It |
| @@ -156,6 +157,37 @@ reads `PORT` and `HOST`, and keeps poses, recordings, settings and metrics in | ||
| 156 | 157 | `$FORGE_APP_DATA/so101` in place of a temp dir. Run it the same way locally |
| 157 | 158 | with `PORT=8102 FORGE_APP_DATA=./data bun run Software/so101/bench/fake-serve.ts`. |
| 158 | 159 | |
| 160 | **Pointing the hosted app at your real arm.** One setting does it: | |
| 161 | ||
| 162 | 1. On the machine with the follower's board (and the cameras), run the | |
| 163 | server as in the Pi section above: | |
| 164 | `SO101_PORT=/dev/ttyACM0 so101 serve --listen 0.0.0.0:8101`, and the | |
| 165 | cameras on 8080 and 8081 if you have them. | |
| 166 | 2. On the app's settings page on the forge, set `ARM_SERVER` to that server: | |
| 167 | `ws://<host>:8101` (or `http://<host>:8101`, or `https://<host>` behind a | |
| 168 | TLS proxy). | |
| 169 | 3. Reload the page. The top bar says **Live arm · \<host\>**. | |
| 170 | ||
| 171 | The forge's server relays everything (the arm API, the `/follow` stream and | |
| 172 | the camera streams), so the browser only ever talks to the app, wherever it | |
| 173 | is. The forge has to be able to reach `<host>`: put the arm machine on the | |
| 174 | same tailnet as the forge (Tailscale, `ws://<machine>.<tailnet>.ts.net:8101`) | |
| 175 | or on an address the forge can route to. Don't put it on the open internet: | |
| 176 | `so101 serve` has no login. The cameras default to ports 8080 and 8081 on the | |
| 177 | same host. `ARM_CAMERA_1` and `ARM_CAMERA_2` take full stream URLs | |
| 178 | (`http://<host>:8080/stream`) when they live elsewhere. | |
| 179 | ||
| 180 | With `ARM_SERVER` unset, the app runs the simulated arm and says | |
| 181 | **Simulated**. When it is set but the server doesn't answer `/status` within | |
| 182 | 1.5 s, the top bar says **Simulated · \<host\> unreachable (why)**, and the | |
| 183 | page runs on the simulation instead of waiting on the dead link. The app | |
| 184 | checks again every 5 s and switches back by itself. The page reloads on | |
| 185 | either switch, because the other arm has its own limits and poses. A move | |
| 186 | that was in flight to the real arm when the link dropped comes back as an | |
| 187 | error; it is not replayed on the simulation. Changes (`POST`, `PUT`, the | |
| 188 | stream) are only accepted from the app's own page, so another site can't | |
| 189 | drive the arm through it. | |
| 190 | ||
| 159 | 191 | **The safe zone.** The server loads `Simulation/SO101/so101_new_calib.urdf` |
| 160 | 192 | and its meshes (`--model <dir>`, or the repo layout, or `model/` beside the |
| 161 | 193 | package) and runs every target through the arm's geometry: a pose that puts |
Software/so101/bench/fake-serve.ts +15 −2
| @@ -3,12 +3,15 @@ | ||
| 3 | 3 | * node bench/fake-serve.ts [port] then open http://127.0.0.1:8102/?stream=<a camera stream url> |
| 4 | 4 | * Goals become positions at once, so every move "arrives"; poses and recordings go to a temp dir. |
| 5 | 5 | * PORT, HOST and FORGE_APP_DATA override the port, the 127.0.0.1 bind and the temp dir (how the forge runs it). |
| 6 | * ARM_SERVER=ws://<host>:8101 puts a real arm's `so101 serve` behind the same page: the relay in src/relay.ts | |
| 7 | * sends the arm API, /follow and the cameras there while it answers, and to the simulation when it does not. | |
| 6 | 8 | */ |
| 7 | 9 | import { mkdirSync, mkdtempSync } from "node:fs"; |
| 8 | 10 | import { tmpdir } from "node:os"; |
| 9 | 11 | import { join } from "node:path"; |
| 10 | 12 | import { FeetechBus } from "../src/bus.ts"; |
| 11 | 13 | import { FakeBus } from "../src/fake-bus.ts"; |
| 14 | import { parseRemote, startRelay, type Remote } from "../src/relay.ts"; | |
| 12 | 15 | import { startServer } from "../src/server.ts"; |
| 13 | 16 | |
| 14 | 17 | const fake = new FakeBus([1, 2, 3, 4, 5, 6]); |
| @@ -22,5 +25,15 @@ const dir = appData ? join(appData, "so101") : mkdtempSync(join(tmpdir(), "so101 | ||
| 22 | 25 | mkdirSync(dir, { recursive: true }); |
| 23 | 26 | const port = Number(process.env.PORT ?? process.argv[2] ?? 8102); |
| 24 | 27 | const host = process.env.HOST ?? "127.0.0.1"; |
| 25 | const server = await startServer(new FeetechBus(fake), { host, port, posesFile: join(dir, "poses.json"), modelDir: new URL("../../../Simulation/SO101", import.meta.url).pathname }); | |
| 26 | console.log(`fake arm on http://${host}:${server.port} (data in ${dir})`); | |
| 28 | const simulation = await startServer(new FeetechBus(fake), { host: "127.0.0.1", port: 0, posesFile: join(dir, "poses.json"), modelDir: new URL("../../../Simulation/SO101", import.meta.url).pathname }); | |
| 29 | let remote: Remote | null = null; | |
| 30 | let problem: string | null = null; | |
| 31 | try { | |
| 32 | remote = parseRemote(process.env); | |
| 33 | } catch (err) { | |
| 34 | problem = (err as Error).message; | |
| 35 | console.error(`${problem}; simulating`); | |
| 36 | } | |
| 37 | const relay = await startRelay({ host, port, simulation: `http://127.0.0.1:${simulation.port}`, remote, problem }); | |
| 38 | const { mode, host: armHost } = relay.link(); | |
| 39 | console.log(`arm on http://${host}:${relay.port} (${mode === "live" ? `live arm ${armHost}` : "simulated"}; data in ${dir})`); |
Software/so101/src/fake-serve.test.ts +1 −1
| @@ -13,7 +13,7 @@ afterEach(() => { | ||
| 13 | 13 | child = dir = undefined; |
| 14 | 14 | }); |
| 15 | 15 | |
| 16 | /** Start the fake arm with this environment and wait for its "fake arm on http://host:port" line. */ | |
| 16 | /** Start the fake arm with this environment and wait for its "arm on http://host:port" line. */ | |
| 17 | 17 | async function start(env: Record<string, string>): Promise<{ host: string; port: number; line: string }> { |
| 18 | 18 | child = Bun.spawn(["bun", "run", script], { env: { ...process.env, ...env }, stdout: "pipe", stderr: "inherit" }); |
| 19 | 19 | const reader = (child.stdout as ReadableStream<Uint8Array>).getReader(); |
Software/so101/src/relay.test.ts added +174
| @@ -0,0 +1,174 @@ | ||
| 1 | import { afterAll, afterEach, beforeAll, describe, expect, test } from "bun:test"; | |
| 2 | import { mkdtempSync } from "node:fs"; | |
| 3 | import { createServer, type IncomingMessage, type Server } from "node:http"; | |
| 4 | import { createServer as createTcpServer, type Server as TcpServer } from "node:net"; | |
| 5 | import { tmpdir } from "node:os"; | |
| 6 | import { join } from "node:path"; | |
| 7 | import WebSocket from "ws"; | |
| 8 | import { FeetechBus } from "./bus.ts"; | |
| 9 | import { FakeBus } from "./fake-bus.ts"; | |
| 10 | import { parseRemote, sameOrigin, startRelay, type Remote, type RunningRelay } from "./relay.ts"; | |
| 11 | import { startServer, type RunningServer } from "./server.ts"; | |
| 12 | ||
| 13 | const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms)); | |
| 14 | // Bun's node:http can keep a server-closed connection from ending (see server.test.ts): cap every close. | |
| 15 | const shut = (s: { close(): Promise<void> } | undefined) => (s ? Promise.race([s.close(), sleep(500)]) : undefined); | |
| 16 | const listening = (server: Server | TcpServer) => new Promise<number>((r) => server.listen(0, "127.0.0.1", () => r((server.address() as { port: number }).port))); | |
| 17 | ||
| 18 | /** An arm server on a fake bus; `ids` tells the simulation (six joints) and the "real arm" (three) apart. */ | |
| 19 | async function arm(ids: number[]): Promise<RunningServer> { | |
| 20 | const fake = new FakeBus(ids); | |
| 21 | return startServer(new FeetechBus(fake), { host: "127.0.0.1", port: 0, ids, posesFile: join(mkdtempSync(join(tmpdir(), "so101-relay-")), "poses.json"), watchdogMs: 60 }); | |
| 22 | } | |
| 23 | const remoteAt = (port: number, camera?: number): Remote => parseRemote({ ARM_SERVER: `ws://127.0.0.1:${port}`, ...(camera ? { ARM_CAMERA_1: `http://127.0.0.1:${camera}/stream` } : {}) })!; | |
| 24 | const JPEG = Buffer.from([0xff, 0xd8, 0x4a, 0x50, 0x45, 0x47, 0xff, 0xd9]); | |
| 25 | const joints = async (base: string) => ((await (await fetch(`${base}/status`)).json()) as { joints: unknown[] }).joints.length; | |
| 26 | const link = async (base: string) => (await (await fetch(`${base}/link`)).json()) as { mode: string; host: string | null; problem: string | null }; | |
| 27 | ||
| 28 | let simulation: RunningServer; | |
| 29 | let simBase: string; | |
| 30 | beforeAll(async () => { | |
| 31 | simulation = await arm([1, 2, 3, 4, 5, 6]); | |
| 32 | simBase = `http://127.0.0.1:${simulation.port}`; | |
| 33 | }); | |
| 34 | afterAll(() => shut(simulation)); | |
| 35 | ||
| 36 | describe("ARM_SERVER", () => { | |
| 37 | test("unset is no remote; ws, http and https name the server and the cameras beside it", () => { | |
| 38 | expect(parseRemote({})).toBeNull(); | |
| 39 | expect(parseRemote({ ARM_SERVER: " " })).toBeNull(); | |
| 40 | expect(parseRemote({ ARM_SERVER: "ws://arm.tailnet:8101" })).toEqual({ | |
| 41 | http: "http://arm.tailnet:8101", | |
| 42 | ws: "ws://arm.tailnet:8101", | |
| 43 | host: "arm.tailnet:8101", | |
| 44 | cameras: ["http://arm.tailnet:8080/stream", "http://arm.tailnet:8081/stream"], | |
| 45 | }); | |
| 46 | expect(parseRemote({ ARM_SERVER: "https://arm.example/so101/" })).toMatchObject({ http: "https://arm.example/so101", ws: "wss://arm.example/so101", host: "arm.example" }); | |
| 47 | expect(parseRemote({ ARM_SERVER: "100.64.0.7:8101" })?.http).toBe("http://100.64.0.7:8101"); | |
| 48 | expect(parseRemote({ ARM_SERVER: "ws://pi:8101", ARM_CAMERA_2: "https://cams.example/two/stream" })?.cameras[1]).toBe("https://cams.example/two/stream"); | |
| 49 | }); | |
| 50 | ||
| 51 | test("anything else is refused with a reason", () => { | |
| 52 | expect(() => parseRemote({ ARM_SERVER: "ftp://pi:8101" })).toThrow(/must start with/); | |
| 53 | expect(() => parseRemote({ ARM_SERVER: "ws://user:pw@pi:8101" })).toThrow(/nothing else/); | |
| 54 | expect(() => parseRemote({ ARM_SERVER: "http://pi:8101/?x=1" })).toThrow(/nothing else/); | |
| 55 | expect(() => parseRemote({ ARM_SERVER: "http://" })).toThrow(/not a URL/); | |
| 56 | expect(() => parseRemote({ ARM_SERVER: "ws://pi:8101", ARM_CAMERA_1: "ws://pi:8080" })).toThrow(/ARM_CAMERA_1/); | |
| 57 | }); | |
| 58 | }); | |
| 59 | ||
| 60 | describe("same origin", () => { | |
| 61 | const req = (headers: Record<string, string>) => ({ headers }) as unknown as IncomingMessage; | |
| 62 | test("a page on another site may not drive the arm", () => { | |
| 63 | expect(sameOrigin(req({}))).toBe(true); | |
| 64 | expect(sameOrigin(req({ "sec-fetch-site": "same-origin" }))).toBe(true); | |
| 65 | expect(sameOrigin(req({ "sec-fetch-site": "cross-site" }))).toBe(false); | |
| 66 | expect(sameOrigin(req({ origin: "https://arm.forge", host: "arm.forge" }))).toBe(true); | |
| 67 | expect(sameOrigin(req({ origin: "https://arm.forge", host: "127.0.0.1:9", "x-forwarded-host": "arm.forge" }))).toBe(true); | |
| 68 | expect(sameOrigin(req({ origin: "https://evil.example", host: "arm.forge" }))).toBe(false); | |
| 69 | }); | |
| 70 | }); | |
| 71 | ||
| 72 | describe("relay", () => { | |
| 73 | let relay: RunningRelay | undefined; | |
| 74 | let remote: RunningServer | undefined; | |
| 75 | let camera: Server | undefined; | |
| 76 | let hang: TcpServer | undefined; | |
| 77 | afterEach(async () => { | |
| 78 | await shut(relay); | |
| 79 | await shut(remote); | |
| 80 | camera?.closeAllConnections(); | |
| 81 | camera?.close(); | |
| 82 | hang?.close(); | |
| 83 | relay = remote = camera = hang = undefined; | |
| 84 | }); | |
| 85 | ||
| 86 | test("unset: everything is the simulation, and the page says so", async () => { | |
| 87 | relay = await startRelay({ port: 0, simulation: simBase }); | |
| 88 | const base = `http://127.0.0.1:${relay.port}`; | |
| 89 | expect(await link(base)).toEqual({ mode: "simulated", host: null, problem: null }); | |
| 90 | expect(await joints(base)).toBe(6); | |
| 91 | const page = await fetch(`${base}/`); | |
| 92 | expect(page.status).toBe(200); | |
| 93 | expect(await page.text()).toContain("<title>SO-101</title>"); | |
| 94 | expect(page.headers.get("x-content-type-options")).toBe("nosniff"); | |
| 95 | expect(page.headers.get("content-security-policy")).toContain("frame-ancestors 'none'"); | |
| 96 | expect((await fetch(`${base}/camera/1/stream`)).status).toBe(404); | |
| 97 | }); | |
| 98 | ||
| 99 | test("set: status, poses and /follow go to the arm server; the camera's bytes come through", async () => { | |
| 100 | remote = await arm([1, 2, 6]); | |
| 101 | camera = createServer((req, res) => { | |
| 102 | if (req.url?.startsWith("/state")) return res.writeHead(200, { "content-type": "application/json" }).end('{"ok":true}'); | |
| 103 | res.writeHead(200, { "content-type": "multipart/x-mixed-replace;boundary=frame" }); | |
| 104 | res.write(Buffer.concat([Buffer.from("--frame\r\ncontent-type: image/jpeg\r\n\r\n"), JPEG])); // and the stream stays open, like µStreamer | |
| 105 | }); | |
| 106 | const cameraPort = await listening(camera); | |
| 107 | relay = await startRelay({ port: 0, simulation: simBase, remote: remoteAt(remote.port, cameraPort) }); | |
| 108 | const base = `http://127.0.0.1:${relay.port}`; | |
| 109 | expect(await link(base)).toEqual({ mode: "live", host: `127.0.0.1:${remote.port}`, problem: null }); | |
| 110 | expect(await joints(base)).toBe(3); | |
| 111 | ||
| 112 | const saved = await fetch(`${base}/pose/save`, { method: "POST", body: JSON.stringify({ name: "real" }) }); | |
| 113 | expect(saved.status).toBe(200); | |
| 114 | expect(Object.keys((await (await fetch(`http://127.0.0.1:${remote.port}/poses`)).json()) as object)).toEqual(["real"]); | |
| 115 | expect(Object.keys((await (await fetch(`${simBase}/poses`)).json()) as object)).not.toContain("real"); | |
| 116 | ||
| 117 | const ws = new WebSocket(`ws://127.0.0.1:${relay.port}/follow`); | |
| 118 | await new Promise((r, j) => (ws.once("open", r), ws.once("error", j))); | |
| 119 | await sleep(50); | |
| 120 | const status = (await (await fetch(`http://127.0.0.1:${remote.port}/status`)).json()) as { driving: string | null }; | |
| 121 | expect(status.driving).toBe("stream"); // the remote arm took the stream, not the simulation | |
| 122 | ws.close(); | |
| 123 | ||
| 124 | const stream = await fetch(`${base}/camera/1/stream`); | |
| 125 | expect(stream.headers.get("content-type")).toBe("multipart/x-mixed-replace;boundary=frame"); | |
| 126 | const reader = stream.body!.getReader(); | |
| 127 | const { value } = await reader.read(); | |
| 128 | expect(Buffer.from(value!).includes(JPEG)).toBe(true); | |
| 129 | await reader.cancel(); | |
| 130 | expect(await (await fetch(`${base}/camera/1/state`)).json()).toEqual({ ok: true }); | |
| 131 | }); | |
| 132 | ||
| 133 | test("a mutation from another site is refused before it reaches the arm", async () => { | |
| 134 | remote = await arm([1, 2, 6]); | |
| 135 | relay = await startRelay({ port: 0, simulation: simBase, remote: remoteAt(remote.port) }); | |
| 136 | const refused = await fetch(`http://127.0.0.1:${relay.port}/torque`, { method: "POST", headers: { origin: "https://evil.example" }, body: '{"on":false}' }); | |
| 137 | expect(refused.status).toBe(403); | |
| 138 | }); | |
| 139 | ||
| 140 | test("unreachable: the simulation answers within 3 s and the page is told why", async () => { | |
| 141 | hang = createTcpServer(() => {}); // accepts and never answers | |
| 142 | const port = await listening(hang); | |
| 143 | const started = Date.now(); | |
| 144 | relay = await startRelay({ port: 0, simulation: simBase, remote: remoteAt(port) }); | |
| 145 | const base = `http://127.0.0.1:${relay.port}`; | |
| 146 | expect(await joints(base)).toBe(6); | |
| 147 | const l = await link(base); | |
| 148 | expect(Date.now() - started).toBeLessThan(3000); | |
| 149 | expect(l.mode).toBe("simulated"); | |
| 150 | expect(l.host).toBe(`127.0.0.1:${port}`); | |
| 151 | expect(l.problem).toContain("unreachable"); | |
| 152 | }); | |
| 153 | ||
| 154 | test("an arm server that goes away mid-session falls back on the next request, /follow included", async () => { | |
| 155 | remote = await arm([1, 2, 6]); | |
| 156 | relay = await startRelay({ port: 0, simulation: simBase, remote: remoteAt(remote.port), probeEveryMs: 60_000 }); | |
| 157 | const base = `http://127.0.0.1:${relay.port}`; | |
| 158 | expect(await joints(base)).toBe(3); | |
| 159 | await shut(remote); | |
| 160 | remote = undefined; | |
| 161 | const started = Date.now(); | |
| 162 | expect(await joints(base)).toBe(6); | |
| 163 | expect(Date.now() - started).toBeLessThan(3000); | |
| 164 | expect((await link(base)).mode).toBe("simulated"); | |
| 165 | const ws = new WebSocket(`ws://127.0.0.1:${relay.port}/follow`); | |
| 166 | await new Promise((r, j) => (ws.once("open", r), ws.once("error", j))); | |
| 167 | ws.close(); | |
| 168 | }); | |
| 169 | ||
| 170 | test("a bad setting is reported and the arm is simulated", async () => { | |
| 171 | relay = await startRelay({ port: 0, simulation: simBase, problem: 'ARM_SERVER "ftp://x" must start with http://' }); | |
| 172 | expect(await link(`http://127.0.0.1:${relay.port}`)).toEqual({ mode: "simulated", host: null, problem: 'ARM_SERVER "ftp://x" must start with http://' }); | |
| 173 | }); | |
| 174 | }); |
Software/so101/src/relay.ts added +323
| @@ -0,0 +1,323 @@ | ||
| 1 | /** | |
| 2 | * The hosted app's front door: one origin for the browser, whichever arm stands behind it. | |
| 3 | * | |
| 4 | * With ARM_SERVER unset, everything goes to the simulated arm. With it set (ws://<host>:8101, http://…, https://…), | |
| 5 | * the arm API, the WebSocket /follow and the cameras go to that `so101 serve`, as long as it answers; when it does | |
| 6 | * not, the relay says so on /link and answers from the simulation instead, so the page never hangs on a dead link. | |
| 7 | * The page itself, the 3D model and the vendored scripts always come from here. | |
| 8 | * | |
| 9 | * GET /link {"mode": "live"|"simulated", "host": "<host:port>"|null, "problem": "…"|null} | |
| 10 | * GET /camera/<1|2>/<stream|state|snapshot> the arm machine's camera (µStreamer on :8080 and :8081 by default) | |
| 11 | */ | |
| 12 | ||
| 13 | import { createServer, type IncomingMessage, type ServerResponse } from "node:http"; | |
| 14 | import type { Duplex } from "node:stream"; | |
| 15 | import { Readable } from "node:stream"; | |
| 16 | import { pipeline } from "node:stream/promises"; | |
| 17 | import WebSocket, { WebSocketServer } from "ws"; | |
| 18 | ||
| 19 | export interface Remote { | |
| 20 | /** The arm server's HTTP base, no trailing slash. */ | |
| 21 | http: string; | |
| 22 | /** Its WebSocket base. */ | |
| 23 | ws: string; | |
| 24 | /** host:port, for the page and the logs. */ | |
| 25 | host: string; | |
| 26 | /** Camera 1 and 2 stream URLs. */ | |
| 27 | cameras: string[]; | |
| 28 | } | |
| 29 | ||
| 30 | export interface Link { | |
| 31 | mode: "live" | "simulated"; | |
| 32 | /** The arm server the setting names, live or not. */ | |
| 33 | host: string | null; | |
| 34 | /** Why a configured arm server is not in use right now. */ | |
| 35 | problem: string | null; | |
| 36 | } | |
| 37 | ||
| 38 | export interface RelayOptions { | |
| 39 | host?: string; | |
| 40 | port?: number; | |
| 41 | /** The simulated arm's HTTP base, e.g. http://127.0.0.1:41234. */ | |
| 42 | simulation: string; | |
| 43 | remote?: Remote | null; | |
| 44 | /** A setting that could not be used (shown on /link while simulating). */ | |
| 45 | problem?: string | null; | |
| 46 | /** How long the arm server has to answer /status before it counts as unreachable. */ | |
| 47 | probeTimeoutMs?: number; | |
| 48 | /** How often the arm server is checked, whichever way the last check went. */ | |
| 49 | probeEveryMs?: number; | |
| 50 | /** How long a read through to the arm server may wait for its answer before the simulation answers instead. */ | |
| 51 | readTimeoutMs?: number; | |
| 52 | } | |
| 53 | ||
| 54 | export interface RunningRelay { | |
| 55 | readonly port: number; | |
| 56 | link(): Link; | |
| 57 | close(): Promise<void>; | |
| 58 | } | |
| 59 | ||
| 60 | const SCHEMES: Record<string, "http:" | "https:"> = { "http:": "http:", "https:": "https:", "ws:": "http:", "wss:": "https:" }; | |
| 61 | const MAX_BODY = 1_000_000; | |
| 62 | /** Served from this process whichever arm is live: they describe the app, not the arm. */ | |
| 63 | const LOCAL = /^\/(?:index\.html)?$|^\/(?:model|vendor|printables)(?:\/|$)/; | |
| 64 | const CAMERA = /^\/camera\/([12])\/(stream|state|snapshot)$/; | |
| 65 | const FLOOR = { | |
| 66 | "x-content-type-options": "nosniff", | |
| 67 | "referrer-policy": "same-origin", | |
| 68 | "content-security-policy": "frame-ancestors 'none'; base-uri 'none'; object-src 'none'; form-action 'self'", | |
| 69 | }; | |
| 70 | ||
| 71 | const absolute = (text: string, what: string, schemes: Record<string, string>): URL => { | |
| 72 | let url: URL; | |
| 73 | try { | |
| 74 | url = new URL(/^[a-z][a-z0-9+.-]*:\/\//i.test(text) ? text : `http://${text}`); | |
| 75 | } catch { | |
| 76 | throw new Error(`${what} "${text}" is not a URL`); | |
| 77 | } | |
| 78 | if (!Object.hasOwn(schemes, url.protocol)) throw new Error(`${what} must start with ${Object.keys(schemes).map((s) => `${s}//`).join(", ")}`); | |
| 79 | if (url.username || url.password || url.search || url.hash) throw new Error(`${what} takes a scheme, a host, a port and a path, nothing else`); | |
| 80 | return url; | |
| 81 | }; | |
| 82 | ||
| 83 | /** The arm server named by ARM_SERVER (and the cameras, ARM_CAMERA_1 / ARM_CAMERA_2), or null when unset. Throws on a bad value. */ | |
| 84 | export function parseRemote(env: Record<string, string | undefined>): Remote | null { | |
| 85 | const text = env.ARM_SERVER?.trim(); | |
| 86 | if (!text) return null; | |
| 87 | const url = absolute(text, "ARM_SERVER", SCHEMES); | |
| 88 | const scheme = SCHEMES[url.protocol]!; | |
| 89 | const path = url.pathname.replace(/\/+$/, ""); | |
| 90 | const cameras = [1, 2].map((n) => { | |
| 91 | const override = env[`ARM_CAMERA_${n}`]?.trim(); | |
| 92 | return override ? absolute(override, `ARM_CAMERA_${n}`, { "http:": "", "https:": "" }).href : `http://${url.hostname}:${8079 + n}/stream`; | |
| 93 | }); | |
| 94 | return { http: `${scheme}//${url.host}${path}`, ws: `${scheme === "https:" ? "wss:" : "ws:"}//${url.host}${path}`, host: url.host, cameras }; | |
| 95 | } | |
| 96 | ||
| 97 | /** A mutation or a stream is only taken from this page's own origin; requests with no browser markers (curl, the CLI) pass. */ | |
| 98 | export function sameOrigin(req: IncomingMessage): boolean { | |
| 99 | const site = req.headers["sec-fetch-site"]; | |
| 100 | if (typeof site === "string") return site === "same-origin" || site === "none"; | |
| 101 | const origin = req.headers.origin; | |
| 102 | if (!origin) return true; | |
| 103 | const host = String(req.headers["x-forwarded-host"] ?? req.headers.host ?? "").split(",")[0]!.trim(); | |
| 104 | try { | |
| 105 | return new URL(origin).host === host; | |
| 106 | } catch { | |
| 107 | return false; | |
| 108 | } | |
| 109 | } | |
| 110 | ||
| 111 | const reason = (err: unknown) => { | |
| 112 | const e = err as { message?: string; cause?: { code?: string }; code?: string }; | |
| 113 | return e?.cause?.code ?? e?.code ?? e?.message ?? String(err); | |
| 114 | }; | |
| 115 | ||
| 116 | export async function startRelay(options: RelayOptions): Promise<RunningRelay> { | |
| 117 | const remote = options.remote ?? null; | |
| 118 | const simulation = options.simulation.replace(/\/+$/, ""); | |
| 119 | const simulationWs = simulation.replace(/^http/, "ws"); | |
| 120 | const probeTimeoutMs = options.probeTimeoutMs ?? 1500; | |
| 121 | const probeEveryMs = options.probeEveryMs ?? 5000; | |
| 122 | const readTimeoutMs = options.readTimeoutMs ?? 2500; | |
| 123 | ||
| 124 | // ------------------------------------------------------------------ the link | |
| 125 | let live = false; | |
| 126 | let problem: string | null = options.problem ?? null; | |
| 127 | /** Requests in flight to the arm server: cut when it is found gone, so nothing waits on a dead link. */ | |
| 128 | const inFlight = new Set<AbortController>(); | |
| 129 | const up = () => { | |
| 130 | if (!live && remote) console.log(`arm server ${remote.host}: live`); | |
| 131 | live = true; | |
| 132 | problem = null; | |
| 133 | }; | |
| 134 | const down = (why: string) => { | |
| 135 | if (!remote) return; | |
| 136 | if (live || problem === null) console.error(`arm server ${remote.host}: ${why}; simulating`); | |
| 137 | live = false; | |
| 138 | problem = `${remote.host} unreachable (${why})`; | |
| 139 | for (const c of inFlight) c.abort(new Error(why)); | |
| 140 | }; | |
| 141 | let probing: Promise<void> | null = null; | |
| 142 | const probe = () => | |
| 143 | (probing ??= (async () => { | |
| 144 | try { | |
| 145 | const r = await fetch(`${remote!.http}/status`, { signal: AbortSignal.timeout(probeTimeoutMs), cache: "no-store" }); | |
| 146 | await r.body?.cancel(); | |
| 147 | if (r.ok) up(); | |
| 148 | else down(`/status answered ${r.status}`); | |
| 149 | } catch (err) { | |
| 150 | down((err as Error)?.name === "TimeoutError" ? `no answer in ${probeTimeoutMs} ms` : reason(err)); | |
| 151 | } finally { | |
| 152 | probing = null; | |
| 153 | } | |
| 154 | })()); | |
| 155 | const link = (): Link => ({ mode: live ? "live" : "simulated", host: remote?.host ?? null, problem }); | |
| 156 | ||
| 157 | // ------------------------------------------------------------------ HTTP | |
| 158 | const json = (res: ServerResponse, status: number, body: unknown) => { | |
| 159 | res.writeHead(status, { "content-type": "application/json", "cache-control": "no-store" }); | |
| 160 | res.end(JSON.stringify(body)); | |
| 161 | }; | |
| 162 | const readBody = (req: IncomingMessage) => | |
| 163 | new Promise<Buffer>((resolve, reject) => { | |
| 164 | const chunks: Buffer[] = []; | |
| 165 | let size = 0; | |
| 166 | req.on("data", (c: Buffer) => { | |
| 167 | size += c.length; | |
| 168 | if (size > MAX_BODY) { | |
| 169 | req.destroy(); | |
| 170 | reject(new Error("body too large")); | |
| 171 | } else chunks.push(c); | |
| 172 | }); | |
| 173 | req.on("end", () => resolve(Buffer.concat(chunks))); | |
| 174 | req.on("error", reject); | |
| 175 | }); | |
| 176 | /** Ask one upstream; resolves once its headers are in. The controller stays live for the body. */ | |
| 177 | const ask = async (url: string, req: IncomingMessage, body: Buffer | undefined, timeoutMs: number | null, tracked: boolean) => { | |
| 178 | const controller = new AbortController(); | |
| 179 | const timer = timeoutMs === null ? undefined : setTimeout(() => controller.abort(new Error(`no answer in ${timeoutMs} ms`)), timeoutMs); | |
| 180 | if (tracked) inFlight.add(controller); | |
| 181 | const headers: Record<string, string> = {}; | |
| 182 | for (const name of ["content-type", "accept", "range"]) if (typeof req.headers[name] === "string") headers[name] = req.headers[name]; | |
| 183 | try { | |
| 184 | const response = await fetch(url, { method: req.method, headers, body: body?.length ? body : undefined, signal: controller.signal, redirect: "manual", cache: "no-store" }); | |
| 185 | return { response, controller }; | |
| 186 | } catch (err) { | |
| 187 | inFlight.delete(controller); | |
| 188 | throw controller.signal.aborted ? controller.signal.reason : err; | |
| 189 | } finally { | |
| 190 | clearTimeout(timer); | |
| 191 | } | |
| 192 | }; | |
| 193 | /** Hand an upstream answer to the browser, streaming (an MJPEG stream never ends), and let go of it when either side leaves. */ | |
| 194 | const pass = async (res: ServerResponse, { response, controller }: Awaited<ReturnType<typeof ask>>) => { | |
| 195 | const headers: Record<string, string> = { "cache-control": response.headers.get("cache-control") ?? "no-store" }; | |
| 196 | for (const name of ["content-type", "content-length", "content-range", "accept-ranges", "last-modified", "etag"]) { | |
| 197 | const value = response.headers.get(name); | |
| 198 | if (value) headers[name] = value; | |
| 199 | } | |
| 200 | res.writeHead(response.status, headers); | |
| 201 | res.on("close", () => controller.abort()); | |
| 202 | try { | |
| 203 | if (response.body) await pipeline(Readable.fromWeb(response.body as import("node:stream/web").ReadableStream), res); | |
| 204 | else res.end(); | |
| 205 | } catch { | |
| 206 | res.destroy(); | |
| 207 | } finally { | |
| 208 | inFlight.delete(controller); | |
| 209 | } | |
| 210 | }; | |
| 211 | ||
| 212 | const http = createServer(async (req, res) => { | |
| 213 | for (const [name, value] of Object.entries(FLOOR)) res.setHeader(name, value); | |
| 214 | try { | |
| 215 | const url = new URL(req.url ?? "/", "http://x"); | |
| 216 | const path = url.pathname + url.search; | |
| 217 | const reading = req.method === "GET" || req.method === "HEAD"; | |
| 218 | if (!reading && !sameOrigin(req)) return json(res, 403, { error: "only this page may change the arm" }); | |
| 219 | if (req.method === "GET" && url.pathname === "/link") return json(res, 200, link()); | |
| 220 | ||
| 221 | const camera = CAMERA.exec(url.pathname); | |
| 222 | if (camera) { | |
| 223 | if (!reading) return json(res, 405, { error: "cameras are read-only" }); | |
| 224 | if (!remote || !live) return json(res, 404, { error: "no camera: the arm is simulated" }); | |
| 225 | const stream = remote.cameras[Number(camera[1]) - 1]!; | |
| 226 | const target = stream.replace(/\/[^/]*$/, `/${camera[2]}`) + url.search; | |
| 227 | try { | |
| 228 | return await pass(res, await ask(target, req, undefined, readTimeoutMs, true)); | |
| 229 | } catch (err) { | |
| 230 | return json(res, 502, { error: `camera ${camera[1]} did not answer: ${reason(err)}` }); | |
| 231 | } | |
| 232 | } | |
| 233 | ||
| 234 | const body = reading ? undefined : await readBody(req); | |
| 235 | if (LOCAL.test(url.pathname) || !live || !remote) return await pass(res, await ask(simulation + path, req, body, null, false)); | |
| 236 | try { | |
| 237 | // A read must answer fast or fall back; a move or a playback answers when the arm arrives, and is only cut when the link is. | |
| 238 | return await pass(res, await ask(remote.http + path, req, body, reading ? readTimeoutMs : null, true)); | |
| 239 | } catch (err) { | |
| 240 | down(reason(err)); | |
| 241 | if (!reading) return json(res, 502, { error: `the arm server at ${remote.host} did not answer (${reason(err)}); the page is now on the simulation` }); | |
| 242 | return await pass(res, await ask(simulation + path, req, body, null, false)); | |
| 243 | } | |
| 244 | } catch (err) { | |
| 245 | if (!res.headersSent) json(res, 502, { error: reason(err) }); | |
| 246 | else res.destroy(); | |
| 247 | } | |
| 248 | }); | |
| 249 | ||
| 250 | // ------------------------------------------------------------------ WebSocket /follow | |
| 251 | const wss = new WebSocketServer({ noServer: true }); | |
| 252 | const open = (url: string, timeoutMs: number) => | |
| 253 | new Promise<WebSocket>((resolve, reject) => { | |
| 254 | const ws = new WebSocket(url); | |
| 255 | const timer = setTimeout(() => { | |
| 256 | ws.terminate(); | |
| 257 | reject(new Error(`no answer in ${timeoutMs} ms`)); | |
| 258 | }, timeoutMs); | |
| 259 | ws.once("open", () => { | |
| 260 | clearTimeout(timer); | |
| 261 | resolve(ws); | |
| 262 | }); | |
| 263 | ws.once("error", (err) => { | |
| 264 | clearTimeout(timer); | |
| 265 | reject(err); | |
| 266 | }); | |
| 267 | }); | |
| 268 | /** Close codes a server may send on; the rest (1005, 1006, 1015) only describe a connection that died. */ | |
| 269 | const sendable = (code: number) => (code >= 1000 && code <= 1003) || (code >= 1007 && code <= 1014) || (code >= 3000 && code <= 4999); | |
| 270 | const bridge = (browser: WebSocket, arm: WebSocket) => { | |
| 271 | arm.on("message", (data, binary) => browser.readyState === WebSocket.OPEN && browser.send(data, { binary })); | |
| 272 | browser.on("message", (data, binary) => arm.readyState === WebSocket.OPEN && arm.send(data, { binary })); | |
| 273 | arm.on("close", (code, why) => browser.close(sendable(code) ? code : 1011, why.length ? why.toString() : "the arm server went away")); | |
| 274 | browser.on("close", () => arm.close()); | |
| 275 | arm.on("error", () => browser.terminate()); | |
| 276 | browser.on("error", () => arm.terminate()); | |
| 277 | }; | |
| 278 | const refuse = (socket: Duplex, status: string) => { | |
| 279 | socket.end(`HTTP/1.1 ${status}\r\nconnection: close\r\ncontent-length: 0\r\n\r\n`); | |
| 280 | }; | |
| 281 | http.on("upgrade", (req: IncomingMessage, socket: Duplex, head: Buffer) => { | |
| 282 | const url = new URL(req.url ?? "/", "http://x"); | |
| 283 | if (url.pathname !== "/follow") return refuse(socket, "404 Not Found"); | |
| 284 | if (!sameOrigin(req)) return refuse(socket, "403 Forbidden"); | |
| 285 | socket.on("error", () => socket.destroy()); | |
| 286 | void (async () => { | |
| 287 | let arm: WebSocket; | |
| 288 | try { | |
| 289 | if (!remote || !live) throw null; | |
| 290 | arm = await open(`${remote.ws}/follow`, readTimeoutMs); | |
| 291 | } catch (err) { | |
| 292 | if (err) down(reason(err)); | |
| 293 | try { | |
| 294 | arm = await open(`${simulationWs}/follow`, readTimeoutMs); | |
| 295 | } catch { | |
| 296 | return refuse(socket, "502 Bad Gateway"); | |
| 297 | } | |
| 298 | } | |
| 299 | if (socket.destroyed) return arm.close(); | |
| 300 | wss.handleUpgrade(req, socket, head, (browser) => bridge(browser, arm)); | |
| 301 | })(); | |
| 302 | }); | |
| 303 | ||
| 304 | // ------------------------------------------------------------------ start | |
| 305 | if (remote) await probe(); // bounded by the probe timeout: the first page already knows which arm it has | |
| 306 | const timer = remote ? setInterval(() => void probe(), probeEveryMs) : undefined; | |
| 307 | timer?.unref?.(); | |
| 308 | await new Promise<void>((resolve) => http.listen(options.port ?? 8102, options.host ?? "127.0.0.1", resolve)); | |
| 309 | const address = http.address(); | |
| 310 | return { | |
| 311 | port: typeof address === "object" && address ? address.port : options.port ?? 8102, | |
| 312 | link, | |
| 313 | close: () => | |
| 314 | new Promise<void>((resolve) => { | |
| 315 | clearInterval(timer); | |
| 316 | for (const c of inFlight) c.abort(); | |
| 317 | for (const client of wss.clients) client.terminate(); | |
| 318 | wss.close(); | |
| 319 | http.close(() => resolve()); | |
| 320 | http.closeAllConnections(); | |
| 321 | }), | |
| 322 | }; | |
| 323 | } |
Software/so101/src/viewer.html +23 −2
| @@ -28,6 +28,7 @@ | ||
| 28 | 28 | @media (max-aspect-ratio: 1/1) { #stage { flex-direction: column; } } |
| 29 | 29 | #top { position: absolute; top: 8px; right: 10px; z-index: 5; display: flex; gap: 8px; align-items: center; font-size: 13px; background: rgba(0,0,0,.65); padding: 5px 6px 5px 10px; border-radius: 8px; flex-wrap: wrap; justify-content: flex-end; } |
| 30 | 30 | #top .seg button { padding: 5px 9px; min-height: 30px; } #top .seg button.on { background: var(); border-color: var(); } |
| 31 | #link { color: var(); white-space: nowrap; overflow: hidden; text-overflow: ellipsis; max-width: min(46ch, 60vw); } #link.on { color: var(); } #link.warn { color: var(); } | |
| 31 | 32 | .dot { width: 8px; height: 8px; border-radius: 50%; background: var(); display: inline-block; } .dot.on { background: var(); } |
| 32 | 33 | #banner { position: absolute; top: 54px; left: 50%; transform: translateX(-50%); z-index: 5; max-width: min(640px, 90vw); padding: 8px 14px; border-radius: 8px; font-size: 13px; background: rgba(0,0,0,.75); border: 1px solid var(); display: none; } |
| 33 | 34 | #banner.bad { display: block; border-color: var(); color: #ffb3ad; } #banner.warn { display: block; border-color: var(); color: #f2cc60; } |
| @@ -86,7 +87,7 @@ | ||
| 86 | 87 | </div> |
| 87 | 88 | <div id="top"> |
| 88 | 89 | <span class="seg"><button id="v1" class="on">Cam 1</button><button id="v2" hidden>Cam 2</button><button id="v3" class="on">3D</button></span> |
| 89 | <span id="live"></span><span class="dot" id="dot"></span><span id="state">…</span> | |
| 90 | <span id="link" hidden></span><span id="live"></span><span class="dot" id="dot"></span><span id="state">…</span> | |
| 90 | 91 | <button id="zoneBadge" title="safe zone settings">zone</button> |
| 91 | 92 | <button id="stop" class="danger" hidden title="space">Stop</button><button id="fs" title="F">Fullscreen</button> |
| 92 | 93 | </div> |
| @@ -210,9 +211,28 @@ | ||
| 210 | 211 | const NAMES = { 1: "shoulder_pan", 2: "shoulder_lift", 3: "elbow_flex", 4: "wrist_flex", 5: "wrist_roll", 6: "gripper" }; |
| 211 | 212 | const IDS = Object.fromEntries(Object.entries(NAMES).map(([id, n]) => [n, Number(id)])); |
| 212 | 213 | |
| 214 | // ---------------------------------------------------------------- which arm | |
| 215 | // Behind the hosted app's relay (/link answers), the arm is a live `so101 serve` or the simulation, and the cameras come through the relay. | |
| 216 | const relayed = await fetch("/link", { cache: "no-store" }).then((r) => r.ok, () => false); | |
| 217 | let linkMode = null; | |
| 218 | async function pollLink() { | |
| 219 | const l = await api("/link").catch(() => null); | |
| 220 | if (l?.ok) { | |
| 221 | const el = $("link"); | |
| 222 | el.hidden = false; | |
| 223 | el.textContent = l.mode === "live" ? `Live arm · ${l.host}` : l.problem ? `Simulated · ${l.problem}` : "Simulated"; | |
| 224 | el.title = el.textContent; | |
| 225 | el.className = l.mode === "live" ? "on" : l.problem ? "warn" : ""; | |
| 226 | // Another arm has other limits, poses and joints: start the page over on it, once nothing on the page is driving. | |
| 227 | if (linkMode !== null && l.mode !== linkMode && !stream.ws && !live) return location.reload(); | |
| 228 | if (linkMode === null) linkMode = l.mode; | |
| 229 | } | |
| 230 | setTimeout(pollLink, 3000); | |
| 231 | } | |
| 232 | ||
| 213 | 233 | // ---------------------------------------------------------------- cameras |
| 214 | 234 | // Plain <img> streams work in every browser; a canvas comparison once a second notices a frozen picture and reconnects. |
| 215 | const camUrl = (n) => q.get(`stream${n}`) || q.get(n === 1 ? "stream" : "") || `${location.protocol}//${location.hostname}:${8079 + n}/stream`; | |
| 235 | const camUrl = (n) => q.get(`stream${n}`) || q.get(n === 1 ? "stream" : "") || (relayed ? `/camera/${n}/stream` : `${location.protocol}//${location.hostname}:${8079 + n}/stream`); | |
| 216 | 236 | const cams = [1, 2].map((n) => ({ n, img: $(`cam${n}`), url: camUrl(n), sig: "", unchanged: 0, live: false })); |
| 217 | 237 | const canvas = document.createElement("canvas"); canvas.width = 32; canvas.height = 18; |
| 218 | 238 | const ctx = canvas.getContext("2d", { willReadFrequently: true }); |
| @@ -749,6 +769,7 @@ | ||
| 749 | 769 | if (wanted.length) for (const [btn, pane] of Object.entries(panes)) { const on = wanted.includes(pane); $(btn).classList.toggle("on", on); if (pane === "pane3") $(pane).classList.toggle("on", on); else showCam(cams[pane === "pane1" ? 0 : 1], on); } |
| 750 | 770 | } catch {} |
| 751 | 771 | poll(); |
| 772 | if (relayed) pollLink(); | |
| 752 | 773 | })(); |
| 753 | 774 | </script> |
| 754 | 775 | </body> |
new above
offline