Public: anyone can read this; the crew can change it.

Migrated from github.com/Trevato/SO-ARM100

workbenchthis machine

Point the app at my real arm: a setting for the server with the camera and the SO-101, simulation when unset #4

Merged opened this in answer to

Check · Review · Land done · Live — Landed; nothing deploys from this branch, so this is the end of its path.

Merged

quill/arm:quill/3-point-the-app-at-my-real-arm-a-setting-f → main 1 commit fork

  • Merged. quill/3-point-the-app-at-my-real-arm-a-setting-f was merged into main as 3462817.

Review

Sign in to review this change.
6 files changed +569 −6
Software/README.md +33 −1
@@ -148,7 +148,8 @@ play, a stop button (space bar), fullscreen (F), a metrics panel, a settings
148148 panel for the safe zone, and a 3D-calibration panel that explains the vision
149149 quest and shows its status. `bench/fake-serve.ts` runs the same server on a
150150 simulated bus for working on the page. The page expects MJPEG streams on
151 ports 8080 and 8081 of the same host (`?stream=<url>` overrides the first).
151 ports 8080 and 8081 of the same host (`?stream=<url>` overrides the first),
152 or, behind the hosted app's relay, at `/camera/1/stream` and `/camera/2/stream`.
152153
153154 On the forge (`forge.json` at the repository root), the app is that same
154155 simulated arm: no servo board is attached, so every move arrives at once. It
@@ -156,6 +157,37 @@ reads `PORT` and `HOST`, and keeps poses, recordings, settings and metrics in
156157 `$FORGE_APP_DATA/so101` in place of a temp dir. Run it the same way locally
157158 with `PORT=8102 FORGE_APP_DATA=./data bun run Software/so101/bench/fake-serve.ts`.
158159
160 **Pointing the hosted app at your real arm.** One setting does it:
161
162 1. On the machine with the follower's board (and the cameras), run the
163 server as in the Pi section above:
164 `SO101_PORT=/dev/ttyACM0 so101 serve --listen 0.0.0.0:8101`, and the
165 cameras on 8080 and 8081 if you have them.
166 2. On the app's settings page on the forge, set `ARM_SERVER` to that server:
167 `ws://<host>:8101` (or `http://<host>:8101`, or `https://<host>` behind a
168 TLS proxy).
169 3. Reload the page. The top bar says **Live arm · \<host\>**.
170
171 The forge's server relays everything (the arm API, the `/follow` stream and
172 the camera streams), so the browser only ever talks to the app, wherever it
173 is. The forge has to be able to reach `<host>`: put the arm machine on the
174 same tailnet as the forge (Tailscale, `ws://<machine>.<tailnet>.ts.net:8101`)
175 or on an address the forge can route to. Don't put it on the open internet:
176 `so101 serve` has no login. The cameras default to ports 8080 and 8081 on the
177 same host. `ARM_CAMERA_1` and `ARM_CAMERA_2` take full stream URLs
178 (`http://<host>:8080/stream`) when they live elsewhere.
179
180 With `ARM_SERVER` unset, the app runs the simulated arm and says
181 **Simulated**. When it is set but the server doesn't answer `/status` within
182 1.5 s, the top bar says **Simulated · \<host\> unreachable (why)**, and the
183 page runs on the simulation instead of waiting on the dead link. The app
184 checks again every 5 s and switches back by itself. The page reloads on
185 either switch, because the other arm has its own limits and poses. A move
186 that was in flight to the real arm when the link dropped comes back as an
187 error; it is not replayed on the simulation. Changes (`POST`, `PUT`, the
188 stream) are only accepted from the app's own page, so another site can't
189 drive the arm through it.
190
159191 **The safe zone.** The server loads `Simulation/SO101/so101_new_calib.urdf`
160192 and its meshes (`--model <dir>`, or the repo layout, or `model/` beside the
161193 package) and runs every target through the arm's geometry: a pose that puts
Software/so101/bench/fake-serve.ts +15 −2
@@ -3,12 +3,15 @@
33 * node bench/fake-serve.ts [port] then open http://127.0.0.1:8102/?stream=<a camera stream url>
44 * Goals become positions at once, so every move "arrives"; poses and recordings go to a temp dir.
55 * PORT, HOST and FORGE_APP_DATA override the port, the 127.0.0.1 bind and the temp dir (how the forge runs it).
6 * ARM_SERVER=ws://<host>:8101 puts a real arm's `so101 serve` behind the same page: the relay in src/relay.ts
7 * sends the arm API, /follow and the cameras there while it answers, and to the simulation when it does not.
68 */
79 import { mkdirSync, mkdtempSync } from "node:fs";
810 import { tmpdir } from "node:os";
911 import { join } from "node:path";
1012 import { FeetechBus } from "../src/bus.ts";
1113 import { FakeBus } from "../src/fake-bus.ts";
14 import { parseRemote, startRelay, type Remote } from "../src/relay.ts";
1215 import { startServer } from "../src/server.ts";
1316
1417 const fake = new FakeBus([1, 2, 3, 4, 5, 6]);
@@ -22,5 +25,15 @@ const dir = appData ? join(appData, "so101") : mkdtempSync(join(tmpdir(), "so101
2225 mkdirSync(dir, { recursive: true });
2326 const port = Number(process.env.PORT ?? process.argv[2] ?? 8102);
2427 const host = process.env.HOST ?? "127.0.0.1";
25 const server = await startServer(new FeetechBus(fake), { host, port, posesFile: join(dir, "poses.json"), modelDir: new URL("../../../Simulation/SO101", import.meta.url).pathname });
26 console.log(`fake arm on http://${host}:${server.port} (data in ${dir})`);
28 const simulation = await startServer(new FeetechBus(fake), { host: "127.0.0.1", port: 0, posesFile: join(dir, "poses.json"), modelDir: new URL("../../../Simulation/SO101", import.meta.url).pathname });
29 let remote: Remote | null = null;
30 let problem: string | null = null;
31 try {
32 remote = parseRemote(process.env);
33 } catch (err) {
34 problem = (err as Error).message;
35 console.error(`${problem}; simulating`);
36 }
37 const relay = await startRelay({ host, port, simulation: `http://127.0.0.1:${simulation.port}`, remote, problem });
38 const { mode, host: armHost } = relay.link();
39 console.log(`arm on http://${host}:${relay.port} (${mode === "live" ? `live arm ${armHost}` : "simulated"}; data in ${dir})`);
Software/so101/src/fake-serve.test.ts +1 −1
@@ -13,7 +13,7 @@ afterEach(() => {
1313 child = dir = undefined;
1414 });
1515
16 /** Start the fake arm with this environment and wait for its "fake arm on http://host:port" line. */
16 /** Start the fake arm with this environment and wait for its "arm on http://host:port" line. */
1717 async function start(env: Record<string, string>): Promise<{ host: string; port: number; line: string }> {
1818 child = Bun.spawn(["bun", "run", script], { env: { ...process.env, ...env }, stdout: "pipe", stderr: "inherit" });
1919 const reader = (child.stdout as ReadableStream<Uint8Array>).getReader();
Software/so101/src/relay.test.ts added +174
@@ -0,0 +1,174 @@
1 import { afterAll, afterEach, beforeAll, describe, expect, test } from "bun:test";
2 import { mkdtempSync } from "node:fs";
3 import { createServer, type IncomingMessage, type Server } from "node:http";
4 import { createServer as createTcpServer, type Server as TcpServer } from "node:net";
5 import { tmpdir } from "node:os";
6 import { join } from "node:path";
7 import WebSocket from "ws";
8 import { FeetechBus } from "./bus.ts";
9 import { FakeBus } from "./fake-bus.ts";
10 import { parseRemote, sameOrigin, startRelay, type Remote, type RunningRelay } from "./relay.ts";
11 import { startServer, type RunningServer } from "./server.ts";
12
13 const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
14 // Bun's node:http can keep a server-closed connection from ending (see server.test.ts): cap every close.
15 const shut = (s: { close(): Promise<void> } | undefined) => (s ? Promise.race([s.close(), sleep(500)]) : undefined);
16 const listening = (server: Server | TcpServer) => new Promise<number>((r) => server.listen(0, "127.0.0.1", () => r((server.address() as { port: number }).port)));
17
18 /** An arm server on a fake bus; `ids` tells the simulation (six joints) and the "real arm" (three) apart. */
19 async function arm(ids: number[]): Promise<RunningServer> {
20 const fake = new FakeBus(ids);
21 return startServer(new FeetechBus(fake), { host: "127.0.0.1", port: 0, ids, posesFile: join(mkdtempSync(join(tmpdir(), "so101-relay-")), "poses.json"), watchdogMs: 60 });
22 }
23 const remoteAt = (port: number, camera?: number): Remote => parseRemote({ ARM_SERVER: `ws://127.0.0.1:${port}`, ...(camera ? { ARM_CAMERA_1: `http://127.0.0.1:${camera}/stream` } : {}) })!;
24 const JPEG = Buffer.from([0xff, 0xd8, 0x4a, 0x50, 0x45, 0x47, 0xff, 0xd9]);
25 const joints = async (base: string) => ((await (await fetch(`${base}/status`)).json()) as { joints: unknown[] }).joints.length;
26 const link = async (base: string) => (await (await fetch(`${base}/link`)).json()) as { mode: string; host: string | null; problem: string | null };
27
28 let simulation: RunningServer;
29 let simBase: string;
30 beforeAll(async () => {
31 simulation = await arm([1, 2, 3, 4, 5, 6]);
32 simBase = `http://127.0.0.1:${simulation.port}`;
33 });
34 afterAll(() => shut(simulation));
35
36 describe("ARM_SERVER", () => {
37 test("unset is no remote; ws, http and https name the server and the cameras beside it", () => {
38 expect(parseRemote({})).toBeNull();
39 expect(parseRemote({ ARM_SERVER: " " })).toBeNull();
40 expect(parseRemote({ ARM_SERVER: "ws://arm.tailnet:8101" })).toEqual({
41 http: "http://arm.tailnet:8101",
42 ws: "ws://arm.tailnet:8101",
43 host: "arm.tailnet:8101",
44 cameras: ["http://arm.tailnet:8080/stream", "http://arm.tailnet:8081/stream"],
45 });
46 expect(parseRemote({ ARM_SERVER: "https://arm.example/so101/" })).toMatchObject({ http: "https://arm.example/so101", ws: "wss://arm.example/so101", host: "arm.example" });
47 expect(parseRemote({ ARM_SERVER: "100.64.0.7:8101" })?.http).toBe("http://100.64.0.7:8101");
48 expect(parseRemote({ ARM_SERVER: "ws://pi:8101", ARM_CAMERA_2: "https://cams.example/two/stream" })?.cameras[1]).toBe("https://cams.example/two/stream");
49 });
50
51 test("anything else is refused with a reason", () => {
52 expect(() => parseRemote({ ARM_SERVER: "ftp://pi:8101" })).toThrow(/must start with/);
53 expect(() => parseRemote({ ARM_SERVER: "ws://user:pw@pi:8101" })).toThrow(/nothing else/);
54 expect(() => parseRemote({ ARM_SERVER: "http://pi:8101/?x=1" })).toThrow(/nothing else/);
55 expect(() => parseRemote({ ARM_SERVER: "http://" })).toThrow(/not a URL/);
56 expect(() => parseRemote({ ARM_SERVER: "ws://pi:8101", ARM_CAMERA_1: "ws://pi:8080" })).toThrow(/ARM_CAMERA_1/);
57 });
58 });
59
60 describe("same origin", () => {
61 const req = (headers: Record<string, string>) => ({ headers }) as unknown as IncomingMessage;
62 test("a page on another site may not drive the arm", () => {
63 expect(sameOrigin(req({}))).toBe(true);
64 expect(sameOrigin(req({ "sec-fetch-site": "same-origin" }))).toBe(true);
65 expect(sameOrigin(req({ "sec-fetch-site": "cross-site" }))).toBe(false);
66 expect(sameOrigin(req({ origin: "https://arm.forge", host: "arm.forge" }))).toBe(true);
67 expect(sameOrigin(req({ origin: "https://arm.forge", host: "127.0.0.1:9", "x-forwarded-host": "arm.forge" }))).toBe(true);
68 expect(sameOrigin(req({ origin: "https://evil.example", host: "arm.forge" }))).toBe(false);
69 });
70 });
71
72 describe("relay", () => {
73 let relay: RunningRelay | undefined;
74 let remote: RunningServer | undefined;
75 let camera: Server | undefined;
76 let hang: TcpServer | undefined;
77 afterEach(async () => {
78 await shut(relay);
79 await shut(remote);
80 camera?.closeAllConnections();
81 camera?.close();
82 hang?.close();
83 relay = remote = camera = hang = undefined;
84 });
85
86 test("unset: everything is the simulation, and the page says so", async () => {
87 relay = await startRelay({ port: 0, simulation: simBase });
88 const base = `http://127.0.0.1:${relay.port}`;
89 expect(await link(base)).toEqual({ mode: "simulated", host: null, problem: null });
90 expect(await joints(base)).toBe(6);
91 const page = await fetch(`${base}/`);
92 expect(page.status).toBe(200);
93 expect(await page.text()).toContain("<title>SO-101</title>");
94 expect(page.headers.get("x-content-type-options")).toBe("nosniff");
95 expect(page.headers.get("content-security-policy")).toContain("frame-ancestors 'none'");
96 expect((await fetch(`${base}/camera/1/stream`)).status).toBe(404);
97 });
98
99 test("set: status, poses and /follow go to the arm server; the camera's bytes come through", async () => {
100 remote = await arm([1, 2, 6]);
101 camera = createServer((req, res) => {
102 if (req.url?.startsWith("/state")) return res.writeHead(200, { "content-type": "application/json" }).end('{"ok":true}');
103 res.writeHead(200, { "content-type": "multipart/x-mixed-replace;boundary=frame" });
104 res.write(Buffer.concat([Buffer.from("--frame\r\ncontent-type: image/jpeg\r\n\r\n"), JPEG])); // and the stream stays open, like µStreamer
105 });
106 const cameraPort = await listening(camera);
107 relay = await startRelay({ port: 0, simulation: simBase, remote: remoteAt(remote.port, cameraPort) });
108 const base = `http://127.0.0.1:${relay.port}`;
109 expect(await link(base)).toEqual({ mode: "live", host: `127.0.0.1:${remote.port}`, problem: null });
110 expect(await joints(base)).toBe(3);
111
112 const saved = await fetch(`${base}/pose/save`, { method: "POST", body: JSON.stringify({ name: "real" }) });
113 expect(saved.status).toBe(200);
114 expect(Object.keys((await (await fetch(`http://127.0.0.1:${remote.port}/poses`)).json()) as object)).toEqual(["real"]);
115 expect(Object.keys((await (await fetch(`${simBase}/poses`)).json()) as object)).not.toContain("real");
116
117 const ws = new WebSocket(`ws://127.0.0.1:${relay.port}/follow`);
118 await new Promise((r, j) => (ws.once("open", r), ws.once("error", j)));
119 await sleep(50);
120 const status = (await (await fetch(`http://127.0.0.1:${remote.port}/status`)).json()) as { driving: string | null };
121 expect(status.driving).toBe("stream"); // the remote arm took the stream, not the simulation
122 ws.close();
123
124 const stream = await fetch(`${base}/camera/1/stream`);
125 expect(stream.headers.get("content-type")).toBe("multipart/x-mixed-replace;boundary=frame");
126 const reader = stream.body!.getReader();
127 const { value } = await reader.read();
128 expect(Buffer.from(value!).includes(JPEG)).toBe(true);
129 await reader.cancel();
130 expect(await (await fetch(`${base}/camera/1/state`)).json()).toEqual({ ok: true });
131 });
132
133 test("a mutation from another site is refused before it reaches the arm", async () => {
134 remote = await arm([1, 2, 6]);
135 relay = await startRelay({ port: 0, simulation: simBase, remote: remoteAt(remote.port) });
136 const refused = await fetch(`http://127.0.0.1:${relay.port}/torque`, { method: "POST", headers: { origin: "https://evil.example" }, body: '{"on":false}' });
137 expect(refused.status).toBe(403);
138 });
139
140 test("unreachable: the simulation answers within 3 s and the page is told why", async () => {
141 hang = createTcpServer(() => {}); // accepts and never answers
142 const port = await listening(hang);
143 const started = Date.now();
144 relay = await startRelay({ port: 0, simulation: simBase, remote: remoteAt(port) });
145 const base = `http://127.0.0.1:${relay.port}`;
146 expect(await joints(base)).toBe(6);
147 const l = await link(base);
148 expect(Date.now() - started).toBeLessThan(3000);
149 expect(l.mode).toBe("simulated");
150 expect(l.host).toBe(`127.0.0.1:${port}`);
151 expect(l.problem).toContain("unreachable");
152 });
153
154 test("an arm server that goes away mid-session falls back on the next request, /follow included", async () => {
155 remote = await arm([1, 2, 6]);
156 relay = await startRelay({ port: 0, simulation: simBase, remote: remoteAt(remote.port), probeEveryMs: 60_000 });
157 const base = `http://127.0.0.1:${relay.port}`;
158 expect(await joints(base)).toBe(3);
159 await shut(remote);
160 remote = undefined;
161 const started = Date.now();
162 expect(await joints(base)).toBe(6);
163 expect(Date.now() - started).toBeLessThan(3000);
164 expect((await link(base)).mode).toBe("simulated");
165 const ws = new WebSocket(`ws://127.0.0.1:${relay.port}/follow`);
166 await new Promise((r, j) => (ws.once("open", r), ws.once("error", j)));
167 ws.close();
168 });
169
170 test("a bad setting is reported and the arm is simulated", async () => {
171 relay = await startRelay({ port: 0, simulation: simBase, problem: 'ARM_SERVER "ftp://x" must start with http://' });
172 expect(await link(`http://127.0.0.1:${relay.port}`)).toEqual({ mode: "simulated", host: null, problem: 'ARM_SERVER "ftp://x" must start with http://' });
173 });
174 });
Software/so101/src/relay.ts added +323
@@ -0,0 +1,323 @@
1 /**
2 * The hosted app's front door: one origin for the browser, whichever arm stands behind it.
3 *
4 * With ARM_SERVER unset, everything goes to the simulated arm. With it set (ws://<host>:8101, http://…, https://…),
5 * the arm API, the WebSocket /follow and the cameras go to that `so101 serve`, as long as it answers; when it does
6 * not, the relay says so on /link and answers from the simulation instead, so the page never hangs on a dead link.
7 * The page itself, the 3D model and the vendored scripts always come from here.
8 *
9 * GET /link {"mode": "live"|"simulated", "host": "<host:port>"|null, "problem": "…"|null}
10 * GET /camera/<1|2>/<stream|state|snapshot> the arm machine's camera (µStreamer on :8080 and :8081 by default)
11 */
12
13 import { createServer, type IncomingMessage, type ServerResponse } from "node:http";
14 import type { Duplex } from "node:stream";
15 import { Readable } from "node:stream";
16 import { pipeline } from "node:stream/promises";
17 import WebSocket, { WebSocketServer } from "ws";
18
19 export interface Remote {
20 /** The arm server's HTTP base, no trailing slash. */
21 http: string;
22 /** Its WebSocket base. */
23 ws: string;
24 /** host:port, for the page and the logs. */
25 host: string;
26 /** Camera 1 and 2 stream URLs. */
27 cameras: string[];
28 }
29
30 export interface Link {
31 mode: "live" | "simulated";
32 /** The arm server the setting names, live or not. */
33 host: string | null;
34 /** Why a configured arm server is not in use right now. */
35 problem: string | null;
36 }
37
38 export interface RelayOptions {
39 host?: string;
40 port?: number;
41 /** The simulated arm's HTTP base, e.g. http://127.0.0.1:41234. */
42 simulation: string;
43 remote?: Remote | null;
44 /** A setting that could not be used (shown on /link while simulating). */
45 problem?: string | null;
46 /** How long the arm server has to answer /status before it counts as unreachable. */
47 probeTimeoutMs?: number;
48 /** How often the arm server is checked, whichever way the last check went. */
49 probeEveryMs?: number;
50 /** How long a read through to the arm server may wait for its answer before the simulation answers instead. */
51 readTimeoutMs?: number;
52 }
53
54 export interface RunningRelay {
55 readonly port: number;
56 link(): Link;
57 close(): Promise<void>;
58 }
59
60 const SCHEMES: Record<string, "http:" | "https:"> = { "http:": "http:", "https:": "https:", "ws:": "http:", "wss:": "https:" };
61 const MAX_BODY = 1_000_000;
62 /** Served from this process whichever arm is live: they describe the app, not the arm. */
63 const LOCAL = /^\/(?:index\.html)?$|^\/(?:model|vendor|printables)(?:\/|$)/;
64 const CAMERA = /^\/camera\/([12])\/(stream|state|snapshot)$/;
65 const FLOOR = {
66 "x-content-type-options": "nosniff",
67 "referrer-policy": "same-origin",
68 "content-security-policy": "frame-ancestors 'none'; base-uri 'none'; object-src 'none'; form-action 'self'",
69 };
70
71 const absolute = (text: string, what: string, schemes: Record<string, string>): URL => {
72 let url: URL;
73 try {
74 url = new URL(/^[a-z][a-z0-9+.-]*:\/\//i.test(text) ? text : `http://${text}`);
75 } catch {
76 throw new Error(`${what} "${text}" is not a URL`);
77 }
78 if (!Object.hasOwn(schemes, url.protocol)) throw new Error(`${what} must start with ${Object.keys(schemes).map((s) => `${s}//`).join(", ")}`);
79 if (url.username || url.password || url.search || url.hash) throw new Error(`${what} takes a scheme, a host, a port and a path, nothing else`);
80 return url;
81 };
82
83 /** The arm server named by ARM_SERVER (and the cameras, ARM_CAMERA_1 / ARM_CAMERA_2), or null when unset. Throws on a bad value. */
84 export function parseRemote(env: Record<string, string | undefined>): Remote | null {
85 const text = env.ARM_SERVER?.trim();
86 if (!text) return null;
87 const url = absolute(text, "ARM_SERVER", SCHEMES);
88 const scheme = SCHEMES[url.protocol]!;
89 const path = url.pathname.replace(/\/+$/, "");
90 const cameras = [1, 2].map((n) => {
91 const override = env[`ARM_CAMERA_${n}`]?.trim();
92 return override ? absolute(override, `ARM_CAMERA_${n}`, { "http:": "", "https:": "" }).href : `http://${url.hostname}:${8079 + n}/stream`;
93 });
94 return { http: `${scheme}//${url.host}${path}`, ws: `${scheme === "https:" ? "wss:" : "ws:"}//${url.host}${path}`, host: url.host, cameras };
95 }
96
97 /** A mutation or a stream is only taken from this page's own origin; requests with no browser markers (curl, the CLI) pass. */
98 export function sameOrigin(req: IncomingMessage): boolean {
99 const site = req.headers["sec-fetch-site"];
100 if (typeof site === "string") return site === "same-origin" || site === "none";
101 const origin = req.headers.origin;
102 if (!origin) return true;
103 const host = String(req.headers["x-forwarded-host"] ?? req.headers.host ?? "").split(",")[0]!.trim();
104 try {
105 return new URL(origin).host === host;
106 } catch {
107 return false;
108 }
109 }
110
111 const reason = (err: unknown) => {
112 const e = err as { message?: string; cause?: { code?: string }; code?: string };
113 return e?.cause?.code ?? e?.code ?? e?.message ?? String(err);
114 };
115
116 export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
117 const remote = options.remote ?? null;
118 const simulation = options.simulation.replace(/\/+$/, "");
119 const simulationWs = simulation.replace(/^http/, "ws");
120 const probeTimeoutMs = options.probeTimeoutMs ?? 1500;
121 const probeEveryMs = options.probeEveryMs ?? 5000;
122 const readTimeoutMs = options.readTimeoutMs ?? 2500;
123
124 // ------------------------------------------------------------------ the link
125 let live = false;
126 let problem: string | null = options.problem ?? null;
127 /** Requests in flight to the arm server: cut when it is found gone, so nothing waits on a dead link. */
128 const inFlight = new Set<AbortController>();
129 const up = () => {
130 if (!live && remote) console.log(`arm server ${remote.host}: live`);
131 live = true;
132 problem = null;
133 };
134 const down = (why: string) => {
135 if (!remote) return;
136 if (live || problem === null) console.error(`arm server ${remote.host}: ${why}; simulating`);
137 live = false;
138 problem = `${remote.host} unreachable (${why})`;
139 for (const c of inFlight) c.abort(new Error(why));
140 };
141 let probing: Promise<void> | null = null;
142 const probe = () =>
143 (probing ??= (async () => {
144 try {
145 const r = await fetch(`${remote!.http}/status`, { signal: AbortSignal.timeout(probeTimeoutMs), cache: "no-store" });
146 await r.body?.cancel();
147 if (r.ok) up();
148 else down(`/status answered ${r.status}`);
149 } catch (err) {
150 down((err as Error)?.name === "TimeoutError" ? `no answer in ${probeTimeoutMs} ms` : reason(err));
151 } finally {
152 probing = null;
153 }
154 })());
155 const link = (): Link => ({ mode: live ? "live" : "simulated", host: remote?.host ?? null, problem });
156
157 // ------------------------------------------------------------------ HTTP
158 const json = (res: ServerResponse, status: number, body: unknown) => {
159 res.writeHead(status, { "content-type": "application/json", "cache-control": "no-store" });
160 res.end(JSON.stringify(body));
161 };
162 const readBody = (req: IncomingMessage) =>
163 new Promise<Buffer>((resolve, reject) => {
164 const chunks: Buffer[] = [];
165 let size = 0;
166 req.on("data", (c: Buffer) => {
167 size += c.length;
168 if (size > MAX_BODY) {
169 req.destroy();
170 reject(new Error("body too large"));
171 } else chunks.push(c);
172 });
173 req.on("end", () => resolve(Buffer.concat(chunks)));
174 req.on("error", reject);
175 });
176 /** Ask one upstream; resolves once its headers are in. The controller stays live for the body. */
177 const ask = async (url: string, req: IncomingMessage, body: Buffer | undefined, timeoutMs: number | null, tracked: boolean) => {
178 const controller = new AbortController();
179 const timer = timeoutMs === null ? undefined : setTimeout(() => controller.abort(new Error(`no answer in ${timeoutMs} ms`)), timeoutMs);
180 if (tracked) inFlight.add(controller);
181 const headers: Record<string, string> = {};
182 for (const name of ["content-type", "accept", "range"]) if (typeof req.headers[name] === "string") headers[name] = req.headers[name];
183 try {
184 const response = await fetch(url, { method: req.method, headers, body: body?.length ? body : undefined, signal: controller.signal, redirect: "manual", cache: "no-store" });
185 return { response, controller };
186 } catch (err) {
187 inFlight.delete(controller);
188 throw controller.signal.aborted ? controller.signal.reason : err;
189 } finally {
190 clearTimeout(timer);
191 }
192 };
193 /** Hand an upstream answer to the browser, streaming (an MJPEG stream never ends), and let go of it when either side leaves. */
194 const pass = async (res: ServerResponse, { response, controller }: Awaited<ReturnType<typeof ask>>) => {
195 const headers: Record<string, string> = { "cache-control": response.headers.get("cache-control") ?? "no-store" };
196 for (const name of ["content-type", "content-length", "content-range", "accept-ranges", "last-modified", "etag"]) {
197 const value = response.headers.get(name);
198 if (value) headers[name] = value;
199 }
200 res.writeHead(response.status, headers);
201 res.on("close", () => controller.abort());
202 try {
203 if (response.body) await pipeline(Readable.fromWeb(response.body as import("node:stream/web").ReadableStream), res);
204 else res.end();
205 } catch {
206 res.destroy();
207 } finally {
208 inFlight.delete(controller);
209 }
210 };
211
212 const http = createServer(async (req, res) => {
213 for (const [name, value] of Object.entries(FLOOR)) res.setHeader(name, value);
214 try {
215 const url = new URL(req.url ?? "/", "http://x");
216 const path = url.pathname + url.search;
217 const reading = req.method === "GET" || req.method === "HEAD";
218 if (!reading && !sameOrigin(req)) return json(res, 403, { error: "only this page may change the arm" });
219 if (req.method === "GET" && url.pathname === "/link") return json(res, 200, link());
220
221 const camera = CAMERA.exec(url.pathname);
222 if (camera) {
223 if (!reading) return json(res, 405, { error: "cameras are read-only" });
224 if (!remote || !live) return json(res, 404, { error: "no camera: the arm is simulated" });
225 const stream = remote.cameras[Number(camera[1]) - 1]!;
226 const target = stream.replace(/\/[^/]*$/, `/${camera[2]}`) + url.search;
227 try {
228 return await pass(res, await ask(target, req, undefined, readTimeoutMs, true));
229 } catch (err) {
230 return json(res, 502, { error: `camera ${camera[1]} did not answer: ${reason(err)}` });
231 }
232 }
233
234 const body = reading ? undefined : await readBody(req);
235 if (LOCAL.test(url.pathname) || !live || !remote) return await pass(res, await ask(simulation + path, req, body, null, false));
236 try {
237 // A read must answer fast or fall back; a move or a playback answers when the arm arrives, and is only cut when the link is.
238 return await pass(res, await ask(remote.http + path, req, body, reading ? readTimeoutMs : null, true));
239 } catch (err) {
240 down(reason(err));
241 if (!reading) return json(res, 502, { error: `the arm server at ${remote.host} did not answer (${reason(err)}); the page is now on the simulation` });
242 return await pass(res, await ask(simulation + path, req, body, null, false));
243 }
244 } catch (err) {
245 if (!res.headersSent) json(res, 502, { error: reason(err) });
246 else res.destroy();
247 }
248 });
249
250 // ------------------------------------------------------------------ WebSocket /follow
251 const wss = new WebSocketServer({ noServer: true });
252 const open = (url: string, timeoutMs: number) =>
253 new Promise<WebSocket>((resolve, reject) => {
254 const ws = new WebSocket(url);
255 const timer = setTimeout(() => {
256 ws.terminate();
257 reject(new Error(`no answer in ${timeoutMs} ms`));
258 }, timeoutMs);
259 ws.once("open", () => {
260 clearTimeout(timer);
261 resolve(ws);
262 });
263 ws.once("error", (err) => {
264 clearTimeout(timer);
265 reject(err);
266 });
267 });
268 /** Close codes a server may send on; the rest (1005, 1006, 1015) only describe a connection that died. */
269 const sendable = (code: number) => (code >= 1000 && code <= 1003) || (code >= 1007 && code <= 1014) || (code >= 3000 && code <= 4999);
270 const bridge = (browser: WebSocket, arm: WebSocket) => {
271 arm.on("message", (data, binary) => browser.readyState === WebSocket.OPEN && browser.send(data, { binary }));
272 browser.on("message", (data, binary) => arm.readyState === WebSocket.OPEN && arm.send(data, { binary }));
273 arm.on("close", (code, why) => browser.close(sendable(code) ? code : 1011, why.length ? why.toString() : "the arm server went away"));
274 browser.on("close", () => arm.close());
275 arm.on("error", () => browser.terminate());
276 browser.on("error", () => arm.terminate());
277 };
278 const refuse = (socket: Duplex, status: string) => {
279 socket.end(`HTTP/1.1 ${status}\r\nconnection: close\r\ncontent-length: 0\r\n\r\n`);
280 };
281 http.on("upgrade", (req: IncomingMessage, socket: Duplex, head: Buffer) => {
282 const url = new URL(req.url ?? "/", "http://x");
283 if (url.pathname !== "/follow") return refuse(socket, "404 Not Found");
284 if (!sameOrigin(req)) return refuse(socket, "403 Forbidden");
285 socket.on("error", () => socket.destroy());
286 void (async () => {
287 let arm: WebSocket;
288 try {
289 if (!remote || !live) throw null;
290 arm = await open(`${remote.ws}/follow`, readTimeoutMs);
291 } catch (err) {
292 if (err) down(reason(err));
293 try {
294 arm = await open(`${simulationWs}/follow`, readTimeoutMs);
295 } catch {
296 return refuse(socket, "502 Bad Gateway");
297 }
298 }
299 if (socket.destroyed) return arm.close();
300 wss.handleUpgrade(req, socket, head, (browser) => bridge(browser, arm));
301 })();
302 });
303
304 // ------------------------------------------------------------------ start
305 if (remote) await probe(); // bounded by the probe timeout: the first page already knows which arm it has
306 const timer = remote ? setInterval(() => void probe(), probeEveryMs) : undefined;
307 timer?.unref?.();
308 await new Promise<void>((resolve) => http.listen(options.port ?? 8102, options.host ?? "127.0.0.1", resolve));
309 const address = http.address();
310 return {
311 port: typeof address === "object" && address ? address.port : options.port ?? 8102,
312 link,
313 close: () =>
314 new Promise<void>((resolve) => {
315 clearInterval(timer);
316 for (const c of inFlight) c.abort();
317 for (const client of wss.clients) client.terminate();
318 wss.close();
319 http.close(() => resolve());
320 http.closeAllConnections();
321 }),
322 };
323 }
Software/so101/src/viewer.html +23 −2
@@ -28,6 +28,7 @@
2828 @media (max-aspect-ratio: 1/1) { #stage { flex-direction: column; } }
2929 #top { position: absolute; top: 8px; right: 10px; z-index: 5; display: flex; gap: 8px; align-items: center; font-size: 13px; background: rgba(0,0,0,.65); padding: 5px 6px 5px 10px; border-radius: 8px; flex-wrap: wrap; justify-content: flex-end; }
3030 #top .seg button { padding: 5px 9px; min-height: 30px; } #top .seg button.on { background: var(--blue); border-color: var(--blue); }
31 #link { color: var(--dim); white-space: nowrap; overflow: hidden; text-overflow: ellipsis; max-width: min(46ch, 60vw); } #link.on { color: var(--ok); } #link.warn { color: var(--warn); }
3132 .dot { width: 8px; height: 8px; border-radius: 50%; background: var(--bad); display: inline-block; } .dot.on { background: var(--ok); }
3233 #banner { position: absolute; top: 54px; left: 50%; transform: translateX(-50%); z-index: 5; max-width: min(640px, 90vw); padding: 8px 14px; border-radius: 8px; font-size: 13px; background: rgba(0,0,0,.75); border: 1px solid var(--line); display: none; }
3334 #banner.bad { display: block; border-color: var(--bad); color: #ffb3ad; } #banner.warn { display: block; border-color: var(--warn); color: #f2cc60; }
@@ -86,7 +87,7 @@
8687 </div>
8788 <div id="top">
8889 <span class="seg"><button id="v1" class="on">Cam 1</button><button id="v2" hidden>Cam 2</button><button id="v3" class="on">3D</button></span>
89 <span id="live"></span><span class="dot" id="dot"></span><span id="state">…</span>
90 <span id="link" hidden></span><span id="live"></span><span class="dot" id="dot"></span><span id="state">…</span>
9091 <button id="zoneBadge" title="safe zone settings">zone</button>
9192 <button id="stop" class="danger" hidden title="space">Stop</button><button id="fs" title="F">Fullscreen</button>
9293 </div>
@@ -210,9 +211,28 @@
210211 const NAMES = { 1: "shoulder_pan", 2: "shoulder_lift", 3: "elbow_flex", 4: "wrist_flex", 5: "wrist_roll", 6: "gripper" };
211212 const IDS = Object.fromEntries(Object.entries(NAMES).map(([id, n]) => [n, Number(id)]));
212213
214 // ---------------------------------------------------------------- which arm
215 // Behind the hosted app's relay (/link answers), the arm is a live `so101 serve` or the simulation, and the cameras come through the relay.
216 const relayed = await fetch("/link", { cache: "no-store" }).then((r) => r.ok, () => false);
217 let linkMode = null;
218 async function pollLink() {
219 const l = await api("/link").catch(() => null);
220 if (l?.ok) {
221 const el = $("link");
222 el.hidden = false;
223 el.textContent = l.mode === "live" ? `Live arm · ${l.host}` : l.problem ? `Simulated · ${l.problem}` : "Simulated";
224 el.title = el.textContent;
225 el.className = l.mode === "live" ? "on" : l.problem ? "warn" : "";
226 // Another arm has other limits, poses and joints: start the page over on it, once nothing on the page is driving.
227 if (linkMode !== null && l.mode !== linkMode && !stream.ws && !live) return location.reload();
228 if (linkMode === null) linkMode = l.mode;
229 }
230 setTimeout(pollLink, 3000);
231 }
232
213233 // ---------------------------------------------------------------- cameras
214234 // Plain <img> streams work in every browser; a canvas comparison once a second notices a frozen picture and reconnects.
215 const camUrl = (n) => q.get(`stream${n}`) || q.get(n === 1 ? "stream" : "") || `${location.protocol}//${location.hostname}:${8079 + n}/stream`;
235 const camUrl = (n) => q.get(`stream${n}`) || q.get(n === 1 ? "stream" : "") || (relayed ? `/camera/${n}/stream` : `${location.protocol}//${location.hostname}:${8079 + n}/stream`);
216236 const cams = [1, 2].map((n) => ({ n, img: $(`cam${n}`), url: camUrl(n), sig: "", unchanged: 0, live: false }));
217237 const canvas = document.createElement("canvas"); canvas.width = 32; canvas.height = 18;
218238 const ctx = canvas.getContext("2d", { willReadFrequently: true });
@@ -749,6 +769,7 @@
749769 if (wanted.length) for (const [btn, pane] of Object.entries(panes)) { const on = wanted.includes(pane); $(btn).classList.toggle("on", on); if (pane === "pane3") $(pane).classList.toggle("on", on); else showCam(cams[pane === "pane1" ? 0 : 1], on); }
750770 } catch {}
751771 poll();
772 if (relayed) pollLink();
752773 })();
753774 </script>
754775 </body>

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs