Public: anyone can read this; the crew can change it.

Migrated from github.com/Trevato/SO-ARM100

workbenchthis machine

Drive with a leader arm: plug it into a computer (or Android phone) and the browser streams its pose as your commands during your turn #29

Merged opened this in answer to

Check · Review · Land done · Live — Landed; nothing deploys from this branch, so this is the end of its path.

Merged

quill/arm:quill/18-drive-with-a-leader-arm-plug-it-into-a-c → main 3 commits fork

  • Merged. quill/18-drive-with-a-leader-arm-plug-it-into-a-c was merged into main as 056e57a.

Review

Sign in to review this change.
10 files changed +669 −210
Software/README.md +37 −8
@@ -188,6 +188,34 @@ error; it is not replayed on the simulation. Changes (`POST`, `PUT`, the
188188 stream) are only accepted from the app's own page, so another site can't
189189 drive the arm through it.
190190
191 **Drive with your own leader arm.** Plug an SO-101 leader (its
192 bus board on USB, and on its own power supply) into the computer or Android
193 phone you are watching from, and press **Connect leader arm**, then pick the
194 board from the list. Chrome or Edge on a computer can open its serial port
195 from the page (Web Serial). So can Chrome on Android, on versions that offer
196 Web Serial, through a USB-C OTG adapter. The page checks whether the browser
197 has Web Serial, not which browser it is.
198
199 - Before your turn, the 3D model and the joint rows mirror the leader
200 (`leader 2048` beside each joint, in follower ticks), so you can line it up
201 with the arm while you wait.
202 - During your turn, **Drive with leader** streams its pose at 60 Hz over the
203 same `/follow` stream the sliders use. The arm first eases over to the
204 leader, and the server applies the same limits, safe zone and jam
205 watch as to any other stream. **■ Stop leader**, **Hand back**, or the end of
206 your turn stop it, and the arm holds.
207 - Frames that arrive after your turn ends are dropped at the relay, even
208 before your stream has closed.
209 - iPhone and iPad browsers can't open USB ports (WebKit has no Web Serial).
210 The button explains this there, as on Firefox and desktop Safari, and points
211 to the sliders, the − and + buttons, and dragging the 3D model.
212
213 The page reads the leader with the CLI's own bus code: `src/leader-arm.ts`
214 puts `FeetechBus` on a Web Serial port, and the server hands it and the
215 modules it imports to the page at `/app/<file>`, with the types stripped
216 (by Bun's transpiler, or by Node's own on Node 22.13 and later). Nothing is
217 bundled, and no other file is served.
218
191219 **Watch, take turns, chat.** Anyone with the link can watch: the 3D arm,
192220 its live pose and the cameras. Driving needs a forge account and a turn.
193221
@@ -220,10 +248,9 @@ viewing needs the app set to public on the forge.
220248
221249 **Your own leader arm, calibrated once.** No two leaders read the same: a
222250 servo sits a few hundred ticks round its dial, a horn goes on the other way
223 round, a joint stops short. **Calibrate leader** (Chrome, with the leader
224 plugged into your computer) walks you through it. You don't need the turn,
225 and the real arm doesn't move. First connect the leader; its motors go limp.
226 Then hold it in the middle pose the 3D model shows. Then, joint by joint,
251 round, a joint stops short. Once a leader is connected,
252 **Calibrate leader** walks you through it. You don't need the turn, the real
253 arm doesn't move, and a leader that is driving stops first. Hold it in the middle pose the 3D model shows. Then, joint by joint,
227254 take it to each end the model shows. The last step shows the model following
228255 your leader through the new calibration, with a **Flip** per joint in case
229256 one turned the wrong way. **Save** keeps it with your forge account, so it
@@ -235,9 +262,11 @@ joint's offset (the centre lands on the middle of the follower's range), its
235262 direction (`low` above `high` means the joint is mounted backwards), and its
236263 scale, set separately on each side of the centre. A short or lopsided leader
237264 still reaches both ends of the follower, and goes no further. The gripper
238 maps closed to closed and open to open. **Live from leader** applies this in
239 the browser before a frame leaves. Without a calibration, it falls back to
240 the leader servos' stored limits. The server still clamps every frame to the
265 maps closed to closed and open to open. The connected leader maps through
266 it in the browser (`LeaderArm.targets`), for the mirror and for **Drive with
267 leader** alike. Without a calibration, it falls back to the leader servos'
268 stored ranges; a leader that has none still connects, and drives once
269 calibrated. The server still clamps every frame to the
241270 follower's limits. `GET`, `PUT` and `DELETE /leader/calibration` act on the
242271 signed-in person's own calibration. A `PUT` is validated against the same
243272 module: whole ticks only, and each end at least 100 ticks from the centre on
@@ -360,4 +389,4 @@ The methods behind the vision kit, so you can read the originals:
360389 model follows; [three.js](https://threejs.org) and
361390 [urdf-loader](https://github.com/gkjohnson/urdf-loaders) for the 3D view;
362391 the [Web Serial API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Serial_API)
363 for live mode; [µStreamer](https://github.com/pikvm/ustreamer) for the cameras.
392 for the leader arm in the browser; [µStreamer](https://github.com/pikvm/ustreamer) for the cameras.
Software/so101/src/browser-modules.ts added +43
@@ -0,0 +1,43 @@
1 /**
2 * The few modules of src/ the page imports itself, served as JavaScript at /app/<file>.ts: the leader arm over Web Serial
3 * and the bus code under it, so the browser speaks the Feetech protocol with the very code the CLI and the tests use.
4 * Types are stripped (Bun's transpiler, or Node's own on Node 22.13 and later); imports stay as they are, since each
5 * file they name is served here too. Nothing is bundled and nothing outside this list is ever read.
6 */
7
8 import { readFileSync, statSync } from "node:fs";
9 import { dirname, join } from "node:path";
10 import { fileURLToPath } from "node:url";
11
12 export const BROWSER_MODULES: ReadonlySet<string> = new Set(["leader-arm.ts", "bus.ts", "protocol.ts", "sts3215.ts", "mapping.ts", "leader-map.js"]);
13
14 const here = dirname(fileURLToPath(import.meta.url));
15 const cache = new Map<string, { stamp: string; js: string }>();
16
17 type Strip = (source: string) => string;
18 let strip: Strip | null | undefined;
19 async function stripper(): Promise<Strip | null> {
20 if (strip !== undefined) return strip;
21 const bun = (globalThis as { Bun?: { Transpiler: new (o: { loader: "ts" }) => { transformSync(code: string): string } } }).Bun;
22 if (bun) {
23 const transpiler = new bun.Transpiler({ loader: "ts" });
24 return (strip = (source) => transpiler.transformSync(source));
25 }
26 const node = (await import("node:module")) as { stripTypeScriptTypes?: (code: string, options?: { mode: "strip" }) => string };
27 return (strip = node.stripTypeScriptTypes ? (source) => node.stripTypeScriptTypes!(source, { mode: "strip" }) : null);
28 }
29
30 /** The module as JavaScript with a tag that changes when the file does, or null for a name not on the list (or no way to strip types). */
31 export async function browserModule(name: string): Promise<{ js: string; etag: string } | null> {
32 if (!BROWSER_MODULES.has(name)) return null;
33 const transform = await stripper();
34 if (!transform) return null;
35 const path = join(here, name);
36 const { mtimeMs, size } = statSync(path);
37 const stamp = `"${size.toString(36)}-${Math.floor(mtimeMs).toString(36)}"`;
38 const hit = cache.get(name);
39 if (hit?.stamp === stamp) return { js: hit.js, etag: stamp };
40 const js = transform(readFileSync(path, "utf8"));
41 cache.set(name, { stamp, js });
42 return { js, etag: stamp };
43 }
Software/so101/src/bus.ts +2 −1
@@ -1,4 +1,3 @@
1 import { SerialPort } from "serialport";
21 import {
32 BROADCAST_ID,
43 FrameParser,
@@ -97,6 +96,8 @@ export class FeetechBus {
9796 }
9897
9998 static async open(path: string, { baudRate = 1_000_000 }: OpenOptions = {}): Promise<FeetechBus> {
99 // Loaded here, not at the top: the browser runs this same module over Web Serial (src/leader-arm.ts).
100 const { SerialPort } = await import("serialport");
100101 const port = new SerialPort({ path, baudRate, autoOpen: false });
101102 await new Promise<void>((resolve, reject) => port.open((err) => (err ? reject(err) : resolve())));
102103 const bus = new FeetechBus(port);
Software/so101/src/leader-arm.test.ts added +262
@@ -0,0 +1,262 @@
1 import { afterAll, afterEach, beforeAll, describe, expect, test } from "bun:test";
2 import { mkdtempSync } from "node:fs";
3 import { tmpdir } from "node:os";
4 import { join } from "node:path";
5 import WebSocket from "ws";
6 import { BROWSER_MODULES } from "./browser-modules.ts";
7 import { FeetechBus } from "./bus.ts";
8 import { FakeBus, type FakeOptions } from "./fake-bus.ts";
9 import { sign } from "./identity.ts";
10 import { LeaderArm, leaderSupport, type SerialPortLike } from "./leader-arm.ts";
11 import { Lobby } from "./lobby.ts";
12 import { startRelay, type RunningRelay } from "./relay.ts";
13 import { startServer, type RunningServer } from "./server.ts";
14 import { STS3215 } from "./sts3215.ts";
15
16 const KEY = "test-identity-key";
17 const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
18 const shut = (s: { close(): Promise<void> } | undefined) => (s ? Promise.race([s.close(), sleep(500)]) : undefined);
19 const ALICE = (() => {
20 const identity = `u-alice|alice|user|${Date.now() + 3_600_000}`;
21 return { "x-forge-identity": identity, "x-forge-signature": sign(identity, KEY) };
22 })();
23
24 /** A Web Serial port with a leader arm's servos behind it, as Chrome hands one to the page. */
25 class FakeSerialPort implements SerialPortLike {
26 readable: ReadableStream<Uint8Array> | null = null;
27 writable: WritableStream<Uint8Array> | null = null;
28 opened: number | null = null;
29 closed = false;
30 #controller: ReadableStreamDefaultController<Uint8Array> | null = null;
31 readonly servos: FakeBus;
32 constructor(servos: FakeBus) {
33 this.servos = servos;
34 }
35 async open({ baudRate }: { baudRate: number }) {
36 this.opened = baudRate;
37 this.readable = new ReadableStream({ start: (c) => void (this.#controller = c) });
38 this.servos.on("data", (chunk: Uint8Array) => this.#controller?.enqueue(chunk));
39 this.writable = new WritableStream({ write: (chunk) => this.servos.write(chunk, () => undefined) });
40 }
41 async close() {
42 this.closed = true;
43 }
44 unplug() {
45 this.#controller?.error(new Error("The device has been lost."));
46 }
47 }
48
49 /** A calibrated leader: servo id → [min, max, present]. */
50 function leaderServos(arm: Record<number, [number, number, number]>, options: FakeOptions = {}): FakeBus {
51 const fake = new FakeBus(Object.keys(arm).map(Number), options);
52 for (const [id, [min, max]] of Object.entries(arm)) fake.servos.get(Number(id))!.set([min & 0xff, min >> 8, max & 0xff, max >> 8], 9);
53 for (const [id, [, , at]] of Object.entries(arm)) place(fake, Number(id), at);
54 return fake;
55 }
56 const place = (fake: FakeBus, id: number, ticks: number) => fake.servos.get(id)!.set([ticks & 0xff, ticks >> 8], 56);
57
58 describe("the leader arm over Web Serial", () => {
59 test("opens at 1 Mbaud, reads each range, lets the arm go limp, and reads its pose", async () => {
60 const servos = leaderServos({ 1: [1000, 3000, 2500], 2: [900, 3100, 1200], 6: [2000, 3000, 2250] });
61 const port = new FakeSerialPort(servos);
62 const leader = await LeaderArm.open(port, { ids: [1, 2, 6] });
63 expect(port.opened).toBe(1_000_000);
64 expect(Object.fromEntries(leader.limits)).toEqual({ 1: { min: 1000, max: 3000 }, 2: { min: 900, max: 3100 }, 6: { min: 2000, max: 3000 } });
65 const torqueOff = servos.writes.filter((w) => w.address === STS3215.Torque_Enable.address);
66 expect(torqueOff.map((w) => [w.id, w.bytes[0]])).toEqual([[1, 0], [2, 0], [6, 0]]);
67 expect(Object.fromEntries(await leader.pose())).toEqual({ 1: 2500, 2: 1200, 6: 2250 });
68 // Joints keep their angle from the middle of the range; the gripper maps travel onto travel.
69 const follower = { 1: { min: 0, max: 4095 }, 2: { min: 0, max: 4095 }, 6: { min: 2000, max: 3600 } };
70 expect(leader.targets(await leader.pose(), follower)).toEqual({ 1: 2548, 2: 1248, 6: 2400 });
71 await leader.close();
72 expect(port.closed).toBe(true);
73 });
74
75 test("an adapter that echoes what it sends is recognised, as the CLI's bus does", async () => {
76 const servos = leaderServos({ 1: [1000, 3000, 1800] }, { echo: true });
77 const leader = await LeaderArm.open(new FakeSerialPort(servos), { ids: [1] });
78 expect(Object.fromEntries(await leader.pose())).toEqual({ 1: 1800 });
79 await leader.close();
80 });
81
82 test("a leader with no power says so, and lets go of the port", async () => {
83 const port = new FakeSerialPort(leaderServos({ 1: [1000, 3000, 2000], 2: [1000, 3000, 2000] }, { drop: new Set([2]) }));
84 await expect(LeaderArm.open(port, { ids: [1, 2] })).rejects.toThrow(/leader servo 2 did not answer: is the leader board powered/);
85 expect(port.closed).toBe(true);
86 });
87
88 test("follow reads until stopped, and ends with an error when the leader is unplugged", async () => {
89 const servos = leaderServos({ 1: [1000, 3000, 2000] });
90 const port = new FakeSerialPort(servos);
91 const leader = await LeaderArm.open(port, { ids: [1] });
92 const seen: number[] = [];
93 const stop = new AbortController();
94 const running = leader.follow((pose) => {
95 seen.push(pose.get(1)!);
96 place(servos, 1, 2000 + seen.length);
97 if (seen.length === 5) stop.abort();
98 }, { hz: 200, signal: stop.signal });
99 await running;
100 expect(seen).toEqual([2000, 2001, 2002, 2003, 2004]);
101
102 const unplugged = leader.follow(() => port.unplug(), { hz: 200 });
103 await expect(unplugged).rejects.toThrow(/device has been lost/);
104 });
105
106 test("a servo with no stored range still opens and is read, but only a calibration maps it", async () => {
107 const leader = await LeaderArm.open(new FakeSerialPort(leaderServos({ 1: [1000, 3000, 2500], 2: [0, 0, 1500] })), { ids: [1, 2] });
108 expect([...leader.limits.keys()]).toEqual([1]);
109 const pose = await leader.pose();
110 expect(Object.fromEntries(pose)).toEqual({ 1: 2500, 2: 1500 });
111 const follower = { 1: { min: 0, max: 4095 }, 2: { min: 1000, max: 3000 } };
112 expect(leader.targets(pose, follower)).toEqual({ 1: 2548 });
113 // The driver's own calibration: joint 2 reads 1500 at its centre, so it lands mid-range; joint 1 is reversed.
114 const mine = { joints: { 1: { low: 3000, centre: 2000, high: 1000 }, 2: { low: 1000, centre: 1500, high: 2000 } } };
115 expect(leader.targets(pose, follower, mine)).toEqual({ 1: 1024, 2: 2000 });
116 await leader.close();
117 });
118 });
119
120 describe("which browsers can", () => {
121 const chrome = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/140.0 Safari/537.36";
122 test("Web Serial on a secure page is all it takes", () => {
123 expect(leaderSupport({ serial: true, secure: true, userAgent: chrome })).toEqual({ ok: true });
124 expect(leaderSupport({ serial: true, secure: true, userAgent: "Mozilla/5.0 (Linux; Android 15) Chrome/140.0 Mobile" })).toEqual({ ok: true });
125 });
126 test("everyone else is told why, and what to drive with instead", () => {
127 const iphone = leaderSupport({ serial: false, secure: true, userAgent: "Mozilla/5.0 (iPhone; CPU iPhone OS 18_0 like Mac OS X) AppleWebKit/605.1.15 Version/18.0 Mobile/15E148 Safari/604.1" });
128 expect(iphone.ok).toBe(false);
129 expect(!iphone.ok && iphone.why).toContain("iPhone and iPad");
130 expect(!iphone.ok && iphone.why).toContain("sliders");
131 const ipad = leaderSupport({ serial: false, secure: true, userAgent: "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 Version/18.0 Safari/605.1.15", touchPoints: 5 });
132 expect(!ipad.ok && ipad.why).toContain("iPhone and iPad");
133 const mac = leaderSupport({ serial: false, secure: true, userAgent: "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 Version/18.0 Safari/605.1.15", touchPoints: 0 });
134 expect(!mac.ok && mac.why).toContain("Chrome or Edge on a computer");
135 const android = leaderSupport({ serial: false, secure: true, userAgent: "Mozilla/5.0 (Android 14; Mobile; rv:130.0) Gecko/130.0 Firefox/130.0" });
136 expect(!android.ok && android.why).toContain("Android browser");
137 const plain = leaderSupport({ serial: true, secure: false, userAgent: chrome });
138 expect(!plain.ok && plain.why).toContain("https");
139 });
140 });
141
142 describe("driving the follower with it", () => {
143 let simulation: RunningServer;
144 let sim: FakeBus;
145 let relay: RunningRelay | undefined;
146 let lobby: Lobby | undefined;
147 const sockets: WebSocket[] = [];
148 beforeAll(async () => {
149 sim = new FakeBus([1, 2, 6]);
150 sim.servos.get(1)!.set([1000 & 0xff, 1000 >> 8, 3000 & 0xff, 3000 >> 8], 9); // the follower's shoulder: 1000..3000
151 simulation = await startServer(new FeetechBus(sim), { host: "127.0.0.1", port: 0, ids: [1, 2, 6], posesFile: join(mkdtempSync(join(tmpdir(), "so101-leader-")), "poses.json"), watchdogMs: 2000 });
152 });
153 afterAll(() => shut(simulation));
154 afterEach(async () => {
155 for (const s of sockets.splice(0)) s.terminate();
156 await shut(relay);
157 lobby?.close();
158 relay = lobby = undefined;
159 await sleep(100); // the stream's close handler frees the arm
160 });
161 const start = async () => {
162 lobby = new Lobby({ identityKey: KEY });
163 relay = await startRelay({ port: 0, simulation: `http://127.0.0.1:${simulation.port}`, lobby });
164 lobby.turns.join({ id: "u-alice", name: "alice", owner: false });
165 return relay.port;
166 };
167 const stream = async (port: number) => {
168 const ws = new WebSocket(`ws://127.0.0.1:${port}/follow`, { headers: ALICE });
169 sockets.push(ws);
170 await new Promise((r, j) => (ws.once("open", r), ws.once("error", j)));
171 return ws;
172 };
173 const followerAt = async (port: number) => {
174 const s = (await (await fetch(`http://127.0.0.1:${port}/status`)).json()) as { joints: Array<{ id: number; position: number }> };
175 return Object.fromEntries(s.joints.map((j) => [j.id, j.position]));
176 };
177 const until = async (check: () => Promise<boolean>, ms = 3000) => {
178 const deadline = Date.now() + ms;
179 while (Date.now() < deadline) if (await check()) return true;
180 else await sleep(20);
181 return false;
182 };
183
184 test("a fake serial port drives the simulated arm, and the limits still clamp", async () => {
185 const port = await start();
186 const servos = leaderServos({ 1: [1000, 3000, 2600], 2: [1000, 3000, 2100], 6: [2000, 3000, 2500] });
187 const leader = await LeaderArm.open(new FakeSerialPort(servos), { ids: [1, 2, 6] });
188 const followerLimits = (await (await fetch(`http://127.0.0.1:${port}/limits`)).json()) as Record<string, { min: number; max: number }>;
189 const ws = await stream(port);
190 const stop = new AbortController();
191 const running = leader.follow((pose) => ws.send(JSON.stringify(leader.targets(pose, followerLimits))), { hz: 50, signal: stop.signal });
192
193 expect(await until(async () => (await followerAt(port))[1] === 2600)).toBe(true);
194 expect(await followerAt(port)).toEqual({ 1: 2600, 2: 2148, 6: 2048 }); // the gripper's 0..4095 travel, half open
195
196 // The leader near its stop: the mapping holds the shoulder 30 ticks inside the follower's range.
197 place(servos, 1, 2950);
198 expect(await until(async () => (await followerAt(port))[1] === 2950)).toBe(true);
199 place(servos, 1, 2990);
200 expect(await until(async () => (await followerAt(port))[1] === 2970)).toBe(true);
201 stop.abort();
202 await running;
203
204 // A page that skips the mapping is clamped by the server all the same.
205 ws.send(JSON.stringify({ 1: 2700, 2: 2148, 6: 2048 }));
206 expect(await until(async () => (await followerAt(port))[1] === 2700)).toBe(true);
207 ws.send(JSON.stringify({ 1: 4000, 2: 2148, 6: 2048 }));
208 await sleep(150);
209 expect((await followerAt(port))[1]).toBe(2970);
210 await leader.close();
211 });
212
213 test("frames from someone whose turn is over are dropped, even before their stream is closed", async () => {
214 const port = await start();
215 const ws = await stream(port);
216 ws.send(JSON.stringify({ 1: 2000, 2: 2048, 6: 2048 }));
217 expect(await until(async () => (await followerAt(port))[1] === 2000)).toBe(true);
218 ws.send(JSON.stringify({ 1: 2100, 2: 2048, 6: 2048 }));
219 expect(await until(async () => (await followerAt(port))[1] === 2100)).toBe(true);
220
221 // The turn is over as far as the lobby knows, and the close has not reached this page yet.
222 lobby!.turns.driver = null;
223 for (const t of [2200, 2300, 2400]) ws.send(JSON.stringify({ 1: t, 2: 2048, 6: 2048 }));
224 await sleep(200);
225 expect((await followerAt(port))[1]).toBe(2100);
226 expect(ws.readyState).toBe(WebSocket.OPEN);
227 });
228
229 test("when the turn ends the leader's stream is closed with the reason", async () => {
230 const port = await start();
231 const ws = await stream(port);
232 const closed = new Promise<string>((r) => ws.once("close", (code, why) => r(`${code} ${why}`)));
233 lobby!.turns.end("u-alice");
234 expect(await closed).toBe("1000 your turn ended");
235 });
236 });
237
238 describe("the modules the page imports", () => {
239 let server: RunningServer;
240 beforeAll(async () => {
241 server = await startServer(new FeetechBus(new FakeBus([1])), { host: "127.0.0.1", port: 0, ids: [1], posesFile: join(mkdtempSync(join(tmpdir(), "so101-app-")), "poses.json") });
242 });
243 afterAll(() => shut(server));
244 const get = (path: string, headers: Record<string, string> = {}) => fetch(`http://127.0.0.1:${server.port}${path}`, { headers });
245
246 test("are served as JavaScript that only imports each other", async () => {
247 for (const name of BROWSER_MODULES) {
248 const r = await get(`/app/${name}`);
249 expect(r.status).toBe(200);
250 expect(r.headers.get("content-type")).toStartWith("text/javascript");
251 const js = await r.text();
252 expect(js).not.toMatch(/^\s*(?:import type|export (?:type|interface)) /m);
253 for (const [, from] of js.matchAll(/^\s*(?:import|export)\s[^;]*?from\s+"([^"]+)"/gm)) expect(BROWSER_MODULES.has(from!.replace(/^\.\//, ""))).toBe(true);
254 const again = await get(`/app/${name}`, { "if-none-match": r.headers.get("etag")! });
255 expect(again.status).toBe(304);
256 }
257 });
258
259 test("and nothing else", async () => {
260 for (const path of ["/app/server.ts", "/app/../package.json", "/app/%2e%2e/package.json", "/app/identity.ts", "/app/"]) expect((await get(path)).status).toBe(404);
261 });
262 });
Software/so101/src/leader-arm.ts added +169
@@ -0,0 +1,169 @@
1 /**
2 * The leader arm, read in the browser: Web Serial hands the page the leader board's USB serial port, and the same
3 * FeetechBus the CLI uses talks to the servos over it. The page imports this module itself (served by src/browser-modules.ts),
4 * so it touches nothing of Node's and nothing of the DOM's beyond the port it is given.
5 *
6 * Opening does what `so101 leader` does: read each servo's calibrated range, then torque off, since the leader is moved by
7 * hand. After that `pose()` reads the six positions and `targets()` maps them onto the follower's ranges: through the driver's own
8 * calibration when they have one (src/leader-map.js), otherwise from the middle of each servo's stored range (src/mapping.ts).
9 * Whether a frame is sent is not decided here: the page streams only during its turn, and the relay drops anything else.
10 */
11
12 import { FeetechBus, NoResponseError, type Transport } from "./bus.ts";
13 import { mapLeaderFrame, type LeaderCalibration } from "./leader-map.js";
14 import { mapFrame, type Frame, type Range } from "./mapping.ts";
15 import { STS3215 } from "./sts3215.ts";
16
17 /** The part of a Web Serial `SerialPort` the leader needs; a test hands in a fake with the same shape. */
18 export interface SerialPortLike {
19 open(options: { baudRate: number }): Promise<void>;
20 close(): Promise<void>;
21 readonly readable: ReadableStream<Uint8Array> | null;
22 readonly writable: WritableStream<Uint8Array> | null;
23 }
24
25 export interface LeaderOptions {
26 ids?: readonly number[];
27 gripperId?: number;
28 /** STS3215s ship at 1 Mbaud; nothing here uses any other rate. */
29 baudRate?: number;
30 }
31
32 /** The USB serial chips on the Waveshare and Feetech bus boards (WCH CH340/CH343), the ports the browser's picker offers. */
33 export const BOARD_FILTERS = [{ usbVendorId: 0x1a86 }];
34
35 /** An open Web Serial port as the bus's Transport: bytes in through the readable side, out through the writable side. */
36 export function serialTransport(port: SerialPortLike, baudRate = 1_000_000): Transport & { readonly closed: boolean } {
37 if (!port.readable || !port.writable) throw new Error("the serial port is not open");
38 const reader = port.readable.getReader();
39 const writer = port.writable.getWriter();
40 const onData = new Set<(chunk: Uint8Array) => void>();
41 const onError = new Set<(error: Error) => void>();
42 let closed = false;
43 void (async () => {
44 try {
45 for (;;) {
46 const { value, done } = await reader.read();
47 if (done) break;
48 if (value?.length) for (const listener of onData) listener(value);
49 }
50 if (!closed) throw new Error("the leader's serial port closed");
51 } catch (err) {
52 if (closed) return;
53 const error = err instanceof Error ? err : new Error(String(err));
54 for (const listener of onError) listener(error); // unplugged: the bus fails what is in flight and everything after it
55 }
56 })();
57 const done = (callback: (error: Error | null | undefined) => void) => [() => callback(null), (err: unknown) => callback(err instanceof Error ? err : new Error(String(err)))] as const;
58 return {
59 path: "web-serial",
60 baudRate,
61 get closed() {
62 return closed;
63 },
64 write(data, callback) {
65 writer.write(data).then(...done(callback));
66 },
67 drain(callback) {
68 writer.ready.then(...done(callback));
69 },
70 close(callback) {
71 closed = true;
72 void (async () => {
73 await reader.cancel().catch(() => undefined);
74 reader.releaseLock();
75 await writer.close().catch(() => undefined);
76 writer.releaseLock();
77 await port.close().catch(() => undefined);
78 callback(null);
79 })();
80 },
81 on(event: "data" | "error", listener: ((chunk: Uint8Array) => void) | ((error: Error) => void)) {
82 if (event === "data") onData.add(listener as (chunk: Uint8Array) => void);
83 else onError.add(listener as (error: Error) => void);
84 },
85 };
86 }
87
88 export class LeaderArm {
89 readonly ids: readonly number[];
90 readonly gripperId: number;
91 /** Each leader servo's calibrated Min/Max_Position_Limit; a servo that has none is left out, and only a calibration maps it. */
92 readonly limits: ReadonlyMap<number, Range>;
93 readonly #bus: FeetechBus;
94
95 private constructor(bus: FeetechBus, limits: Map<number, Range>, ids: readonly number[], gripperId: number) {
96 this.#bus = bus;
97 this.limits = limits;
98 this.ids = ids;
99 this.gripperId = gripperId;
100 }
101
102 /** Open the port, find every leader servo's range and let the arm go limp. Closes the port again if any of that fails. */
103 static async open(port: SerialPortLike, { ids = [1, 2, 3, 4, 5, 6], gripperId = 6, baudRate = 1_000_000 }: LeaderOptions = {}): Promise<LeaderArm> {
104 await port.open({ baudRate });
105 const bus = new FeetechBus(serialTransport(port, baudRate));
106 try {
107 await bus.detectEcho();
108 const limits = new Map<number, Range>();
109 for (const id of ids) {
110 const min = await bus.read(id, STS3215.Min_Position_Limit);
111 const max = await bus.read(id, STS3215.Max_Position_Limit);
112 if (max > min) limits.set(id, { min, max });
113 }
114 await bus.syncWrite(STS3215.Torque_Enable, ids.map((id) => [id, 0] as const));
115 return new LeaderArm(bus, limits, ids, gripperId);
116 } catch (err) {
117 await bus.close();
118 if (err instanceof NoResponseError) throw new Error(`leader servo ${err.id} did not answer: is the leader board powered (its own supply, not just USB) and is this the leader's port?`);
119 throw err;
120 }
121 }
122
123 /** Present position of every leader servo that answered, in ticks. Throws once the port is gone. */
124 pose(): Promise<Map<number, number>> {
125 return this.#bus.syncRead(this.ids, STS3215.Present_Position);
126 }
127
128 /** The follower targets this pose asks for, mapped onto the follower's calibrated ranges, through `calibration` when given. */
129 targets(pose: ReadonlyMap<number, number>, followerLimits: Record<string, Range>, calibration?: LeaderCalibration | null): Frame {
130 if (calibration) return mapLeaderFrame(pose, calibration, followerLimits);
131 return mapFrame(pose, new Map(this.limits), followerLimits, this.gripperId);
132 }
133
134 /**
135 * Read poses at `hz` until `signal` aborts or the port fails, handing each to `onPose`. Resolves when stopped;
136 * rejects with the port's error when it is unplugged. A read that comes back short is passed on as it is.
137 */
138 async follow(onPose: (pose: Map<number, number>) => void, { hz = 60, signal }: { hz?: number; signal?: AbortSignal } = {}): Promise<void> {
139 const period = 1000 / hz;
140 while (!signal?.aborted) {
141 const started = Date.now();
142 const pose = await this.pose();
143 if (signal?.aborted) break;
144 if (pose.size) onPose(pose);
145 const wait = period - (Date.now() - started);
146 if (wait > 0) await new Promise((r) => setTimeout(r, wait));
147 }
148 }
149
150 close(): Promise<void> {
151 return this.#bus.close();
152 }
153 }
154
155 export type Support = { ok: true } | { ok: false; why: string };
156
157 /** Whether this browser can read a leader arm, and if not, what to say: told apart by what the browser has, not by its name, except to say why. */
158 export function leaderSupport(env: { serial: boolean; secure: boolean; userAgent: string; touchPoints?: number }): Support {
159 if (env.serial && env.secure) return { ok: true };
160 const instead = "Meanwhile you can drive with the sliders, the − and + buttons, or by dragging a joint of the 3D model.";
161 if (!env.secure && env.serial) return { ok: false, why: `The browser only opens USB ports on a secure page. Open this page over https. ${instead}` };
162 const ua = env.userAgent;
163 // iPadOS asks for desktop pages and calls itself a Mac; a Mac has no touch points.
164 if (/iPhone|iPad|iPod/.test(ua) || (/Macintosh/.test(ua) && (env.touchPoints ?? 0) > 1)) {
165 return { ok: false, why: `iPhone and iPad browsers cannot open USB serial ports: Apple's WebKit has no Web Serial, whichever browser app you use. Plug the leader arm into a computer with Chrome or Edge, or an Android phone with Chrome. ${instead}` };
166 }
167 if (/Android/.test(ua)) return { ok: false, why: `This Android browser cannot open USB serial ports. Chrome on Android can where it offers Web Serial (update it, and use a USB-C OTG adapter); a computer with Chrome or Edge always can. ${instead}` };
168 return { ok: false, why: `This browser cannot open USB serial ports (Web Serial). Use Chrome or Edge on a computer, or Chrome on Android where it offers Web Serial. ${instead}` };
169 }
Software/so101/src/leader.ts +3 −13
@@ -2,7 +2,9 @@
22
33 import WebSocket from "ws";
44 import type { FeetechBus } from "./bus.ts";
5 import { mapGripper, mapJoint, type Frame, type Range } from "./mapping.ts";
5 import { mapFrame, type Frame, type Range } from "./mapping.ts";
6
7 export { mapFrame };
68 import { STS3215 } from "./sts3215.ts";
79
810 export interface LeaderOptions {
@@ -20,18 +22,6 @@ export async function fetchLimits(serverUrl: string): Promise<Record<string, Ran
2022 return (await res.json()) as Record<string, Range>;
2123 }
2224
23 /** Pure: leader ticks → follower frame, given both arms' calibrated ranges. */
24 export function mapFrame(leader: ReadonlyMap<number, number>, leaderLimits: Map<number, Range>, followerLimits: Record<string, Range>, gripperId: number): Frame {
25 const frame: Frame = {};
26 for (const [id, ticks] of leader) {
27 const l = leaderLimits.get(id);
28 const f = followerLimits[String(id)];
29 if (!l || !f) continue;
30 frame[String(id)] = id === gripperId ? mapGripper(ticks, l, f) : mapJoint(ticks, l, f);
31 }
32 return frame;
33 }
34
3525 export async function streamLeader(bus: FeetechBus, serverUrl: string, { hz = 60, ids = [1, 2, 3, 4, 5, 6], gripperId = 6, signal, onFrame }: LeaderOptions = {}): Promise<number> {
3626 const followerLimits = await fetchLimits(serverUrl);
3727 const leaderLimits = new Map<number, Range>();
Software/so101/src/mapping.ts +12
@@ -24,6 +24,18 @@ export function mapGripper(leaderTicks: number, leader: Range, follower: Range,
2424 return clamp(target, follower.min + margin, follower.max - margin);
2525 }
2626
27 /** Pure: leader ticks → follower frame, given both arms' calibrated ranges. */
28 export function mapFrame(leader: ReadonlyMap<number, number>, leaderLimits: Map<number, Range>, followerLimits: Record<string, Range>, gripperId: number): Frame {
29 const frame: Frame = {};
30 for (const [id, ticks] of leader) {
31 const l = leaderLimits.get(id);
32 const f = followerLimits[String(id)];
33 if (!l || !f) continue;
34 frame[String(id)] = id === gripperId ? mapGripper(ticks, l, f) : mapJoint(ticks, l, f);
35 }
36 return frame;
37 }
38
2739 export const clamp = (v: number, lo: number, hi: number): number => Math.min(hi, Math.max(lo, v));
2840
2941 /** A target frame on the wire: servo id → ticks. */
Software/so101/src/relay.ts +7 −3
@@ -75,7 +75,7 @@ export interface RunningRelay {
7575 const SCHEMES: Record<string, "http:" | "https:"> = { "http:": "http:", "https:": "https:", "ws:": "http:", "wss:": "https:" };
7676 const MAX_BODY = 1_000_000;
7777 /** Served from this process whichever arm is live: they describe the app, not the arm. */
78 const LOCAL = /^\/(?:index\.html|leader-map\.js)?$|^\/(?:model|vendor|printables)(?:\/|$)/;
78 const LOCAL = /^\/(?:index\.html|leader-map\.js)?$|^\/(?:model|vendor|printables|app)(?:\/|$)/;
7979 const LEADER = "/leader/calibration";
8080 const CAMERA = /^\/camera\/([12])\/(stream|state|snapshot)$/;
8181 const FLOOR = {
@@ -197,7 +197,7 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
197197 const timer = timeoutMs === null ? undefined : setTimeout(() => controller.abort(new Error(`no answer in ${timeoutMs} ms`)), timeoutMs);
198198 if (tracked) inFlight.add(controller);
199199 const headers: Record<string, string> = {};
200 for (const name of ["content-type", "accept", "range"]) if (typeof req.headers[name] === "string") headers[name] = req.headers[name];
200 for (const name of ["content-type", "accept", "range", "if-none-match"]) if (typeof req.headers[name] === "string") headers[name] = req.headers[name];
201201 try {
202202 const response = await fetch(url, { method: req.method, headers, body: body?.length ? body : undefined, signal: controller.signal, redirect: "manual", cache: "no-store" });
203203 return { response, controller };
@@ -329,7 +329,11 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
329329 }
330330 arm.on("message", (data, binary) => browser.readyState === WebSocket.OPEN && browser.send(data, { binary }));
331331 browser.on("message", (data, binary) => {
332 if (driver !== null) lobby?.activity(driver);
332 if (driver !== null) {
333 // The turn may have ended while this frame was on its way, before the close reaches the page: it is not passed on.
334 if (lobby && !lobby.turns.isDriver(driver) && !lobby.local) return;
335 lobby?.activity(driver);
336 }
333337 if (arm.readyState === WebSocket.OPEN) arm.send(data, { binary });
334338 });
335339 arm.on("close", (code, why) => browser.close(sendable(code) ? code : 1011, why.length ? why.toString() : "the arm server went away"));
Software/so101/src/server.ts +12
@@ -26,6 +26,7 @@
2626 * GET /model.json the arm model: joints, the table plane, where the URDF is served
2727 * GET /leader-map.js the leader calibration mapping, for the browser (src/leader-map.js)
2828 * GET /model/<file> the URDF and its meshes; GET /vendor/<pkg>/<file> three.js and the URDF loader
29 * GET /app/<file> the modules the page imports itself (the leader arm over Web Serial), as JavaScript
2930 * GET /printables the sheets to print for the vision quest (board, cube markers, gripper marker); GET /printables/<file>
3031 * WS /follow client sends {"1": ticks, "2": ticks, ...} frames; the server drives the arm
3132 *
@@ -40,6 +41,7 @@ import { createServer, type IncomingMessage, type ServerResponse } from "node:ht
4041 import { dirname, extname, join, resolve } from "node:path";
4142 import { fileURLToPath } from "node:url";
4243 import { WebSocketServer, type WebSocket } from "ws";
44 import { browserModule } from "./browser-modules.ts";
4345 import type { FeetechBus } from "./bus.ts";
4446 import { ArmModel, gripperTicksToRadians, ticksToRadians, type Violation } from "./kinematics.ts";
4547 import { clamp, type Frame } from "./mapping.ts";
@@ -479,6 +481,16 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
479481 if (!path || ![".js", ".mjs", ".json"].includes(extname(path).toLowerCase())) return json(res, 404, { error: "no such vendored file" });
480482 return file(res, path, "max-age=86400");
481483 }
484 if (req.method === "GET" && url.pathname.startsWith("/app/")) {
485 const served = await browserModule(decode(url.pathname.slice("/app/".length)));
486 if (!served) return json(res, 404, { error: "no such module" });
487 if (req.headers["if-none-match"] === served.etag) {
488 res.writeHead(304, { etag: served.etag, "cache-control": "no-cache" });
489 return res.end();
490 }
491 res.writeHead(200, { "content-type": "text/javascript; charset=utf-8", "cache-control": "no-cache", etag: served.etag });
492 return res.end(served.js);
493 }
482494 if (req.method === "PUT" && url.pathname.startsWith("/calibration/")) {
483495 const kind = url.pathname.slice("/calibration/".length);
484496 if (!/^[a-z][a-z0-9_-]{0,31}$/.test(kind)) throw new HttpError(400, "calibration kind: lowercase letters, digits, - and _");
Software/so101/src/viewer.html +122 −185
@@ -40,7 +40,7 @@
4040 .j.jam { background: rgba(248,81,73,.18); } .j.drag { background: rgba(56,139,253,.15); }
4141 .j b { grid-area: name; font-size: 11px; color: #c9d1d9; letter-spacing: .03em; text-transform: uppercase; font-weight: 600; }
4242 .j .v { grid-area: val; display: flex; align-items: center; gap: 6px; font-variant-numeric: tabular-nums; font-size: 15px; }
43 .j small { color: var(--dim); font-size: 11px; }
43 .j small { color: var(--dim); font-size: 11px; } .j small.ld { color: #58a6ff; } .j small.ld:empty { display: none; }
4444 .j .slide { grid-area: slide; display: flex; align-items: center; gap: 4px; }
4545 .j input[type=range] { flex: 1; min-width: 0; accent-color: var(--blue); height: 22px; }
4646 .j .jog button { min-width: 30px; min-height: 28px; padding: 0 6px; font-size: 15px; line-height: 1; touch-action: none; user-select: none; }
@@ -117,7 +117,7 @@
117117 <div id="stage">
118118 <div class="pane on" id="pane1"><img id="cam1" alt="camera 1"><span class="tag" id="tag1">camera 1</span></div>
119119 <div class="pane" id="pane2"><img id="cam2" alt="camera 2"><span class="tag" id="tag2">camera 2</span></div>
120 <div class="pane on" id="pane3"><urdf-viewer id="model" up="Z" ignore-limits display-shadow ambient-color="#b9c0c8"></urdf-viewer><span class="tag">3D · drag a joint to move the arm · drag empty space to orbit</span></div>
120 <div class="pane on" id="pane3"><urdf-viewer id="model" up="Z" ignore-limits display-shadow ambient-color="#b9c0c8"></urdf-viewer><span class="tag" id="tag3">3D · drag a joint to move the arm · drag empty space to orbit</span></div>
121121 </div>
122122 <div id="top">
123123 <span class="seg"><button id="v1" class="on">Cam 1</button><button id="v2" hidden>Cam 2</button><button id="v3" class="on">3D</button></span>
@@ -136,8 +136,9 @@
136136 <button id="releaseBtn" class="danger" title="torque off">Release</button>
137137 <button id="record" class="rec">● Record</button>
138138 <button id="play">▶ Play…</button>
139 <button id="liveBtn" class="go" hidden title="read the leader arm through this computer's USB port and stream it (Chrome)">Live from leader</button>
139 <button id="leaderDrive" class="go" hidden title="stream the leader arm's pose to the arm during your turn">Drive with leader</button>
140140 </span>
141 <button id="leaderBtn" title="read a leader arm plugged into this computer or phone (Chrome, Edge)">Connect leader arm</button>
141142 <button id="leaderCal" hidden title="teach this page your own leader arm: once, kept with your sign-in">Calibrate leader</button>
142143 <button id="metricsBtn">Metrics</button>
143144 <button id="settingsBtn">Settings</button>
@@ -224,13 +225,20 @@
224225 <div class="actions"><button value="cancel">Cancel</button><button value="start" class="rec">Start</button></div>
225226 </form>
226227 </dialog>
228 <dialog id="leaderHelp">
229 <form method="dialog">
230 <h3>Leader arm</h3>
231 <p id="leaderWhy"></p>
232 <div class="actions"><button value="ok" class="go">OK</button></div>
233 </form>
234 </dialog>
227235 <dialog id="leaderDlg" aria-labelledby="lcTitle">
228236 <h3 id="lcTitle">Calibrate your leader arm</h3>
229237 <p id="lcStep"></p>
230238 <div id="lcRead" aria-live="off"></div>
231239 <div id="lcFlips" hidden></div>
232240 <p id="lcErr" class="bad" role="alert"></p>
233 <div class="actions"><button id="lcCancel">Cancel</button><button id="lcBack" hidden>Back</button><button id="lcNext" class="go">Connect</button></div>
241 <div class="actions"><button id="lcCancel">Cancel</button><button id="lcBack" hidden>Back</button><button id="lcNext" class="go">Capture</button></div>
234242 </dialog>
235243 <dialog id="playDlg">
236244 <h3>Play a recording</h3>
@@ -284,7 +292,7 @@
284292 el.title = el.textContent;
285293 el.className = l.mode === "live" ? "on" : l.problem ? "warn" : "";
286294 // Another arm has other limits, poses and joints: start the page over on it, once nothing on the page is driving.
287 if (linkMode !== null && l.mode !== linkMode && !stream.ws && !live) return location.reload();
295 if (linkMode !== null && l.mode !== linkMode && !stream.ws && !leader) return location.reload();
288296 if (linkMode === null) linkMode = l.mode;
289297 }
290298 setTimeout(pollLink, 3000);
@@ -462,143 +470,87 @@
462470 $("play").onclick = () => { loadRecordings(); $("playDlg").showModal(); };
463471 $("playClose").onclick = () => $("playDlg").close();
464472
465 // ---------------------------------------------------------------- live mode: the leader arm through Web Serial
466 // Chrome can open the leader board's USB serial port from the page. The page then does what `so101 leader` does:
467 // torque off, read the six positions at 60 Hz, map them onto the follower's ranges, and stream them.
468 const REG = { Min_Position_Limit: [9, 2], Max_Position_Limit: [11, 2], Torque_Enable: [40, 1], Present_Position: [56, 2, 15] };
469 const decode = (params, size, signBit) => {
470 let v = 0; for (let k = 0; k < size; k++) v |= params[k] << (8 * k);
471 return signBit !== undefined && v & (1 << signBit) ? -(v & ((1 << signBit) - 1)) : v;
472 };
473 class SerialBus {
474 constructor(port) { this.port = port; this.buf = new Uint8Array(0); this.wake = null; }
475 async open() {
476 await this.port.open({ baudRate: 1000000 });
477 this.writer = this.port.writable.getWriter();
478 this.reader = this.port.readable.getReader();
479 this.pump();
480 }
481 async pump() {
482 try {
483 for (;;) {
484 const { value, done } = await this.reader.read();
485 if (done) break;
486 const joined = new Uint8Array(this.buf.length + value.length); joined.set(this.buf); joined.set(value, this.buf.length); this.buf = joined;
487 this.wake?.();
488 }
489 } catch { /* the port went away */ }
490 }
491 async close() {
492 try { await this.reader?.cancel(); } catch {}
493 try { this.writer?.releaseLock(); } catch {}
494 try { await this.port.close(); } catch {}
495 }
496 packet(id, instruction, params) {
497 const body = [id, params.length + 2, instruction, ...params];
498 return new Uint8Array([0xff, 0xff, ...body, ~body.reduce((a, b) => a + b, 0) & 0xff]);
499 }
500 frames() {
501 const out = []; const b = this.buf; let i = 0;
502 while (i + 4 <= b.length) {
503 if (b[i] !== 0xff || b[i + 1] !== 0xff) { i++; continue; }
504 const id = b[i + 2], len = b[i + 3];
505 if (i + 4 + len > b.length) break;
506 out.push({ id, error: b[i + 4], params: b.slice(i + 5, i + 3 + len) });
507 i += 4 + len;
508 }
509 this.buf = b.slice(i);
510 return out;
511 }
512 tick() { return new Promise((r) => { this.wake = r; setTimeout(r, 2); }); }
513 async syncRead(ids, [address, size, signBit]) {
514 this.buf = new Uint8Array(0);
515 await this.writer.write(this.packet(0xfe, 0x82, [address, size, ...ids]));
516 const got = new Map(); const deadline = performance.now() + 15 + ids.length * 3;
517 while (got.size < ids.length && performance.now() < deadline) {
518 for (const f of this.frames()) if (ids.includes(f.id) && f.params.length === size) got.set(f.id, decode(f.params, size, signBit));
519 if (got.size < ids.length) await this.tick();
520 }
521 return got;
522 }
523 async syncWrite([address, size], pairs) {
524 const params = [address, size];
525 for (const [id, v] of pairs) { params.push(id); for (let k = 0; k < size; k++) params.push((v >> (8 * k)) & 0xff); }
526 await this.writer.write(this.packet(0xfe, 0x83, params));
527 }
528 async read(id, [address, size, signBit]) {
529 this.buf = new Uint8Array(0);
530 await this.writer.write(this.packet(id, 0x02, [address, size]));
531 const deadline = performance.now() + 100;
532 for (;;) {
533 for (const f of this.frames()) if (f.id === id && f.params.length === size) return decode(f.params, size, signBit);
534 if (performance.now() > deadline) throw new Error(`no reply from leader servo ${id}: is the leader board powered and plugged in here?`);
535 await this.tick();
536 }
537 }
473 // ---------------------------------------------------------------- the leader arm, plugged into this computer or phone
474 // src/leader-arm.ts, served at /app/: the CLI's own bus code over Web Serial. Connected, the leader is mirrored by the 3D model
475 // and the joint rows; during your turn "Drive with leader" streams its pose like any stream, through the server's limits and zone.
476 let leader = null; // { arm, port, stop, pose, targets, ws, frames, calibration }
477 let calib = null; // the calibration card, while it is open
478 const leaderStreaming = () => Boolean(leader?.ws);
479 const leaderMirrored = () => Object.keys(leader?.targets ?? {}).length > 0 && !leaderStreaming();
480 const explain = (text) => { $("leaderWhy").textContent = text; $("leaderHelp").showModal(); };
481 function renderLeader() {
482 const t = leader?.targets;
483 for (const [id, row] of built) row.ld.textContent = t && t[id] !== undefined ? `leader ${t[id]}` : "";
484 if (!t || leaderStreaming() || !viewer.robot || calib) return; // while calibrating, the model shows the pose to copy
485 for (const [id, ticks] of Object.entries(t)) if (NAMES[id] && limits[id] && NAMES[id] !== dragging) viewer.setJointValue(NAMES[id], toRad(Number(id), ticks));
538486 }
539 let live = null, calib = null;
540 const midOf = (r) => (r.min + r.max) / 2;
541 const clampTo = (v, lo, hi) => Math.min(hi, Math.max(lo, v));
542 const mapLeader = (positions, leaderLimits) => {
543 const frame = {};
544 for (const [id, ticks] of positions) {
545 const L = leaderLimits[id], F = limits[id]; if (!L || !F) continue;
546 if (id === 6) { const f = clampTo((ticks - L.min) / Math.max(1, L.max - L.min), 0, 1); frame[id] = Math.round(F.min + f * (F.max - F.min)); }
547 else frame[id] = clampTo(Math.round(ticks - midOf(L) + midOf(F)), F.min + 30, F.max - 30);
548 }
549 return frame;
550 };
551 async function startLive() {
552 if (calib) return msg("finish or cancel the leader calibration first", true);
553 const port = await navigator.serial.requestPort({ filters: [{ usbVendorId: 0x1a86 }] }).catch(() => null);
487 function renderLeaderButtons() {
488 $("leaderBtn").textContent = leader ? "Disconnect leader" : "Connect leader arm";
489 $("leaderDrive").hidden = !leader;
490 $("leaderCal").hidden = !leader;
491 $("leaderDrive").textContent = leaderStreaming() ? "■ Stop leader" : "Drive with leader";
492 $("tag3").textContent = leader && !leaderStreaming() ? "3D · mirroring your leader arm" : "3D · drag a joint to move the arm · drag empty space to orbit";
493 }
494 async function connectLeader() {
495 let lib;
496 try { lib = await import("/app/leader-arm.ts"); }
497 catch { return explain("This server could not hand the page the leader arm's code. Meanwhile you can drive with the sliders, the − and + buttons, or by dragging a joint of the 3D model."); }
498 const support = lib.leaderSupport({ serial: "serial" in navigator, secure: isSecureContext, userAgent: navigator.userAgent, touchPoints: navigator.maxTouchPoints });
499 if (!support.ok) return explain(support.why);
500 const port = await navigator.serial.requestPort({ filters: lib.BOARD_FILTERS }).catch(() => null);
554501 if (!port) return;
555 const bus = new SerialBus(port);
556 try {
557 await bus.open();
558 const ids = [1, 2, 3, 4, 5, 6];
559 // The driver's own calibration when they have one; otherwise the leader servos' stored limits, LeRobot's way.
560 const mine = await api("/leader/calibration").catch(() => null);
561 const calibration = mine?.ok && mine.joints ? mine : null;
562 const leaderLimits = {};
563 if (!calibration) for (const id of ids) leaderLimits[id] = { min: await bus.read(id, REG.Min_Position_Limit), max: await bus.read(id, REG.Max_Position_Limit) };
564 await bus.syncWrite(REG.Torque_Enable, ids.map((id) => [id, 0])); // the leader is moved by hand
565 const ws = new WebSocket(`${location.protocol === "https:" ? "wss" : "ws"}://${location.host}/follow`);
566 await new Promise((resolve, reject) => { ws.onopen = resolve; ws.onerror = () => reject(new Error("no stream socket")); });
567 live = { bus, ws, ids, leaderLimits, frames: 0, busy: false, timer: null };
568 ws.onclose = (e) => { if (live?.ws === ws) stopLive(e.reason || (e.code === 1013 ? "the arm is busy" : "the server closed the stream")); };
569 live.timer = setInterval(async () => {
570 if (!live || live.busy) return;
571 live.busy = true;
572 try {
573 const positions = await bus.syncRead(ids, REG.Present_Position);
574 const frame = calibration ? mapLeaderFrame(positions, calibration, limits) : mapLeader(positions, leaderLimits);
575 if (ws.readyState === 1 && Object.keys(frame).length) { ws.send(JSON.stringify(frame)); live.frames++; }
576 } catch { /* a missed read; the next tick tries again */ }
577 live.busy = false;
578 }, 16);
579 $("liveBtn").textContent = "■ Stop live";
580 msg(`live${calibration ? ", with your calibration" : "; Calibrate leader for a clean mapping"}: the follower follows the leader. CAUTION: it first eases over to the leader's pose.`);
581 } catch (err) {
582 await bus.close();
583 msg(`live mode: ${err.message}`, true);
584 }
502 msg("leader: connecting…");
503 let arm;
504 try { arm = await lib.LeaderArm.open(port); }
505 catch (err) { return msg(`leader: ${err.message}`, true); }
506 // The driver's own calibration when they have one; otherwise the leader servos' stored ranges, LeRobot's way.
507 const mine = await api("/leader/calibration").catch(() => null);
508 const l = { arm, port, stop: new AbortController(), pose: null, targets: null, ws: null, frames: 0, calibration: mine?.ok && mine.joints ? mine : null };
509 leader = l;
510 renderLeaderButtons();
511 const how = l.calibration ? ", with your calibration" : arm.limits.size ? "; Calibrate leader for a clean mapping" : "; it has no stored ranges, so Calibrate leader before it can drive";
512 msg(`leader connected and mirrored in 3D${how}. ${mayDrive() ? "Press Drive with leader to move the arm with it." : "It drives the arm once you have control."}`);
513 l.arm.follow((pose) => {
514 l.pose = pose;
515 l.targets = l.arm.targets(pose, limits, l.calibration);
516 if (calib) return calibReading(pose);
517 if (l.ws?.readyState === 1 && Object.keys(l.targets).length) { l.ws.send(JSON.stringify(l.targets)); l.frames++; }
518 renderLeader();
519 }, { hz: 60, signal: l.stop.signal }).catch((err) => disconnectLeader(`the leader stopped answering (${err.message})`));
585520 }
586 async function stopLive(reason) {
587 const l = live; if (!l) return;
588 live = null;
589 clearInterval(l.timer);
590 try { l.ws.close(); } catch {}
591 await l.bus.close();
592 $("liveBtn").textContent = "Live from leader";
593 msg(`live mode ended${reason ? `: ${reason}` : ""}; the follower holds (${l.frames} frames)`);
521 async function disconnectLeader(reason) {
522 const l = leader; if (!l) return;
523 stopLeaderDrive();
524 closeCalib();
525 leader = null;
526 l.stop.abort();
527 await l.arm.close().catch(() => {});
528 renderLeaderButtons(); renderLeader();
529 if (status) applyModel(status);
530 msg(`leader disconnected${reason ? `: ${reason}` : ""}`, Boolean(reason));
531 }
532 function driveLeader() {
533 const l = leader; if (!l || l.ws) return;
534 if (calib) return msg("finish or cancel the leader calibration first", true);
535 if (!mayDrive()) return msg("take control first: the leader drives the arm only during your turn", true);
536 const ws = new WebSocket(`${location.protocol === "https:" ? "wss" : "ws"}://${location.host}/follow`);
537 l.ws = ws; l.frames = 0;
538 ws.onopen = () => msg("driving with the leader. CAUTION: the arm first eases over to the leader's pose.");
539 ws.onclose = (e) => {
540 if (l.ws !== ws) return;
541 l.ws = null;
542 renderLeaderButtons();
543 msg(`the leader stopped driving${e.reason ? `: ${e.reason}` : ""}; the arm holds, and the 3D model mirrors the leader again`, e.code !== 1000);
544 };
545 renderLeaderButtons();
594546 }
595547 // ---------------------------------------------------------------- leader calibration (src/leader-map.js)
596548 // Once per driver: the leader in the follower's middle pose, then each joint at each end the model shows. The ends
597549 // carry the direction (a joint whose low end reads higher is mounted backwards) and each side's length; the centre
598 // carries the offset. Kept on the server under the driver's sign-in; Live from leader applies it before sending.
550 // carries the offset. Kept on the server under the driver's sign-in; the connected leader maps through it from then on.
599551 const LABELS = { 1: "Base rotation", 2: "Shoulder", 3: "Elbow", 4: "Wrist bend", 5: "Wrist roll", 6: "Gripper trigger" };
600552 const LEADER_IDS = [1, 2, 3, 4, 5, 6];
601 const STEPS = ["connect", "centre", ...LEADER_IDS.flatMap((id) => [{ id, end: "low" }, { id, end: "high" }]), "check"];
553 const STEPS = ["centre", ...LEADER_IDS.flatMap((id) => [{ id, end: "low" }, { id, end: "high" }]), "check"];
602554 /** Follower ticks the model shows for a step: every joint in the middle, the one being taught at its end. */
603555 const previewOf = (step) => Object.fromEntries(Object.keys(limits).map((k) => {
604556 const id = Number(k);
@@ -614,15 +566,13 @@
614566 const step = STEPS[calib.step];
615567 const t = typeof step === "object" ? step : null;
616568 $("lcErr").textContent = calib.error ?? "";
617 $("lcBack").hidden = calib.step < 2;
569 $("lcBack").hidden = calib.step < 1;
618570 $("lcFlips").hidden = step !== "check";
619 $("lcRead").hidden = step === "connect";
620 $("lcNext").textContent = step === "connect" ? "Connect" : step === "check" ? "Save" : "Capture";
571 $("lcNext").textContent = step === "check" ? "Save" : "Capture";
621572 $("lcStep").textContent =
622 step === "connect" ? "Plug your leader arm into this computer and power its board, then press Connect and pick its port. Its motors go limp so you can move it by hand. Nothing moves the real arm while you calibrate."
623 : step === "centre" ? "Hold the leader in the pose the 3D model shows: every joint in the middle of its travel, the gripper half open. Then press Capture."
573 step === "centre" ? "Nothing moves the real arm while you calibrate. Hold the leader in the pose the 3D model shows: every joint in the middle of its travel, the gripper half open. Then press Capture."
624574 : step === "check" ? "Move the leader: the model follows through your calibration. If a joint turns the wrong way, flip it. Save keeps it with your sign-in, for every computer you drive from."
625 : `${LABELS[t.id]} (${calib.step - 1} of 12): move it to the ${t.id === 6 ? (t.end === "low" ? "closed" : "open") : "end"} the model shows, as far as it goes, and press Capture.`;
575 : `${LABELS[t.id]} (${calib.step} of 12): move it to the ${t.id === 6 ? (t.end === "low" ? "closed" : "open") : "end"} the model shows, as far as it goes, and press Capture.`;
626576 $("lcRead").replaceChildren(...LEADER_IDS.map((id) => el("span", { className: t?.id === id ? "on" : "", textContent: `${LABELS[id].split(" ")[0]} ${calib.reading.get(id) ?? "–"}` })));
627577 if (step === "check") {
628578 $("lcFlips").replaceChildren(...LEADER_IDS.flatMap((id) => {
@@ -634,59 +584,40 @@
634584 if (step !== "check") showOnModel(previewOf(step));
635585 }
636586 async function openCalib() {
587 if (!leader) return;
637588 if (!Object.keys(limits).length) return msg("the arm's limits have not loaded yet", true);
638589 const mine = await api("/leader/calibration").catch(() => null);
639590 if (mine?.status === 401) return msg("sign in to calibrate your leader: the calibration is kept with your account", true);
640 if (live) await stopLive("calibrating the leader");
641 calib = { step: 0, joints: {}, reading: new Map(), bus: null, timer: null, busy: false, error: null, had: Boolean(mine?.joints) };
591 if (!leader) return;
592 stopLeaderDrive();
593 calib = { step: 0, joints: {}, reading: new Map(leader.pose ?? []), error: null, had: Boolean(mine?.joints) };
642594 $("leaderDlg").show();
643595 renderCalib();
644596 }
645 async function closeCalib(note) {
646 const c = calib; if (!c) return;
597 function closeCalib(note) {
598 if (!calib) return;
647599 calib = null;
648 clearInterval(c.timer);
649 await c.bus?.close();
650600 $("leaderDlg").close();
651601 if (status) applyModel(status);
602 renderLeader();
652603 if (note) msg(note);
653604 }
654 async function connectCalib() {
655 const port = await navigator.serial.requestPort({ filters: [{ usbVendorId: 0x1a86 }] }).catch(() => null);
656 if (!port || !calib) return;
657 const bus = new SerialBus(port);
658 try {
659 await bus.open();
660 await bus.syncWrite(REG.Torque_Enable, LEADER_IDS.map((id) => [id, 0]));
661 } catch (err) {
662 await bus.close();
663 calib.error = `could not open the leader: ${err.message}`;
664 return renderCalib();
665 }
666 calib.bus = bus;
667 calib.timer = setInterval(async () => {
668 const c = calib; if (!c || c.busy) return;
669 c.busy = true;
670 try {
671 const got = await bus.syncRead(LEADER_IDS, REG.Present_Position);
672 for (const [id, t] of got) c.reading.set(id, t);
673 if (calib !== c) return;
674 if (STEPS[c.step] === "check") showOnModel(mapLeaderFrame(c.reading, draft(), limits));
675 for (const [i, span] of [...$("lcRead").children].entries()) span.textContent = `${LABELS[LEADER_IDS[i]].split(" ")[0]} ${c.reading.get(LEADER_IDS[i]) ?? "–"}`;
676 } catch { /* a missed read; the next tick tries again */ } finally { c.busy = false; }
677 }, 33);
678 calib.step = 1;
679 renderCalib();
605 /** Each read of the connected leader while calibrating: the numbers in the card, and in the last step the model through the draft. */
606 function calibReading(pose) {
607 const c = calib;
608 for (const [id, t] of pose) c.reading.set(id, t);
609 if (STEPS[c.step] === "check") showOnModel(mapLeaderFrame(c.reading, draft(), limits));
610 for (const [i, span] of [...$("lcRead").children].entries()) span.textContent = `${LABELS[LEADER_IDS[i]].split(" ")[0]} ${c.reading.get(LEADER_IDS[i]) ?? "–"}`;
680611 }
681612 async function nextCalib() {
682613 const c = calib; if (!c) return;
683614 const step = STEPS[c.step];
684615 c.error = null;
685 if (step === "connect") return connectCalib();
686616 if (step === "check") {
687617 const r = await api("/leader/calibration", draft(), "PUT");
688618 if (!r.ok) { c.error = r.error ?? "the calibration was not saved"; return renderCalib(); }
689 return closeCalib("leader calibrated and saved; Live from leader now uses it");
619 if (leader) leader.calibration = draft();
620 return closeCalib("leader calibrated and saved; the leader now maps through it");
690621 }
691622 const missing = LEADER_IDS.filter((id) => !c.reading.has(id));
692623 if (missing.length) { c.error = `no reading from leader servo ${missing.join(", ")}: is every joint plugged in?`; return renderCalib(); }
@@ -703,17 +634,22 @@
703634 renderCalib();
704635 }
705636 $("lcNext").onclick = () => void nextCalib();
706 $("lcBack").onclick = () => { if (calib && calib.step > 1) { calib.step--; calib.error = null; renderCalib(); } };
637 $("lcBack").onclick = () => { if (calib && calib.step > 0) { calib.step--; calib.error = null; renderCalib(); } };
707638 $("lcCancel").onclick = () => void closeCalib(calib?.had ? "calibration cancelled; your saved one still applies" : "calibration cancelled");
708639 $("leaderDlg").addEventListener("cancel", (e) => { e.preventDefault(); void closeCalib("calibration cancelled"); });
709640
710 if (navigator.serial) {
711 $("leaderCal").hidden = false;
712 $("leaderCal").onclick = () => (calib ? undefined : void openCalib());
713 $("liveBtn").hidden = false;
714 $("liveBtn").onclick = () => (live ? stopLive() : startLive());
715 navigator.serial.addEventListener("disconnect", () => { if (live) stopLive("the leader was unplugged"); if (calib?.bus) closeCalib("calibration stopped: the leader was unplugged"); });
641 $("leaderCal").onclick = () => (calib ? undefined : void openCalib());
642
643 function stopLeaderDrive() {
644 const ws = leader?.ws; if (!ws) return;
645 leader.ws = null;
646 ws.close();
647 renderLeaderButtons();
648 msg(`the leader stopped driving after ${leader.frames} frames; the arm holds`);
716649 }
650 $("leaderBtn").onclick = () => (leader ? disconnectLeader() : connectLeader());
651 $("leaderDrive").onclick = () => (leaderStreaming() ? stopLeaderDrive() : driveLeader());
652 navigator.serial?.addEventListener("disconnect", (e) => { if (e.target === leader?.port) disconnectLeader("the leader was unplugged"); });
717653
718654 // ---------------------------------------------------------------- the 3D model
719655 const viewer = $("model");
@@ -758,7 +694,7 @@
758694 });
759695 function applyModel(d) {
760696 if (!viewer.robot || calib) return; // while calibrating, the model shows the pose to copy
761 for (const j of d.joints) {
697 for (const j of leaderMirrored() ? [] : d.joints) {
762698 const name = NAMES[j.id];
763699 if (!name || j.position == null || name === dragging || !limits[j.id]) continue;
764700 viewer.setJointValue(name, toRad(j.id, j.position));
@@ -837,6 +773,7 @@
837773 $("stop").inert = !(can || isOwner());
838774 for (const part of $("settingsPanel").querySelectorAll(".row, .trim")) part.inert = !can;
839775 viewer.disableDragging = !can;
776 if (!can && leaderStreaming()) stopLeaderDrive(); // the server closes it too; the page need not wait for that
840777 }
841778 const serverNow = () => Date.now() + lobby.offset;
842779 const minutes = (ms) => { const t = Math.max(0, Math.ceil(ms / 1000)); return `${Math.floor(t / 60)}:${String(t % 60).padStart(2, "0")}`; };
@@ -972,7 +909,7 @@
972909 if (!el) {
973910 const lim = limits[j.id] ?? { min: 0, max: 4095 };
974911 const div = document.createElement("div"); div.className = "j";
975 div.innerHTML = `<b>${j.joint ?? j.id}</b><div class="v"><span class="pos"></span><small class="t"></small></div>
912 div.innerHTML = `<b>${j.joint ?? j.id}</b><div class="v"><span class="pos"></span><small class="ld"></small><small class="t"></small></div>
976913 <div class="slide"><span class="jog"><button data-d="-1">−</button></span><input type="range" min="${lim.min + 30}" max="${lim.max - 30}" step="1"><span class="jog"><button data-d="1">+</button></span></div>`;
977914 const range = div.querySelector("input");
978915 range.addEventListener("pointerdown", () => { sliding = j.id; stream.open(); });
@@ -998,7 +935,7 @@
998935 b.addEventListener("contextmenu", (e) => e.preventDefault());
999936 }
1000937 joints.append(div);
1001 el = { div, pos: div.querySelector(".pos"), t: div.querySelector(".t"), range, buttons: div.querySelectorAll("button") };
938 el = { div, pos: div.querySelector(".pos"), ld: div.querySelector(".ld"), t: div.querySelector(".t"), range, buttons: div.querySelectorAll("button") };
1002939 built.set(j.id, el);
1003940 }
1004941 const hot = j.temperature >= 55, load = Math.abs(j.load ?? 0);
@@ -1008,7 +945,7 @@
1008945 if (sliding !== j.id && j.position != null) el.range.value = j.position;
1009946 el.div.classList.toggle("jam", d.jammed?.includes(j.id));
1010947 el.div.classList.toggle("drag", dragging === j.joint || sliding === j.id);
1011 const locked = (d.driving !== null && d.driving !== "stream") || live !== null; // the page's own stream may run; a leader's may not be fought
948 const locked = (d.driving !== null && d.driving !== "stream") || leaderStreaming(); // the page's own stream may run; a leader's may not be fought
1012949 el.range.disabled = locked; for (const b of el.buttons) b.disabled = locked;
1013950 }
1014951 }
@@ -1033,7 +970,7 @@
1033970 if (!d.ok) throw new Error();
1034971 status = d;
1035972 $("dot").className = "dot on";
1036 $("state").textContent = d.driving === null ? "idle" : d.driving === "record" ? `recording ${d.recording}` : live ? `live · ${live.frames} frames` : d.driving;
973 $("state").textContent = d.driving === null ? "idle" : d.driving === "record" ? `recording ${d.recording}` : leaderStreaming() ? `leader · ${leader.frames} frames` : d.driving;
1037974 $("stop").hidden = d.driving === null;
1038975 $("record").textContent = d.driving === "record" ? "■ Stop recording" : "● Record";
1039976 for (const id of ["hold", "releaseBtn", "play"]) $(id).disabled = d.driving !== null;

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs