Public: anyone can read this; the crew can change it.

Migrated from github.com/Trevato/SO-ARM100

workbenchthis machine

Mobile first: the arm is fantastic on a phone - watch, queue, drive, chat and moderate with one thumb #39

Merged opened this in answer to

Check · Review · Land done · Live — Landed; nothing deploys from this branch, so this is the end of its path.

Merged

quill/arm:quill/32-mobile-first-the-arm-is-fantastic-on-a-p → main 3 commits fork 1 thread · 1 unresolved

  • Merged. quill/32-mobile-first-the-arm-is-fantastic-on-a-p was merged into main as 9e5e048.

Review

Sign in to review this change.
13 files changed +1103 −311
Software/README.md +39 −4
@@ -249,7 +249,7 @@ bundled, and no other file is served.
249249 **Watch, take turns, chat.** Anyone with the link can watch: the 3D arm,
250250 its live pose and the cameras. Driving needs a name and a turn.
251251
252 - **Join to take control**: pick a name (3-20 letters, digits, `_` or `-`)
252 - **Join to drive** (in the status strip, or the Queue tab): pick a name (3-20 letters, digits, `_` or `-`)
253253 and press the button. That is the whole account: no email, no password,
254254 no forge account. The app sets a random session token in an HttpOnly,
255255 Secure, SameSite=Lax cookie for a year, so the same browser stays the
@@ -257,7 +257,8 @@ its live pose and the cameras. Driving needs a name and a turn.
257257 owner, …), offensive words and the names of forge users are refused. One
258258 address may make at most 3 new names an hour. People who have a forge
259259 account can sign in there instead (the link under the field).
260 - **Take control** then joins the queue in the panel beside the arm. When nobody is driving, you drive at once.
260 - **Take control** then joins the queue from the status strip under the view.
261 When nobody is driving, you drive at once.
261262 - A turn lasts 3 minutes (the owner can change it), with a countdown. It ends
262263 early when you press **Hand back**, after 30 s with no command (also the
263264 owner's to change), or 10 s after your connection
@@ -283,7 +284,8 @@ its live pose and the cameras. Driving needs a name and a turn.
283284 Settings and log: `$FORGE_APP_DATA/so101/crew.json`.
284285 - The owner (anyone who may write this repository) can take control at
285286 any time (this skips the queue), end anyone's turn, take people out of
286 the queue, delete messages, mute people and ban them (hover a chat line).
287 the queue, delete messages, mute people and ban them (long-press or
288 right-click a chat line).
287289 The owner can also press **Stop** during anyone's turn.
288290 - A ban bars the person's account, their name and the address they were
289291 last seen at: their turn ends, they leave the queue, and they cannot
@@ -295,7 +297,7 @@ its live pose and the cameras. Driving needs a name and a turn.
295297 share one browser for it to tell them apart.
296298
297299 **Admin.** The owner runs the room from `/admin` (plain forms, no script),
298 or from the compact **Admin** bar that only the owner sees beside the queue:
300 or from the **Admin** tab that only the owner sees:
299301
300302 - **Pause the queue** for maintenance. The driver's turn ends (unless it is
301303 the owner's), nobody new gets a turn, and everyone sees "Paused for
@@ -329,6 +331,39 @@ appended to `admin.log.jsonl` next to it, with who did it. `/admin` shows
329331 the latest entries. Changes are accepted only from the owner, and only from
330332 the app's own page.
331333
334 **On a phone.** The page is laid out for one thumb, phone first. The view
335 (camera or 3D) sits on top at full width. Under it a status strip shows who
336 drives, your place in the line, the time left, the turn's one button, and
337 the red **Stop**. Stop is there whenever you drive, and for the owner. Below
338 that is a sheet with tabs at the bottom edge: **Chat**, **Queue**,
339 **Controls** (only on your turn, and it opens by itself when your turn
340 starts) and **Admin** (the owner: the room, the safe zone, the trims,
341 metrics and 3D calibration).
342
343 - Controls: a joystick moves the gripper. Up reaches out, sideways swings
344 around the base, and ▲/▼ raise and lower it. The page solves that onto
345 pan, lift and elbow against the URDF (`src/ui/reach.js`), and the server
346 checks every frame against the zone as usual. Below the joystick come the
347 joint sliders with hold-to-jog −/+, the poses, and the rest.
348 - Chat: the page follows the visual viewport. The keyboard shrinks the
349 sheet and the view, and never covers the box or the last message.
350 Long-press a message (or right-click it) to delete it, or mute or ban its
351 author.
352 - Phone on its side: the view fills the screen, the strip and the sheet
353 float over it, and **Hide** clears them away. A wide screen puts the
354 sheet in a column beside the view.
355 - Every tap target is at least 44 px. The safe-area insets are respected,
356 the page follows the system's light or dark mode, and nothing moves for
357 people who ask for reduced motion. The bottom-right corner belongs to the
358 forge pill, so no control sits under it.
359 - Mobile data: the first view needs no three.js. The model and the
360 joystick arrive after it. The relay gzips the page, three.js, the URDF
361 and the meshes (`src/compress.ts`). The page's modules are cached for
362 good under a name that changes with their content. On a slow link, a
363 slow `/status` round trip or the browser's Save-Data, the camera drops
364 from the MJPEG stream to one snapshot at a time, 0.5 to 4 a second
365 (`src/ui/camera-plan.js`), and goes back once the link recovers.
366
332367 The queue, the countdown and the chat all come over one WebSocket (`/live`):
333368 no polling. People who joined are kept in
334369 `$FORGE_APP_DATA/so101/accounts.sqlite` (`src/accounts.ts`, bun's built-in
Software/so101/src/compress.test.ts added +32
@@ -0,0 +1,32 @@
1 import { describe, expect, test } from "bun:test";
2 import { gunzipSync } from "node:zlib";
3 import { acceptsGzip, GzipCache, worthCompressing } from "./compress.ts";
4
5 describe("gzip for the page's files", () => {
6 test("only a browser that takes gzip gets it, and q=0 is a no", () => {
7 expect(acceptsGzip({ "accept-encoding": "gzip, deflate, br" })).toBe(true);
8 expect(acceptsGzip({ "accept-encoding": "br;q=1, GZIP;q=0.5" })).toBe(true);
9 expect(acceptsGzip({ "accept-encoding": "gzip;q=0" })).toBe(false);
10 expect(acceptsGzip({ "accept-encoding": "x-gzip, identity" })).toBe(false);
11 expect(acceptsGzip({})).toBe(false);
12 });
13
14 test("text, scripts, the URDF and meshes are worth it; pictures and crumbs are not", () => {
15 expect(worthCompressing("text/javascript", 5000)).toBe(true);
16 expect(worthCompressing("model/stl", 5000)).toBe(true);
17 expect(worthCompressing("application/xml", 5000)).toBe(true);
18 expect(worthCompressing("image/png", 5000)).toBe(false);
19 expect(worthCompressing("text/html", 200)).toBe(false);
20 expect(worthCompressing(null, 5000)).toBe(false);
21 });
22
23 test("a cacheable answer is compressed once; an uncacheable one every time", () => {
24 const cache = new GzipCache();
25 const body = new TextEncoder().encode("three ".repeat(1000));
26 const first = cache.gzip("/vendor/three.js", body);
27 expect(gunzipSync(first).toString()).toBe("three ".repeat(1000));
28 expect(first.length).toBeLessThan(body.length / 10);
29 expect(cache.gzip("/vendor/three.js", new Uint8Array(0))).toBe(first);
30 expect(cache.gzip(null, body)).not.toBe(cache.gzip(null, body));
31 });
32 });
Software/so101/src/compress.ts added +44
@@ -0,0 +1,44 @@
1 /**
2 * Gzip for the page's own files on their way out of the relay: three.js, the URDF and its meshes are most of
3 * what a phone downloads on the first visit, and they shrink to about a quarter. Files the browser may cache
4 * are compressed once and kept; the rest are compressed per answer.
5 */
6 import type { IncomingHttpHeaders } from "node:http";
7 import { gzipSync } from "node:zlib";
8
9 const COMPRESSIBLE = /^(?:text\/|application\/(?:javascript|json|xml)|model\/(?:stl|vnd\.collada\+xml))/;
10 /** Below this the headers outweigh the saving. */
11 const MIN_BYTES = 1024;
12 /** What the cache may hold in all; the arm's model and three.js together are well under it. */
13 const MAX_CACHED_BYTES = 16_000_000;
14
15 export function acceptsGzip(headers: IncomingHttpHeaders): boolean {
16 return String(headers["accept-encoding"] ?? "")
17 .split(",")
18 .some((part) => {
19 const [name, ...params] = part.split(";").map((s) => s.trim().toLowerCase());
20 const q = params.find((p) => p.startsWith("q="));
21 return name === "gzip" && (q === undefined || Number(q.slice(2)) > 0);
22 });
23 }
24
25 export function worthCompressing(contentType: string | null, bytes: number): boolean {
26 return bytes >= MIN_BYTES && COMPRESSIBLE.test(contentType ?? "");
27 }
28
29 export class GzipCache {
30 private readonly kept = new Map<string, Buffer>();
31 private bytes = 0;
32
33 /** The body gzipped; kept under `key` when the answer may be cached (`key` null: never kept). */
34 gzip(key: string | null, body: Uint8Array): Buffer {
35 const hit = key === null ? undefined : this.kept.get(key);
36 if (hit) return hit;
37 const packed = gzipSync(body, { level: 6 });
38 if (key !== null && this.bytes + packed.length <= MAX_CACHED_BYTES) {
39 this.kept.set(key, packed);
40 this.bytes += packed.length;
41 }
42 return packed;
43 }
44 }
Software/so101/src/relay.test.ts +22
@@ -4,6 +4,7 @@ import { createServer, type IncomingMessage, type Server } from "node:http";
44 import { createServer as createTcpServer, type Server as TcpServer } from "node:net";
55 import { tmpdir } from "node:os";
66 import { join } from "node:path";
7 import { gunzipSync } from "node:zlib";
78 import WebSocket from "ws";
89 import { FeetechBus } from "./bus.ts";
910 import { FakeBus } from "./fake-bus.ts";
@@ -128,6 +129,27 @@ describe("relay", () => {
128129 expect((await fetch(`${base}/camera/1/state`)).status).toBe(404);
129130 });
130131
132 test("the page and its modules leave gzipped for a browser that takes gzip, and are kept for good under a versioned name", async () => {
133 relay = await startRelay({ port: 0, simulation: simBase });
134 const base = `http://127.0.0.1:${relay.port}`;
135 const raw = (path: string) => fetch(`${base}${path}`, { headers: { "accept-encoding": "gzip, br" }, decompress: false } as RequestInit);
136 const page = await raw("/");
137 expect(page.headers.get("content-encoding")).toBe("gzip");
138 expect(page.headers.get("x-content-type-options")).toBe("nosniff");
139 const html = gunzipSync(Buffer.from(await page.arrayBuffer())).toString();
140 const module = /\/ui\/[0-9a-f]{12}\/reach\.js/.exec(html)?.[0];
141 expect(module).toBeDefined();
142 const js = await raw(module!);
143 expect(js.status).toBe(200);
144 expect(js.headers.get("cache-control")).toContain("immutable");
145 expect(js.headers.get("content-encoding")).toBe("gzip");
146 expect(gunzipSync(Buffer.from(await js.arrayBuffer())).toString()).toContain("export function reachStep");
147 // A client that asks for nothing compressed gets the plain bytes.
148 const plain = await fetch(`${base}${module}`, { headers: { "accept-encoding": "identity" }, decompress: false } as RequestInit);
149 expect(plain.headers.get("content-encoding")).toBeNull();
150 expect(await plain.text()).toContain("export function reachStep");
151 });
152
131153 test("set: status, poses and /follow go to the arm server; the camera's bytes come through", async () => {
132154 remote = await arm([1, 2, 6]);
133155 camera = createServer((req, res) => {
Software/so101/src/relay.ts +22 −1
@@ -32,6 +32,7 @@ import type { Duplex } from "node:stream";
3232 import { Readable } from "node:stream";
3333 import { pipeline } from "node:stream/promises";
3434 import WebSocket, { WebSocketServer } from "ws";
35 import { acceptsGzip, GzipCache, worthCompressing } from "./compress.ts";
3536 import { AccountError, sessionCookie, visitor } from "./accounts.ts";
3637 import { AdminError, changeFromForm } from "./admin.ts";
3738 import { ADMIN_CSP, DONE, renderAdmin, renderRefusal, type Done } from "./admin-page.ts";
@@ -99,7 +100,9 @@ const ADMIN_CAMERA = /^\/admin\/cameras(?:\/(\d{1,9}))?$/;
99100 const CAMERA_PAGE = readFileSync(new URL("./camera-list.html", import.meta.url), "utf8");
100101 const MAX_FORM = 16_384;
101102 /** Served from this process whichever arm is live: they describe the app, not the arm. */
102 const LOCAL = /^\/(?:index\.html|leader-map\.js)?$|^\/(?:model|vendor|printables|app)(?:\/|$)/;
103 const LOCAL = /^\/(?:index\.html|leader-map\.js)?$|^\/(?:model|vendor|printables|ui|app)(?:\/|$)/;
104 /** The page's static files: what a first visit downloads, and what gzip is for. */
105 const STATIC = /^\/(?:index\.html)?$|^\/(?:model|vendor|ui)\//;
103106 const LEADER = "/leader/calibration";
104107 const MAX_JOIN_BODY = 1024;
105108 const FLOOR = {
@@ -264,6 +267,23 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
264267 }
265268 };
266269
270 /** A static answer, gzipped: whole, so the length is known; kept when the browser may cache it too. */
271 const gzipped = new GzipCache();
272 const packed = async (res: ServerResponse, { response }: Awaited<ReturnType<typeof ask>>, path: string) => {
273 const body = new Uint8Array(await response.arrayBuffer());
274 const type = response.headers.get("content-type");
275 const cache = response.headers.get("cache-control") ?? "no-store";
276 const headers: Record<string, string> = { "cache-control": cache, vary: "accept-encoding" };
277 if (type) headers["content-type"] = type;
278 if (response.status !== 200 || !worthCompressing(type, body.length)) {
279 res.writeHead(response.status, headers);
280 return res.end(body);
281 }
282 const out = gzipped.gzip(/max-age=[1-9]/.test(cache) ? path : null, body);
283 res.writeHead(200, { ...headers, "content-encoding": "gzip", "content-length": String(out.length) });
284 res.end(out);
285 };
286
267287 /** The owner's changes to the camera list. */
268288 const administer = async (req: IncomingMessage, res: ServerResponse, id: number | null) => {
269289 const who = lobby ? lobby.who(req) : null;
@@ -437,6 +457,7 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
437457 }
438458 }
439459
460 if (req.method === "GET" && STATIC.test(url.pathname) && acceptsGzip(req.headers)) return await packed(res, await ask(simulation + path, req, undefined, null, false), path);
440461 const raw = reading ? undefined : await readBody(req);
441462 const body = raw && lobby ? capBody(url.pathname, raw, lobby.maxSpeedPercent) : raw;
442463 if (LOCAL.test(url.pathname) || !live || !remote) return await pass(res, await ask(simulation + path, req, body, null, false));
Software/so101/src/server.test.ts +25
@@ -99,6 +99,31 @@ describe("viewer page", () => {
9999 expect(r.headers.get("content-type")).toContain("text/html");
100100 expect(await r.text()).toContain("Fullscreen");
101101 });
102
103 test("names its modules by their content, serves them for good, and serves nothing else from src/ui", async () => {
104 const page = await (await fetch(`${base}/`)).text();
105 expect(page).not.toContain("__VERSION__");
106 const path = /\/ui\/[0-9a-f]{12}\/camera-plan\.js/.exec(page)?.[0];
107 expect(path).toBeDefined();
108 const r = await fetch(`${base}${path}`);
109 expect(r.status).toBe(200);
110 expect(r.headers.get("content-type")).toBe("text/javascript");
111 expect(r.headers.get("cache-control")).toContain("immutable");
112 const version = path!.split("/")[2];
113 expect((await fetch(`${base}/ui/${version}/reach.test.ts`)).status).toBe(404);
114 expect((await fetch(`${base}/ui/${version}/nothing.js`)).status).toBe(404);
115 expect((await fetch(`${base}/ui/${version}/..%2Fserver.ts`)).status).toBe(404);
116 });
117
118 test("is laid out for a phone: tabs under the thumb, an E-stop beside the status, room for the forge pill", async () => {
119 const page = await (await fetch(`${base}/`)).text();
120 for (const tab of ["chat", "queue", "controls", "admin"]) expect(page).toContain(`data-tab="${tab}"`);
121 expect(page).toContain('class="estop"');
122 expect(page).toContain("interactive-widget=resizes-content");
123 expect(page).toContain("prefers-color-scheme: light");
124 expect(page).toContain("prefers-reduced-motion: reduce");
125 expect(page).toMatch(/#tabs \{[^}]*var\(--pill\)/);
126 });
102127 });
103128
104129 describe("moves over HTTP", () => {
Software/so101/src/server.ts +16 −1
@@ -39,6 +39,7 @@
3939 import { existsSync, mkdirSync, readdirSync, readFileSync, renameSync, statSync, unlinkSync, writeFileSync } from "node:fs";
4040 import { createServer, type IncomingMessage, type ServerResponse } from "node:http";
4141 import { dirname, extname, join, resolve } from "node:path";
42 import { createHash } from "node:crypto";
4243 import { fileURLToPath } from "node:url";
4344 import { WebSocketServer, type WebSocket } from "ws";
4445 import { browserModule } from "./browser-modules.ts";
@@ -106,6 +107,8 @@ const JAM_WINDOW = 6;
106107 /** Consecutive too-far frames for one joint before it is eased over to the leader instead of left behind. */
107108 const RESUME_AFTER_DROPS = 20;
108109 const VENDORED = new Set(["three", "urdf-loader"]);
110 /** A module the page loads from src/ui: plain JavaScript, never its tests. */
111 const isUiModule = (name: string) => /^[a-z][a-z0-9-]*\.js$/.test(name);
109112 const MIME: Record<string, string> = { ".js": "text/javascript", ".mjs": "text/javascript", ".json": "application/json", ".urdf": "application/xml", ".stl": "model/stl", ".dae": "model/vnd.collada+xml", ".png": "image/png", ".jpg": "image/jpeg" };
110113
111114 export async function startServer(bus: FeetechBus, options: ServerOptions = {}): Promise<RunningServer> {
@@ -444,6 +447,13 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
444447
445448 // ------------------------------------------------------------------ routes
446449 const viewerPath = join(here, "viewer.html");
450 const uiDir = join(here, "ui");
451 /** The page's own modules, under a name that changes with their content: cached for good, and a deploy is seen at once. */

This reads and hashes all of src/ui on every GET /, the common request. Compute the version once at startup, or cache it keyed on the directory mtime.

452 const uiVersion = () => {
453 const hash = createHash("sha256");
454 for (const f of readdirSync(uiDir).filter(isUiModule).sort()) hash.update(f).update(readFileSync(join(uiDir, f)));
455 return hash.digest("hex").slice(0, 12);
456 };
447457 const nodeModules = resolve(here, "../node_modules");
448458 const http = createServer(async (req, res) => {
449459 try {
@@ -481,6 +491,11 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
481491 if (!path || ![".js", ".mjs", ".json"].includes(extname(path).toLowerCase())) return json(res, 404, { error: "no such vendored file" });
482492 return file(res, path, "max-age=86400");
483493 }
494 const ui = /^\/ui\/[0-9a-f]{12}\/([a-z][a-z0-9-]*\.js)$/.exec(url.pathname);
495 if (req.method === "GET" && ui) {
496 if (!isUiModule(ui[1]!) || !existsSync(join(uiDir, ui[1]!))) return json(res, 404, { error: "no such module" });
497 return file(res, join(uiDir, ui[1]!), "public, max-age=31536000, immutable");
498 }
484499 if (req.method === "GET" && url.pathname.startsWith("/app/")) {
485500 const served = await browserModule(decode(url.pathname.slice("/app/".length)));
486501 if (!served) return json(res, 404, { error: "no such module" });
@@ -502,7 +517,7 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
502517 case "GET /":
503518 case "GET /index.html":
504519 res.writeHead(200, { "content-type": "text/html; charset=utf-8", "cache-control": "no-store" });
505 return res.end(readFileSync(viewerPath)); // read per request: a viewer change deploys with rsync alone
520 return res.end(readFileSync(viewerPath, "utf8").replaceAll("/ui/__VERSION__/", `/ui/${uiVersion()}/`)); // read per request: a viewer change deploys with rsync alone
506521 case "GET /leader-map.js":
507522 return file(res, join(here, "leader-map.js"), "no-cache");
508523 case "GET /status":
Software/so101/src/ui/camera-plan.js added +36
@@ -0,0 +1,36 @@
1 /**
2 * How the page fetches a camera on the connection it has: the full MJPEG stream on a good link, or one
3 * snapshot at a time, further apart the slower the link. Plain ESM, served at /ui/ and imported by the tests.
4 *
5 * The signals are the browser's own estimate where it gives one (navigator.connection: Chrome on Android)
6 * and the round trips of the page's /status polls, which every browser has.
7 */
8
9 /** Round trips above this mean the stream would queue up behind itself. */
10 export const SLOW_RTT_MS = 400;
11 /** And below this the stream comes back. The gap keeps the plan from flapping. */
12 export const FAST_RTT_MS = 200;
13
14 /**
15 * @param {{ effectiveType?: string, saveData?: boolean, downlink?: number } | null | undefined} net
16 * @param {number[]} rtts recent /status round trips, milliseconds
17 * @param {"stream" | "snapshot"} [current]
18 * @returns {{ mode: "stream" } | { mode: "snapshot", everyMs: number }}
19 */
20 export function cameraPlan(net, rtts, current = "stream") {
21 const rtt = median(rtts);
22 const told = net?.saveData === true || net?.effectiveType === "slow-2g" || net?.effectiveType === "2g" || net?.effectiveType === "3g" || (typeof net?.downlink === "number" && net.downlink > 0 && net.downlink < 1.5);
23 const slow = told || (rtt !== null && rtt > (current === "snapshot" ? FAST_RTT_MS : SLOW_RTT_MS));
24 if (!slow) return { mode: "stream" };
25 // A frame every four round trips, between 4 and 0.5 frames a second.
26 const everyMs = Math.round(Math.max(250, Math.min(2000, 4 * (rtt ?? SLOW_RTT_MS))));
27 return { mode: "snapshot", everyMs };
28 }
29
30 /** @param {number[]} values */
31 function median(values) {
32 const v = values.filter((x) => Number.isFinite(x) && x >= 0).sort((a, b) => a - b);
33 if (v.length < 3) return null; // too few to judge
34 const m = v.length >> 1;
35 return v.length % 2 ? /** @type {number} */ (v[m]) : (/** @type {number} */ (v[m - 1]) + /** @type {number} */ (v[m])) / 2;
36 }
Software/so101/src/ui/camera-plan.test.ts added +27
@@ -0,0 +1,27 @@
1 import { describe, expect, test } from "bun:test";
2 import { cameraPlan } from "./camera-plan.js";
3
4 describe("the camera plan", () => {
5 test("a good link, or too little to judge, gets the full stream", () => {
6 expect(cameraPlan(null, [])).toEqual({ mode: "stream" });
7 expect(cameraPlan({ effectiveType: "4g", downlink: 10 }, [40, 60, 50])).toEqual({ mode: "stream" });
8 expect(cameraPlan(undefined, [900, 900])).toEqual({ mode: "stream" }); // two samples are not a verdict
9 });
10
11 test("slow round trips drop to snapshots, further apart the slower they are", () => {
12 expect(cameraPlan(null, [500, 520, 480])).toEqual({ mode: "snapshot", everyMs: 2000 });
13 expect(cameraPlan(null, [420, 410, 430, 2000])).toEqual({ mode: "snapshot", everyMs: 1700 });
14 });
15
16 test("the browser saying the link is slow, or that data is to be saved, is believed", () => {
17 expect(cameraPlan({ saveData: true }, [30, 30, 30]).mode).toBe("snapshot");
18 expect(cameraPlan({ effectiveType: "3g" }, [30, 30, 30])).toEqual({ mode: "snapshot", everyMs: 250 });
19 expect(cameraPlan({ downlink: 0.8 }, []).mode).toBe("snapshot");
20 });
21
22 test("the stream comes back only once the link is clearly fast again", () => {
23 expect(cameraPlan(null, [300, 300, 300], "snapshot").mode).toBe("snapshot");
24 expect(cameraPlan(null, [300, 300, 300], "stream").mode).toBe("stream");
25 expect(cameraPlan(null, [120, 150, 130], "snapshot").mode).toBe("stream");
26 });
27 });
Software/so101/src/ui/reach.js added +79
@@ -0,0 +1,79 @@
1 /**
2 * The touch joystick's arithmetic: move the gripper, not a joint. Plain ESM, served to the page at /ui/ and
3 * imported by the tests. The page owns the arm's geometry (the URDF in three.js) and hands it in as `tip`:
4 * joint angles in, the gripper's position in the base frame (metres, z up) out.
5 */
6
7 /**
8 * The joystick and the height buttons as a displacement of the gripper, in the base frame.
9 * Up on the pad reaches away from the base, sideways swings around it; `lift` is -1, 0 or 1.
10 * `tip` is where the gripper is now; the step is `speed` m/s for `dt` seconds at full deflection.
11 * @param {{ x: number, y: number, lift: number }} stick
12 * @param {number[]} tip
13 * @param {number} speed
14 * @param {number} dt
15 * @returns {number[]}
16 */
17 export function stickToDelta(stick, tip, speed, dt) {
18 const [tx = 0, ty = 0] = tip;
19 const r = Math.hypot(tx, ty);
20 // Straight above the base there is no "away": take the base's own forward, +x.
21 const fx = r > 1e-3 ? tx / r : 1;
22 const fy = r > 1e-3 ? ty / r : 0;
23 const x = clampUnit(stick.x), y = clampUnit(stick.y), lift = clampUnit(stick.lift);
24 const step = speed * dt;
25 // Sideways is the forward direction turned a quarter to the right, seen from above.
26 return [(fx * y + fy * x) * step, (fy * y - fx * x) * step, lift * step];
27 }
28
29 /**
30 * One damped-least-squares step: the joint changes that move the tip by `delta`.
31 * The Jacobian comes from `tip` by finite differences, so any chain the page can evaluate works;
32 * the damping keeps the step small near a stretched-out or folded-up arm instead of letting it whip.
33 * @param {(angles: number[]) => number[]} tip
34 * @param {number[]} angles
35 * @param {number[]} delta
36 * @param {{ h?: number, damping?: number, maxStep?: number }} [options]
37 * @returns {number[]}
38 */
39 export function reachStep(tip, angles, delta, options = {}) {
40 const h = options.h ?? 1e-4, damping = options.damping ?? 0.02, maxStep = options.maxStep ?? 0.08;
41 const here = tip(angles);
42 const n = angles.length;
43 /** @type {number[][]} J[row][joint] */
44 const J = [[], [], []];
45 for (let j = 0; j < n; j++) {
46 const moved = angles.slice();
47 moved[j] = (moved[j] ?? 0) + h;
48 const there = tip(moved);
49 for (let r = 0; r < 3; r++) /** @type {number[]} */ (J[r])[j] = ((there[r] ?? 0) - (here[r] ?? 0)) / h;
50 }
51 // dq = Jᵀ (J Jᵀ + λ² I)⁻¹ delta
52 const A = [0, 1, 2].map((r) => [0, 1, 2].map((c) => dot(/** @type {number[]} */ (J[r]), /** @type {number[]} */ (J[c])) + (r === c ? damping * damping : 0)));
53 const w = solve3(A, delta);
54 const dq = angles.map((_, j) => w.reduce((s, wr, r) => s + wr * (/** @type {number[]} */ (J[r])[j] ?? 0), 0));
55 // Never more than maxStep radians on any joint in one step.
56 const biggest = Math.max(...dq.map(Math.abs));
57 return biggest > maxStep ? dq.map((d) => (d * maxStep) / biggest) : dq;
58 }
59
60 /** @param {number} v */
61 const clampUnit = (v) => (Number.isFinite(v) ? Math.max(-1, Math.min(1, v)) : 0);
62 /** @param {number[]} a @param {number[]} b */
63 const dot = (a, b) => a.reduce((s, v, i) => s + v * (b[i] ?? 0), 0);
64
65 /**
66 * Cramer's rule for a 3×3 system; a singular matrix gives no motion.
67 * @param {number[][]} A
68 * @param {number[]} b
69 */
70 function solve3(A, b) {
71 /** @param {number[][]} m */
72 const det = (m) => {
73 const [[a = 0, b = 0, c = 0] = [], [d = 0, e = 0, f = 0] = [], [g = 0, h = 0, i = 0] = []] = m;
74 return a * (e * i - f * h) - b * (d * i - f * g) + c * (d * h - e * g);
75 };
76 const d = det(A);
77 if (!Number.isFinite(d) || Math.abs(d) < 1e-12) return [0, 0, 0];
78 return [0, 1, 2].map((c) => det(A.map((row, r) => row.map((v, k) => (k === c ? b[r] ?? 0 : v)))) / d);
79 }
Software/so101/src/ui/reach.test.ts added +58
@@ -0,0 +1,58 @@
1 import { describe, expect, test } from "bun:test";
2 import { reachStep, stickToDelta } from "./reach.js";
3
4 // A yaw joint and a two-link planar arm on top of it: the SO-101's pan, shoulder lift and elbow, roughly.
5 const L1 = 0.116, L2 = 0.135;
6 const tip = ([pan = 0, lift = 0, elbow = 0]: number[]) => {
7 const reach = L1 * Math.cos(lift) + L2 * Math.cos(lift + elbow);
8 const z = L1 * Math.sin(lift) + L2 * Math.sin(lift + elbow);
9 return [reach * Math.cos(pan), reach * Math.sin(pan), z];
10 };
11 const distance = (a: number[], b: number[]) => Math.hypot(...a.map((v, i) => v - b[i]!));
12
13 describe("the joystick as a displacement", () => {
14 test("up on the pad reaches away from the base, whichever way the arm faces", () => {
15 const [dx, dy, dz] = stickToDelta({ x: 0, y: 1, lift: 0 }, [0, 0.2, 0.1], 0.05, 0.1);
16 expect(dx).toBeCloseTo(0);
17 expect(dy).toBeCloseTo(0.005);
18 expect(dz).toBe(0);
19 });
20
21 test("right on the pad swings clockwise seen from above, and lift is straight up", () => {
22 const [dx, dy, dz] = stickToDelta({ x: 1, y: 0, lift: 1 }, [0.2, 0, 0.1], 0.05, 0.1);
23 expect(dx).toBeCloseTo(0);
24 expect(dy).toBeCloseTo(-0.005);
25 expect(dz).toBeCloseTo(0.005);
26 });
27
28 test("a deflection past the edge or a broken number is clamped, never amplified", () => {
29 const [dx, dy, dz] = stickToDelta({ x: 0, y: 9, lift: Number.NaN }, [0.2, 0, 0], 0.05, 0.1);
30 expect(dx).toBeCloseTo(0.005);
31 expect(dy).toBeCloseTo(0);
32 expect(dz).toBe(0);
33 });
34 });
35
36 describe("a reach step", () => {
37 test("moves the tip the way it was asked, and the error shrinks step by step", () => {
38 let angles = [0.3, 0.6, -1.2];
39 const start = tip(angles);
40 const goal = [start[0]! + 0.03, start[1]! - 0.02, start[2]! + 0.02];
41 let before = distance(start, goal);
42 for (let i = 0; i < 20; i++) {
43 const here = tip(angles);
44 const delta = goal.map((g, k) => g - here[k]!);
45 angles = angles.map((a, k) => a + reachStep(tip, angles, delta)[k]!);
46 const after = distance(tip(angles), goal);
47 expect(after).toBeLessThanOrEqual(before + 1e-9);
48 before = after;
49 }
50 expect(before).toBeLessThan(0.001);
51 });
52
53 test("a stretched-out arm asked to reach further barely moves instead of whipping", () => {
54 const angles = [0, 0, 0]; // straight out: no joint can take the tip further away
55 const dq = reachStep(tip, angles, [0.05, 0, 0]);
56 for (const d of dq) expect(Math.abs(d)).toBeLessThanOrEqual(0.08);
57 });
58 });
Software/so101/src/viewer.html +702 −304

Diff not shown — 103,005 bytes of diff (limit 100,000). +702 −304

View file
Software/so101/tsconfig.json +1 −1
@@ -5,9 +5,9 @@
55 "module": "Preserve",
66 "moduleResolution": "bundler",
77 "allowImportingTsExtensions": true,
8 "allowJs": true,
89 "verbatimModuleSyntax": true,
910 "noEmit": true,
10 "allowJs": true,
1111 "checkJs": true,
1212 "strict": true,
1313 "skipLibCheck": true,

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs