Public: anyone can read this; the crew can change it.
Migrated from github.com/Trevato/SO-ARM100
workbenchthis machine
Mobile driving: pick a joint and drag one big slider (portrait), two thumb sticks (landscape); zero motion the instant a finger lifts #81
vesper/arm:vesper/50-mobile-driving-pick-a-joint-and-drag-one → main 2 commits fork
15 files
changed +658
−1175
Software/README.md +52 −53
| @@ -168,7 +168,7 @@ arm (its own URDF and meshes, drag a joint and the arm follows), a slider and | ||
| 168 | 168 | hold-to-jog buttons per joint that stream targets like a leader does, a |
| 169 | 169 | button per saved pose plus save and delete, hold and release, record and |
| 170 | 170 | play, one STOP button (pressed, not released; space or Esc) with a separate |
| 171 | two-tap **Resume driving** in the stopped banner (STOP is also how a pose | |
| 171 | two-tap **Resume driving** in the stopped note (STOP is also how a pose | |
| 172 | 172 | move or a playback is ended early; there is no separate "End move" button, |
| 173 | 173 | so driving again after it takes a Resume), fullscreen (F), a metrics panel, a settings |
| 174 | 174 | panel for the safe zone, and a 3D-calibration panel that explains the vision |
| @@ -353,7 +353,7 @@ has Web Serial, not which browser it is. | ||
| 353 | 353 | before your stream has closed. |
| 354 | 354 | - iPhone and iPad browsers can't open USB ports (WebKit has no Web Serial). |
| 355 | 355 | The button explains this there, as on Firefox and desktop Safari, and points |
| 356 | to the sliders, the − and + buttons, and dragging the 3D model. | |
| 356 | to dragging the 3D model and the sliders. | |
| 357 | 357 | |
| 358 | 358 | The page reads the leader with the CLI's own bus code: `src/leader-arm.ts` |
| 359 | 359 | puts `FeetechBus` on a Web Serial port, and the server hands it and the |
| @@ -510,67 +510,66 @@ appended to `admin.log.jsonl` next to it, with who did it. `/admin` shows | ||
| 510 | 510 | the latest entries. Changes are accepted only from the owner, and only from |
| 511 | 511 | the app's own page. |
| 512 | 512 | |
| 513 | **On a phone.** The page is laid out for one thumb, phone first. The view | |
| 514 | (camera or 3D) sits on top at full width. Under it a status strip shows who | |
| 515 | drives, your place in the line, the time left, the turn's one button, and | |
| 516 | the red **Stop**. Stop is there whenever you drive, and for the owner. Below | |
| 517 | that is a sheet with tabs at the bottom edge: **Chat**, **Queue**, | |
| 513 | **On a phone.** The page is laid out for one thumb, phone first. Held | |
| 514 | upright, the camera sits on top at full width (16:9) and the 3D model right | |
| 515 | under it, both always on screen. Under them a status strip shows who drives, | |
| 516 | your place in the line, the time left, the turn's one button, the notes pill | |
| 517 | and the red **Stop**. Stop is there whenever you drive, and for the owner. | |
| 518 | Below that is a tray with tabs at the bottom edge: **Chat**, **Queue**, | |
| 518 | 519 | **Controls** (only on your turn, and it opens by itself when your turn |
| 519 | starts) and **Admin** (the owner: the room, the safe zone, the trims, | |
| 520 | metrics and 3D calibration). | |
| 520 | starts), **Admin** (the owner: the room, the safe zone, the trims, metrics | |
| 521 | and 3D calibration) and **Machine**. The grabber on top of the tray, or the | |
| 522 | open tab tapped again, folds the tray away and gives the model the room. | |
| 521 | 523 | |
| 522 | - Controls, phone upright: a row of chips (J1 base, J2 shoulder, J3 elbow, | |
| 523 | J4 wrist, J5 roll, Grip) picks one joint. One tall lever drives it: push | |
| 524 | up or down, and the further you push the faster it goes. Let go and it | |
| 525 | stops. ▲/▼ beside the lever nudge one step per tap and jog while held. | |
| 526 | Under them come the poses, Hold, Record and the rest. This layout is for | |
| 527 | screens up to 640 px wide. A tablet held upright gets the joystick and | |
| 528 | sliders of the wider screens. From a keyboard, arrows on the chips pick | |
| 529 | the joint and ↑/↓ on the lever step it. | |
| 530 | - Controls, phone on its side: the sheet tucks away when your turn starts, | |
| 531 | and two thumb sticks sit over the picture. The left stick moves J1 | |
| 532 | sideways and J2 up/down. The right stick moves J4 sideways and J3 up/down, | |
| 533 | and **Wrist** swaps it to J5 roll and the gripper. **Open** and **Close** | |
| 534 | between the sticks are press-and-hold. **Show** brings the sheet back. | |
| 535 | The sticks and the grip buttons take touch and a mouse only. With a | |
| 536 | keyboard or a screen reader, **Show** brings back the sheet, and its joint | |
| 537 | sliders and ±-buttons drive every joint, the gripper included. | |
| 538 | - How touch driving works (`src/ui/drive.js`): a finger's deflection is a | |
| 539 | speed, with a dead band in the middle, never a position. Each finger is | |
| 540 | tracked by its own pointer, so two thumbs drive two sets of joints | |
| 541 | independently. The instant a finger lifts, its joints' targets are sent | |
| 542 | as they stand, without waiting for the next frame (`src/ui/frames.js` | |
| 543 | sends whatever is still pending with it). The arm's governor | |
| 544 | settles on them. The same happens on pointercancel, on a lost capture, | |
| 545 | when the window loses focus, when the page is hidden, and when driving | |
| 546 | stops being allowed: your turn ends, the arm is stopped, or it is | |
| 547 | reconnecting (`canDrive` in `src/ui/drive.js`). Frames go out | |
| 548 | at 25 Hz. A target never runs more than 0.6 s ahead of where the arm | |
| 549 | last said it was, so a slow link or a speed cap never queues up motion. | |
| 550 | The server's limits clamp every frame as before. Android buzzes on a | |
| 551 | press and at the end of a joint's range. Every press also shows on | |
| 552 | screen. The screen stays awake during your turn (Screen Wake Lock), and | |
| 553 | the status line shows the round trip to the arm once it passes 250 ms. | |
| 554 | On a phone the strip carries one stop button, the E-stop. | |
| 555 | - Controls, wider screens: a joystick moves the gripper. Up reaches out, | |
| 556 | sideways swings around the base, and ▲/▼ raise and lower it. The page | |
| 557 | solves that onto pan, lift and elbow against the URDF | |
| 558 | (`src/ui/reach.js`), and the server checks every frame against the zone | |
| 559 | as usual. Below the joystick come the joint sliders with hold-to-jog −/+, | |
| 560 | the poses, and the rest. | |
| 524 | - Driving is touching the model (`src/ui/grab.js`). Put a finger on a part | |
| 525 | of the arm and drag: that part's joint turns, and only that joint goes to | |
| 526 | the arm, live, through the same latest-wins stream as the sliders (at most | |
| 527 | one frame per 33 ms, `src/ui/frames.js`). A finger on empty space orbits | |
| 528 | the view, and two fingers orbit and pinch to zoom. None of that sends | |
| 529 | anything. The model is drawn where you asked the arm to go; a faint blue | |
| 530 | ghost shows where the real arm is, whenever it lags behind by more than a | |
| 531 | few ticks. Lifting the finger sends the joint's last target at once, not | |
| 532 | at the next frame. So do pointercancel, a lost capture, a hidden page and | |
| 533 | a window that loses focus. When it is not your turn, or the arm is | |
| 534 | stopped or reconnecting, touching the model only orbits it. The mouse | |
| 535 | works the same way on a wide screen. | |
| 536 | - Controls tab: every joint's slider at once, one compact row each (name · | |
| 537 | slider · value). On each rail a tall blue mark shows your target and a | |
| 538 | thin line shows where the arm is. Wider screens add hold-to-jog −/+ | |
| 539 | buttons per joint. Below the sliders come the poses, Hold, Record and | |
| 540 | the rest. | |
| 541 | - Notes never cover the camera or the model. The stop, the arm's warnings | |
| 542 | (reconnecting, the simulation, a jam, the safe zone), the owner's | |
| 543 | announcement and the page's replies become one pill in the strip ("2 | |
| 544 | notes", red for a stop or an error). Tap it to read them in a sheet, | |
| 545 | where **Resume driving** and **Clear the stop** are too. | |
| 546 | - Android buzzes when a finger takes a joint. The screen stays awake during | |
| 547 | your turn (Screen Wake Lock). The status line shows the round trip to the | |
| 548 | arm once it passes 250 ms. The server's limits clamp every frame as | |
| 549 | before. | |
| 561 | 550 | - Chat: the page follows the visual viewport. The keyboard shrinks the |
| 562 | 551 | sheet and the view, and never covers the box or the last message. |
| 563 | 552 | Long-press a message (or right-click it) to delete it, or mute or ban its |
| 564 | 553 | author. |
| 565 | - Phone on its side: the view fills the screen, the strip and the sheet | |
| 566 | float over it, and **Hide** clears them away. A wide screen puts the | |
| 554 | - Phone on its side: the camera and the model share the screen side by | |
| 555 | side, the strip and the sheet float over them, and **Hide** clears them | |
| 556 | away. A wide screen puts the | |
| 567 | 557 | sheet in a column beside the view. |
| 568 | - Every tap target is at least 44 px. The safe-area insets are respected, | |
| 558 | - Checked in a browser: `src/layout.test.ts` loads the page in headless | |
| 559 | Chromium (over the DevTools protocol, `src/browser.ts`, no dependency) at | |
| 560 | 390×844 and 412×915, light and dark. It checks that the camera is on top | |
| 561 | at 16:9 with the model under it, that all six sliders fit in the tray, | |
| 562 | and that no note overlaps either view. Without a Chromium that can start, | |
| 563 | the suite is skipped. `bun bench/shots.ts [dir]` takes the screenshots: | |
| 564 | both phones, upright and on their side, light and dark, with the tray | |
| 565 | open, folded, stopped and with the notes sheet. | |
| 566 | - Every button is at least 44 px. The tray's grabber is thinner, and | |
| 567 | tapping the open tab does the same. The safe-area insets are respected, | |
| 569 | 568 | the page follows the system's light or dark mode, and nothing moves for |
| 570 | 569 | people who ask for reduced motion. The bottom-right corner belongs to the |
| 571 | 570 | forge pill, so no control sits under it. |
| 572 | - Mobile data: the first view needs no three.js. The model and the | |
| 573 | joystick arrive after it. The relay gzips the page, three.js, the URDF | |
| 571 | - Mobile data: the first view needs no three.js. The model and its touch | |
| 572 | handling arrive after it. The relay gzips the page, three.js, the URDF | |
| 574 | 573 | and the meshes (`src/compress.ts`). The page's modules are cached for |
| 575 | 574 | good under a name that changes with their content. On a slow link, a |
| 576 | 575 | slow `/status` round trip or the browser's Save-Data, the camera drops |
Software/so101/bench/shots.ts added +55
| @@ -0,0 +1,55 @@ | ||
| 1 | /** | |
| 2 | * Screenshots of the page on phones, for a pull: iPhone (390x844) and Android (412x915), upright and on its side, | |
| 3 | * light and dark, each with the tray open, the tray folded and the notes sheet open (after a Stop). | |
| 4 | * bun bench/shots.ts [out-dir] (default ./shots; CHROMIUM=/path/to/chrome picks the browser) | |
| 5 | * The arm is the simulated bus, the model the repository's URDF; there is no camera, so its pane is black. | |
| 6 | */ | |
| 7 | import { mkdirSync, mkdtempSync, writeFileSync } from "node:fs"; | |
| 8 | import { tmpdir } from "node:os"; | |
| 9 | import { join } from "node:path"; | |
| 10 | import { launch } from "../src/browser.ts"; | |
| 11 | import { FeetechBus } from "../src/bus.ts"; | |
| 12 | import { FakeBus } from "../src/fake-bus.ts"; | |
| 13 | import { startServer } from "../src/server.ts"; | |
| 14 | ||
| 15 | const out = process.argv[2] ?? "shots"; | |
| 16 | mkdirSync(out, { recursive: true }); | |
| 17 | const browser = await launch(); | |
| 18 | if (!browser) throw new Error("no headless Chromium could start: set CHROMIUM to one"); | |
| 19 | const fake = new FakeBus([1, 2, 3, 4, 5, 6]); | |
| 20 | for (const id of [1, 2, 3, 4, 5, 6]) fake.servos.get(id)!.set([0x32, 0x03, 0x9f, 0x0d], 9); | |
| 21 | const server = await startServer(new FeetechBus(fake), { host: "127.0.0.1", port: 0, posesFile: join(mkdtempSync(join(tmpdir(), "so101-shots-")), "poses.json") }); | |
| 22 | const base = `http://127.0.0.1:${server.port}`; | |
| 23 | ||
| 24 | for (const [phone, w, h] of [["iphone", 390, 844], ["android", 412, 915]] as const) { | |
| 25 | for (const [turn, width, height] of [["portrait", w, h], ["landscape", h, w]] as const) { | |
| 26 | for (const dark of [false, true]) { | |
| 27 | const page = await browser.page(); | |
| 28 | await page.view({ width, height, dark, scale: 2 }); | |
| 29 | await page.goto(`${base}/?stream=${encodeURIComponent(`${base}/no-camera/stream`)}`); | |
| 30 | await page.until(`document.querySelectorAll("#joints .j").length === 6`); | |
| 31 | await page.until(`document.querySelector("#pane3 canvas")`, 15_000).catch(() => {}); | |
| 32 | await Bun.sleep(1500); // the meshes | |
| 33 | const name = (state: string) => join(out, `${phone}-${turn}-${dark ? "dark" : "light"}-${state}.png`); | |
| 34 | writeFileSync(name("tray"), await page.shot()); | |
| 35 | if (turn === "portrait") { | |
| 36 | await page.eval(`document.getElementById("fold").click()`); | |
| 37 | await Bun.sleep(200); | |
| 38 | writeFileSync(name("folded"), await page.shot()); | |
| 39 | await page.eval(`document.getElementById("fold").click()`); | |
| 40 | } | |
| 41 | await page.eval(`document.getElementById("stop").dispatchEvent(new PointerEvent("pointerdown", { button: 0, bubbles: true }))`); | |
| 42 | await page.until(`!document.getElementById("notes").hidden`); | |
| 43 | await Bun.sleep(400); | |
| 44 | writeFileSync(name("stopped"), await page.shot()); | |
| 45 | await page.eval(`document.getElementById("notes").click()`); | |
| 46 | await Bun.sleep(200); | |
| 47 | writeFileSync(name("notes"), await page.shot()); | |
| 48 | await page.eval(`fetch("/resume", { method: "POST", body: "{}" })`); | |
| 49 | console.log(`${phone} ${turn} ${dark ? "dark" : "light"}`); | |
| 50 | } | |
| 51 | } | |
| 52 | } | |
| 53 | await browser.close(); | |
| 54 | await Promise.race([server.close(), Bun.sleep(500)]); | |
| 55 | console.log(`screenshots in ${out}`); |
Software/so101/src/browser.ts added +125
| @@ -0,0 +1,125 @@ | ||
| 1 | /** | |
| 2 | * A headless Chromium driven over the DevTools protocol, for the layout test (src/layout.test.ts) and the screenshots | |
| 3 | * (bench/shots.ts); the app never loads it. No dependency: the browser the machine has, a WebSocket and a few commands. | |
| 4 | * `launch` resolves null when there is no browser, or when it cannot start (a sandbox without sockets). | |
| 5 | */ | |
| 6 | import { existsSync, mkdtempSync, rmSync } from "node:fs"; | |
| 7 | import { tmpdir } from "node:os"; | |
| 8 | import { join } from "node:path"; | |
| 9 | ||
| 10 | const BROWSERS = [process.env.CHROMIUM, "/usr/bin/chromium", "/usr/bin/chromium-browser", "/usr/bin/google-chrome", "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"]; | |
| 11 | ||
| 12 | export type View = { width: number; height: number; mobile?: boolean; dark?: boolean; scale?: number }; | |
| 13 | ||
| 14 | export type Page = { | |
| 15 | /** The viewport: a phone (touch, portrait or landscape by its shape) or not, light or dark. */ | |
| 16 | view(v: View): Promise<void>; | |
| 17 | /** Load a page and wait for its load event. */ | |
| 18 | goto(url: string): Promise<void>; | |
| 19 | /** An expression's value, awaited when it is a promise; a thrown error comes back as one. */ | |
| 20 | eval<T>(expression: string): Promise<T>; | |
| 21 | /** Wait until an expression is truthy. */ | |
| 22 | until(expression: string, timeoutMs?: number): Promise<void>; | |
| 23 | /** A PNG of the viewport. */ | |
| 24 | shot(): Promise<Uint8Array>; | |
| 25 | }; | |
| 26 | ||
| 27 | export type Browser = { page(): Promise<Page>; close(): Promise<void> }; | |
| 28 | ||
| 29 | type Message = { id?: number; method?: string; params?: Record<string, unknown>; sessionId?: string; result?: Record<string, unknown>; error?: { message: string } }; | |
| 30 | ||
| 31 | /** Where the browser says its DevTools listen; the rest of its stderr is drained so it never blocks on a full pipe. */ | |
| 32 | async function devtoolsUrl(stream: ReadableStream<Uint8Array>): Promise<string | null> { | |
| 33 | const reader = stream.getReader(), decoder = new TextDecoder(); | |
| 34 | let text = ""; | |
| 35 | for (;;) { | |
| 36 | const { value, done } = await reader.read(); | |
| 37 | if (done) return null; | |
| 38 | text += decoder.decode(value, { stream: true }); | |
| 39 | const url = /DevTools listening on (ws:\/\/\S+)/.exec(text)?.[1]; | |
| 40 | if (url) { | |
| 41 | void (async () => { while (!(await reader.read().catch(() => ({ done: true }))).done); })(); | |
| 42 | return url; | |
| 43 | } | |
| 44 | } | |
| 45 | } | |
| 46 | ||
| 47 | export async function launch(timeoutMs = 15_000): Promise<Browser | null> { | |
| 48 | const bin = BROWSERS.find((p): p is string => Boolean(p) && existsSync(p!)); | |
| 49 | if (!bin) return null; | |
| 50 | const profile = mkdtempSync(join(tmpdir(), "so101-chromium-")); | |
| 51 | const proc = Bun.spawn([bin, "--headless=new", "--no-sandbox", "--no-first-run", "--no-default-browser-check", "--disable-dev-shm-usage", "--disable-crash-reporter", | |
| 52 | "--hide-scrollbars", "--mute-audio", "--use-angle=swiftshader", "--enable-unsafe-swiftshader", `--user-data-dir=${profile}`, "--remote-debugging-port=0", "about:blank"], | |
| 53 | { stdout: "ignore", stderr: "pipe", env: { ...process.env, HOME: profile } }); | |
| 54 | const gone = () => { try { proc.kill(); } catch {} rmSync(profile, { recursive: true, force: true }); }; | |
| 55 | const url = await Promise.race([devtoolsUrl(proc.stderr), Bun.sleep(timeoutMs).then(() => null)]); | |
| 56 | if (!url) return gone(), null; | |
| 57 | const ws = new WebSocket(url); | |
| 58 | if (!(await new Promise<boolean>((resolve) => { ws.onopen = () => resolve(true); ws.onerror = () => resolve(false); }))) return gone(), null; | |
| 59 | let next = 0; | |
| 60 | const pending = new Map<number, { resolve: (r: Record<string, unknown>) => void; reject: (e: Error) => void }>(); | |
| 61 | const events: Array<(m: Message) => void> = []; | |
| 62 | ws.onmessage = (e) => { | |
| 63 | const m = JSON.parse(String(e.data)) as Message; | |
| 64 | if (m.id !== undefined) { | |
| 65 | const p = pending.get(m.id); | |
| 66 | pending.delete(m.id); | |
| 67 | if (m.error) p?.reject(new Error(m.error.message)); else p?.resolve(m.result ?? {}); | |
| 68 | } else for (const f of [...events]) f(m); | |
| 69 | }; | |
| 70 | const send = (method: string, params: Record<string, unknown> = {}, sessionId?: string) => new Promise<Record<string, unknown>>((resolve, reject) => { | |
| 71 | const id = ++next; | |
| 72 | pending.set(id, { resolve, reject }); | |
| 73 | ws.send(JSON.stringify({ id, method, params, ...(sessionId ? { sessionId } : {}) })); | |
| 74 | }); | |
| 75 | ||
| 76 | async function page(): Promise<Page> { | |
| 77 | const { targetId } = await send("Target.createTarget", { url: "about:blank" }); | |
| 78 | const { sessionId } = (await send("Target.attachToTarget", { targetId, flatten: true })) as { sessionId: string }; | |
| 79 | const on = (method: string, params: Record<string, unknown> = {}) => send(method, params, sessionId); | |
| 80 | await on("Page.enable"); | |
| 81 | await on("Runtime.enable"); | |
| 82 | const self: Page = { | |
| 83 | async view({ width, height, mobile = true, dark = false, scale = 1 }) { | |
| 84 | await on("Emulation.setDeviceMetricsOverride", { width, height, deviceScaleFactor: scale, mobile, screenOrientation: width > height ? { type: "landscapePrimary", angle: 90 } : { type: "portraitPrimary", angle: 0 } }); | |
| 85 | await on("Emulation.setTouchEmulationEnabled", { enabled: mobile, maxTouchPoints: mobile ? 5 : 0 }); | |
| 86 | await on("Emulation.setEmulatedMedia", { features: [{ name: "prefers-color-scheme", value: dark ? "dark" : "light" }] }); | |
| 87 | }, | |
| 88 | async goto(url) { | |
| 89 | const loaded = new Promise<void>((resolve) => { | |
| 90 | const f = (m: Message) => { if (m.sessionId === sessionId && m.method === "Page.loadEventFired") { events.splice(events.indexOf(f), 1); resolve(); } }; | |
| 91 | events.push(f); | |
| 92 | }); | |
| 93 | await on("Page.navigate", { url }); | |
| 94 | await Promise.race([loaded, Bun.sleep(15_000)]); | |
| 95 | }, | |
| 96 | async eval<T>(expression: string) { | |
| 97 | const r = (await on("Runtime.evaluate", { expression, returnByValue: true, awaitPromise: true })) as { result: { value?: T }; exceptionDetails?: { text: string; exception?: { description?: string } } }; | |
| 98 | if (r.exceptionDetails) throw new Error(r.exceptionDetails.exception?.description ?? r.exceptionDetails.text); | |
| 99 | return r.result.value as T; | |
| 100 | }, | |
| 101 | async until(expression, timeoutMs = 10_000) { | |
| 102 | const end = Date.now() + timeoutMs; | |
| 103 | while (!(await self.eval<boolean>(`Boolean(${expression})`))) { | |
| 104 | if (Date.now() > end) throw new Error(`timed out waiting for ${expression}`); | |
| 105 | await Bun.sleep(50); | |
| 106 | } | |
| 107 | }, | |
| 108 | async shot() { | |
| 109 | const { data } = (await on("Page.captureScreenshot", { format: "png" })) as { data: string }; | |
| 110 | return Buffer.from(data, "base64"); | |
| 111 | }, | |
| 112 | }; | |
| 113 | return self; | |
| 114 | } | |
| 115 | ||
| 116 | return { | |
| 117 | page, | |
| 118 | async close() { | |
| 119 | await send("Browser.close").catch(() => {}); | |
| 120 | ws.close(); | |
| 121 | await Promise.race([proc.exited, Bun.sleep(2000)]); | |
| 122 | gone(); | |
| 123 | }, | |
| 124 | }; | |
| 125 | } |
Software/so101/src/layout.test.ts added +71
| @@ -0,0 +1,71 @@ | ||
| 1 | import { afterAll, beforeAll, describe, expect, test } from "bun:test"; | |
| 2 | import { mkdtempSync } from "node:fs"; | |
| 3 | import { tmpdir } from "node:os"; | |
| 4 | import { join } from "node:path"; | |
| 5 | import { launch, type Browser } from "./browser.ts"; | |
| 6 | import { FeetechBus } from "./bus.ts"; | |
| 7 | import { FakeBus } from "./fake-bus.ts"; | |
| 8 | import { startServer, type RunningServer } from "./server.ts"; | |
| 9 | ||
| 10 | // The page in a real browser at a phone's size: where the camera, the model, the sliders and the notes end up. | |
| 11 | // Without a Chromium that can start (none installed, or a sandbox without sockets) this suite is skipped. | |
| 12 | const browser: Browser | null = await launch(); | |
| 13 | if (!browser) console.warn("layout.test.ts: skipped, no headless Chromium could start here"); | |
| 14 | ||
| 15 | type Box = { x: number; y: number; w: number; h: number }; | |
| 16 | const overlap = (a: Box, b: Box) => a.w > 0 && a.h > 0 && b.w > 0 && b.h > 0 && a.x < b.x + b.w - 0.5 && b.x < a.x + a.w - 0.5 && a.y < b.y + b.h - 0.5 && b.y < a.y + a.h - 0.5; | |
| 17 | /** In the page: the boxes of the camera, the model, the strip, the tray, each slider row, and every note on screen. */ | |
| 18 | const BOXES = `(() => { | |
| 19 | const box = (e) => { const r = e.getBoundingClientRect(); return { x: r.left, y: r.top, w: r.width, h: r.height }; }; | |
| 20 | const shown = (e) => e.checkVisibility ? e.checkVisibility() : e.getClientRects().length > 0; | |
| 21 | return { | |
| 22 | camera: box(document.querySelector(".pane.on:not(#pane3)")), model: box(document.getElementById("pane3")), | |
| 23 | strip: box(document.getElementById("strip")), tray: box(document.getElementById("tab-controls")), | |
| 24 | rows: [...document.querySelectorAll("#joints .j")].map(box), | |
| 25 | notes: [...document.querySelectorAll("#notes, #banner, #msg, #notice, [role=alert], [role=status], dialog")].filter(shown).map((e) => ({ id: e.id, ...box(e) })), | |
| 26 | }; | |
| 27 | })()`; | |
| 28 | ||
| 29 | let fake: FakeBus; | |
| 30 | let server: RunningServer; | |
| 31 | let base: string; | |
| 32 | ||
| 33 | describe.skipIf(!browser)("on a phone held upright", () => { | |
| 34 | beforeAll(async () => { | |
| 35 | fake = new FakeBus([1, 2, 3, 4, 5, 6]); | |
| 36 | for (const id of [1, 2, 3, 4, 5, 6]) fake.servos.get(id)!.set([0x32, 0x03, 0x9f, 0x0d], 9); // limits 818-3487 | |
| 37 | server = await startServer(new FeetechBus(fake), { host: "127.0.0.1", port: 0, posesFile: join(mkdtempSync(join(tmpdir(), "so101-layout-")), "poses.json") }); | |
| 38 | base = `http://127.0.0.1:${server.port}`; | |
| 39 | }); | |
| 40 | afterAll(async () => { | |
| 41 | await browser?.close(); | |
| 42 | await Promise.race([server?.close(), Bun.sleep(500)]); | |
| 43 | }); | |
| 44 | ||
| 45 | for (const [width, height] of [[390, 844], [412, 915]] as const) { | |
| 46 | for (const dark of [false, true]) { | |
| 47 | test(`${width}x${height} ${dark ? "dark" : "light"}: the camera on top, the model under it, every slider in the tray, and no note over either`, async () => { | |
| 48 | const page = await browser!.page(); | |
| 49 | await page.view({ width, height, dark }); | |
| 50 | await page.goto(`${base}/?stream=${encodeURIComponent(`${base}/no-camera/stream`)}`); | |
| 51 | await page.until(`document.querySelectorAll(".pane.on").length >= 2 && document.querySelectorAll("#joints .j").length === 6`); | |
| 52 | let b = await page.eval<{ camera: Box; model: Box; strip: Box; tray: Box; rows: Box[]; notes: Array<Box & { id: string }> }>(BOXES); | |
| 53 | expect(b.camera.y).toBeLessThan(1); | |
| 54 | expect(b.camera.w).toBeCloseTo(width, 0); | |
| 55 | expect(Math.abs(b.camera.h - (width * 9) / 16)).toBeLessThan(2); | |
| 56 | expect(b.model.y).toBeGreaterThanOrEqual(b.camera.y + b.camera.h - 0.5); // under it, both whole | |
| 57 | expect(b.model.h).toBeGreaterThan(150); | |
| 58 | expect(b.strip.y).toBeGreaterThanOrEqual(b.model.y + b.model.h - 0.5); | |
| 59 | expect(b.rows.length).toBe(6); | |
| 60 | for (const row of b.rows) expect(row.y + row.h).toBeLessThanOrEqual(b.tray.y + b.tray.h + 0.5); // all six at once, none scrolled away | |
| 61 | // A stop raises notes: the held banner and the reply. They show as one pill in the strip, never over the view. | |
| 62 | await page.eval(`document.getElementById("stop").dispatchEvent(new PointerEvent("pointerdown", { button: 0, bubbles: true }))`); | |
| 63 | await page.until(`!document.getElementById("notes").hidden`); | |
| 64 | b = await page.eval(BOXES); | |
| 65 | expect(b.notes.map((n) => n.id)).toContain("notes"); | |
| 66 | for (const n of b.notes) for (const view of [b.camera, b.model]) expect({ note: n.id, overlaps: overlap(n, view) }).toEqual({ note: n.id, overlaps: false }); | |
| 67 | await page.eval(`fetch("/resume", { method: "POST", body: "{}" })`); | |
| 68 | }); | |
| 69 | } | |
| 70 | } | |
| 71 | }); |
Software/so101/src/relay.test.ts +3 −3
| @@ -161,17 +161,17 @@ describe("relay", () => { | ||
| 161 | 161 | expect(page.headers.get("content-encoding")).toBe("gzip"); |
| 162 | 162 | expect(page.headers.get("x-content-type-options")).toBe("nosniff"); |
| 163 | 163 | const html = gunzipSync(Buffer.from(await page.arrayBuffer())).toString(); |
| 164 | const module = /\/ui\/[0-9a-f]{12}\/reach\.js/.exec(html)?.[0]; | |
| 164 | const module = /\/ui\/[0-9a-f]{12}\/grab\.js/.exec(html)?.[0]; | |
| 165 | 165 | expect(module).toBeDefined(); |
| 166 | 166 | const js = await raw(module!); |
| 167 | 167 | expect(js.status).toBe(200); |
| 168 | 168 | expect(js.headers.get("cache-control")).toContain("immutable"); |
| 169 | 169 | expect(js.headers.get("content-encoding")).toBe("gzip"); |
| 170 | expect(gunzipSync(Buffer.from(await js.arrayBuffer())).toString()).toContain("export function reachStep"); | |
| 170 | expect(gunzipSync(Buffer.from(await js.arrayBuffer())).toString()).toContain("export class Grab"); | |
| 171 | 171 | // A client that asks for nothing compressed gets the plain bytes. |
| 172 | 172 | const plain = await fetch(`${base}${module}`, { headers: { "accept-encoding": "identity" }, decompress: false } as RequestInit); |
| 173 | 173 | expect(plain.headers.get("content-encoding")).toBeNull(); |
| 174 | expect(await plain.text()).toContain("export function reachStep"); | |
| 174 | expect(await plain.text()).toContain("export class Grab"); | |
| 175 | 175 | }); |
| 176 | 176 | |
| 177 | 177 | test("set: status, poses and /follow go to the arm server; the camera's bytes come through", async () => { |
Software/so101/src/server.test.ts +26 −8
| @@ -160,7 +160,7 @@ describe("viewer page", () => { | ||
| 160 | 160 | } |
| 161 | 161 | expect(bytes).toBeLessThan(50_000); |
| 162 | 162 | // three.js, the leader's mapping, the calibration panel and the Machine panel are imported where they are used, not up front. |
| 163 | for (const lazy of ["three", "/leader-map.js", "calibration.js", "reach.js", "machine-panel.js", "machine.js"]) expect([...upfront].some((p) => p === lazy || p.endsWith(`/${lazy}`))).toBe(false); | |
| 163 | for (const lazy of ["three", "/leader-map.js", "calibration.js", "grab.js", "machine-panel.js", "machine.js"]) expect([...upfront].some((p) => p === lazy || p.endsWith(`/${lazy}`))).toBe(false); | |
| 164 | 164 | expect(script).toContain('import("/leader-map.js")'); |
| 165 | 165 | expect(script).toMatch(/import\("\/ui\/[0-9a-f]{12}\/calibration\.js"\)/); |
| 166 | 166 | expect(script).toMatch(/import\("\/ui\/[0-9a-f]{12}\/machine-panel\.js"\)/); |
| @@ -176,7 +176,7 @@ describe("viewer page", () => { | ||
| 176 | 176 | expect(r.headers.get("content-type")).toBe("text/javascript"); |
| 177 | 177 | expect(r.headers.get("cache-control")).toContain("immutable"); |
| 178 | 178 | const version = path!.split("/")[2]; |
| 179 | expect((await fetch(`${base}/ui/${version}/reach.test.ts`)).status).toBe(404); | |
| 179 | expect((await fetch(`${base}/ui/${version}/grab.test.ts`)).status).toBe(404); | |
| 180 | 180 | expect((await fetch(`${base}/ui/${version}/nothing.js`)).status).toBe(404); |
| 181 | 181 | expect((await fetch(`${base}/ui/${version}/..%2Fserver.ts`)).status).toBe(404); |
| 182 | 182 | }); |
| @@ -191,19 +191,37 @@ describe("viewer page", () => { | ||
| 191 | 191 | expect(page).toMatch(/#tabs \{[^}]*var\(--pill\)/); |
| 192 | 192 | }); |
| 193 | 193 | |
| 194 | test("serves the touch-driving modules, and the phone layout keeps the browser's gestures off the controls", async () => { | |
| 194 | test("notes never cover the camera or the model: they sit in a sheet the strip's pill opens", async () => { | |
| 195 | 195 | const page = await (await fetch(`${base}/`)).text(); |
| 196 | for (const name of ["drive", "frames"]) { | |
| 196 | const view = page.slice(page.indexOf('<div id="app">'), page.indexOf('<section id="strip"')); | |
| 197 | const sheet = page.slice(page.indexOf('<dialog id="notesDlg"'), page.indexOf("</dialog>", page.indexOf('<dialog id="notesDlg"'))); | |
| 198 | for (const id of ["banner", "msg", "notice"]) { | |
| 199 | expect(view).not.toContain(`id="${id}"`); | |
| 200 | expect(sheet).toContain(`id="${id}"`); | |
| 201 | } | |
| 202 | expect(view).not.toMatch(/role="(alert|status)"/); | |
| 203 | expect(page.slice(page.indexOf('<section id="strip"'), page.indexOf("</section>", page.indexOf('<section id="strip"')))).toContain('<button id="notes"'); | |
| 204 | // A phone held upright stacks them: the camera at 16:9 on top, the model under it. | |
| 205 | const upright = page.slice(page.indexOf("@media (max-width: 640px) and (orientation: portrait)")); | |
| 206 | expect(upright).toMatch(/#stage \{ flex-direction: column; \}/); | |
| 207 | expect(upright).toMatch(/\.pane:not\(#pane3\) \{ flex: none; aspect-ratio: 16 \/ 9; \}/); | |
| 208 | }); | |
| 209 | ||
| 210 | test("drives by touching the model: its module is served, the canvas keeps the browser's gestures, and no joystick remains", async () => { | |
| 211 | const page = await (await fetch(`${base}/`)).text(); | |
| 212 | for (const name of ["frames", "grab"]) { | |
| 197 | 213 | const path = new RegExp(`/ui/[0-9a-f]{12}/${name}\\.js`).exec(page)?.[0]; |
| 198 | 214 | expect(path).toBeDefined(); |
| 199 | 215 | const r = await fetch(`${base}${path}`); |
| 200 | 216 | expect(r.status).toBe(200); |
| 201 | 217 | expect(r.headers.get("cache-control")).toContain("immutable"); |
| 202 | 218 | } |
| 203 | expect(page).toMatch(/#lever \{[^}]*touch-action: none/); | |
| 204 | expect(page).toMatch(/\.stick \{[^}]*touch-action: none/); | |
| 205 | expect(page).toMatch(/#grips button \{[^}]*min-height: (7[2-9]|[89]\d)px/); | |
| 206 | expect(page).toMatch(/#chips button small \{[^}]*white-space: nowrap/); // a long label never wraps and shifts the row | |
| 219 | expect(page).toContain('canvas.style.touchAction = "none"'); | |
| 220 | expect(page).toMatch(/import\("\/ui\/[0-9a-f]{12}\/grab\.js"\)/); // with three.js, after the first view | |
| 221 | // The joysticks are gone, on a phone and on a wide screen alike, and so are the one-joint lever and its chips. | |
| 222 | for (const gone of ['id="sticks"', 'class="stick"', 'id="pad"', 'id="reach"', 'id="lever"', 'id="chips"', "drive.js", "reach.js", "joystick"]) expect(page).not.toContain(gone); | |
| 223 | // Every joint has its slider at once. | |
| 224 | expect(page).toContain('<div id="joints"></div>'); | |
| 207 | 225 | // One STOP in every layout: one stop control in the markup, and no layout hides it. Resume is its own control. |
| 208 | 226 | expect(page.match(/<button[^>]*data-stop/g)?.length).toBe(1); |
| 209 | 227 | expect(page).not.toMatch(/#stop \{[^}]*display: none/); |
Software/so101/src/ui/calibration.js +1 −1
| @@ -35,7 +35,7 @@ const ARTICLE = ` | ||
| 35 | 35 | <p>This page drives an <a href="https://github.com/TheRobotStudio/SO-ARM100" target="_blank" rel="noopener">SO-101</a>, the open robot arm by The Robot Studio, with the calibration conventions of <a href="https://huggingface.co/docs/lerobot/so101" target="_blank" rel="noopener">Hugging Face LeRobot</a>. The 3D view uses the arm's own URDF through <a href="https://github.com/gkjohnson/urdf-loaders" target="_blank" rel="noopener">urdf-loader</a> and three.js. The code, the build notes and the quest are in the repository's <code>Software/</code> folder.</p> |
| 36 | 36 | </article>`; |
| 37 | 37 | |
| 38 | /** The page's document: this module only runs there (the type check has no DOM, as for src/ui/drive.js). @type {any} */ | |
| 38 | /** The page's document: this module only runs there (the type check has no DOM, as for src/ui/intent.js). @type {any} */ | |
| 39 | 39 | const doc = /** @type {any} */ (globalThis).document; |
| 40 | 40 | |
| 41 | 41 | /** @param {string} tag @param {Record<string, unknown>} [props] @param {...any} children @returns {any} */ |
Software/so101/src/ui/drive.js deleted −356
| @@ -1,356 +0,0 @@ | ||
| 1 | /** | |
| 2 | * Touch driving: a finger's deflection is a joint's velocity, never its position. Plain ESM, served to the page at | |
| 3 | * /ui/ and imported by the tests. The wire stays what /follow always took, a target in ticks per joint: each held | |
| 4 | * joint's target runs ahead at the deflection's share of the jog speed, and the moment the last finger on it lifts | |
| 5 | * the target stops where it is and goes out at once. That frame is the zero: the arm's governor settles on it. | |
| 6 | * The server's limits still clamp every frame; this only keeps the page from asking for more. | |
| 7 | */ | |
| 8 | ||
| 9 | /** The tick direction that raises the gripper, swings the base right, rolls the wrist or opens the jaw (from the URDF). */ | |
| 10 | export const UP = /** @type {Record<number, number>} */ ({ 1: 1, 2: -1, 3: -1, 4: -1, 5: 1, 6: 1 }); | |
| 11 | ||
| 12 | /** | |
| 13 | * A raw deflection (-1..1) through a dead band and a gentle curve: nothing near the middle, fine control | |
| 14 | * a little way out, full speed at the edge. | |
| 15 | * @param {number} v | |
| 16 | * @param {number} [dead] | |
| 17 | */ | |
| 18 | export function shape(v, dead = 0.12) { | |
| 19 | if (!Number.isFinite(v)) return 0; | |
| 20 | const a = Math.min(1, Math.abs(v)); | |
| 21 | if (a <= dead) return 0; | |
| 22 | return Math.sign(v) * ((a - dead) / (1 - dead)) ** 1.5; | |
| 23 | } | |
| 24 | ||
| 25 | /** @typedef {Record<number, { min: number, max: number }>} Limits */ | |
| 26 | /** @typedef {Record<number, number>} Frame */ | |
| 27 | ||
| 28 | /** | |
| 29 | * Whether a finger may drive now: it is this person's turn, the arm is not stopped and not reconnecting, and its | |
| 30 | * limits are known. The page asks before a press starts and on every status poll, letting go of whatever is held | |
| 31 | * the moment the answer turns false. | |
| 32 | * @param {{ mayDrive: boolean, estop?: unknown, held?: unknown, reconnecting?: unknown, limits?: Limits | null }} state | |
| 33 | */ | |
| 34 | export function canDrive({ mayDrive, estop, held, reconnecting, limits }) { | |
| 35 | return Boolean(mayDrive) && !estop && !held && !reconnecting && Object.keys(limits ?? {}).length > 0; | |
| 36 | } | |
| 37 | ||
| 38 | /** | |
| 39 | * The velocity integrator. Each holder (a pointer, a key) holds some joints at some deflections; a joint's velocity is | |
| 40 | * the sum of its holders', capped at 1. `step` advances the held joints and returns their targets; `release` drops a | |
| 41 | * holder and returns the targets of the joints it was moving, which is what stops them. | |
| 42 | */ | |
| 43 | export class Drive { | |
| 44 | /** @param {{ margin?: number, lead?: number }} [options] */ | |
| 45 | constructor(options = {}) { | |
| 46 | /** Ticks kept clear of each end of a joint's range, as the page's sliders always did. */ | |
| 47 | this.margin = options.margin ?? 30; | |
| 48 | /** How far a target may run ahead of where the arm last said it was: a slow arm (a speed cap) never builds up a backlog. */ | |
| 49 | this.lead = options.lead ?? 0.6; | |
| 50 | /** @type {Map<string, Map<number, number>>} holder → joint → deflection */ | |
| 51 | this.holders = new Map(); | |
| 52 | /** @type {Map<number, number>} the targets of joints being driven, in fractional ticks */ | |
| 53 | this.targets = new Map(); | |
| 54 | /** @type {Map<number, number>} where the arm last said each joint was */ | |
| 55 | this.present = new Map(); | |
| 56 | } | |
| 57 | ||
| 58 | /** Where the arm is. A joint nobody holds starts from here when it is next held. @param {Iterable<readonly [number, number]>} positions */ | |
| 59 | seed(positions) { | |
| 60 | for (const [id, p] of positions) if (Number.isFinite(p)) this.present.set(id, p); | |
| 61 | } | |
| 62 | ||
| 63 | /** Whether any joint is held off the middle. */ | |
| 64 | get moving() { | |
| 65 | for (const axes of this.holders.values()) for (const v of axes.values()) if (v !== 0) return true; | |
| 66 | return false; | |
| 67 | } | |
| 68 | ||
| 69 | /** The joints held, by anyone. */ | |
| 70 | held() { | |
| 71 | const out = new Set(); | |
| 72 | for (const axes of this.holders.values()) for (const id of axes.keys()) out.add(id); | |
| 73 | return out; | |
| 74 | } | |
| 75 | ||
| 76 | /** | |
| 77 | * A holder's joints and deflections, replacing what it held before; a joint it lets go of this way stops like a release. | |
| 78 | * @param {string} holder | |
| 79 | * @param {Record<number, number>} axes joint id → deflection, -1..1, already shaped | |
| 80 | * @returns {Frame | null} targets for joints that stopped, to send at once | |
| 81 | */ | |
| 82 | hold(holder, axes) { | |
| 83 | const before = this.holders.get(holder); | |
| 84 | const next = new Map(); | |
| 85 | for (const [k, v] of Object.entries(axes)) { | |
| 86 | const id = Number(k); | |
| 87 | if (!Number.isInteger(id) || !Number.isFinite(v)) continue; | |
| 88 | if (!this.targets.has(id)) { | |
| 89 | const p = this.present.get(id); | |
| 90 | if (p === undefined) continue; // no position yet: nothing to run ahead of | |
| 91 | this.targets.set(id, p); | |
| 92 | } | |
| 93 | next.set(id, Math.max(-1, Math.min(1, v))); | |
| 94 | } | |
| 95 | this.holders.set(holder, next); | |
| 96 | const dropped = before ? [...before.keys()].filter((id) => !next.has(id)) : []; | |
| 97 | return this.(dropped); | |
| 98 | } | |
| 99 | ||
| 100 | /** @param {string} holder @returns {Frame | null} the zero: the targets of the joints this holder moved, now still */ | |
| 101 | release(holder) { | |
| 102 | const axes = this.holders.get(holder); | |
| 103 | if (!axes) return null; | |
| 104 | this.holders.delete(holder); | |
| 105 | return this.([...axes.keys()]); | |
| 106 | } | |
| 107 | ||
| 108 | /** Every holder at once: the window lost focus, the page was hidden. @returns {Frame | null} */ | |
| 109 | releaseAll() { | |
| 110 | const ids = [...this.held()]; | |
| 111 | this.holders.clear(); | |
| 112 | return this.(ids); | |
| 113 | } | |
| 114 | ||
| 115 | /** | |
| 116 | * One small step of one joint, for a tap on − or +. | |
| 117 | * @param {number} id @param {number} ticks @param {Limits} limits | |
| 118 | * @returns {Frame | null} | |
| 119 | */ | |
| 120 | nudge(id, ticks, limits) { | |
| 121 | const from = this.targets.get(id) ?? this.present.get(id); | |
| 122 | if (from === undefined || !limits[id]) return null; | |
| 123 | const t = this.(id, from + ticks, limits); | |
| 124 | if (!this.held().has(id)) this.targets.delete(id); else this.targets.set(id, t); | |
| 125 | this.present.set(id, t); // the next nudge goes on from here, not from a stale reading | |
| 126 | return { [id]: Math.round(t) }; | |
| 127 | } | |
| 128 | ||
| 129 | /** | |
| 130 | * Advance every held joint by `dt` seconds at up to `speed` ticks/s. | |
| 131 | * @param {number} dt @param {number} speed @param {Limits} limits | |
| 132 | * @returns {{ frame: Frame | null, stops: number[] }} the targets to send, and joints that just met an end of their range | |
| 133 | */ | |
| 134 | step(dt, speed, limits) { | |
| 135 | /** @type {Map<number, number>} */ | |
| 136 | const v = new Map(); | |
| 137 | for (const axes of this.holders.values()) for (const [id, d] of axes) v.set(id, (v.get(id) ?? 0) + d); | |
| 138 | /** @type {Frame} */ | |
| 139 | const frame = {}; | |
| 140 | const stops = []; | |
| 141 | let any = false; | |
| 142 | for (const [id, sum] of v) { | |
| 143 | const t0 = this.targets.get(id); | |
| 144 | if (t0 === undefined || !limits[id]) continue; | |
| 145 | const vel = Math.max(-1, Math.min(1, sum)); | |
| 146 | if (vel === 0) continue; | |
| 147 | const want = t0 + vel * speed * dt; | |
| 148 | let t = this.(id, want, limits); | |
| 149 | const ended = t !== want; | |
| 150 | // Never more than `lead` seconds of travel ahead of the arm: a slow arm (a speed cap) is waited for, not queued | |
| 151 | // up behind. Only further travel is held back; a target never steps back toward a stale reading. | |
| 152 | const p = this.present.get(id); | |
| 153 | if (p !== undefined) { | |
| 154 | const room = Math.max(speed * this.lead, 20); | |
| 155 | t = vel > 0 ? Math.min(t, Math.max(t0, p + room)) : Math.max(t, Math.min(t0, p - room)); | |
| 156 | } | |
| 157 | if (t === t0) continue; // nowhere to go: nothing to send | |
| 158 | if (ended) stops.push(id); | |
| 159 | this.targets.set(id, t); | |
| 160 | frame[id] = Math.round(t); | |
| 161 | any = true; | |
| 162 | } | |
| 163 | return { frame: any ? frame : null, stops }; | |
| 164 | } | |
| 165 | ||
| 166 | /** @param {number} id @param {number} t @param {Limits} limits */ | |
| 167 | (id, t, limits) { | |
| 168 | const l = limits[id]; | |
| 169 | if (!l) return t; | |
| 170 | return Math.max(l.min + this.margin, Math.min(l.max - this.margin, t)); | |
| 171 | } | |
| 172 | ||
| 173 | /** @param {number[]} ids */ | |
| 174 | (ids) { | |
| 175 | const still = this.held(); | |
| 176 | /** @type {Frame} */ | |
| 177 | const frame = {}; | |
| 178 | let any = false; | |
| 179 | for (const id of ids) { | |
| 180 | if (still.has(id)) continue; // another finger still drives it | |
| 181 | const t = this.targets.get(id); | |
| 182 | this.targets.delete(id); | |
| 183 | if (t === undefined) continue; | |
| 184 | frame[id] = Math.round(t); | |
| 185 | this.present.set(id, t); // until the arm says otherwise, it is where it was told to stop | |
| 186 | any = true; | |
| 187 | } | |
| 188 | return any ? frame : null; | |
| 189 | } | |
| 190 | } | |
| 191 | ||
| 192 | /** | |
| 193 | * The page's driver: a Drive, a clock at the send rate while anything is held, and the ways a press ends. | |
| 194 | * `send` takes frames at the clock's rate (the page may coalesce them); `flush` must put a frame on the wire now, | |
| 195 | * because it is the zero; `end` says nothing is held any more. | |
| 196 | */ | |
| 197 | export class Driver { | |
| 198 | /** | |
| 199 | * @param {{ | |
| 200 | * send: (frame: Frame) => void, | |
| 201 | * flush: (frame: Frame) => void, | |
| 202 | * end: () => void, | |
| 203 | * begin?: () => boolean | void, | |
| 204 | * limits: () => Limits, | |
| 205 | * speed: () => number, | |
| 206 | * hz?: number, | |
| 207 | * onStop?: (id: number) => void, | |
| 208 | * timers?: { every: (fn: () => void, ms: number) => unknown, cancel: (handle: unknown) => void, now: () => number }, | |
| 209 | * }} options | |
| 210 | */ | |
| 211 | constructor(options) { | |
| 212 | this.o = options; | |
| 213 | this.drive = new Drive(); | |
| 214 | this.hz = Math.max(20, Math.min(30, options.hz ?? 25)); | |
| 215 | this.timers = options.timers ?? { every: (fn, ms) => setInterval(fn, ms), cancel: (h) => clearInterval(/** @type {any} */ (h)), now: () => performance.now() }; | |
| 216 | /** @type {unknown} */ | |
| 217 | this.clock = null; | |
| 218 | this.last = 0; | |
| 219 | } | |
| 220 | ||
| 221 | /** Whether a finger (or a key) is down on anything. */ | |
| 222 | get active() { | |
| 223 | return this.clock !== null; | |
| 224 | } | |
| 225 | ||
| 226 | /** @param {Iterable<readonly [number, number]>} positions */ | |
| 227 | seed(positions) { | |
| 228 | this.drive.seed(positions); | |
| 229 | } | |
| 230 | ||
| 231 | /** @param {string} holder @param {Record<number, number>} axes @returns {boolean} false when driving is refused */ | |
| 232 | hold(holder, axes) { | |
| 233 | if (!this.clock) { | |
| 234 | if (this.o.begin?.() === false) return false; | |
| 235 | this.last = this.timers.now(); | |
| 236 | this.clock = this.timers.every(() => this.tick(), 1000 / this.hz); | |
| 237 | } | |
| 238 | const stopped = this.drive.hold(holder, axes); | |
| 239 | if (stopped) this.o.flush(stopped); | |
| 240 | return true; | |
| 241 | } | |
| 242 | ||
| 243 | /** A holder lets go: its joints' zero goes out now, not at the next tick. @param {string} holder */ | |
| 244 | release(holder) { | |
| 245 | const frame = this.drive.release(holder); | |
| 246 | if (frame) this.o.flush(frame); | |
| 247 | if (this.clock && this.drive.holders.size === 0) this.(); | |
| 248 | } | |
| 249 | ||
| 250 | /** Everything lets go at once: the window lost focus, the page was hidden. */ | |
| 251 | releaseAll() { | |
| 252 | const frame = this.drive.releaseAll(); | |
| 253 | if (frame) this.o.flush(frame); | |
| 254 | if (this.clock) this.(); | |
| 255 | } | |
| 256 | ||
| 257 | /** One step of one joint, for a tap. @param {number} id @param {number} ticks signed */ | |
| 258 | nudge(id, ticks) { | |
| 259 | if (this.o.begin?.() === false) return; | |
| 260 | const frame = this.drive.nudge(id, ticks, this.o.limits()); | |
| 261 | if (frame) this.o.flush(frame); | |
| 262 | if (!this.clock) this.o.end(); | |
| 263 | } | |
| 264 | ||
| 265 | tick() { | |
| 266 | const now = this.timers.now(); | |
| 267 | const dt = Math.min(0.1, Math.max(0, (now - this.last) / 1000)); // a late tick never turns into a jump | |
| 268 | this.last = now; | |
| 269 | const { frame, stops } = this.drive.step(dt, this.o.speed(), this.o.limits()); | |
| 270 | if (frame) this.o.send(frame); | |
| 271 | for (const id of stops) this.o.onStop?.(id); | |
| 272 | } | |
| 273 | ||
| 274 | () { | |
| 275 | this.timers.cancel(this.clock); | |
| 276 | this.clock = null; | |
| 277 | this.o.end(); | |
| 278 | } | |
| 279 | ||
| 280 | /** | |
| 281 | * Every way a hold can end without a pointerup: the window loses focus, the page is hidden or put away. | |
| 282 | * @param {EventTarget} win @param {EventTarget & { hidden?: boolean }} doc | |
| 283 | */ | |
| 284 | watch(win, doc) { | |
| 285 | const all = () => this.releaseAll(); | |
| 286 | win.addEventListener("blur", all); | |
| 287 | win.addEventListener("pagehide", all); | |
| 288 | doc.addEventListener("visibilitychange", () => { if (doc.hidden) all(); }); | |
| 289 | } | |
| 290 | } | |
| 291 | ||
| 292 | /** | |
| 293 | * @typedef {EventTarget & { | |
| 294 | * setPointerCapture?: (id: number) => void, | |
| 295 | * getBoundingClientRect: () => { left: number, top: number, width: number, height: number }, | |
| 296 | * classList: { toggle: (name: string, on?: boolean) => unknown }, | |
| 297 | * }} Surface | |
| 298 | * @typedef {Event & { pointerId: number, clientX: number, clientY: number }} Pointer | |
| 299 | */ | |
| 300 | ||
| 301 | /** | |
| 302 | * One finger on a surface, tracked by its pointerId: `place` maps where it is to joint deflections, every way the | |
| 303 | * finger can leave ends the hold, and a second finger on the same surface is ignored until the first one lifts. | |
| 304 | * Two surfaces are two holders, so two thumbs drive two sets of joints independently. | |
| 305 | * @param {Surface} el | |
| 306 | * @param {Driver} driver | |
| 307 | * @param {string} holder | |
| 308 | * @param {(x: number, y: number) => Record<number, number> | null} place x, y from -1 to 1 across the surface, y up | |
| 309 | * @param {{ show?: (x: number, y: number) => void, haptic?: () => void }} [view] | |
| 310 | * @returns {{ end: () => void }} | |
| 311 | */ | |
| 312 | export function bindSurface(el, driver, holder, place, view = {}) { | |
| 313 | /** @type {number | null} */ | |
| 314 | let pointer = null; | |
| 315 | /** @param {Pointer} e */ | |
| 316 | const at = (e) => { | |
| 317 | const r = el.getBoundingClientRect(); | |
| 318 | const x = r.width ? ((e.clientX - r.left) / r.width) * 2 - 1 : 0; | |
| 319 | const y = r.height ? 1 - ((e.clientY - r.top) / r.height) * 2 : 0; | |
| 320 | return [Math.max(-1, Math.min(1, x)), Math.max(-1, Math.min(1, y))]; | |
| 321 | }; | |
| 322 | /** @param {Pointer} e */ | |
| 323 | const apply = (e) => { | |
| 324 | const [x = 0, y = 0] = at(e); | |
| 325 | view.show?.(x, y); | |
| 326 | const axes = place(x, y); | |
| 327 | if (!axes) return false; | |
| 328 | return driver.hold(holder, axes); | |
| 329 | }; | |
| 330 | const end = () => { | |
| 331 | if (pointer === null) return; | |
| 332 | pointer = null; | |
| 333 | el.classList.toggle("held", false); | |
| 334 | view.show?.(0, 0); | |
| 335 | driver.release(holder); | |
| 336 | }; | |
| 337 | el.addEventListener("pointerdown", (ev) => { | |
| 338 | const e = /** @type {Pointer} */ (ev); | |
| 339 | if (pointer !== null) return; | |
| 340 | e.preventDefault(); | |
| 341 | pointer = e.pointerId; | |
| 342 | try { el.setPointerCapture?.(e.pointerId); } catch { /* a synthetic pointer */ } | |
| 343 | el.classList.toggle("held", true); | |
| 344 | view.haptic?.(); | |
| 345 | if (apply(e) === false) end(); | |
| 346 | }); | |
| 347 | el.addEventListener("pointermove", (ev) => { | |
| 348 | const e = /** @type {Pointer} */ (ev); | |
| 349 | if (e.pointerId === pointer) apply(e); | |
| 350 | }); | |
| 351 | for (const type of ["pointerup", "pointercancel", "lostpointercapture"]) { | |
| 352 | el.addEventListener(type, (ev) => { if (/** @type {Pointer} */ (ev).pointerId === pointer) end(); }); | |
| 353 | } | |
| 354 | el.addEventListener("contextmenu", (e) => e.preventDefault()); | |
| 355 | return { end }; | |
| 356 | } |
Software/so101/src/ui/drive.test.ts deleted −293
| @@ -1,293 +0,0 @@ | ||
| 1 | import { describe, expect, test } from "bun:test"; | |
| 2 | import { bindSurface, canDrive, Drive, Driver, shape } from "./drive.js"; | |
| 3 | ||
| 4 | const LIMITS = { 1: { min: 1000, max: 3000 }, 2: { min: 1000, max: 3000 }, 3: { min: 1000, max: 3000 }, 4: { min: 1000, max: 3000 }, 6: { min: 2000, max: 3500 } }; | |
| 5 | const AT = [[1, 2000], [2, 2000], [3, 2000], [4, 2000], [6, 2500]] as const; | |
| 6 | ||
| 7 | /** A driver on a hand-wound clock that records what goes on the wire, and when. */ | |
| 8 | function rig() { | |
| 9 | const wire: Array<{ kind: "send" | "flush" | "end"; frame?: Record<number, number>; at: number }> = []; | |
| 10 | let now = 0; | |
| 11 | let tick: (() => void) | null = null; | |
| 12 | const driver = new Driver({ | |
| 13 | send: (frame) => wire.push({ kind: "send", frame, at: now }), | |
| 14 | flush: (frame) => wire.push({ kind: "flush", frame, at: now }), | |
| 15 | end: () => wire.push({ kind: "end", at: now }), | |
| 16 | limits: () => LIMITS, | |
| 17 | speed: () => 300, | |
| 18 | timers: { every: (fn) => ((tick = fn), 1), cancel: () => (tick = null), now: () => now }, | |
| 19 | }); | |
| 20 | driver.seed(AT); | |
| 21 | /** Advance the clock by one 40 ms frame and run the tick, when one is running. */ | |
| 22 | const frame = () => { | |
| 23 | now += 40; | |
| 24 | tick?.(); | |
| 25 | }; | |
| 26 | return { driver, wire, frame, running: () => tick !== null }; | |
| 27 | } | |
| 28 | ||
| 29 | /** A surface 100 px square at the origin, built on EventTarget like the page's elements. */ | |
| 30 | function surface() { | |
| 31 | const el = Object.assign(new EventTarget(), { | |
| 32 | captured: [] as number[], | |
| 33 | held: false, | |
| 34 | setPointerCapture(id: number) { this.captured.push(id); }, | |
| 35 | getBoundingClientRect: () => ({ left: 0, top: 0, width: 100, height: 100 }), | |
| 36 | classList: { toggle: (_: string, on?: boolean) => (el.held = Boolean(on)) }, | |
| 37 | }); | |
| 38 | return el; | |
| 39 | } | |
| 40 | const pointer = (type: string, pointerId: number, clientX = 50, clientY = 50) => Object.assign(new Event(type, { cancelable: true }), { pointerId, clientX, clientY }); | |
| 41 | ||
| 42 | describe("the stick's shape", () => { | |
| 43 | test("a dead band in the middle, the full speed at the edge, never more", () => { | |
| 44 | expect(shape(0.05)).toBe(0); | |
| 45 | expect(shape(-0.12)).toBe(0); | |
| 46 | expect(shape(1)).toBe(1); | |
| 47 | expect(shape(-3)).toBe(-1); | |
| 48 | expect(shape(Number.NaN)).toBe(0); | |
| 49 | expect(shape(0.5)).toBeGreaterThan(0); | |
| 50 | expect(shape(0.5)).toBeLessThan(0.5); // fine control a little way out | |
| 51 | expect(shape(-0.5)).toBe(-shape(0.5)); | |
| 52 | }); | |
| 53 | }); | |
| 54 | ||
| 55 | describe("the velocity integrator", () => { | |
| 56 | test("a held joint runs ahead at the deflection's share of the speed, from where the arm is", () => { | |
| 57 | const d = new Drive(); | |
| 58 | d.seed(AT); | |
| 59 | d.hold("a", { 1: 0.5 }); | |
| 60 | expect(d.step(0.1, 300, LIMITS).frame).toEqual({ 1: 2015 }); | |
| 61 | expect(d.step(0.1, 300, LIMITS).frame).toEqual({ 1: 2030 }); | |
| 62 | }); | |
| 63 | ||
| 64 | test("the target stops at the range's margin and says so once", () => { | |
| 65 | const d = new Drive(); | |
| 66 | d.seed([[1, 2960]]); | |
| 67 | d.hold("a", { 1: 1 }); | |
| 68 | expect(d.step(0.1, 300, LIMITS)).toEqual({ frame: { 1: 2970 }, stops: [1] }); | |
| 69 | expect(d.step(0.1, 300, LIMITS)).toEqual({ frame: null, stops: [] }); | |
| 70 | }); | |
| 71 | ||
| 72 | test("never runs more than the lead ahead of an arm that lags, and never steps back toward a stale reading", () => { | |
| 73 | const d = new Drive({ lead: 0.5 }); | |
| 74 | d.seed([[1, 2000]]); | |
| 75 | d.hold("a", { 1: 1 }); | |
| 76 | for (let i = 0; i < 20; i++) d.step(0.1, 300, LIMITS); | |
| 77 | expect(d.targets.get(1)).toBe(2150); // 0.5 s of 300 ticks/s ahead of where the arm said it was | |
| 78 | d.seed([[1, 1900]]); // a reading older than the target | |
| 79 | expect(d.step(0.1, 300, LIMITS).frame).toBeNull(); | |
| 80 | expect(d.targets.get(1)).toBe(2150); | |
| 81 | }); | |
| 82 | ||
| 83 | test("a joint without a position is not driven", () => { | |
| 84 | const d = new Drive(); | |
| 85 | d.hold("a", { 5: 1 }); | |
| 86 | expect(d.step(0.1, 300, LIMITS).frame).toBeNull(); | |
| 87 | expect(d.release("a")).toBeNull(); | |
| 88 | }); | |
| 89 | ||
| 90 | test("a tap nudges from the last target, inside the range", () => { | |
| 91 | const d = new Drive(); | |
| 92 | d.seed([[6, 3450]]); | |
| 93 | expect(d.nudge(6, 20, LIMITS)).toEqual({ 6: 3470 }); | |
| 94 | expect(d.nudge(6, 20, LIMITS)).toEqual({ 6: 3470 }); | |
| 95 | expect(d.nudge(6, -20, LIMITS)).toEqual({ 6: 3450 }); | |
| 96 | }); | |
| 97 | }); | |
| 98 | ||
| 99 | describe("the driver", () => { | |
| 100 | test("lifting the finger sends zero within one frame: the target as it stands, flushed at once, then nothing", () => { | |
| 101 | const { driver, wire, frame, running } = rig(); | |
| 102 | driver.hold("lever", { 2: 1 }); | |
| 103 | frame(); frame(); | |
| 104 | const last = wire.at(-1)!; | |
| 105 | expect(last.kind).toBe("send"); | |
| 106 | const sentAt = wire.length; | |
| 107 | driver.release("lever"); | |
| 108 | // Before any further tick of the clock: the zero is already on the wire. | |
| 109 | expect(wire.slice(sentAt)).toEqual([{ kind: "flush", frame: last.frame, at: last.at }, { kind: "end", at: last.at }]); | |
| 110 | expect(running()).toBe(false); | |
| 111 | frame(); frame(); | |
| 112 | expect(wire.length).toBe(sentAt + 2); // and nothing after it | |
| 113 | }); | |
| 114 | ||
| 115 | test("two thumbs drive independent joints: lifting one stops only its joints", () => { | |
| 116 | const { driver, wire, frame } = rig(); | |
| 117 | driver.hold("left", { 1: 1, 2: 0 }); | |
| 118 | driver.hold("right", { 3: -1 }); | |
| 119 | frame(); | |
| 120 | expect(wire.at(-1)).toEqual({ kind: "send", frame: { 1: 2012, 3: 1988 }, at: 40 }); | |
| 121 | driver.release("left"); | |
| 122 | expect(wire.at(-1)).toEqual({ kind: "flush", frame: { 1: 2012, 2: 2000 }, at: 40 }); | |
| 123 | frame(); | |
| 124 | expect(wire.at(-1)).toEqual({ kind: "send", frame: { 3: 1976 }, at: 80 }); // the right thumb goes on | |
| 125 | driver.release("right"); | |
| 126 | expect(wire.slice(-2)).toEqual([{ kind: "flush", frame: { 3: 1976 }, at: 80 }, { kind: "end", at: 80 }]); | |
| 127 | }); | |
| 128 | ||
| 129 | test("two fingers on one joint: it stops only when both have lifted", () => { | |
| 130 | const { driver, wire, frame } = rig(); | |
| 131 | driver.hold("lever", { 4: 1 }); | |
| 132 | driver.hold("plus", { 4: 1 }); | |
| 133 | frame(); | |
| 134 | driver.release("plus"); | |
| 135 | expect(wire.at(-1)!.kind).toBe("send"); | |
| 136 | frame(); | |
| 137 | expect(wire.at(-1)).toEqual({ kind: "send", frame: { 4: 2024 }, at: 80 }); | |
| 138 | driver.release("lever"); | |
| 139 | expect(wire.at(-1)!.kind).toBe("end"); | |
| 140 | }); | |
| 141 | ||
| 142 | test("blur, pagehide and a hidden page stop every joint at once", () => { | |
| 143 | for (const how of ["blur", "pagehide", "hidden"] as const) { | |
| 144 | const { driver, wire, frame, running } = rig(); | |
| 145 | const win = new EventTarget(), doc = Object.assign(new EventTarget(), { hidden: false }); | |
| 146 | driver.watch(win, doc); | |
| 147 | driver.hold("left", { 1: 1 }); | |
| 148 | driver.hold("right", { 3: 1 }); | |
| 149 | frame(); | |
| 150 | if (how === "hidden") { doc.hidden = true; doc.dispatchEvent(new Event("visibilitychange")); } | |
| 151 | else win.dispatchEvent(new Event(how)); | |
| 152 | expect(wire.slice(-2)).toEqual([{ kind: "flush", frame: { 1: 2012, 3: 2012 }, at: 40 }, { kind: "end", at: 40 }]); | |
| 153 | expect(running()).toBe(false); | |
| 154 | } | |
| 155 | }); | |
| 156 | ||
| 157 | test("a hidden page that was not driving sends nothing", () => { | |
| 158 | const { driver, wire } = rig(); | |
| 159 | const win = new EventTarget(), doc = Object.assign(new EventTarget(), { hidden: true }); | |
| 160 | driver.watch(win, doc); | |
| 161 | doc.dispatchEvent(new Event("visibilitychange")); | |
| 162 | win.dispatchEvent(new Event("blur")); | |
| 163 | expect(wire).toEqual([]); | |
| 164 | }); | |
| 165 | ||
| 166 | test("refused by the page (not your turn), a press starts nothing", () => { | |
| 167 | const wire: unknown[] = []; | |
| 168 | const driver = new Driver({ send: (f) => wire.push(f), flush: (f) => wire.push(f), end: () => wire.push("end"), begin: () => false, limits: () => LIMITS, speed: () => 300 }); | |
| 169 | driver.seed(AT); | |
| 170 | expect(driver.hold("x", { 1: 1 })).toBe(false); | |
| 171 | expect(driver.active).toBe(false); | |
| 172 | driver.nudge(1, 20); | |
| 173 | expect(wire).toEqual([]); | |
| 174 | }); | |
| 175 | ||
| 176 | test("a tap nudges and ends at once", () => { | |
| 177 | const { driver, wire } = rig(); | |
| 178 | driver.nudge(6, -20); | |
| 179 | expect(wire).toEqual([{ kind: "flush", frame: { 6: 2480 }, at: 0 }, { kind: "end", at: 0 }]); | |
| 180 | }); | |
| 181 | }); | |
| 182 | ||
| 183 | describe("a surface under a finger", () => { | |
| 184 | const lever = (y: number) => ({ 2: y }); | |
| 185 | ||
| 186 | test("pointerup sends the zero before the next frame, and the surface lets go", () => { | |
| 187 | const { driver, wire, frame } = rig(); | |
| 188 | const el = surface(); | |
| 189 | bindSurface(el, driver, "lever", (_, y) => lever(y)); | |
| 190 | el.dispatchEvent(pointer("pointerdown", 7, 50, 0)); // the top edge: full speed up | |
| 191 | expect(el.captured).toEqual([7]); | |
| 192 | expect(el.held).toBe(true); | |
| 193 | frame(); | |
| 194 | expect(wire.at(-1)).toEqual({ kind: "send", frame: { 2: 2012 }, at: 40 }); | |
| 195 | el.dispatchEvent(pointer("pointerup", 7)); | |
| 196 | expect(wire.slice(-2)).toEqual([{ kind: "flush", frame: { 2: 2012 }, at: 40 }, { kind: "end", at: 40 }]); | |
| 197 | expect(el.held).toBe(false); | |
| 198 | }); | |
| 199 | ||
| 200 | for (const type of ["pointercancel", "lostpointercapture"]) { | |
| 201 | test(`${type} ends the hold like a lift`, () => { | |
| 202 | const { driver, wire, frame, running } = rig(); | |
| 203 | const el = surface(); | |
| 204 | bindSurface(el, driver, "lever", (_, y) => lever(y)); | |
| 205 | el.dispatchEvent(pointer("pointerdown", 3, 50, 100)); | |
| 206 | frame(); | |
| 207 | el.dispatchEvent(pointer(type, 3)); | |
| 208 | expect(wire.at(-1)!.kind).toBe("end"); | |
| 209 | expect(running()).toBe(false); | |
| 210 | }); | |
| 211 | } | |
| 212 | ||
| 213 | test("a second finger on the same surface neither steers it nor ends it", () => { | |
| 214 | const { driver, wire, frame } = rig(); | |
| 215 | const el = surface(); | |
| 216 | bindSurface(el, driver, "lever", (_, y) => lever(y)); | |
| 217 | el.dispatchEvent(pointer("pointerdown", 1, 50, 0)); | |
| 218 | el.dispatchEvent(pointer("pointerdown", 2, 50, 100)); | |
| 219 | el.dispatchEvent(pointer("pointermove", 2, 50, 100)); | |
| 220 | el.dispatchEvent(pointer("pointerup", 2)); | |
| 221 | frame(); | |
| 222 | expect(wire.at(-1)).toEqual({ kind: "send", frame: { 2: 2012 }, at: 40 }); | |
| 223 | }); | |
| 224 | ||
| 225 | test("two sticks under two thumbs, each with its own pointer, drive their own joints", () => { | |
| 226 | const { driver, wire, frame } = rig(); | |
| 227 | const left = surface(), right = surface(); | |
| 228 | bindSurface(left, driver, "left", (x, y) => ({ 1: x, 2: y })); | |
| 229 | bindSurface(right, driver, "right", (x, y) => ({ 4: x, 3: y })); | |
| 230 | left.dispatchEvent(pointer("pointerdown", 11, 100, 50)); // full right | |
| 231 | right.dispatchEvent(pointer("pointerdown", 12, 50, 100)); // full down | |
| 232 | frame(); | |
| 233 | expect(wire.at(-1)).toEqual({ kind: "send", frame: { 1: 2012, 3: 1988 }, at: 40 }); | |
| 234 | right.dispatchEvent(pointer("pointerup", 11)); // the other thumb's id on this stick: ignored | |
| 235 | left.dispatchEvent(pointer("pointerup", 11)); | |
| 236 | expect(wire.at(-1)).toEqual({ kind: "flush", frame: { 1: 2012, 2: 2000 }, at: 40 }); | |
| 237 | frame(); | |
| 238 | expect(wire.at(-1)).toEqual({ kind: "send", frame: { 3: 1976 }, at: 80 }); | |
| 239 | }); | |
| 240 | ||
| 241 | test("a press the page refuses lets go of the surface at once", () => { | |
| 242 | const driver = new Driver({ send: () => {}, flush: () => {}, end: () => {}, begin: () => false, limits: () => LIMITS, speed: () => 300 }); | |
| 243 | const el = surface(); | |
| 244 | bindSurface(el, driver, "lever", (_, y) => lever(y)); | |
| 245 | el.dispatchEvent(pointer("pointerdown", 1, 50, 0)); | |
| 246 | expect(el.held).toBe(false); | |
| 247 | }); | |
| 248 | ||
| 249 | test("the arm drops while a thumb is held: the poll's releaseAll flushes the zero, and the still-pressed stick sends nothing more", () => { | |
| 250 | const wire: Array<{ kind: string; frame?: Record<number, number> }> = []; | |
| 251 | let tick: (() => void) | null = null, now = 0; | |
| 252 | const state = { mayDrive: true, estop: false, reconnecting: false, limits: LIMITS }; | |
| 253 | const driver = new Driver({ | |
| 254 | send: (frame) => wire.push({ kind: "send", frame }), | |
| 255 | flush: (frame) => wire.push({ kind: "flush", frame }), | |
| 256 | end: () => wire.push({ kind: "end" }), | |
| 257 | begin: () => canDrive(state), | |
| 258 | limits: () => LIMITS, | |
| 259 | speed: () => 300, | |
| 260 | timers: { every: (fn) => ((tick = fn), 1), cancel: () => (tick = null), now: () => now }, | |
| 261 | }); | |
| 262 | driver.seed(AT); | |
| 263 | const stick = surface(); | |
| 264 | bindSurface(stick, driver, "left", (x, y) => ({ 1: x, 2: y })); | |
| 265 | stick.dispatchEvent(pointer("pointerdown", 7, 100, 50)); | |
| 266 | now += 40; tick!(); | |
| 267 | expect(wire.at(-1)).toEqual({ kind: "send", frame: { 1: 2012 } }); | |
| 268 | // The status poll: reconnecting, so driving is refused and everything held lets go. | |
| 269 | state.reconnecting = true; | |
| 270 | if (!canDrive(state)) driver.releaseAll(); | |
| 271 | expect(wire.slice(-2)).toEqual([{ kind: "flush", frame: { 1: 2012, 2: 2000 } }, { kind: "end" }]); | |
| 272 | expect(tick).toBeNull(); | |
| 273 | const after = wire.length; | |
| 274 | stick.dispatchEvent(pointer("pointermove", 7, 100, 0)); // the thumb is still down and moving | |
| 275 | expect(wire.length).toBe(after); | |
| 276 | expect(tick).toBeNull(); | |
| 277 | }); | |
| 278 | }); | |
| 279 | ||
| 280 | describe("canDrive", () => { | |
| 281 | const ok = { mayDrive: true, estop: false, reconnecting: false, limits: LIMITS }; | |
| 282 | test("drives on your turn, with the arm up and its limits known", () => { | |
| 283 | expect(canDrive(ok)).toBe(true); | |
| 284 | }); | |
| 285 | test("refuses when it is not your turn, the arm is stopped, the arm is reconnecting, or no limits have come", () => { | |
| 286 | expect(canDrive({ ...ok, mayDrive: false })).toBe(false); | |
| 287 | expect(canDrive({ ...ok, estop: true })).toBe(false); | |
| 288 | expect(canDrive({ ...ok, held: { by: "ann", at: "2026-10-20T22:05:00Z" } })).toBe(false); // STOP, until someone resumes | |
| 289 | expect(canDrive({ ...ok, reconnecting: true })).toBe(false); | |
| 290 | expect(canDrive({ ...ok, limits: {} })).toBe(false); | |
| 291 | expect(canDrive({ ...ok, limits: null })).toBe(false); | |
| 292 | }); | |
| 293 | }); |
Software/so101/src/ui/grab.js added +106
| @@ -0,0 +1,106 @@ | ||
| 1 | /** | |
| 2 | * Driving by touching the 3D model. One finger (or the mouse) that lands on a part of the arm takes that part's | |
| 3 | * joint, and only that joint goes to the arm while it is held; a finger that lands on empty space, a second finger, | |
| 4 | * a pinch, are the view's (orbit and zoom) and never send anything. Lifting the finger sends the joint's last target | |
| 5 | * at once, and so do pointercancel, a lost capture, a hidden page and a blurred window. Plain ESM, served at /ui/ | |
| 6 | * and imported by the tests: the page hands it plain pointer data and the model's hooks. | |
| 7 | */ | |
| 8 | ||
| 9 | /** @typedef {{ id: number, x: number, y: number }} Point */ | |
| 10 | /** @typedef {Record<number, number>} Frame */ | |
| 11 | ||
| 12 | export class Grab { | |
| 13 | /** | |
| 14 | * @param {{ | |
| 15 | * allowed: () => boolean, | |
| 16 | * pick: (x: number, y: number) => string | null, | |
| 17 | * drag: (x: number, y: number) => void, | |
| 18 | * drop: (x: number, y: number) => void, | |
| 19 | * ticks: (joint: string, angle: number) => [number, number] | null, | |
| 20 | * send: (frame: Frame) => void, | |
| 21 | * flush: (frame: Frame) => void, | |
| 22 | * begin?: (joint: string) => void, | |
| 23 | * end?: (joint: string) => void, | |
| 24 | * }} io allowed: whether this page may drive now; pick: the joint under a point, taken (null: none); drag/drop: the | |
| 25 | * held finger moved/lifted there; ticks: a joint's angle as [servo id, target ticks], or null when it cannot go | |
| 26 | */ | |
| 27 | constructor(io) { | |
| 28 | this.io = io; | |
| 29 | /** @type {number | null} the pointer holding a joint */ | |
| 30 | this.pointer = null; | |
| 31 | /** @type {string | null} the joint it holds */ | |
| 32 | this.joint = null; | |
| 33 | /** @type {Frame | null} the last target sent for it */ | |
| 34 | this.last = null; | |
| 35 | /** @type {[number, number]} */ | |
| 36 | this.at = [0, 0]; | |
| 37 | } | |
| 38 | ||
| 39 | /** | |
| 40 | * A finger or the mouse comes down. True when the model keeps it: it took a joint, or a joint is already held and | |
| 41 | * this second finger must neither orbit nor drive. False: the view's, to orbit or zoom. | |
| 42 | * @param {Point} p | |
| 43 | */ | |
| 44 | down(p) { | |
| 45 | if (this.pointer !== null) return true; | |
| 46 | if (!this.io.allowed()) return false; | |
| 47 | const joint = this.io.pick(p.x, p.y); | |
| 48 | if (!joint) return false; | |
| 49 | this.pointer = p.id; this.joint = joint; this.last = null; this.at = [p.x, p.y]; | |
| 50 | this.io.begin?.(joint); | |
| 51 | return true; | |
| 52 | } | |
| 53 | ||
| 54 | /** @param {Point} p */ | |
| 55 | move(p) { | |
| 56 | if (p.id !== this.pointer) return false; | |
| 57 | this.at = [p.x, p.y]; | |
| 58 | this.io.drag(p.x, p.y); | |
| 59 | return true; | |
| 60 | } | |
| 61 | ||
| 62 | /** The finger lifted, or was cancelled or lost. @param {Point} p */ | |
| 63 | up(p) { | |
| 64 | if (p.id !== this.pointer) return false; | |
| 65 | this.at = [p.x, p.y]; | |
| 66 | this.release(); | |
| 67 | return true; | |
| 68 | } | |
| 69 | ||
| 70 | /** Let go of the held joint, wherever the finger is: its last target goes now, not at the next slot. */ | |
| 71 | release() { | |
| 72 | if (this.pointer === null) return; | |
| 73 | const joint = /** @type {string} */ (this.joint); | |
| 74 | this.io.drop(...this.at); // the model's last turn of the joint still goes out, as the held joint's | |
| 75 | this.pointer = null; this.joint = null; | |
| 76 | if (this.last) this.io.flush(this.last); | |
| 77 | this.last = null; | |
| 78 | this.io.end?.(joint); | |
| 79 | } | |
| 80 | ||
| 81 | /** | |
| 82 | * The model turned a joint. Only the held one is sent; anything else (the arm's own pose, a mirrored leader) is not. | |
| 83 | * @param {string} joint @param {number} angle | |
| 84 | * @returns {[number, number] | null} the [id, ticks] sent | |
| 85 | */ | |
| 86 | angle(joint, angle) { | |
| 87 | if (this.joint === null || joint !== this.joint) return null; | |
| 88 | const t = this.io.ticks(joint, angle); | |
| 89 | if (!t) return null; | |
| 90 | this.last = { [t[0]]: t[1] }; | |
| 91 | this.io.send(this.last); | |
| 92 | return t; | |
| 93 | } | |
| 94 | ||
| 95 | /** | |
| 96 | * Blur, pagehide and a hidden page let go. | |
| 97 | * @param {{ addEventListener: (type: string, f: () => void) => void }} win | |
| 98 | * @param {{ addEventListener: (type: string, f: () => void) => void, hidden?: boolean }} doc | |
| 99 | */ | |
| 100 | watch(win, doc) { | |
| 101 | const all = () => this.release(); | |
| 102 | win.addEventListener("blur", all); | |
| 103 | win.addEventListener("pagehide", all); | |
| 104 | doc.addEventListener("visibilitychange", () => { if (doc.hidden) all(); }); | |
| 105 | } | |
| 106 | } |
Software/so101/src/ui/grab.test.ts added +103
| @@ -0,0 +1,103 @@ | ||
| 1 | import { describe, expect, test } from "bun:test"; | |
| 2 | import { Grab } from "./grab.js"; | |
| 3 | ||
| 4 | const IDS: Record<string, number> = { shoulder_pan: 1, shoulder_lift: 2, elbow_flex: 3 }; | |
| 5 | ||
| 6 | /** | |
| 7 | * A stand-in for the 3D model: the elbow's link fills x 0..100, the shoulder's x 100..200, and the rest is empty | |
| 8 | * space. A held finger turns its joint by its sideways travel (0.01 rad a pixel), and every turn of any joint is an | |
| 9 | * angle-change, as the URDF element fires them, which the page hands to the grab. The wire records what goes out. | |
| 10 | */ | |
| 11 | function rig(allowed = true) { | |
| 12 | const wire: Array<{ kind: "send" | "flush" | "begin" | "end"; frame?: Record<number, number>; joint?: string }> = []; | |
| 13 | const angles: Record<string, number> = { shoulder_pan: 0, shoulder_lift: 0, elbow_flex: 0 }; | |
| 14 | let held: { joint: string; x: number } | null = null; | |
| 15 | const turn = (joint: string, angle: number) => { angles[joint] = angle; grab.angle(joint, angle); }; | |
| 16 | const grab: Grab = new Grab({ | |
| 17 | allowed: () => allowed, | |
| 18 | pick: (x) => { | |
| 19 | const joint = x < 0 || x >= 200 ? null : x < 100 ? "elbow_flex" : "shoulder_lift"; | |
| 20 | if (joint) held = { joint, x }; | |
| 21 | return joint; | |
| 22 | }, | |
| 23 | drag: (x) => { if (held) { turn(held.joint, angles[held.joint]! + (x - held.x) * 0.01); held.x = x; } }, | |
| 24 | drop: (x) => { if (held) { turn(held.joint, angles[held.joint]! + (x - held.x) * 0.01); held = null; } }, | |
| 25 | ticks: (joint, angle) => (IDS[joint] ? [IDS[joint]!, Math.round(2048 + angle * 100)] : null), | |
| 26 | send: (frame) => wire.push({ kind: "send", frame }), | |
| 27 | flush: (frame) => wire.push({ kind: "flush", frame }), | |
| 28 | begin: (joint) => wire.push({ kind: "begin", joint }), | |
| 29 | end: (joint) => wire.push({ kind: "end", joint }), | |
| 30 | }); | |
| 31 | return { grab, wire, turn, frames: () => wire.filter((w) => w.kind === "send" || w.kind === "flush").map((w) => w.frame!) }; | |
| 32 | } | |
| 33 | ||
| 34 | describe("touching the model", () => { | |
| 35 | test("a touch-drag on a joint's link streams targets for that joint only", () => { | |
| 36 | const { grab, wire, turn, frames } = rig(); | |
| 37 | expect(grab.down({ id: 7, x: 150, y: 40 })).toBe(true); | |
| 38 | expect(wire[0]).toEqual({ kind: "begin", joint: "shoulder_lift" }); | |
| 39 | grab.move({ id: 7, x: 160, y: 40 }); | |
| 40 | turn("shoulder_pan", 0.4); // the arm's own pose arriving meanwhile is drawn, never sent | |
| 41 | turn("elbow_flex", -0.2); | |
| 42 | grab.move({ id: 7, x: 180, y: 90 }); | |
| 43 | expect(frames()).toEqual([{ 2: 2058 }, { 2: 2078 }]); | |
| 44 | expect(frames().every((f) => Object.keys(f).join() === "2")).toBe(true); | |
| 45 | }); | |
| 46 | ||
| 47 | test("lifting the finger sends its last target at once, then nothing", () => { | |
| 48 | const { grab, wire, turn, frames } = rig(); | |
| 49 | grab.down({ id: 1, x: 50, y: 0 }); | |
| 50 | grab.move({ id: 1, x: 60, y: 0 }); | |
| 51 | expect(grab.up({ id: 1, x: 70, y: 0 })).toBe(true); | |
| 52 | // The model's last turn at the lift point, then that same target flushed (not left for the next slot), then the end. | |
| 53 | expect(wire.slice(-3)).toEqual([{ kind: "send", frame: { 3: 2068 } }, { kind: "flush", frame: { 3: 2068 } }, { kind: "end", joint: "elbow_flex" }]); | |
| 54 | const sent = wire.length; | |
| 55 | turn("elbow_flex", 1); | |
| 56 | grab.move({ id: 1, x: 90, y: 0 }); | |
| 57 | expect(wire.length).toBe(sent); | |
| 58 | expect(frames().at(-1)).toEqual({ 3: 2068 }); | |
| 59 | }); | |
| 60 | ||
| 61 | for (const how of ["blur", "pagehide", "hidden"] as const) { | |
| 62 | test(`${how} lets go of a held joint like a lift`, () => { | |
| 63 | const { grab, wire } = rig(); | |
| 64 | const win = new EventTarget(), doc = Object.assign(new EventTarget(), { hidden: false }); | |
| 65 | grab.watch(win, doc); | |
| 66 | grab.down({ id: 3, x: 120, y: 0 }); | |
| 67 | grab.move({ id: 3, x: 130, y: 0 }); | |
| 68 | if (how === "hidden") { doc.hidden = true; doc.dispatchEvent(new Event("visibilitychange")); } | |
| 69 | else win.dispatchEvent(new Event(how)); | |
| 70 | expect(wire.at(-1)).toEqual({ kind: "end", joint: "shoulder_lift" }); | |
| 71 | expect(wire.at(-2)!.kind).toBe("flush"); | |
| 72 | expect(grab.joint).toBeNull(); | |
| 73 | }); | |
| 74 | } | |
| 75 | ||
| 76 | test("orbit and pinch do not send commands: fingers on empty space are the view's", () => { | |
| 77 | const { grab, wire } = rig(); | |
| 78 | expect(grab.down({ id: 1, x: 300, y: 10 })).toBe(false); // one finger on empty space: orbit | |
| 79 | expect(grab.down({ id: 2, x: 400, y: 200 })).toBe(false); // the second: a pinch or a two-finger orbit | |
| 80 | for (const id of [1, 2]) expect(grab.move({ id, x: 350, y: 120 })).toBe(false); | |
| 81 | expect(grab.up({ id: 1, x: 0, y: 0 })).toBe(false); | |
| 82 | expect(wire).toEqual([]); | |
| 83 | }); | |
| 84 | ||
| 85 | test("a second finger while a joint is held neither orbits nor drives, and its lift ends nothing", () => { | |
| 86 | const { grab, wire, frames } = rig(); | |
| 87 | grab.down({ id: 1, x: 150, y: 0 }); | |
| 88 | expect(grab.down({ id: 2, x: 50, y: 0 })).toBe(true); // kept from the view, and the elbow is not taken | |
| 89 | expect(grab.move({ id: 2, x: 90, y: 0 })).toBe(false); | |
| 90 | expect(grab.up({ id: 2, x: 90, y: 0 })).toBe(false); | |
| 91 | grab.move({ id: 1, x: 160, y: 0 }); | |
| 92 | expect(frames()).toEqual([{ 2: 2058 }]); | |
| 93 | expect(wire.filter((w) => w.kind === "end")).toEqual([]); | |
| 94 | }); | |
| 95 | ||
| 96 | test("not your turn: touching a part orbits instead, and nothing is picked or sent", () => { | |
| 97 | let picked = 0; | |
| 98 | const grab = new Grab({ allowed: () => false, pick: () => (picked++, "elbow_flex"), drag() {}, drop() {}, ticks: () => [3, 1], send: () => { throw new Error("sent"); }, flush() {} }); | |
| 99 | expect(grab.down({ id: 1, x: 50, y: 0 })).toBe(false); | |
| 100 | expect(picked).toBe(0); | |
| 101 | expect(grab.angle("elbow_flex", 1)).toBeNull(); | |
| 102 | }); | |
| 103 | }); |
Software/so101/src/ui/machine-panel.js +1 −1
| @@ -7,7 +7,7 @@ | ||
| 7 | 7 | import { copyText, entryLine, jointRows, linkRows, stateRows } from "./machine.js"; |
| 8 | 8 | |
| 9 | 9 | const KEEP = 500; |
| 10 | /** The page's document and navigator: this module only runs there (the type check has no DOM, as for src/ui/drive.js). @type {any} */ | |
| 10 | /** The page's document and navigator: this module only runs there (the type check has no DOM, as for src/ui/intent.js). @type {any} */ | |
| 11 | 11 | const doc = /** @type {any} */ (globalThis).document; |
| 12 | 12 | const nav = /** @type {any} */ (globalThis).navigator; |
| 13 | 13 | const STYLE = ` |
Software/so101/src/ui/reach.js deleted −79
| @@ -1,79 +0,0 @@ | ||
| 1 | /** | |
| 2 | * The touch joystick's arithmetic: move the gripper, not a joint. Plain ESM, served to the page at /ui/ and | |
| 3 | * imported by the tests. The page owns the arm's geometry (the URDF in three.js) and hands it in as `tip`: | |
| 4 | * joint angles in, the gripper's position in the base frame (metres, z up) out. | |
| 5 | */ | |
| 6 | ||
| 7 | /** | |
| 8 | * The joystick and the height buttons as a displacement of the gripper, in the base frame. | |
| 9 | * Up on the pad reaches away from the base, sideways swings around it; `lift` is -1, 0 or 1. | |
| 10 | * `tip` is where the gripper is now; the step is `speed` m/s for `dt` seconds at full deflection. | |
| 11 | * @param {{ x: number, y: number, lift: number }} stick | |
| 12 | * @param {number[]} tip | |
| 13 | * @param {number} speed | |
| 14 | * @param {number} dt | |
| 15 | * @returns {number[]} | |
| 16 | */ | |
| 17 | export function stickToDelta(stick, tip, speed, dt) { | |
| 18 | const [tx = 0, ty = 0] = tip; | |
| 19 | const r = Math.hypot(tx, ty); | |
| 20 | // Straight above the base there is no "away": take the base's own forward, +x. | |
| 21 | const fx = r > 1e-3 ? tx / r : 1; | |
| 22 | const fy = r > 1e-3 ? ty / r : 0; | |
| 23 | const x = clampUnit(stick.x), y = clampUnit(stick.y), lift = clampUnit(stick.lift); | |
| 24 | const step = speed * dt; | |
| 25 | // Sideways is the forward direction turned a quarter to the right, seen from above. | |
| 26 | return [(fx * y + fy * x) * step, (fy * y - fx * x) * step, lift * step]; | |
| 27 | } | |
| 28 | ||
| 29 | /** | |
| 30 | * One damped-least-squares step: the joint changes that move the tip by `delta`. | |
| 31 | * The Jacobian comes from `tip` by finite differences, so any chain the page can evaluate works; | |
| 32 | * the damping keeps the step small near a stretched-out or folded-up arm instead of letting it whip. | |
| 33 | * @param {(angles: number[]) => number[]} tip | |
| 34 | * @param {number[]} angles | |
| 35 | * @param {number[]} delta | |
| 36 | * @param {{ h?: number, damping?: number, maxStep?: number }} [options] | |
| 37 | * @returns {number[]} | |
| 38 | */ | |
| 39 | export function reachStep(tip, angles, delta, options = {}) { | |
| 40 | const h = options.h ?? 1e-4, damping = options.damping ?? 0.02, maxStep = options.maxStep ?? 0.08; | |
| 41 | const here = tip(angles); | |
| 42 | const n = angles.length; | |
| 43 | /** @type {number[][]} J[row][joint] */ | |
| 44 | const J = [[], [], []]; | |
| 45 | for (let j = 0; j < n; j++) { | |
| 46 | const moved = angles.slice(); | |
| 47 | moved[j] = (moved[j] ?? 0) + h; | |
| 48 | const there = tip(moved); | |
| 49 | for (let r = 0; r < 3; r++) /** @type {number[]} */ (J[r])[j] = ((there[r] ?? 0) - (here[r] ?? 0)) / h; | |
| 50 | } | |
| 51 | // dq = Jᵀ (J Jᵀ + λ² I)⁻¹ delta | |
| 52 | const A = [0, 1, 2].map((r) => [0, 1, 2].map((c) => dot(/** @type {number[]} */ (J[r]), /** @type {number[]} */ (J[c])) + (r === c ? damping * damping : 0))); | |
| 53 | const w = solve3(A, delta); | |
| 54 | const dq = angles.map((_, j) => w.reduce((s, wr, r) => s + wr * (/** @type {number[]} */ (J[r])[j] ?? 0), 0)); | |
| 55 | // Never more than maxStep radians on any joint in one step. | |
| 56 | const biggest = Math.max(...dq.map(Math.abs)); | |
| 57 | return biggest > maxStep ? dq.map((d) => (d * maxStep) / biggest) : dq; | |
| 58 | } | |
| 59 | ||
| 60 | /** @param {number} v */ | |
| 61 | const clampUnit = (v) => (Number.isFinite(v) ? Math.max(-1, Math.min(1, v)) : 0); | |
| 62 | /** @param {number[]} a @param {number[]} b */ | |
| 63 | const dot = (a, b) => a.reduce((s, v, i) => s + v * (b[i] ?? 0), 0); | |
| 64 | ||
| 65 | /** | |
| 66 | * Cramer's rule for a 3×3 system; a singular matrix gives no motion. | |
| 67 | * @param {number[][]} A | |
| 68 | * @param {number[]} b | |
| 69 | */ | |
| 70 | function solve3(A, b) { | |
| 71 | /** @param {number[][]} m */ | |
| 72 | const det = (m) => { | |
| 73 | const [[a = 0, b = 0, c = 0] = [], [d = 0, e = 0, f = 0] = [], [g = 0, h = 0, i = 0] = []] = m; | |
| 74 | return a * (e * i - f * h) - b * (d * i - f * g) + c * (d * h - e * g); | |
| 75 | }; | |
| 76 | const d = det(A); | |
| 77 | if (!Number.isFinite(d) || Math.abs(d) < 1e-12) return [0, 0, 0]; | |
| 78 | return [0, 1, 2].map((c) => det(A.map((row, r) => row.map((v, k) => (k === c ? b[r] ?? 0 : v)))) / d); | |
| 79 | } |
Software/so101/src/ui/reach.test.ts deleted −58
| @@ -1,58 +0,0 @@ | ||
| 1 | import { describe, expect, test } from "bun:test"; | |
| 2 | import { reachStep, stickToDelta } from "./reach.js"; | |
| 3 | ||
| 4 | // A yaw joint and a two-link planar arm on top of it: the SO-101's pan, shoulder lift and elbow, roughly. | |
| 5 | const L1 = 0.116, L2 = 0.135; | |
| 6 | const tip = ([pan = 0, lift = 0, elbow = 0]: number[]) => { | |
| 7 | const reach = L1 * Math.cos(lift) + L2 * Math.cos(lift + elbow); | |
| 8 | const z = L1 * Math.sin(lift) + L2 * Math.sin(lift + elbow); | |
| 9 | return [reach * Math.cos(pan), reach * Math.sin(pan), z]; | |
| 10 | }; | |
| 11 | const distance = (a: number[], b: number[]) => Math.hypot(...a.map((v, i) => v - b[i]!)); | |
| 12 | ||
| 13 | describe("the joystick as a displacement", () => { | |
| 14 | test("up on the pad reaches away from the base, whichever way the arm faces", () => { | |
| 15 | const [dx, dy, dz] = stickToDelta({ x: 0, y: 1, lift: 0 }, [0, 0.2, 0.1], 0.05, 0.1); | |
| 16 | expect(dx).toBeCloseTo(0); | |
| 17 | expect(dy).toBeCloseTo(0.005); | |
| 18 | expect(dz).toBe(0); | |
| 19 | }); | |
| 20 | ||
| 21 | test("right on the pad swings clockwise seen from above, and lift is straight up", () => { | |
| 22 | const [dx, dy, dz] = stickToDelta({ x: 1, y: 0, lift: 1 }, [0.2, 0, 0.1], 0.05, 0.1); | |
| 23 | expect(dx).toBeCloseTo(0); | |
| 24 | expect(dy).toBeCloseTo(-0.005); | |
| 25 | expect(dz).toBeCloseTo(0.005); | |
| 26 | }); | |
| 27 | ||
| 28 | test("a deflection past the edge or a broken number is clamped, never amplified", () => { | |
| 29 | const [dx, dy, dz] = stickToDelta({ x: 0, y: 9, lift: Number.NaN }, [0.2, 0, 0], 0.05, 0.1); | |
| 30 | expect(dx).toBeCloseTo(0.005); | |
| 31 | expect(dy).toBeCloseTo(0); | |
| 32 | expect(dz).toBe(0); | |
| 33 | }); | |
| 34 | }); | |
| 35 | ||
| 36 | describe("a reach step", () => { | |
| 37 | test("moves the tip the way it was asked, and the error shrinks step by step", () => { | |
| 38 | let angles = [0.3, 0.6, -1.2]; | |
| 39 | const start = tip(angles); | |
| 40 | const goal = [start[0]! + 0.03, start[1]! - 0.02, start[2]! + 0.02]; | |
| 41 | let before = distance(start, goal); | |
| 42 | for (let i = 0; i < 20; i++) { | |
| 43 | const here = tip(angles); | |
| 44 | const delta = goal.map((g, k) => g - here[k]!); | |
| 45 | angles = angles.map((a, k) => a + reachStep(tip, angles, delta)[k]!); | |
| 46 | const after = distance(tip(angles), goal); | |
| 47 | expect(after).toBeLessThanOrEqual(before + 1e-9); | |
| 48 | before = after; | |
| 49 | } | |
| 50 | expect(before).toBeLessThan(0.001); | |
| 51 | }); | |
| 52 | ||
| 53 | test("a stretched-out arm asked to reach further barely moves instead of whipping", () => { | |
| 54 | const angles = [0, 0, 0]; // straight out: no joint can take the tip further away | |
| 55 | const dq = reachStep(tip, angles, [0.05, 0, 0]); | |
| 56 | for (const d of dq) expect(Math.abs(d)).toBeLessThanOrEqual(0.08); | |
| 57 | }); | |
| 58 | }); |
Software/so101/src/viewer.html +115 −323
| @@ -18,7 +18,6 @@ | ||
| 18 | 18 | } } |
| 19 | 19 | </script> |
| 20 | 20 | <link rel="modulepreload" href="/ui/__VERSION__/camera-plan.js"> |
| 21 | <link rel="modulepreload" href="/ui/__VERSION__/drive.js"> | |
| 22 | 21 | <link rel="modulepreload" href="/ui/__VERSION__/frames.js"> |
| 23 | 22 | <style> |
| 24 | 23 | /* Phone portrait first: the view on top, a status strip with the E-stop under it, a sheet of tabs below that. |
| @@ -59,10 +58,8 @@ | ||
| 59 | 58 | /* the view: cameras and the 3D model, always on black */ |
| 60 | 59 | #app { grid-area: view; position: relative; min-height: 0; background: #000; color: #e8eaed; overflow: hidden; } |
| 61 | 60 | #stage { position: absolute; inset: 0; display: flex; gap: 2px; } |
| 62 | /* the emergency stop's held state, and the banner that says who pressed it */ | |
| 61 | /* the emergency stop's held state */ | |
| 63 | 62 | .estop:disabled { opacity: .5; cursor: not-allowed; } .estop.on { background: #7d1410; box-shadow: 0 0 0 3px var(); } |
| 64 | #banner.stop { display: block; border: 2px solid var(); background: rgba(80,10,8,.92); color: #fff; font-weight: 600; } | |
| 65 | #banner button { margin-left: 10px; min-height: 32px; padding: 2px 10px; } | |
| 66 | 63 | .pane { flex: 1; min-width: 0; position: relative; background: #000; display: none; } |
| 67 | 64 | .pane.on { display: block; } |
| 68 | 65 | .pane img { width: 100%; height: 100%; object-fit: contain; display: block; } |
| @@ -75,10 +72,6 @@ | ||
| 75 | 72 | #top .info { display: inline-flex; gap: 6px; align-items: center; background: rgba(0,0,0,.6); padding: 0 10px; min-height: 36px; border-radius: 8px; max-width: 100%; } |
| 76 | 73 | #link { color: #9aa4ae; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; max-width: min(36ch, 46vw); } #link.on { color: #3fb950; } #link.warn { color: #d29922; } #link.bad { color: #f85149; font-weight: 700; } |
| 77 | 74 | .dot { width: 8px; height: 8px; border-radius: 50%; background: #f85149; display: inline-block; flex: none; } .dot.on { background: #3fb950; } .dot.warn { background: #d29922; } |
| 78 | #banner { position: absolute; top: 50px; left: 50%; transform: translateX(-50%); z-index: 5; width: max-content; max-width: min(640px, calc(100% - 16px)); padding: 8px 12px; border-radius: 8px; font-size: 13px; background: rgba(0,0,0,.8); border: 1px solid #30363d; display: none; } | |
| 79 | #banner.bad { display: block; border-color: #f85149; color: #ffb3ad; } #banner.warn { display: block; border-color: #d29922; color: #f2cc60; } | |
| 80 | #msg { position: absolute; left: calc(8px + var()); right: calc(8px + var()); bottom: 8px; z-index: 6; margin: 0 auto; width: max-content; max-width: calc(100% - 16px); padding: 8px 12px; border-radius: 8px; font-size: 13px; background: rgba(13,17,23,.9); color: #e8eaed; border: 1px solid #30363d; } | |
| 81 | #msg:empty { display: none; } #msg.bad { border-color: #f85149; color: #ffb3ad; } | |
| 82 | 75 | |
| 83 | 76 | /* the status strip: who drives, your place, time left, the turn's action and the E-stop */ |
| 84 | 77 | #strip { grid-area: strip; display: flex; align-items: center; gap: 10px; padding: 8px calc(12px + var()) 8px calc(12px + var()); background: var(); border-bottom: 1px solid var(); min-height: 64px; } |
| @@ -124,7 +117,15 @@ | ||
| 124 | 117 | #tabs .badge { display: inline-block; min-width: 18px; margin-left: 4px; padding: 0 5px; border-radius: 9px; background: var(); color: var(); font-size: 11px; line-height: 18px; } |
| 125 | 118 | #tabs .badge:empty { display: none; } |
| 126 | 119 | |
| 127 | #notice { flex: none; padding: 10px calc(12px + var()) 10px calc(12px + var()); border-bottom: 1px solid var(); border-left: 3px solid var(); background: var(); font-size: 14px; overflow-wrap: anywhere; } | |
| 120 | /* notes: the stop, the arm's warnings, the announcement and the page's replies. Never over the camera or the model: | |
| 121 | one pill in the strip says how many, and opens them. */ | |
| 122 | #notes { flex: none; padding: 0 12px; border-radius: 999px; font-size: 13px; font-weight: 650; white-space: nowrap; } | |
| 123 | #notes.warn { border-color: var(); color: var(); } #notes.bad { border-color: var(); color: var(); background: var(); } | |
| 124 | #banner, #msg, #notice { margin: 8px 0 0; padding: 10px 12px; border-radius: 10px; border: 1px solid var(); border-left: 3px solid var(); font-size: 14px; overflow-wrap: anywhere; } | |
| 125 | #banner:not(.bad, .warn, .stop), #msg:empty { display: none; } | |
| 126 | #banner.bad, #banner.stop, #msg.bad { border-color: var(); background: var(); } #banner.warn { border-color: var(); } #banner.stop { font-weight: 600; } | |
| 127 | .sr { position: absolute; width: 1px; height: 1px; overflow: hidden; clip-path: inset(50%); white-space: nowrap; } | |
| 128 | #banner button { margin: 8px 8px 0 0; } | |
| 128 | 129 | |
| 129 | 130 | /* chat */ |
| 130 | 131 | #tab-chat { padding: 0; gap: 0; } |
| @@ -149,14 +150,6 @@ | ||
| 149 | 150 | #forgeSignin { color: var(); font-size: 13px; align-self: flex-start; } |
| 150 | 151 | |
| 151 | 152 | /* controls */ |
| 152 | #reach { display: grid; grid-template-columns: auto 1fr; gap: 12px; align-items: center; } | |
| 153 | #pad { position: relative; width: min(46vw, 180px); aspect-ratio: 1; border-radius: 50%; background: radial-gradient(circle, var() 0 58%, var() 59%); border: 1px solid var(); touch-action: none; -webkit-user-select: none; user-select: none; } | |
| 154 | #pad::before, #pad::after { content: ""; position: absolute; background: var(); left: 50%; top: 12%; bottom: 12%; width: 1px; } #pad::after { left: 12%; right: 12%; top: 50%; bottom: auto; width: auto; height: 1px; } | |
| 155 | #knob { position: absolute; left: 50%; top: 50%; width: 38%; aspect-ratio: 1; margin: -19% 0 0 -19%; border-radius: 50%; background: var(); box-shadow: 0 2px 8px rgba(0,0,0,.35); transition: transform .12s ease-out; pointer-events: none; z-index: 1; } | |
| 156 | #pad.held #knob { transition: none; } | |
| 157 | #reach .side { display: grid; gap: 8px; } | |
| 158 | #reach .side button { width: 100%; touch-action: none; -webkit-user-select: none; user-select: none; } | |
| 159 | #reach .hint { grid-column: 1 / -1; margin: 0; color: var(); font-size: 12px; } | |
| 160 | 153 | #joints { display: grid; grid-template-columns: repeat(auto-fit, minmax(240px, 1fr)); gap: 6px 14px; } |
| 161 | 154 | .j { display: grid; grid-template-columns: 1fr auto; grid-template-areas: "name val" "slide slide"; align-items: center; column-gap: 6px; padding: 4px 6px; border-radius: 10px; } |
| 162 | 155 | .j.jam { background: var(); } .j.drag { background: var(); } |
| @@ -169,40 +162,8 @@ | ||
| 169 | 162 | .pose { display: inline-flex; gap: 4px; align-items: center; } |
| 170 | 163 | #readout { color: var(); font-size: 13px; display: flex; gap: 14px; flex-wrap: wrap; } #readout b { color: var(); font-weight: 550; } |
| 171 | 164 | |
| 172 | /* touch driving (src/ui/drive.js): on a phone held upright, pick a joint and push one big lever; let go and it stops */ | |
| 173 | #touch { display: none; gap: 10px; } | |
| 174 | #chips { display: grid; grid-template-columns: repeat(6, minmax(0, 1fr)); gap: 4px; } | |
| 175 | #chips button { min-height: 50px; padding: 0 2px; border-radius: 10px; display: grid; place-content: center; gap: 1px; line-height: 1.1; } | |
| 176 | #chips button b { font-size: 15px; } #chips button small { font-size: 10.5px; color: var(); overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } | |
| 177 | #chips button[aria-checked=true] { background: var(); border-color: var(); color: var(); } #chips button[aria-checked=true] small { color: inherit; opacity: .85; } | |
| 178 | #one { display: grid; grid-template-columns: minmax(0, 1fr) 84px minmax(0, 1fr); gap: 12px; height: clamp(210px, 36svh, 360px); } | |
| 179 | #one .nudges { display: grid; grid-template-rows: 1fr 1fr; gap: 10px; } | |
| 180 | #one .nudges button { display: grid; place-content: center; gap: 2px; font-size: 22px; line-height: 1; touch-action: none; -webkit-user-select: none; user-select: none; } | |
| 181 | #one .nudges small { font-size: 12px; color: var(); } | |
| 182 | #lever { position: relative; border-radius: 42px; background: var(); border: 1px solid var(); touch-action: none; -webkit-user-select: none; user-select: none; overflow: hidden; cursor: ns-resize; } | |
| 183 | #lever::before { content: ""; position: absolute; left: 16px; right: 16px; top: 50%; height: 2px; margin-top: -1px; background: var(); } | |
| 184 | #leverAt { position: absolute; left: 0; bottom: 0; width: 5px; height: 100%; background: var(); transform-origin: bottom; transform: scaleY(var(, .5)); } | |
| 185 | #leverKnob { position: absolute; left: 7px; right: 7px; top: 50%; height: 68px; margin-top: -34px; border-radius: 34px; background: var(); color: var(); display: grid; place-content: center; font-size: 22px; box-shadow: 0 2px 10px rgba(0,0,0,.3); transition: transform .14s ease-out; pointer-events: none; } | |
| 186 | #lever.held { border-color: var(); background: var(); } #lever.held #leverKnob { transition: none; } | |
| 187 | #lever.end, .stick.end { box-shadow: 0 0 0 3px var(); } | |
| 188 | #one .about { display: grid; align-content: center; gap: 4px; min-width: 0; } | |
| 189 | #oneName { font-size: 20px; font-weight: 700; } #onePos { font-size: 28px; font-weight: 650; font-variant-numeric: tabular-nums; } | |
| 190 | #oneHint { color: var(); font-size: 12px; } | |
| 191 | 165 | #lag { color: #f2cc60; font-variant-numeric: tabular-nums; } |
| 192 | 166 | #chat .earlier { display: block; margin: 2px auto 8px; font-size: 13px; } |
| 193 | /* on a phone on its side: two thumb sticks over the picture, the gripper between them */ | |
| 194 | #sticks { display: none; position: absolute; inset: 0; z-index: 6; pointer-events: none; : min(40vh, 164px); } | |
| 195 | #sticks > * { pointer-events: auto; } | |
| 196 | .stick { position: absolute; bottom: calc(14px + var()); width: var(); height: var(); border-radius: 50%; background: radial-gradient(circle, rgba(255,255,255,.08) 0 54%, rgba(255,255,255,.03) 55%); border: 1px solid rgba(255,255,255,.24); touch-action: none; -webkit-user-select: none; user-select: none; } | |
| 197 | #stickL { left: calc(18px + var()); } #stickR { right: calc(var() + var()); } | |
| 198 | .stick .knob { position: absolute; left: 50%; top: 50%; width: 42%; height: 42%; margin: -21% 0 0 -21%; border-radius: 50%; background: rgba(56,139,253,.8); box-shadow: 0 2px 10px rgba(0,0,0,.45); transition: transform .14s ease-out; pointer-events: none; } | |
| 199 | .stick.held { border-color: #388bfd; } .stick.held .knob { background: #388bfd; transition: none; } | |
| 200 | .stick .cap { position: absolute; left: -20px; right: -20px; bottom: 100%; margin-bottom: 4px; text-align: center; font-size: 11px; color: #e8eaed; text-shadow: 0 1px 2px #000; pointer-events: none; white-space: nowrap; } | |
| 201 | #sticks button { background: rgba(0,0,0,.55); border-color: rgba(255,255,255,.24); color: #e8eaed; touch-action: none; -webkit-user-select: none; user-select: none; } | |
| 202 | #sticks button.held { background: #388bfd; border-color: #388bfd; } | |
| 203 | #swap { position: absolute; right: calc(var() + var()); bottom: calc(var() + 40px + var()); min-height: 40px; font-size: 13px; } | |
| 204 | #grips { position: absolute; left: 50%; bottom: calc(14px + var()); transform: translateX(-50%); display: flex; gap: 10px; } | |
| 205 | #grips button { min-width: 80px; min-height: 72px; font-weight: 650; } | |
| 206 | 167 | |
| 207 | 168 | /* admin */ |
| 208 | 169 | .card { display: grid; gap: 10px; padding: 12px; border: 1px solid var(); border-radius: 12px; background: var(); } |
| @@ -276,18 +237,21 @@ | ||
| 276 | 237 | #shell.bare #sheet { transform: translateX(100%); visibility: hidden; transition: transform .2s ease-out, visibility 0s .2s; } |
| 277 | 238 | #top { top: auto; bottom: 6px; justify-content: flex-start; right: calc(min(380px, 48vw) + 12px + var()); } |
| 278 | 239 | #shell.bare #top { right: calc(var() + var()); } |
| 279 | #banner { top: 64px; } | |
| 280 | #msg { bottom: 52px; } | |
| 281 | 240 | .tab { padding-top: 8px; } |
| 282 | #pad { width: min(30vh, 150px); } | |
| 283 | 241 | } |
| 284 | @media (max-width: 640px) and (orientation: portrait) { #touch { display: grid; } #reach, #joints { display: none ; } } | |
| 285 | /* One STOP in every layout; it holds until Resume. */ | |
| 286 | @media (orientation: landscape) and (max-height: 500px) { | |
| 287 | #shell.driving.bare #sticks { display: block; } | |
| 288 | #shell.driving.bare #top { top: 6px; bottom: auto; left: auto; right: calc(6px + var()); max-width: 42%; justify-content: flex-end; } | |
| 289 | #shell.driving.bare #strip { max-width: calc(58% - 18px - var()); } | |
| 290 | #shell.driving.bare #msg { bottom: auto; top: 110px; } | |
| 242 | /* A phone held upright: the camera on top at 16:9 and the 3D model under it, both always there; the model is the | |
| 243 | main control. The sheet is a tray under them: the grabber, or the open tab tapped again, folds it away. */ | |
| 244 | #fold { display: none; flex: none; width: 100%; min-height: 28px; border: 0; border-radius: 0; background: none; padding: 0; } | |
| 245 | #fold::before { content: ""; display: block; width: 40px; height: 4px; margin: 0 auto; border-radius: 2px; background: var(); } | |
| 246 | @media (max-width: 640px) and (orientation: portrait) { | |
| 247 | #shell, #shell.typing { grid-template-rows: minmax(0, 1fr) auto var(, min(42%, 350px)); } | |
| 248 | #shell.folded { : auto; } #shell.folded .tab { display: none; } | |
| 249 | #stage { flex-direction: column; } | |
| 250 | .pane:not(#pane3) { flex: none; aspect-ratio: 16 / 9; } #stage:not(:has(#pane3.on)) .pane { flex: 1; aspect-ratio: auto; } | |
| 251 | #fold { display: block; } .tab { padding-top: 0; gap: 8px; } | |
| 252 | #joints { grid-template-columns: 100%; gap: 0; } | |
| 253 | .j { grid-template-columns: 6.6em minmax(0, 1fr) 3.2em; grid-template-areas: "name slide val"; padding: 0 2px; } | |
| 254 | .j .jog, .j small { display: none; } .j .v { justify-content: flex-end; } .j input[type=range] { height: 36px; } | |
| 291 | 255 | } |
| 292 | 256 | @media (max-width: 420px) { #top .label { display: none; } #link { max-width: 30vw; } } |
| 293 | 257 | @media (prefers-reduced-motion: reduce) { *, *::before, *::after { transition: none ; animation: none ; scroll-behavior: auto ; } } |
| @@ -306,26 +270,19 @@ | ||
| 306 | 270 | <button id="fs" title="F">Fullscreen</button> |
| 307 | 271 | </div> |
| 308 | 272 | <div id="driverTag" aria-hidden="true"></div> |
| 309 | <div id="banner" role="alert"></div> | |
| 310 | <div id="msg" role="status"></div> | |
| 311 | <div id="sticks" role="group" aria-label="Thumb sticks"> | |
| 312 | <div class="stick" id="stickL" role="group" aria-label="Left stick: sideways turns the base, up and down lifts the shoulder"><span class="cap">J1 base ↔ · J2 shoulder ↕</span><span class="knob"></span></div> | |
| 313 | <div class="stick" id="stickR" role="group" aria-label="Right stick"><span class="cap" id="capR"></span><span class="knob"></span></div> | |
| 314 | <button id="swap" type="button" aria-pressed="false">Wrist</button> | |
| 315 | <div id="grips" role="group" aria-label="Gripper"><button id="gripOpen" type="button">Open</button><button id="gripClose" type="button">Close</button></div> | |
| 316 | </div> | |
| 317 | 273 | </div> |
| 318 | 274 | |
| 319 | 275 | <section id="strip" aria-label="Who drives" aria-live="polite"> |
| 320 | 276 | <div id="ring" hidden aria-hidden="true"><span id="ringN"></span></div> |
| 321 | 277 | <div class="who"><div id="driverLine" class="none">Connecting…</div><div id="clock" aria-live="off"></div><div id="capLine"></div></div> |
| 322 | 278 | <div id="turnActions"></div> |
| 279 | <button id="notes" type="button" hidden aria-haspopup="dialog" aria-controls="notesDlg"></button><span id="said" class="sr" role="status"></span> | |
| 323 | 280 | <button id="sheetToggle" hidden aria-controls="sheet" aria-expanded="true">Hide</button> |
| 324 | 281 | <button id="stop" class="estop" data-stop title="Stop (space or Esc): halt everything, hold with torque, until someone resumes" aria-label="Stop the arm">Stop</button> |
| 325 | 282 | </section> |
| 326 | 283 | |
| 327 | 284 | <section id="sheet" aria-label="Chat, queue and controls"> |
| 328 | <div id="notice" role="status" hidden></div> | |
| 285 | <button id="fold" type="button" aria-controls="sheet" aria-expanded="true" aria-label="Fold the tray away"></button> | |
| 329 | 286 | <div class="tab" id="tab-chat" role="tabpanel" aria-labelledby="t-chat"> |
| 330 | 287 | <div id="chat" role="log" aria-label="Chat"></div> |
| 331 | 288 | <form id="say" hidden><input id="sayText" type="text" maxlength="500" autocomplete="off" enterkeyhint="send" placeholder="Say something" aria-label="Message"><button class="go">Send</button></form> |
| @@ -347,24 +304,6 @@ | ||
| 347 | 304 | <p class="note">Plug an SO-101 leader into this computer or Android phone. The 3D model mirrors it while you wait; switch Teleop on and, on your turn, the arm follows it live. Hiding this tab or unplugging the leader stops the arm where it is.</p> |
| 348 | 305 | </div> |
| 349 | 306 | <div class="tab" id="tab-controls" role="tabpanel" aria-labelledby="t-controls" hidden> |
| 350 | <section id="touch" aria-label="Drive one joint"> | |
| 351 | <div id="chips" role="radiogroup" aria-label="Joint"></div> | |
| 352 | <div id="one"> | |
| 353 | <div class="nudges"><button type="button" data-nudge="1" id="nudgeUp">▲<small></small></button><button type="button" data-nudge="-1" id="nudgeDown">▼<small></small></button></div> | |
| 354 | <div id="lever" role="slider" tabindex="0" aria-orientation="vertical" aria-valuemin="-100" aria-valuemax="100" aria-valuenow="0"><span id="leverAt"></span><span id="leverKnob" aria-hidden="true">⇕</span></div> | |
| 355 | <div class="about"><span id="oneName"></span><span id="onePos">—</span><span id="oneHint">Push the lever: further is faster. Let go and it stops.</span></div> | |
| 356 | </div> | |
| 357 | </section> | |
| 358 | <div id="reach" hidden> | |
| 359 | <div id="pad" role="application" aria-label="Move the gripper: up reaches out, down pulls in, sideways swings around the base"><span id="knob"></span></div> | |
| 360 | <div class="side"> | |
| 361 | <button data-lift="1" aria-label="Gripper up">▲ Up</button> | |
| 362 | <button data-lift="-1" aria-label="Gripper down">▼ Down</button> | |
| 363 | <button data-grip="1">Open</button> | |
| 364 | <button data-grip="-1">Close</button> | |
| 365 | </div> | |
| 366 | <p class="hint">Hold the pad to move the gripper: up reaches out, sideways swings around the base. The jog speed sets how fast.</p> | |
| 367 | </div> | |
| 368 | 307 | <p id="why" role="status"></p> |
| 369 | 308 | <div id="joints"></div> |
| 370 | 309 | <div class="row" id="poses"></div> |
| @@ -467,6 +406,13 @@ | ||
| 467 | 406 | </article> |
| 468 | 407 | </div> |
| 469 | 408 | |
| 409 | <dialog id="notesDlg" aria-labelledby="notesTitle"> | |
| 410 | <h3 id="notesTitle">Notes</h3> | |
| 411 | <div id="banner"></div> | |
| 412 | <div id="notice" hidden></div> | |
| 413 | <div id="msg"></div> | |
| 414 | <form method="dialog" class="actions"><button class="go">Close</button></form> | |
| 415 | </dialog> | |
| 470 | 416 | <dialog id="saveDlg"> |
| 471 | 417 | <form method="dialog"> |
| 472 | 418 | <h3>Save the current pose</h3> |
| @@ -515,7 +461,6 @@ | ||
| 515 | 461 | </div> |
| 516 | 462 | <script type="module"> |
| 517 | 463 | import { cameraPlan, QUALITIES, samePlan } from "/ui/__VERSION__/camera-plan.js"; |
| 518 | import { bindSurface, canDrive, Driver, shape, UP } from "/ui/__VERSION__/drive.js"; | |
| 519 | 464 | import { Frames } from "/ui/__VERSION__/frames.js"; |
| 520 | 465 | |
| 521 | 466 | const $ = (id) => document.getElementById(id); |
| @@ -525,6 +470,20 @@ | ||
| 525 | 470 | $("msg").textContent = text; $("msg").className = bad ? "bad" : ""; |
| 526 | 471 | clearTimeout(msgTimer); if (text) msgTimer = setTimeout(() => ($("msg").textContent = ""), bad ? 9000 : 5000); |
| 527 | 472 | }; |
| 473 | // Notes never cover the camera or the model: the stop, the arm's warnings, the announcement and the page's replies sit | |
| 474 | // in a sheet, and one pill in the strip says how many (red for a stop or an error) and opens it. | |
| 475 | const NOTES = ["banner", "notice", "msg"].map($); | |
| 476 | new MutationObserver((changes) => { | |
| 477 | const shown = NOTES.filter((n) => (n.id === "banner" ? n.className : n.id === "notice" ? !n.hidden : n.textContent)), pill = $("notes"); | |
| 478 | pill.hidden = !shown.length; | |
| 479 | pill.textContent = `${shown.length} note${shown.length === 1 ? "" : "s"}`; | |
| 480 | pill.className = shown.some((n) => /bad|stop/.test(n.className)) ? "bad" : shown.some((n) => n.className === "warn") ? "warn" : ""; | |
| 481 | pill.title = shown.map((n) => n.textContent).join(" · "); | |
| 482 | const latest = NOTES.find((n) => n.contains(changes.at(-1).target)), said = latest && shown.includes(latest) ? latest.textContent : ""; | |
| 483 | if (said && $("said").textContent !== said) $("said").textContent = said; // read out, once | |
| 484 | if (!shown.length && $("notesDlg").open) $("notesDlg").close(); | |
| 485 | }).observe($("notesDlg"), { childList: true, characterData: true, subtree: true, attributeFilter: ["class", "hidden"] }); | |
| 486 | $("notes").onclick = () => $("notesDlg").showModal(); | |
| 528 | 487 | const api = async (path, body, method = body === undefined ? "GET" : "POST") => { |
| 529 | 488 | const r = await fetch(path, { method, body: body === undefined ? undefined : JSON.stringify(body), cache: "no-store" }); |
| 530 | 489 | const data = await r.json().catch(() => ({})); |
| @@ -532,8 +491,8 @@ | ||
| 532 | 491 | }; |
| 533 | 492 | const el = (tag, props = {}, ...children) => { const e = Object.assign(document.createElement(tag), props); e.append(...children); return e; }; |
| 534 | 493 | const NAMES = { 1: "shoulder_pan", 2: "shoulder_lift", 3: "elbow_flex", 4: "wrist_flex", 5: "wrist_roll", 6: "gripper" }; |
| 494 | const SHORT = { 1: "J1 base", 2: "J2 shoulder", 3: "J3 elbow", 4: "J4 wrist", 5: "J5 roll", 6: "Grip" }; | |
| 535 | 495 | const IDS = Object.fromEntries(Object.entries(NAMES).map(([id, n]) => [n, Number(id)])); |
| 536 | const narrow = matchMedia("(max-width: 760px), (max-height: 500px)"); | |
| 537 | 496 | |
| 538 | 497 | // ---------------------------------------------------------------- the screen: the keyboard, the sheet, the tabs |
| 539 | 498 | // The shell follows the visual viewport, so the on-screen keyboard shrinks the sheet instead of covering the chat. |
| @@ -546,7 +505,6 @@ | ||
| 546 | 505 | $("sheetToggle").textContent = bare ? "Show" : "Hide"; $("sheetToggle").setAttribute("aria-expanded", String(!bare)); |
| 547 | 506 | }; |
| 548 | 507 | $("sheetToggle").onclick = () => setBare(!shell.classList.contains("bare")); |
| 549 | const sideways = matchMedia("(orientation: landscape) and (max-height: 500px)"); | |
| 550 | 508 | let tab = "chat"; |
| 551 | 509 | /** The Machine tab's panel, once it is loaded (below); here, because leaving any tab closes it. */ |
| 552 | 510 | const machine = { panel: null }; |
| @@ -564,7 +522,10 @@ | ||
| 564 | 522 | if (name === "admin") loadSettings(); |
| 565 | 523 | if (name === "machine") machineOpen(); else machineClose(); |
| 566 | 524 | } |
| 567 | for (const b of $("tabs").querySelectorAll("[role=tab]")) b.onclick = () => showTab(b.dataset.tab); | |
| 525 | /** The tray (a phone held upright): folded, only the tabs show and the model has the room. */ | |
| 526 | const fold = (on) => { shell.classList.toggle("folded", on); $("fold").setAttribute("aria-expanded", String(!on)); $("fold").ariaLabel = on ? "Unfold the tray" : "Fold the tray away"; }; | |
| 527 | $("fold").onclick = () => fold(!shell.classList.contains("folded")); | |
| 528 | for (const b of $("tabs").querySelectorAll("[role=tab]")) b.onclick = () => { fold(b.dataset.tab === tab && !shell.classList.contains("folded")); showTab(b.dataset.tab); }; | |
| 568 | 529 | $("tabs").addEventListener("keydown", (e) => { |
| 569 | 530 | if (e.key !== "ArrowLeft" && e.key !== "ArrowRight") return; |
| 570 | 531 | const shown = [...$("tabs").querySelectorAll("[role=tab]:not([hidden])")]; |
| @@ -623,10 +584,10 @@ | ||
| 623 | 584 | plan = cameraPlan(navigator.connection, rtts, "stream", { quality, throttle: relayed }); |
| 624 | 585 | /** The stream's address under the plan: through the relay, ?fps= holds it to fewer frames. */ |
| 625 | 586 | const streamUrl = (c) => fresh(plan.mode === "stream" && plan.fps && relayed ? `${c.url}?fps=${plan.fps}` : c.url); |
| 626 | const saved = (() => { try { const s = JSON.parse(localStorage.getItem("so101.panes") || "null"); return Array.isArray(s) ? s : null; } catch { return null; } })(); | |
| 587 | const saved = (() => { try { const s = JSON.parse(localStorage.getItem("so101.views") || "null"); return Array.isArray(s) ? s : null; } catch { return null; } })(); | |
| 627 | 588 | // Pane keys: "cam:<id>" and "pane3". Until the startup below knows the model and the link, cameras are listed but not shown. |
| 628 | 589 | let chosen = saved ? new Set(saved) : null, viewsReady = false, cameraFirst = true; |
| 629 | const keep = () => { try { localStorage.setItem("so101.panes", JSON.stringify([...document.querySelectorAll(".pane.on")].map((p) => p.dataset.key))); } catch {} }; | |
| 590 | const keep = () => { try { localStorage.setItem("so101.views", JSON.stringify([...document.querySelectorAll(".pane.on")].map((p) => p.dataset.key))); } catch {} }; | |
| 630 | 591 | const shown = () => document.querySelectorAll(".pane.on").length; |
| 631 | 592 | const paneOn = (c) => c.pane.classList.contains("on"); |
| 632 | 593 | async function snap(c) { |
| @@ -655,22 +616,20 @@ | ||
| 655 | 616 | if (on) load3D(); |
| 656 | 617 | } else { const c = cams.find((x) => x.pane.dataset.key === key); if (c) showCam(c, on); } |
| 657 | 618 | } |
| 658 | /** A view button: on or off; a phone shows one view at a time, so picking one there puts the other away. */ | |
| 619 | /** A view button: on or off, keeping at least one. */ | |
| 659 | 620 | function pick(key) { |
| 660 | 621 | const pane = key === "pane3" ? $("pane3") : cams.find((x) => x.pane.dataset.key === key)?.pane; |
| 661 | 622 | if (!pane) return; |
| 662 | 623 | const on = !pane.classList.contains("on"); |
| 663 | 624 | if (!on && shown() === 1) return; // keep one pane |
| 664 | if (on && narrow.matches) for (const other of document.querySelectorAll(".pane.on")) setPane(other.dataset.key, false); | |
| 665 | 625 | setPane(key, on); |
| 666 | 626 | keep(); viewer.recenter?.(); |
| 667 | 627 | } |
| 668 | /** Whether a view is on when it first appears: the viewer's last choice, else the camera when there is a real one, and the model beside it on a wide screen. */ | |
| 628 | /** Whether a view is on when it first appears: the viewer's last choice, else the camera when there is a real one and the model with it (under it on a phone). */ | |
| 669 | 629 | function wanted(key) { |
| 670 | if (narrow.matches && shown() >= 1) return false; | |
| 671 | 630 | if (key === "pane3" && !modelInfo?.ok) return false; |
| 672 | 631 | if (chosen) return chosen.has(key); |
| 673 | if (key === "pane3") return !narrow.matches || !cameraFirst || !cams.length; | |
| 632 | if (key === "pane3") return true; | |
| 674 | 633 | return cameraFirst && key === cams[0]?.pane.dataset.key; |
| 675 | 634 | } |
| 676 | 635 | /** At least one view on: the first camera, else the model. */ |
| @@ -780,7 +739,7 @@ | ||
| 780 | 739 | $("v3").onclick = () => pick("pane3"); |
| 781 | 740 | |
| 782 | 741 | // ---------------------------------------------------------------- the stream: the page as a leader |
| 783 | // Opened while a slider, a jog button, the joystick or a model joint is held; frames go at most every 33 ms; closed shortly after letting go. | |
| 742 | // Opened while a slider, a jog button or a model joint is held; frames go at most every 33 ms; closed shortly after letting go. | |
| 784 | 743 | // Stamped targets (src/ui/frames.js): while the socket opens only the newest waits, and goes only if still fresh. |
| 785 | 744 | const stream = { |
| 786 | 745 | ws: null, closeTimer: null, halted: false, waiting: null, |
| @@ -872,8 +831,7 @@ | ||
| 872 | 831 | // STOP: drop what this page would still send, then the server halts and holds the arm until Resume. |
| 873 | 832 | async function stop() { |
| 874 | 833 | stream.halt(); |
| 875 | driver.releaseAll(); | |
| 876 | reachEnd(); | |
| 834 | grab?.release(); | |
| 877 | 835 | stopLeaderDrive("stopped"); |
| 878 | 836 | const r = await api("/stop", {}); |
| 879 | 837 | msg(r.ok ? `STOPPED${r.stopped ? ` ${r.stopped}` : ""}${r.saved ? `; saved recording ${r.saved}` : ""}: the arm holds until you resume` : r.error, true); |
| @@ -1040,7 +998,7 @@ | ||
| 1040 | 998 | async function connectLeader() { |
| 1041 | 999 | let lib, tele; |
| 1042 | 1000 | try { [lib, tele] = await Promise.all([import("/app/leader-arm.ts"), import("/app/teleop.ts")]); } |
| 1043 | catch { return explain("This server could not hand the page the leader arm's code. Meanwhile you can drive with the sliders, the − and + buttons, or by dragging a joint of the 3D model."); } | |
| 1001 | catch { return explain("This server could not hand the page the leader arm's code. Meanwhile you can drive by dragging a part of the 3D model, or with the sliders."); } | |
| 1044 | 1002 | const support = lib.leaderSupport({ serial: "serial" in navigator, secure: isSecureContext, userAgent: navigator.userAgent, touchPoints: navigator.maxTouchPoints }); |
| 1045 | 1003 | if (!support.ok) return explain(support.why); |
| 1046 | 1004 | const port = await navigator.serial.requestPort({ filters: lib.BOARD_FILTERS }).catch(() => null); |
| @@ -1203,28 +1161,54 @@ | ||
| 1203 | 1161 | document.addEventListener("visibilitychange", () => teleop?.update()); |
| 1204 | 1162 | navigator.serial?.addEventListener("disconnect", (e) => { if (e.target === leader?.port) disconnectLeader("the leader was unplugged"); }); |
| 1205 | 1163 | |
| 1206 | // ---------------------------------------------------------------- the 3D model | |
| 1164 | // ---------------------------------------------------------------- the 3D model: the main control | |
| 1207 | 1165 | // three.js is most of what the page downloads, so it comes after the first view: the room and the camera work without it. |
| 1166 | // A finger (or the mouse) on a part of the arm turns that part's joint, live (src/ui/grab.js); on empty space, two | |
| 1167 | // fingers or a pinch, it orbits and zooms the view and sends nothing. The model shows your target; a ghost shows the arm. | |
| 1208 | 1168 | const viewer = $("model"); |
| 1209 | let THREE = null, model3D = null, dragging = null, table = null, marker = null; | |
| 1169 | let THREE = null, model3D = null, dragging = null, table = null, marker = null, grab = null, ghost = null, may = false; | |
| 1210 | 1170 | function load3D() { |
| 1211 | 1171 | if (!modelInfo?.ok) return Promise.resolve(false); |
| 1212 | 1172 | return (model3D ??= (async () => { |
| 1213 | const [three, { STLLoader }, { default: URDFManipulator }, reach] = await Promise.all([ | |
| 1214 | import("three"), import("three/examples/jsm/loaders/STLLoader.js"), import("urdf-loader/manipulator"), import("/ui/__VERSION__/reach.js"), | |
| 1173 | const [three, { STLLoader }, { default: URDFManipulator }, { Grab }] = await Promise.all([ | |
| 1174 | import("three"), import("three/examples/jsm/loaders/STLLoader.js"), import("urdf-loader/manipulator"), import("/ui/__VERSION__/grab.js"), | |
| 1215 | 1175 | ]); |
| 1216 | THREE = three; reachMath = reach; | |
| 1176 | THREE = three; | |
| 1217 | 1177 | if (!customElements.get("urdf-viewer")) customElements.define("urdf-viewer", URDFManipulator); |
| 1218 | 1178 | viewer.loadMeshFunc = (path, manager, material, done) => { |
| 1219 | 1179 | new STLLoader(manager).load(path, (geometry) => done(new THREE.Mesh(geometry, material ?? new THREE.MeshPhongMaterial({ color: 0x9aa4ae, shininess: 30 }))), undefined, (err) => done(null, err)); |
| 1220 | 1180 | }; |
| 1221 | 1181 | const canvas = viewer.renderer.domElement; |
| 1222 | 1182 | canvas.style.touchAction = "none"; |
| 1223 | const asMouse = (type, t) => canvas.dispatchEvent(new MouseEvent(type, { clientX: t.clientX, clientY: t.clientY, bubbles: true })); | |
| 1224 | canvas.addEventListener("touchstart", (e) => { if (e.touches.length === 1) asMouse("mousedown", e.touches[0]); }, { passive: true }); | |
| 1225 | canvas.addEventListener("touchmove", (e) => { if (e.touches.length === 1 && dragging) { e.preventDefault(); asMouse("mousemove", e.touches[0]); } }, { passive: false }); | |
| 1226 | canvas.addEventListener("touchend", (e) => { if (dragging) asMouse("mouseup", e.changedTouches[0]); }, { passive: true }); | |
| 1227 | canvas.addEventListener("touchcancel", (e) => { if (dragging) asMouse("mouseup", e.changedTouches[0]); }, { passive: true }); | |
| 1183 | viewer.controls.touches = { ONE: THREE.TOUCH.ROTATE, TWO: THREE.TOUCH.DOLLY_ROTATE }; | |
| 1184 | // The element's own mouse handling would turn a joint whoever touches it: every real mouse event stops here, and | |
| 1185 | // the grab feeds it made-up ones only while it holds a joint for someone who may drive. | |
| 1186 | const as = (type, x, y) => canvas.dispatchEvent(new MouseEvent(type, { clientX: x, clientY: y })); | |
| 1187 | for (const type of ["mousedown", "mousemove", "mouseup"]) viewer.addEventListener(type, (e) => { if (e.isTrusted) e.stopPropagation(); }, true); | |
| 1188 | let touch = false; | |
| 1189 | grab = new Grab({ | |
| 1190 | allowed: () => may && Boolean(viewer.robot) && !calib && !leaderStreaming(), | |
| 1191 | pick: (x, y) => { as("mousedown", x, y); return viewer.dragControls.manipulating?.name ?? null; }, | |
| 1192 | drag: (x, y) => as("mousemove", x, y), | |
| 1193 | drop: (x, y) => { as("mouseup", x, y); if (touch) as("mousemove", -1e5, -1e5); }, // a finger leaves no hover behind | |
| 1194 | ticks: (name, angle) => { const id = IDS[name]; return id && limits[id] ? [id, clampTicks(id, toTicks(id, angle))] : null; }, | |
| 1195 | send: (f) => stream.send(f), | |
| 1196 | flush: (f) => stream.flush(f), | |
| 1197 | begin: (name) => { dragging = name; stream.open(); navigator.vibrate?.(8); }, | |
| 1198 | end: (name) => { dragging = null; intent.release(IDS[name], performance.now()); stream.end(); if (status) applyModel(status); }, | |
| 1199 | }); | |
| 1200 | grab.watch(window, document); | |
| 1201 | viewer.addEventListener("pointerdown", (e) => { | |
| 1202 | if (e.pointerType === "mouse" && e.button !== 0) return; // the right button pans | |
| 1203 | touch = e.pointerType !== "mouse"; | |
| 1204 | if (grab.down({ id: e.pointerId, x: e.clientX, y: e.clientY })) { e.stopPropagation(); try { canvas.setPointerCapture(e.pointerId); } catch {} } | |
| 1205 | }, true); | |
| 1206 | viewer.addEventListener("pointermove", (e) => { if (!grab.move({ id: e.pointerId, x: e.clientX, y: e.clientY }) && e.pointerType === "mouse" && grab.joint === null) as("mousemove", e.clientX, e.clientY); }); | |
| 1207 | for (const type of ["pointerup", "pointercancel", "lostpointercapture"]) viewer.addEventListener(type, (e) => grab.up({ id: e.pointerId, x: e.clientX, y: e.clientY })); | |
| 1208 | viewer.addEventListener("angle-change", (e) => { | |
| 1209 | const t = viewer.robot && grab.angle(e.detail, viewer.robot.joints[e.detail].angle); | |
| 1210 | if (t) intent.set(t[0], t[1], performance.now()); | |
| 1211 | }); | |
| 1228 | 1212 | applyLock(); |
| 1229 | 1213 | viewer.urdf = modelInfo.urdf; |
| 1230 | 1214 | return true; |
| @@ -1236,6 +1220,12 @@ | ||
| 1236 | 1220 | table.position.z = modelInfo.floorZ - 0.0005; |
| 1237 | 1221 | const grid = new THREE.GridHelper(0.9, 18, 0x4a5058, 0x30363d); grid.rotation.x = Math.PI / 2; grid.position.z = modelInfo.floorZ; |
| 1238 | 1222 | marker = new THREE.Mesh(new THREE.SphereGeometry(0.012, 16, 12), new THREE.MeshBasicMaterial({ color: 0xf85149 })); marker.visible = false; |
| 1223 | // The ghost: the real arm, faint, wherever your target has run ahead of it. It shares the meshes and is never picked. | |
| 1224 | ghost = viewer.robot.clone(); | |
| 1225 | const faint = new THREE.MeshBasicMaterial({ color: 0x8ab4f8, transparent: true, opacity: 0.28, depthWrite: false }); | |
| 1226 | ghost.traverse((o) => { o.raycast = () => {}; if (o.isMesh) o.material = faint; if (o.isURDFJoint) o.ignoreLimits = true; }); | |
| 1227 | ghost.visible = false; | |
| 1228 | viewer.robot.parent.add(ghost); | |
| 1239 | 1229 | for (const o of [table, grid, marker]) o.raycast = () => {}; |
| 1240 | 1230 | viewer.world.add(table, grid, marker); |
| 1241 | 1231 | viewer.plane.visible = false; // the element's own shadow plane sits under the lowest part, which would be below the table |
| @@ -1244,26 +1234,19 @@ | ||
| 1244 | 1234 | viewer.controls.target.set(0.08, 0.1, 0); |
| 1245 | 1235 | viewer.controls.update(); |
| 1246 | 1236 | viewer.recenter(); |
| 1247 | $("reach").hidden = !tipLink(); | |
| 1248 | }); | |
| 1249 | viewer.addEventListener("manipulate-start", (e) => { dragging = e.detail; stream.open(); }); | |
| 1250 | viewer.addEventListener("manipulate-end", () => { const id = IDS[dragging]; if (id) intent.release(id, performance.now()); dragging = null; stream.end(); }); | |
| 1251 | viewer.addEventListener("angle-change", (e) => { | |
| 1252 | if (dragging !== e.detail || !viewer.robot) return; | |
| 1253 | const id = IDS[e.detail]; if (!id || !limits[id]) return; | |
| 1254 | const ticks = clampTicks(id, toTicks(id, viewer.robot.joints[e.detail].angle)); | |
| 1255 | intent.set(id, ticks, performance.now()); | |
| 1256 | stream.send({ [id]: ticks }); | |
| 1257 | 1237 | }); |
| 1258 | 1238 | function applyModel(d) { |
| 1259 | 1239 | if (!viewer.robot || calib) return; // while calibrating, the model shows the pose to copy |
| 1240 | let apart = false; | |
| 1260 | 1241 | for (const j of leaderMirrored() ? [] : d.joints) { |
| 1261 | 1242 | const name = NAMES[j.id]; |
| 1262 | if (!name || j.position == null || name === dragging || !limits[j.id]) continue; | |
| 1263 | if (reachState.timer && j.id <= 3) continue; // the joystick is ahead of the arm; snapping back would make it stutter | |
| 1264 | const want = intent.target(j.id, j.position, performance.now()); | |
| 1265 | viewer.setJointValue(name, toRad(j.id, want ?? j.position)); | |
| 1243 | if (!name || j.position == null || !limits[j.id]) continue; | |
| 1244 | ghost?.joints[name]?.setJointValue(toRad(j.id, j.position)); | |
| 1245 | const want = name === dragging ? toTicks(j.id, viewer.robot.joints[name].angle) : intent.target(j.id, j.position, performance.now()); | |
| 1246 | if (want != null && Math.abs(want - j.position) > 15) apart = true; | |
| 1247 | if (name !== dragging) viewer.setJointValue(name, toRad(j.id, want ?? j.position)); | |
| 1266 | 1248 | } |
| 1249 | if (ghost && (apart || ghost.visible)) { ghost.visible = apart; viewer.redraw(); } | |
| 1267 | 1250 | if (marker) { |
| 1268 | 1251 | const v = d.blocked ?? d.violation; |
| 1269 | 1252 | marker.visible = Boolean(v); |
| @@ -1271,193 +1254,6 @@ | ||
| 1271 | 1254 | } |
| 1272 | 1255 | } |
| 1273 | 1256 | |
| 1274 | // ---------------------------------------------------------------- the joystick: move the gripper, not a joint | |
| 1275 | // The pad's deflection becomes a displacement of the gripper (src/ui/reach.js), solved onto pan, lift and elbow. | |
| 1276 | let reachMath = null; | |
| 1277 | const REACH = [1, 2, 3]; | |
| 1278 | const SPEED = { 100: 0.02, 300: 0.05, 800: 0.1 }; // m/s at full deflection, by jog speed | |
| 1279 | const reachState = { stick: { x: 0, y: 0, lift: 0 }, angles: null, timer: null, holders: 0 }; | |
| 1280 | const tipLink = () => viewer.robot?.links?.gripper_frame_link ?? viewer.robot?.links?.gripper_link ?? null; | |
| 1281 | /** Where the gripper would be with pan, lift and elbow at `angles`, in the arm's base frame: the model is posed, read and put back. */ | |
| 1282 | function tipAt(angles) { | |
| 1283 | const robot = viewer.robot, link = tipLink(); | |
| 1284 | const saved = REACH.map((id) => robot.joints[NAMES[id]].jointValue[0]); | |
| 1285 | REACH.forEach((id, i) => robot.joints[NAMES[id]].setJointValue(angles[i])); | |
| 1286 | robot.updateMatrixWorld(true); | |
| 1287 | const p = robot.worldToLocal(link.getWorldPosition(new THREE.Vector3())); | |
| 1288 | REACH.forEach((id, i) => robot.joints[NAMES[id]].setJointValue(saved[i])); | |
| 1289 | robot.updateMatrixWorld(true); | |
| 1290 | return [p.x, p.y, p.z]; | |
| 1291 | } | |
| 1292 | function reachBegin() { | |
| 1293 | if (reachState.timer) { reachState.holders++; return; } | |
| 1294 | if (!viewer.robot || !reachMath || !tipLink()) return msg("the 3D model is still loading", true); | |
| 1295 | const at = REACH.map((id) => status?.joints.find((j) => j.id === id)?.position); | |
| 1296 | if (at.some((p) => p == null) || REACH.some((id) => !limits[id])) return msg("no position for the arm yet", true); | |
| 1297 | reachState.angles = REACH.map((id, i) => toRad(id, at[i])); | |
| 1298 | reachState.holders = 1; | |
| 1299 | stream.open(); | |
| 1300 | reachState.timer = setInterval(() => { | |
| 1301 | const { stick } = reachState; | |
| 1302 | if (!stick.x && !stick.y && !stick.lift) return; | |
| 1303 | const delta = reachMath.stickToDelta(stick, tipAt(reachState.angles), SPEED[$("rate").value] ?? 0.05, 0.05); | |
| 1304 | const dq = reachMath.reachStep(tipAt, reachState.angles, delta); | |
| 1305 | const frame = {}; | |
| 1306 | REACH.forEach((id, i) => { frame[id] = clampTicks(id, toTicks(id, reachState.angles[i] + dq[i])); }); | |
| 1307 | reachState.angles = REACH.map((id) => toRad(id, frame[id])); // a joint at its limit stays there | |
| 1308 | REACH.forEach((id, i) => viewer.setJointValue(NAMES[id], reachState.angles[i])); | |
| 1309 | stream.send(frame); | |
| 1310 | }, 50); | |
| 1311 | } | |
| 1312 | function reachEnd(all = true) { | |
| 1313 | if (!reachState.timer) return; | |
| 1314 | if (!all && --reachState.holders > 0) return; | |
| 1315 | clearInterval(reachState.timer); reachState.timer = null; reachState.holders = 0; | |
| 1316 | reachState.stick = { x: 0, y: 0, lift: 0 }; | |
| 1317 | stream.end(); | |
| 1318 | } | |
| 1319 | { | |
| 1320 | const pad = $("pad"), knob = $("knob"); | |
| 1321 | let pointer = null; | |
| 1322 | const place = (e) => { | |
| 1323 | const r = pad.getBoundingClientRect(), radius = r.width / 2; | |
| 1324 | let x = (e.clientX - r.left - radius) / radius, y = -(e.clientY - r.top - radius) / radius; | |
| 1325 | const length = Math.hypot(x, y); | |
| 1326 | if (length > 1) { x /= length; y /= length; } | |
| 1327 | reachState.stick.x = Math.abs(x) < 0.12 ? 0 : x; reachState.stick.y = Math.abs(y) < 0.12 ? 0 : y; // a dead zone around the middle | |
| 1328 | knob.style.transform = `translate(${x * radius * 0.62}px, ${-y * radius * 0.62}px)`; | |
| 1329 | }; | |
| 1330 | const release = () => { | |
| 1331 | if (pointer === null) return; | |
| 1332 | pointer = null; pad.classList.remove("held"); knob.style.transform = ""; | |
| 1333 | reachState.stick.x = reachState.stick.y = 0; reachEnd(false); | |
| 1334 | }; | |
| 1335 | pad.addEventListener("pointerdown", (e) => { | |
| 1336 | if (pointer !== null) return; | |
| 1337 | e.preventDefault(); pointer = e.pointerId; | |
| 1338 | try { pad.setPointerCapture(e.pointerId); } catch {} | |
| 1339 | pad.classList.add("held"); place(e); reachBegin(); | |
| 1340 | }); | |
| 1341 | pad.addEventListener("pointermove", (e) => { if (e.pointerId === pointer) place(e); }); | |
| 1342 | for (const ev of ["pointerup", "pointercancel", "lostpointercapture"]) pad.addEventListener(ev, (e) => { if (e.pointerId === pointer) release(); }); | |
| 1343 | addEventListener("blur", release); | |
| 1344 | for (const b of document.querySelectorAll("#reach [data-lift]")) { | |
| 1345 | let held = false; | |
| 1346 | const up = () => { if (!held) return; held = false; reachState.stick.lift = 0; reachEnd(false); }; | |
| 1347 | b.addEventListener("pointerdown", (e) => { e.preventDefault(); if (held) return; held = true; try { b.setPointerCapture(e.pointerId); } catch {} reachState.stick.lift = Number(b.dataset.lift); reachBegin(); }); | |
| 1348 | for (const ev of ["pointerup", "pointercancel", "lostpointercapture"]) b.addEventListener(ev, up); | |
| 1349 | b.addEventListener("contextmenu", (e) => e.preventDefault()); | |
| 1350 | } | |
| 1351 | for (const b of document.querySelectorAll("#reach [data-grip]")) jog(b, 6, () => Number(b.dataset.grip)); | |
| 1352 | } | |
| 1353 | ||
| 1354 | // ---------------------------------------------------------------- touch driving (src/ui/drive.js) | |
| 1355 | // Upright: a chip picks the joint, the lever's deflection is its speed, ▲ and ▼ nudge it and jog when held. | |
| 1356 | const driver = new Driver({ | |
| 1357 | send: (f) => stream.send(f), | |
| 1358 | flush: (f) => stream.flush(f), | |
| 1359 | end: () => stream.end(), | |
| 1360 | begin: () => { | |
| 1361 | if (!canDrive({ mayDrive: mayDrive(), estop: status?.estop, held: status?.held, reconnecting: status?.reconnecting, limits })) return false; | |
| 1362 | stream.open(); | |
| 1363 | return true; | |
| 1364 | }, | |
| 1365 | limits: () => limits, | |
| 1366 | speed: () => Number($("rate").value) || 300, | |
| 1367 | onStop: (id) => { | |
| 1368 | navigator.vibrate?.(30); | |
| 1369 | for (const e of [$("lever"), $("stickL"), $("stickR")]) { e.classList.add("end"); setTimeout(() => e.classList.remove("end"), 400); } | |
| 1370 | msg(`${LABEL[id]?.[1] ?? id} is at the end of its range`); | |
| 1371 | }, | |
| 1372 | }); | |
| 1373 | driver.watch(window, document); | |
| 1374 | // Turning the phone swaps the controls under the thumbs: whatever was held stops, and on its side the sticks come up. | |
| 1375 | sideways.addEventListener("change", () => { driver.releaseAll(); if (sideways.matches && shell.classList.contains("driving")) setBare(true); }); | |
| 1376 | const tap = () => navigator.vibrate?.(8); | |
| 1377 | const LABEL = { 1: ["J1", "Base"], 2: ["J2", "Shoulder"], 3: ["J3", "Elbow"], 4: ["J4", "Wrist"], 5: ["J5", "Roll"], 6: ["Grip", "Gripper"] }; | |
| 1378 | const ENDS = { 1: ["right", "left"], 2: ["up", "down"], 3: ["up", "down"], 4: ["up", "down"], 5: ["roll +", "roll −"], 6: ["open", "close"] }; | |
| 1379 | let picked = (() => { try { const n = Number(localStorage.getItem("so101.joint")); return LABEL[n] ? n : 2; } catch { return 2; } })(); | |
| 1380 | const lever = $("lever"); | |
| 1381 | function pickJoint(id) { | |
| 1382 | if (driver.drive.holders.has("lever")) driver.hold("lever", { [id]: 0 }); // the joint that was moving stops; the finger now drives this one | |
| 1383 | picked = id; | |
| 1384 | try { localStorage.setItem("so101.joint", String(id)); } catch {} | |
| 1385 | for (const b of $("chips").children) { const on = Number(b.dataset.id) === id; b.setAttribute("aria-checked", String(on)); b.tabIndex = on ? 0 : -1; } | |
| 1386 | $("oneName").textContent = LABEL[id][1]; | |
| 1387 | $("nudgeUp").querySelector("small").textContent = ENDS[id][0]; $("nudgeDown").querySelector("small").textContent = ENDS[id][1]; | |
| 1388 | $("nudgeUp").setAttribute("aria-label", `${LABEL[id][1]} a step ${ENDS[id][0]}`); $("nudgeDown").setAttribute("aria-label", `${LABEL[id][1]} a step ${ENDS[id][1]}`); | |
| 1389 | lever.setAttribute("aria-label", `${LABEL[id][1]}: push ${ENDS[id][0]} or ${ENDS[id][1]}, let go to stop`); | |
| 1390 | if (status) renderTouch(status); | |
| 1391 | } | |
| 1392 | $("chips").replaceChildren(...Object.entries(LABEL).map(([id, [short, long]]) => { | |
| 1393 | const b = el("button", { type: "button", onclick: () => { tap(); pickJoint(Number(id)); } }, el("b", { textContent: short }), el("small", { textContent: long })); | |
| 1394 | b.dataset.id = id; b.setAttribute("role", "radio"); b.setAttribute("aria-label", `${short} ${long}`); | |
| 1395 | return b; | |
| 1396 | })); | |
| 1397 | $("chips").addEventListener("keydown", (e) => { | |
| 1398 | const d = e.key === "ArrowRight" || e.key === "ArrowDown" ? 1 : e.key === "ArrowLeft" || e.key === "ArrowUp" ? -1 : 0; | |
| 1399 | if (!d) return; | |
| 1400 | e.preventDefault(); pickJoint(((picked - 1 + d + 6) % 6) + 1); $("chips").children[picked - 1]?.focus(); | |
| 1401 | }); | |
| 1402 | pickJoint(picked); | |
| 1403 | /** Where the picked joint is: degrees from the middle of its range, the gripper as how far open; the green bar on the lever likewise. */ | |
| 1404 | function renderTouch(d) { | |
| 1405 | const j = d.joints.find((x) => x.id === picked), l = limits[picked]; | |
| 1406 | if (!j || j.position == null || !l) { $("onePos").textContent = "—"; return; } | |
| 1407 | const share = (j.position - l.min) / (l.max - l.min); | |
| 1408 | const up = UP[picked] > 0 ? share : 1 - share; // the bar fills toward the lever's top end | |
| 1409 | lever.style.setProperty("--at", String(Math.max(0, Math.min(1, up)).toFixed(3))); | |
| 1410 | $("onePos").textContent = picked === 6 ? `${Math.round(share * 100)}% open` : `${(((j.position - mid(picked)) * 360) / 4096).toFixed(0)}°`; | |
| 1411 | } | |
| 1412 | bindSurface(lever, driver, "lever", (_, y) => ({ [picked]: shape(y) * UP[picked] }), { | |
| 1413 | haptic: tap, | |
| 1414 | show: (_, y) => { | |
| 1415 | $("leverKnob").style.transform = `translateY(${(-y * (lever.clientHeight / 2 - 38)).toFixed(1)}px)`; | |
| 1416 | lever.setAttribute("aria-valuenow", String(Math.round(y * 100))); | |
| 1417 | }, | |
| 1418 | }); | |
| 1419 | // ▲ ▼: a tap is one step, a hold jogs at half speed; arrow keys on the lever step too. | |
| 1420 | const step = () => Math.max(5, Math.round((Number($("rate").value) || 300) / 15)); | |
| 1421 | for (const b of [$("nudgeUp"), $("nudgeDown")]) { | |
| 1422 | let pointer = null, timer = null; | |
| 1423 | const dir = () => Number(b.dataset.nudge) * UP[picked]; | |
| 1424 | b.addEventListener("pointerdown", (e) => { | |
| 1425 | if (pointer !== null) return; | |
| 1426 | e.preventDefault(); pointer = e.pointerId; | |
| 1427 | try { b.setPointerCapture(e.pointerId); } catch {} | |
| 1428 | tap(); b.classList.add("held"); | |
| 1429 | driver.nudge(picked, dir() * step()); | |
| 1430 | timer = setTimeout(() => { timer = null; if (pointer !== null) driver.hold("nudge", { [picked]: dir() * 0.5 }); }, 350); | |
| 1431 | }); | |
| 1432 | const up = (e) => { if (e.pointerId !== pointer) return; pointer = null; clearTimeout(timer); b.classList.remove("held"); driver.release("nudge"); }; | |
| 1433 | for (const ev of ["pointerup", "pointercancel", "lostpointercapture"]) b.addEventListener(ev, up); | |
| 1434 | b.addEventListener("keydown", (e) => { if (e.key === "Enter" || e.key === " ") { e.preventDefault(); e.stopPropagation(); driver.nudge(picked, dir() * step()); } }); | |
| 1435 | b.addEventListener("contextmenu", (e) => e.preventDefault()); | |
| 1436 | } | |
| 1437 | lever.addEventListener("keydown", (e) => { | |
| 1438 | const d = e.key === "ArrowUp" ? 1 : e.key === "ArrowDown" ? -1 : 0; | |
| 1439 | if (d) { e.preventDefault(); driver.nudge(picked, d * UP[picked] * step()); } | |
| 1440 | }); | |
| 1441 | // The sticks: a disc, so a corner is no faster than an edge; each stick has its own finger. | |
| 1442 | const disc = (x, y) => { const r = Math.hypot(x, y); return r > 1 ? [x / r, y / r] : [x, y]; }; | |
| 1443 | const knob = (stick) => (x, y) => { | |
| 1444 | const [dx, dy] = disc(x, y), r = stick.clientWidth * 0.3; | |
| 1445 | stick.querySelector(".knob").style.transform = `translate(${(dx * r).toFixed(1)}px, ${(-dy * r).toFixed(1)}px)`; | |
| 1446 | }; | |
| 1447 | bindSurface($("stickL"), driver, "left", (x, y) => { const [dx, dy] = disc(x, y); return { 1: shape(dx) * UP[1], 2: shape(dy) * UP[2] }; }, { show: knob($("stickL")), haptic: tap }); | |
| 1448 | let wrist = false; | |
| 1449 | const right = () => (wrist ? { x: 5, y: 6 } : { x: 4, y: 3 }); | |
| 1450 | const rightStick = bindSurface($("stickR"), driver, "right", (x, y) => { const [dx, dy] = disc(x, y), m = right(); return { [m.x]: shape(dx) * UP[m.x], [m.y]: shape(dy) * UP[m.y] }; }, { show: knob($("stickR")), haptic: tap }); | |
| 1451 | const labelRight = () => { | |
| 1452 | $("capR").textContent = wrist ? "J5 roll ↔ · grip ↕" : "J4 wrist ↔ · J3 elbow ↕"; | |
| 1453 | $("stickR").setAttribute("aria-label", wrist ? "Right stick: sideways rolls the wrist, up opens the gripper" : "Right stick: sideways bends the wrist, up and down lifts the elbow"); | |
| 1454 | $("swap").textContent = wrist ? "Arm" : "Wrist"; $("swap").setAttribute("aria-pressed", String(wrist)); | |
| 1455 | }; | |
| 1456 | $("swap").onclick = () => { rightStick.end(); wrist = !wrist; tap(); labelRight(); }; | |
| 1457 | labelRight(); | |
| 1458 | bindSurface($("gripOpen"), driver, "open", () => ({ 6: UP[6] }), { haptic: tap }); | |
| 1459 | bindSurface($("gripClose"), driver, "close", () => ({ 6: -UP[6] }), { haptic: tap }); | |
| 1460 | ||
| 1461 | 1257 | // Keep the screen on while it is your turn: a phone that dims mid-drive is a hand off the controls. |
| 1462 | 1258 | let wakeLock = null, waking = false, wantAwake = false; |
| 1463 | 1259 | async function wake(on) { |
| @@ -1571,22 +1367,21 @@ | ||
| 1571 | 1367 | $("t-controls").hidden = !mayDrive(); |
| 1572 | 1368 | $("t-admin").hidden = !owner; |
| 1573 | 1369 | $("t-chat").hidden = $("t-queue").hidden = !relayed; |
| 1574 | for (const id of ["joints", "poses", "drive", "reach", "touch", "sticks"]) $(id).inert = !can; | |
| 1370 | for (const id of ["joints", "poses", "drive"]) $(id).inert = !can; | |
| 1575 | 1371 | shell.classList.toggle("driving", can); |
| 1576 | if (!can) driver.releaseAll(); | |
| 1372 | may = can; | |
| 1373 | if (!can) grab?.release(); | |
| 1577 | 1374 | wake(can); |
| 1578 | 1375 | $("stop").disabled = !(mayDrive() || owner); |
| 1579 | 1376 | $("stop").title = $("stop").disabled ? "Stop: the driver and the admins can press it" : "Stop (space or Esc): halt everything, hold with torque, until someone resumes"; |
| 1580 | 1377 | // The server takes settings, the safety limits among them, from admins only. |
| 1581 | 1378 | for (const id of ["zoneRow", "trims", "safety", "jointLimits"]) $(id).inert = !owner; |
| 1582 | viewer.toggleAttribute("disable-dragging", !can); | |
| 1583 | if (!can) reachEnd(); | |
| 1584 | 1379 | teleop?.update(); // the turn ended: teleop stops at once (the server closes it too; the page need not wait for that) |
| 1585 | 1380 | // Your turn just came: open the controls. It just ended: back to the chat. |
| 1586 | 1381 | if (mayDrive() !== drove) { |
| 1587 | 1382 | drove = mayDrive(); |
| 1588 | 1383 | if (relayed) showTab(drove ? "controls" : "chat"); |
| 1589 | if (drove && sideways.matches) setBare(true); // on its side, a turn starts on the thumb sticks | |
| 1384 | if (drove) fold(false); | |
| 1590 | 1385 | } |
| 1591 | 1386 | else if ($(`t-${tab}`).hidden) showTab(tab); |
| 1592 | 1387 | } |
| @@ -1875,7 +1670,7 @@ | ||
| 1875 | 1670 | const div = document.createElement("div"); div.className = "j"; |
| 1876 | 1671 | div.innerHTML = `<b></b><div class="v"><span class="pos"></span><small class="ld"></small><small class="t"></small></div> |
| 1877 | 1672 | <div class="slide"><span class="jog"><button data-d="-1" aria-label="less">−</button></span><span class="rail"><input type="range" min="${lim.min + 30}" max="${lim.max - 30}" step="1"></span><span class="jog"><button data-d="1" aria-label="more">+</button></span></div>`; |
| 1878 | div.querySelector("b").textContent = j.joint ?? String(j.id); | |
| 1673 | Object.assign(div.querySelector("b"), { textContent: SHORT[j.id] ?? j.joint ?? String(j.id), title: j.joint ?? "" }); | |
| 1879 | 1674 | const range = div.querySelector("input"); |
| 1880 | 1675 | range.setAttribute("aria-label", j.joint ?? `joint ${j.id}`); |
| 1881 | 1676 | range.addEventListener("pointerdown", () => { sliding = j.id; stream.open(); }); |
| @@ -1943,10 +1738,7 @@ | ||
| 1943 | 1738 | { |
| 1944 | 1739 | const was = $("why").textContent; |
| 1945 | 1740 | status = d; |
| 1946 | driver.seed(d.joints.filter((j) => j.position != null).map((j) => [j.id, j.position])); | |
| 1947 | 1741 | if ((explainLock() ?? "") !== was) applyLock(); |
| 1948 | // Stopped, reconnecting or no longer this person's turn: a held thumb lets go, and its zero goes out now. | |
| 1949 | if (!canDrive({ mayDrive: mayDrive(), estop: d.estop, held: d.held, reconnecting: d.reconnecting, limits })) driver.releaseAll(); | |
| 1950 | 1742 | // The round trip to the arm, which every command also makes: shown once it is long enough to feel. |
| 1951 | 1743 | const lag = [...rtts].sort((a, b) => a - b)[rtts.length >> 1] ?? 0; |
| 1952 | 1744 | $("lag").hidden = rtts.length < 3 || lag <= 250; $("lag").textContent = `· ${Math.round(lag)} ms lag`; |
| @@ -1960,7 +1752,7 @@ | ||
| 1960 | 1752 | $("stop").classList.toggle("on", Boolean(d.estop || d.held)); $("stop").textContent = d.estop || d.held ? "Stopped" : "Stop"; |
| 1961 | 1753 | $("record").textContent = d.driving === "record" ? "■ Stop recording" : "● Record"; |
| 1962 | 1754 | for (const id of ["hold", "releaseBtn", "play"]) $(id).disabled = d.driving !== null; |
| 1963 | if (tab === "controls") { render(d); renderTouch(d); } | |
| 1755 | if (tab === "controls") render(d); | |
| 1964 | 1756 | banner(d); applyModel(d); |
| 1965 | 1757 | $("readout").innerHTML = `<span>lowest part <b>${Number.isFinite(d.lowestMm) ? d.lowestMm : "—"} mm</b> above the table</span><span>zone <b>${d.zone === "enforce" || d.zone === "warn" || d.zone === "off" ? d.zone : "—"}</b></span>`; |
| 1966 | 1758 | } |
| @@ -1998,7 +1790,7 @@ | ||
| 1998 | 1790 | if (early) applyStatus(early); |
| 1999 | 1791 | poll(); |
| 2000 | 1792 | if (relayed) pollLink(); |
| 2001 | // The model is fetched once the first view is up, whether or not it is on screen: the joystick needs it. | |
| 1793 | // The model is fetched once the first view is up, whether or not it is on screen: it is the main control. | |
| 2002 | 1794 | if (modelInfo.ok) (window.requestIdleCallback ?? setTimeout)(() => load3D(), { timeout: 3000 }); |
| 2003 | 1795 | })(); |
| 2004 | 1796 | </script> |
new above
offline