Public: anyone can read this; the crew can change it.

Migrated from github.com/Trevato/SO-ARM100

workbenchthis machine

SAFETY P0: pressing STOP never makes the arm move - commands sent while held are dropped, never buffered and replayed #72

Merged opened this in answer to

Check · Review · Land done · Live — Landed; nothing deploys from this branch, so this is the end of its path.

Merged

quill/arm:quill/66-safety-p0-pressing-stop-never-makes-the → main 8 commits fork

  • Merged. quill/66-safety-p0-pressing-stop-never-makes-the was merged into main as 87dd4e8.

Review

Sign in to review this change.
26 files changed +1068 −119
Software/README.md +29 −7
@@ -130,18 +130,31 @@ SO101_PORT=<leader port> so101 leader --to ws://<pi>:8101
130130 The API, all JSON (the full list with bodies is at the top of
131131 `so101/src/server.ts`): `GET /status`, `/limits`, `/poses`, `/recordings`,
132132 `/calibration`; `POST /pose/save`, `/pose/delete`, `/pose/go`, `/move`
133 (explicit `{"targets": {"3": 2048}}`), `/grip`, `/torque`, `/stop`,
133 (explicit `{"targets": {"3": 2048}}`), `/grip`, `/torque`, `/stop`, `/resume`,
134134 `/estop`, `/estop/clear`, `/record/start`, `/record/stop`,
135135 `/recording/delete`, `/play`;
136136 `PUT /calibration/camera|handeye`; the WebSocket `/follow` taking
137 `{"1": ticks, ...}` frames; and the read-only WebSocket `/watch`, which sends
137 `{"1": ticks, ..., "seq": n, "t": ms}` frames; and the read-only WebSocket `/watch`, which sends
138138 the `/status` body every 250 ms, one bus read shared by every watcher, and
139139 between them the machine log: every register written on the bus, every change
140140 of driver, hold and E-stop, and every safety clamp, as it happens. Each joint
141141 in `/status` also carries its last goal (`target`), its calibrated range
142 (`min`, `max`) and the bounds commands are held inside (`lo`, `hi`). One thing drives the arm at a time. Every move
143 goes through the joint supervisor; `/stop` ends a move, a playback, a
144 recording, or a stream, and the arm holds. The stream is clamped to the
142 (`min`, `max`) and the bounds commands are held inside (`lo`, `hi`); `held` is
143 the STOP (who, when; `null` once resumed) and `deadman` the stream's silence
144 hold. One thing drives the arm at a time. Every move
145 goes through the joint supervisor. `/stop` ends a move, a playback, a
146 recording, or a stream, and the arm holds with torque on; it stays held (every
147 stream refused, every frame dropped, every move answered 423) until `/resume`,
148 and after a resume nothing moves until a command sent after it arrives. Stream
149 frames are latest-wins targets, never a queue: one carrying `seq` and `t` is
150 dropped when it is out of order, more than 200 ms late, or sent before the last
151 resume (`so101/src/command-gate.ts`); the hosted app's relay requires both and
152 applies the same gate, so even an arm server too old to hold gets nothing
153 after a STOP. An arm server also takes frames with no `seq` and `t` while it
154 is not held, because `so101 leader` on a LAN may be older than the stamps
155 (the current one stamps). For such a frame the arm server checks only the
156 hold: nothing on the arm server tells a late frame from a fresh one, so for
157 the hosted page the relay is what drops stale frames. The stream is clamped to the
145158 joint limits, eases over to the first frame, drops jumps no hand makes, is
146159 held to the speed and acceleration limits, and holds the arm after a second
147160 of silence (see **Safety limits** below). There is no authentication: keep it
@@ -151,7 +164,10 @@ on a LAN or behind Tailscale.
151164 arm (its own URDF and meshes, drag a joint and the arm follows), a slider and
152165 hold-to-jog buttons per joint that stream targets like a leader does, a
153166 button per saved pose plus save and delete, hold and release, record and
154 play, a stop button while a move or a playback runs (space bar), fullscreen (F), a metrics panel, a settings
167 play, one STOP button (pressed, not released; space or Esc) with a separate
168 two-tap **Resume driving** in the stopped banner (STOP is also how a pose
169 move or a playback is ended early; there is no separate "End move" button,
170 so driving again after it takes a Resume), fullscreen (F), a metrics panel, a settings
155171 panel for the safe zone, and a 3D-calibration panel that explains the vision
156172 quest and shows its status. `bench/fake-serve.ts` runs the same server on a
157173 simulated bus for working on the page. The page expects MJPEG streams on
@@ -418,6 +434,11 @@ watches. A line you type is said in the chat; commands start with `/`:
418434 - `/join`, `/leave`: ask to drive (the owner takes over at once), hand back.
419435 - `/ready`, `/extend`: confirm you are there when your turn is called (15 s,
420436 or it passes on), add 30 s to your turn while nobody waits.
437 - `/stop`, the driver or an owner: the page's STOP, the same `POST /stop`:
438 the arm halts and holds until `/resume`. `/resume`, the same people: ends
439 the hold, and the arm stays put until a new command. Every terminal prints
440 `STOPPED by <who>` when the arm is stopped, from a terminal or a page, and
441 `resumed` when it is resumed.
421442 - `/end [name]`, owner: end that turn (no name: whoever drives), or take
422443 them out of the line.
423444 - `/mute <name>`, `/unmute <name>`, owner.
@@ -433,7 +454,8 @@ the same `/live` socket on `127.0.0.1:$PORT`, as the forge's identity for
433454 the person at the keyboard (`FORGE_APP_USER*`, signed with
434455 `FORGE_APP_IDENTITY_KEY`). The server applies the same rules to both, so a
435456 muted person is muted at the terminal too, only the owner may end turns or
436 mute, and every line, join and hand-back from the terminal reaches every
457 mute, only the driver or an owner may stop or resume (over HTTP to the same
458 app, signed afresh for each request), and every line, join and hand-back from the terminal reaches every
437459 web viewer as it happens. Text from the room is shown with control
438460 characters removed, so a chat line cannot drive your terminal. Locally,
439461 `PORT=8102 bun run Software/so101/src/tty.ts` beside `fake-serve.ts`
Software/so101/src/command-gate.test.ts added +96
@@ -0,0 +1,96 @@
1 import { describe, expect, test } from "bun:test";
2 import { Clock, CommandGate, MAX_AGE_MS, parseCommand, stamper } from "./command-gate.ts";
3
4 const frame = (seq: number, t: number, joints: Record<string, number> = { 1: 2000 }) => JSON.stringify({ ...joints, seq, t });
5 /** The page's clock runs this far behind the server's: only the gaps matter. */
6 const SKEW = 5_000_000;
7
8 describe("parseCommand", () => {
9 test("reads the joints, and seq and t only when both are well formed", () => {
10 expect(parseCommand(frame(3, 1000))).toEqual({ seq: 3, t: 1000, joints: new Map([[1, 2000]]) });
11 expect(parseCommand('{"1": 2000}')).toEqual({ seq: null, t: null, joints: new Map([[1, 2000]]) });
12 for (const bad of ['{"1":2000,"seq":-1,"t":5}', '{"1":2000,"seq":1.5,"t":5}', '{"1":2000,"seq":1}', '{"1":2000,"seq":1,"t":"now"}', '{"1":2000,"seq":1,"t":0}'])
13 expect(parseCommand(bad)).toMatchObject({ seq: null, t: null });
14 expect(parseCommand("[1]")).toBeNull();
15 expect(parseCommand("nope")).toBeNull();
16 });
17 });
18
19 describe("CommandGate", () => {
20 test("passes fresh frames in order, and drops one out of order or sent twice", () => {
21 const gate = new CommandGate();
22 const s = gate.open(new Clock(), true);
23 expect(s.admit(frame(1, 1000), 1000 + SKEW)).toEqual({ ok: true, joints: new Map([[1, 2000]]) });
24 expect(s.admit(frame(2, 1030), 1030 + SKEW).ok).toBe(true);
25 expect(s.admit(frame(2, 1030), 1031 + SKEW)).toEqual({ ok: false, drop: "replayed" });
26 expect(s.admit(frame(1, 1000), 1032 + SKEW)).toEqual({ ok: false, drop: "replayed" });
27 });
28
29 test("a frame that waited in a buffer is stale, whatever the two clocks say", () => {
30 const gate = new CommandGate();
31 const s = gate.open(new Clock(), true);
32 for (let i = 1; i <= 5; i++) expect(s.admit(frame(i, 1000 + i * 30), 1000 + i * 30 + SKEW + 12).ok).toBe(true);
33 // three frames made 30 ms apart that sat in a send queue and arrive together
34 const at = 1500 + SKEW + 12;
35 expect(s.admit(frame(6, 1180), at)).toEqual({ ok: false, drop: "stale" });
36 expect(s.admit(frame(7, 1210), at)).toEqual({ ok: false, drop: "stale" });
37 expect(s.admit(frame(8, 1500 - MAX_AGE_MS + 1), at).ok).toBe(true); // just young enough
38 });
39
40 test("held: everything is dropped, stamped or not; a resume lets through only what was sent after it", () => {
41 const gate = new CommandGate();
42 const clock = new Clock();
43 const s = gate.open(clock, false);
44 const now = 10_000_000;
45 expect(s.admit(frame(1, now - SKEW), now).ok).toBe(true);
46 expect(gate.hold("ann", now + 10)).toBe(true);
47 expect(gate.held?.by).toBe("ann");
48 expect(gate.hold("bob", now + 11)).toBe(false); // the first stop is the one on record
49 expect(s.admit(frame(2, now + 20 - SKEW), now + 20)).toEqual({ ok: false, drop: "held" });
50 expect(s.admit('{"1": 2100}', now + 20)).toEqual({ ok: false, drop: "held" });
51 // Resumed; a stream open across it sends nothing more that counts.
52 expect(gate.resume(now + 100)).toBe(true);
53 expect(s.admit(frame(3, now + 120 - SKEW), now + 120)).toEqual({ ok: false, drop: "replayed" });
54 // The page's next stream: its clock is the same person's, so a frame made before the resume is known as one.
55 const next = gate.open(clock, true);
56 expect(next.admit(frame(4, now + 90 - SKEW), now + 101)).toEqual({ ok: false, drop: "before-resume" });
57 expect(next.admit(frame(5, now + 102 - SKEW), now + 102).ok).toBe(true);
58 expect(gate.resume(now + 200)).toBe(false); // nothing was held
59 });
60
61 test("behind the relay a frame must be stamped; on the arm server an unstamped one passes unless held", () => {
62 const gate = new CommandGate();
63 expect(gate.open(new Clock(), true).admit('{"1": 2000}')).toEqual({ ok: false, drop: "unstamped" });
64 expect(gate.open(new Clock(), false).admit('{"1": 2000}').ok).toBe(true);
65 expect(gate.open(new Clock(), false).admit("{nope")).toEqual({ ok: false, drop: "malformed" });
66 });
67 });
68
69 describe("Clock", () => {
70 test("takes the quickest trip for the skew, so lateness is measured from it", () => {
71 const c = new Clock();
72 expect(c.observe(100, 100 + SKEW + 40)).toBe(0); // the first frame sets it
73 expect(c.observe(130, 130 + SKEW + 10)).toBe(0); // quicker: it becomes the skew
74 expect(c.observe(160, 160 + SKEW + 60)).toBe(50);
75 expect(c.here(160)).toBe(160 + SKEW + 10);
76 });
77
78 test("a page whose clock is stepped is relearnt from one window of its frames, not called stale for good", () => {
79 const c = new Clock();
80 let late = 0;
81 for (let t = 0; t <= 31_000; t += 100) c.observe(t, t + SKEW);
82 // the page's clock jumps back 10 s: every frame now looks 10 s late
83 for (let t = 31_100; t <= 92_000; t += 100) late = c.observe(t - 10_000, t + SKEW);
84 expect(late).toBe(0);
85 });
86 });
87
88 describe("stamper", () => {
89 test("numbers each frame and stamps the time", () => {
90 let now = 50;
91 const stamp = stamper(() => now);
92 expect(JSON.parse(stamp({ 1: 2000 }))).toEqual({ 1: 2000, seq: 1, t: 50 });
93 now = 80;
94 expect(JSON.parse(stamp({ 6: 2500 }))).toEqual({ 6: 2500, seq: 2, t: 80 });
95 });
96 });
Software/so101/src/command-gate.ts added +165
@@ -0,0 +1,165 @@
1 /**
2 * The gate every /follow frame passes before it may move the arm: commands are latest-wins targets, never a queue.
3 *
4 * A frame is `{"seq": n, "t": ms, "1": ticks, ...}`: `seq` counts up in the page that sent it, `t` is the page's clock
5 * when it was made. A frame is dropped, never kept for later, when
6 *
7 * held the arm is stopped: STOP holds it until someone resumes, and nothing sent meanwhile counts
8 * replayed its seq is not above the last one this stream sent (a duplicate, or out of order)
9 * stale it is older than MAX_AGE_MS: it sat in a buffer somewhere and is no longer what the hand meant
10 * before-resume it was sent before the latest resume, however it got here
11 * unstamped it has no seq and t where they are required (behind the relay; the CLI leader on a LAN sends none)
12 *
13 * The page's clock is not the server's. Each sender keeps a `Clock`: the smallest gap seen between a frame's `t` and
14 * its arrival is the skew plus the quickest trip, and a frame's age is how much later than that it came. A buffered
15 * frame comes late by however long it waited, whatever the clocks say. Every REANCHOR_MS the smallest gap is taken
16 * again from that window alone, so a page whose clock is stepped does not have every frame called stale for good.
17 * A stream still open when the arm resumes sends nothing more that counts: the page opens a new one.
18 * Pure: text and times in, verdicts out.
19 */
20
21 import { parseFrame } from "./safety.ts";
22
23 export const MAX_AGE_MS = 200;
24 const REANCHOR_MS = 30_000;
25 /** The close code for a stream ended, or refused, because the arm is stopped. */
26 export const HELD_CLOSE = 4409;
27
28 export interface Held {
29 by: string;
30 at: string;
31 }
32
33 export type Drop = "malformed" | "unstamped" | "replayed" | "held" | "stale" | "before-resume";
34 export type Verdict = { ok: true; joints: Map<number, number> } | { ok: false; drop: Drop };
35
36 export interface Command {
37 seq: number | null;
38 t: number | null;
39 joints: Map<number, number>;
40 }
41
42 /** The frame's joints, and its seq and t when both are well formed; null for anything that is not a frame. */
43 export function parseCommand(text: string): Command | null {
44 const joints = parseFrame(text);
45 if (!joints) return null;
46 const { seq, t } = JSON.parse(text) as { seq?: unknown; t?: unknown };
47 const stamped = Number.isSafeInteger(seq) && (seq as number) >= 0 && typeof t === "number" && Number.isFinite(t) && t > 0;
48 return { seq: stamped ? (seq as number) : null, t: stamped ? (t as number) : null, joints };
49 }
50
51 /** One sender's clock, seen from here: when a frame it stamped would have arrived had nothing held it up. */
52 export class Clock {
53 private offset: number | null = null;
54 /** The smallest gap in the window under way, and when it began: at its end, that becomes the offset. */
55 private windowMin = Infinity;
56 private windowAt = 0;
57
58 /** Learn from a frame stamped `t` that arrived `now`; answers how late it is, ms. */
59 observe(t: number, now: number): number {
60 const gap = now - t;
61 if (this.offset === null) {
62 this.offset = gap;
63 this.windowAt = now;
64 }
65 this.offset = Math.min(this.offset, gap);
66 this.windowMin = Math.min(this.windowMin, gap);
67 if (now - this.windowAt > REANCHOR_MS) {
68 this.offset = this.windowMin;
69 this.windowMin = Infinity;
70 this.windowAt = now;
71 }
72 return gap - this.offset;
73 }
74
75 /** The sender's `t` on this clock, as the earliest it could have arrived. */
76 here(t: number): number {
77 return t + (this.offset ?? 0);
78 }
79 }
80
81 export class CommandGate {
82 /** Who stopped the arm, and when; null while it may move. */
83 held: Held | null = null;
84 /** When the arm was last resumed, on this clock, ms; frames sent before it never count. */
85 private resumedAt = 0;
86 private readonly streams = new Set<GateStream>();
87
88 /** Stop: from now on every frame is dropped until `resume`. Answers whether it was moving. */
89 hold(by: string, now = Date.now()): boolean {
90 if (this.held) return false;
91 this.held = { by, at: new Date(now).toISOString() };
92 return true;
93 }
94
95 /** Answers whether it was held. Nothing moves until a frame sent after this comes in. Not held, it changes nothing. */
96 resume(now = Date.now()): boolean {
97 if (!this.held) return false;
98 this.held = null;
99 this.resumedAt = now;
100 for (const s of this.streams) s.floor();
101 return true;
102 }
103
104 /** A stream's own view of the gate; `stamped` says frames must carry seq and t. Close it when the stream ends. */
105 open(clock: Clock, stamped: boolean): GateStream {
106 const s = new GateStream(this, clock, stamped);
107 this.streams.add(s);
108 return s;
109 }
110
111 /** @internal */
112 forget(s: GateStream) {
113 this.streams.delete(s);
114 }
115
116 /** @internal */
117 get since(): number {
118 return this.resumedAt;
119 }
120 }
121
122 export class GateStream {
123 private lastSeq = -1;
124 private readonly gate: CommandGate;
125 private readonly clock: Clock;
126 private readonly stamped: boolean;
127
128 constructor(gate: CommandGate, clock: Clock, stamped: boolean) {
129 this.gate = gate;
130 this.clock = clock;
131 this.stamped = stamped;
132 }
133
134 /** What may go on to the arm from this text, or why nothing may. */
135 admit(text: string, now = Date.now()): Verdict {
136 const c = parseCommand(text);
137 if (!c) return { ok: false, drop: "malformed" };
138 if (c.seq === null || c.t === null) {
139 if (this.stamped) return { ok: false, drop: "unstamped" };
140 return this.gate.held ? { ok: false, drop: "held" } : { ok: true, joints: c.joints };
141 }
142 if (c.seq <= this.lastSeq) return { ok: false, drop: "replayed" };
143 this.lastSeq = c.seq;
144 const late = this.clock.observe(c.t, now);
145 if (this.gate.held) return { ok: false, drop: "held" };
146 if (late > MAX_AGE_MS) return { ok: false, drop: "stale" };
147 if (this.clock.here(c.t) < this.gate.since) return { ok: false, drop: "before-resume" };
148 return { ok: true, joints: c.joints };
149 }
150
151 /** @internal Nothing this stream sent up to now may count after a resume. */
152 floor() {
153 this.lastSeq = Number.MAX_SAFE_INTEGER;
154 }
155
156 close() {
157 this.gate.forget(this);
158 }
159 }
160
161 /** A sender's side: each frame as text, stamped with the next seq and the time now. */
162 export function stamper(now: () => number = Date.now): (joints: Readonly<Record<string, number>>) => string {
163 let seq = 0;
164 return (joints) => JSON.stringify({ ...joints, seq: ++seq, t: now() });
165 }
Software/so101/src/leader-arm.test.ts +5 −1
@@ -5,6 +5,7 @@ import { join } from "node:path";
55 import WebSocket from "ws";
66 import { BROWSER_MODULES } from "./browser-modules.ts";
77 import { FeetechBus } from "./bus.ts";
8 import { stamper } from "./command-gate.ts";
89 import { FakeBus } from "./fake-bus.ts";
910 import { FakeSerialPort, leaderServos, place } from "./fake-serial.ts";
1011 import { sign } from "./identity.ts";
@@ -135,7 +136,10 @@ describe("driving the follower with it", () => {
135136 const ws = new WebSocket(`ws://127.0.0.1:${port}/follow`, { headers: ALICE });
136137 sockets.push(ws);
137138 await new Promise((r, j) => (ws.once("open", r), ws.once("error", j)));
138 return ws;
139 // Stamped as the page stamps them: the relay drops a frame without a seq and a time.
140 const stamp = stamper();
141 const send = ws.send.bind(ws);
142 return Object.assign(ws, { send: (text: string) => send(stamp(JSON.parse(text))) });
139143 };
140144 const followerAt = async (port: number) => {
141145 const s = (await (await fetch(`http://127.0.0.1:${port}/status`)).json()) as { joints: Array<{ id: number; position: number }> };
Software/so101/src/leader.ts +3 −1
@@ -2,6 +2,7 @@
22
33 import WebSocket from "ws";
44 import type { FeetechBus } from "./bus.ts";
5 import { stamper } from "./command-gate.ts";
56 import { mapFrame, type Frame, type Range } from "./mapping.ts";
67
78 export { mapFrame };
@@ -35,6 +36,7 @@ export async function streamLeader(bus: FeetechBus, serverUrl: string, { hz = 60
3536 ws.once("error", reject);
3637 });
3738 const period = 1000 / hz;
39 const stamp = stamper(); // seq and time: an arm server drops a frame that is stale or out of order
3840 let sent = 0;
3941 const started = performance.now();
4042 try {
@@ -42,7 +44,7 @@ export async function streamLeader(bus: FeetechBus, serverUrl: string, { hz = 60
4244 const leader = await bus.syncRead(ids, STS3215.Present_Position);
4345 const frame = mapFrame(leader, leaderLimits, followerLimits, gripperId);
4446 if (Object.keys(frame).length) {
45 ws.send(JSON.stringify(frame));
47 ws.send(stamp(frame));
4648 sent++;
4749 onFrame?.(frame);
4850 }
Software/so101/src/lobby.test.ts +4 −2
@@ -7,6 +7,7 @@ import WebSocket, { WebSocketServer } from "ws";
77 import { Admin } from "./admin.ts";
88 import { FeetechBus } from "./bus.ts";
99 import { Chat } from "./chat.ts";
10 import { stamper } from "./command-gate.ts";
1011 import { FakeBus } from "./fake-bus.ts";
1112 import { sign } from "./identity.ts";
1213 import { Accounts } from "./accounts.ts";
@@ -362,7 +363,7 @@ describe("lobby", () => {
362363 const ws = new WebSocket(`ws://127.0.0.1:${port}/follow`, { headers: ALICE });
363364 sockets.push(ws);
364365 await new Promise((r) => ws.once("open", r));
365 ws.send(JSON.stringify({ "1": 2300 }));
366 ws.send(stamper()({ "1": 2300 }));
366367 await sleep(80);
367368 return ws;
368369 };
@@ -626,8 +627,9 @@ describe("lobby", () => {
626627 await new Promise((r) => stream.once("open", r));
627628 // The leader starts far from where the arm is (2000), and joint 2's position is not known at all.
628629 const started = Date.now();
630 const stamp = stamper();
629631 for (let i = 0; i < 6; i++) {
630 stream.send('{"1":3900,"2":100}');
632 stream.send(stamp({ 1: 3900, 2: 100 }));
631633 await sleep(30);
632634 }
633635 for (let i = 0; i < 50 && frames.length < 3; i++) await sleep(10);
Software/so101/src/machine-feed.test.ts +1 −1
@@ -94,7 +94,7 @@ test("a clamp shows up in the log the moment it is applied, with the command tha
9494 const p = await panel(port);
9595 const follow = new WebSocket(`ws://127.0.0.1:${port}/follow`);
9696 await new Promise((r, j) => (follow.once("open", r), follow.once("error", j)));
97 follow.send(JSON.stringify({ 1: 9999 }));
97 follow.send(JSON.stringify({ 1: 9999, seq: 1, t: Date.now() })); // the relay takes stamped frames only
9898
9999 const sent = await p.until(() => p.log.find((e) => e.kind === "sent" && e.what === "follow frame"), "frame passed on");
100100 expect(sent.detail.values).toEqual({ 1: 9999 });
Software/so101/src/machine-feed.ts +1 −1
@@ -56,7 +56,7 @@ interface Joint {
5656 /** What makes one status different from the last, for "received": where the joints are, their torque, who drives, the stop. */
5757 function signature(status: Status): string {
5858 const joints = Array.isArray(status.joints) ? (status.joints as Joint[]) : [];
59 return JSON.stringify([joints.map((j) => [j.id, j.position, j.torque]), status.driving, status.estop ? 1 : 0, status.held]);
59 return JSON.stringify([joints.map((j) => [j.id, j.position, j.torque]), status.driving, status.estop ? 1 : 0, status.held, status.deadman]);
6060 }
6161
6262 function positions(status: Status): Record<string, number> {
Software/so101/src/metrics.ts +2 −2
@@ -7,7 +7,7 @@
77 import { appendFileSync, mkdirSync, readFileSync } from "node:fs";
88 import { dirname } from "node:path";
99
10 export type EventKind = "jam" | "block" | "stall" | "missing" | "stream" | "move" | "play" | "record" | "hot" | "estop" | "estop-clear" | "park";
10 export type EventKind = "jam" | "block" | "stall" | "missing" | "stream" | "move" | "play" | "record" | "hot" | "estop" | "estop-clear" | "park" | "hold" | "resume";
1111
1212 export interface MetricEvent {
1313 readonly t: string;
@@ -27,7 +27,7 @@ const HOT = 55;
2727 const COOL = 50;
2828
2929 export class Metrics {
30 readonly counters: Record<EventKind, number> = { jam: 0, block: 0, stall: 0, missing: 0, stream: 0, move: 0, play: 0, record: 0, hot: 0, estop: 0, "estop-clear": 0, park: 0 };
30 readonly counters: Record<EventKind, number> = { jam: 0, block: 0, stall: 0, missing: 0, stream: 0, move: 0, play: 0, record: 0, hot: 0, estop: 0, "estop-clear": 0, park: 0, hold: 0, resume: 0 };
3131 readonly peaks = new Map<number, JointPeak>();
3232 readonly recent: MetricEvent[] = [];
3333 readonly since = new Date().toISOString();
Software/so101/src/relay.test.ts +8 −6
@@ -8,6 +8,7 @@ import { gunzipSync } from "node:zlib";
88 import WebSocket, { WebSocketServer } from "ws";
99 import { FeetechBus } from "./bus.ts";
1010 import { CameraHub } from "./cameras.ts";
11 import { stamper } from "./command-gate.ts";
1112 import { FakeBus } from "./fake-bus.ts";
1213 import { OFFLINE_FRAME } from "./offline-frame.ts";
1314 import { parseRemote, sameOrigin, startRelay, type Remote, type RunningRelay } from "./relay.ts";
@@ -352,22 +353,23 @@ describe("relay", () => {
352353 relay = await startRelay({ port: 0, simulation: simBase, remote: remoteAt(await listening(camera)), link: FAST });
353354 const ws = new WebSocket(`ws://127.0.0.1:${relay.port}/follow`);
354355 await new Promise((r, j) => (ws.once("open", r), ws.once("error", j)));
355 ws.send('{"1": 2050}');
356 const stamp = stamper();
357 ws.send(stamp({ 1: 2050 }));
356358 await sleep(30);
357 expect(got).toEqual(['{"1": 2050}']);
359 expect(got).toEqual(['{"1":2050}']); // the joints alone: the stamps stay here
358360 watching = false;
359361 await sleep(100);
360362 expect(relay.link().mode).toBe("reconnecting");
361 ws.send('{"1": 2060}');
363 ws.send(stamp({ 1: 2060 }));
362364 await sleep(30);
363 expect(got).toEqual(['{"1": 2050}']); // the arm server's dead-man sees silence, and holds
365 expect(got).toEqual(['{"1":2050}']); // the arm server's dead-man sees silence, and holds
364366 expect(ws.readyState).toBe(WebSocket.OPEN);
365367 watching = true;
366368 await sleep(80);
367369 expect(relay.link().mode).toBe("live");
368 ws.send('{"1": 2070}');
370 ws.send(stamp({ 1: 2070 }));
369371 await sleep(30);
370 expect(got).toEqual(['{"1": 2050}', '{"1": 2070}']);
372 expect(got).toEqual(['{"1":2050}', '{"1":2070}']);
371373 ws.close();
372374 } finally {
373375 clearInterval(pump);
Software/so101/src/relay.ts +52 −13
@@ -14,7 +14,14 @@
1414 * With a lobby (src/lobby.ts), anyone may watch but only the person whose turn it is may command the arm: every
1515 * request that is not a read, and the /follow stream, is refused for anyone else before it reaches an arm. When a
1616 * turn ends, that person's streams are closed and whatever they started is stopped. The driver and any admin may hit
17 * the emergency stop (/estop) and /stop; only an admin may clear the stop or change the settings.
17 * the emergency stop (/estop), /stop and /resume; only an admin may clear the emergency stop or change the settings.
18 *
19 * STOP is held here as well as on the arm server (src/command-gate.ts), so an arm server too old to hold, or one the
20 * stop did not reach, still gets nothing: /stop shuts the gate before it is passed on and ends every /follow stream;
21 * while held, a stream is refused (HELD_CLOSE) and no frame goes through; /resume opens it once the arm server agrees,
22 * and only a frame sent after that moves the arm. Every frame from the page carries "seq" and "t": one without them,
23 * out of order, older than 200 ms, or sent before the resume is dropped here, never kept. What goes on to the arm
24 * server is the joints alone. A turn that ends stops what drives without holding ("hold": false).
1825 *
1926 * A /follow stream whose browser goes away without closing it (or stops answering pings) is closed toward the arm
2027 * with LOST_CLOSE: the arm server parks the arm then, where a stream closed on purpose only holds.
@@ -62,6 +69,7 @@ import { proxyFor, whyNot } from "./egress.ts";
6269 import { CalibrationError } from "./leader-map.js";
6370 import type { Lobby } from "./lobby.ts";
6471 import { capBody, FrameCap } from "./speed-cap.ts";
72 import { Clock, CommandGate, HELD_CLOSE } from "./command-gate.ts";
6573
6674 export interface Remote {
6775 /** The arm server's HTTP base, no trailing slash. */
@@ -135,7 +143,7 @@ const LEADER = "/leader/calibration";
135143 /** Admins only: clearing an emergency stop and changing the settings (the safety limits among them). */
136144 const ADMIN_ONLY = new Set(["POST /estop/clear", "PUT /settings"]);
137145 /** The driver's, and any admin's at any time. */
138 const ADMIN_TOO = new Set(["POST /stop", "POST /estop"]);
146 const ADMIN_TOO = new Set(["POST /stop", "POST /resume", "POST /estop"]);
139147 /** What still goes to the arm server while the link is reconnecting: stopping it. */
140148 const STOPS = new Set(["POST /stop", "POST /estop"]);
141149 const RECONNECTING = "reconnecting to the arm";
@@ -344,6 +352,9 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
344352 }
345353 };
346354
355 /** STOP's hold, for every stream through here, and each driver's clock (their stamps, seen from here). */
356 const gate = new CommandGate();
357 const clocks = new Map<string, Clock>();
347358 /**
348359 * The arm's /status, saying which arm it is: from the link while it is live, the last one it had while reconnecting,
349360 * and from the simulation otherwise.
@@ -353,11 +364,11 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
353364 const last = armLink!.last;
354365 const reconnecting = mode() === "reconnecting";
355366 if (!last) return { code: 503, body: { error: `${RECONNECTING}…`, source: source(), reconnecting } };
356 return { code: 200, body: { ...last, source: source(), reconnecting, ...(reconnecting ? { seenAt: new Date(armLink!.lastAt).toISOString() } : {}) } };
367 return { code: 200, body: { ...last, held: gate.held ?? last.held ?? null, source: source(), reconnecting, ...(reconnecting ? { seenAt: new Date(armLink!.lastAt).toISOString() } : {}) } };
357368 }
358369 const answer = await fetch(simulation + path, { cache: "no-store", signal: AbortSignal.timeout(readTimeoutMs) });
359370 const body = (await answer.json()) as Record<string, unknown>;
360 return { code: answer.status, body: { ...body, source: source() } };
371 return { code: answer.status, body: { ...body, held: gate.held ?? body.held ?? null, source: source() } };
361372 };
362373 const status = async (res: ServerResponse, path: string) => {
363374 const { code, body } = await statusOf(path);
@@ -563,6 +574,20 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
563574 }
564575 };
565576
577 /** Resume once the arm server has: one that is still held would refuse the stream anyway. One too old for /resume (404) has no hold of its own. */
578 const resume = async (res: ServerResponse, by: string) => {
579 const real = Boolean(remote && onArm());
580 if (real && mode() === "reconnecting") return json(res, 503, { error: `${RECONNECTING} at ${remote!.host}…; resume when it answers`, reconnecting: true });
581 try {
582 const r = await fetch(`${real ? remote!.http : simulation}/resume`, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ by }), signal: AbortSignal.timeout(readTimeoutMs), cache: "no-store" });
583 if (!r.ok && r.status !== 404) return json(res, r.status, { error: await whyNot(r, "/resume") });
584 await r.body?.cancel();
585 } catch (err) {
586 return json(res, 502, { error: `the arm server did not answer (${reason(err)})` });
587 }
588 return json(res, 200, { resumed: gate.resume(), held: null });
589 };
590
566591 const http = createServer(async (req, res) => {
567592 for (const [name, value] of Object.entries(FLOOR)) res.setHeader(name, value);
568593 try {
@@ -620,11 +645,17 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
620645 if (req.method === "GET" && PAGE.test(url.pathname)) return await page(req, res, await ask(simulation + path, req, undefined, null, false));
621646 if (req.method === "GET" && STATIC.test(url.pathname) && acceptsGzip(req.headers)) return await packed(res, await ask(simulation + path, req, undefined, null, false), path);
622647 // Who stopped or cleared the arm is the relay's to say, never the page's; every other command is held to the admin's speed cap.
623 const raw = reading || route === "POST /estop" || route === "POST /estop/clear" ? undefined : await readBody(req);
648 const raw = reading || route === "POST /estop" || route === "POST /estop/clear" || route === "POST /stop" || route === "POST /resume" ? undefined : await readBody(req);
624649 const body = reading ? undefined : !raw ? Buffer.from(JSON.stringify({ by })) : lobby ? capBody(url.pathname, raw, lobby.maxSpeedPercent) : raw;
625650 const logged = !reading && !LOCAL.test(url.pathname);
626651 if (logged) machine.note("sent", route, { by });
627652 const answered = (status: number) => logged && machine.note("received", route, { status });
653 if (route === "POST /stop") {
654 // Shut before anything is asked of an arm: a frame already on its way finds the gate closed.
655 gate.hold(by);
656 for (const browser of streams.keys()) browser.close(HELD_CLOSE, `stopped by ${by}`);
657 }
658 if (route === "POST /resume") return await resume(res, by);
628659 if (LOCAL.test(url.pathname) || !onArm() || !remote) {
629660 const answer = await ask(simulation + path, req, body, null, false);
630661 answered(answer.response.status);
@@ -680,10 +711,14 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
680711 */
681712 const bridge = (browser: WebSocket, arm: WebSocket, base: string, driver: string | null) => {
682713 const real = base !== simulation;
683 if (driver !== null) {
684 streams.set(browser, driver);
685 browser.on("close", () => streams.delete(browser));
686 }
714 streams.set(browser, driver ?? "");
715 const clock = clocks.get(driver ?? "") ?? new Clock();
716 clocks.set(driver ?? "", clock);
717 const admission = gate.open(clock, true);
718 browser.on("close", () => {
719 streams.delete(browser);
720 admission.close();
721 });
687722 arm.on("message", (data, binary) => browser.readyState === WebSocket.OPEN && browser.send(data, { binary }));
688723 const cap = new FrameCap();
689724 let seeded = false;
@@ -707,20 +742,23 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
707742 seed();
708743 /** A frame as the panel shows it: servo id → ticks. */
709744 const values = (text: string) => perServo(parseFrame(text) ?? []);
710 browser.on("message", (data, binary) => {
745 browser.on("message", (raw) => {
711746 if (driver !== null) {
712747 // The turn may have ended while this frame was on its way, before the close reaches the page: it is not passed on.
713748 if (lobby && !lobby.turns.isDriver(driver) && !lobby.local) return void machine.note("clamp", "not passed on", { why: "the turn ended" });
714749 lobby?.activity(driver);
715750 }
751 const verdict = admission.admit(String(raw));
752 if (!verdict.ok) return void machine.note("clamp", "not passed on", { why: verdict.drop });
716753 if (arm.readyState !== WebSocket.OPEN) return;
717754 if (real && mode() !== "live") return void machine.note("clamp", "not passed on", { why: RECONNECTING });
755 const data = JSON.stringify(Object.fromEntries(verdict.joints));
718756 const percent = lobby?.maxSpeedPercent ?? 100;
719757 const text = String(data);
720758 if (percent >= 100) {
721759 if (lobby) cap.cap(text, percent); // keeps each joint's last target, so a cap set mid-stream starts from there
722760 machine.note("sent", "follow frame", { values: values(text) });
723 arm.send(data, { binary });
761 arm.send(data);
724762 return;
725763 }
726764 if (!seeded) seed();
@@ -732,7 +770,7 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
732770 });
733771 arm.on("close", (code, why) => browser.close(sendable(code) ? code : 1011, why.length ? why.toString() : "the arm server went away"));
734772 // A browser that closed the stream on purpose leaves the arm holding; one that vanished leaves it parking.
735 browser.on("close", (code) => (code === 1000 || code === 1005 ? arm.close(1000) : arm.close(LOST_CLOSE, "the driver's connection was lost")));
773 browser.on("close", (code) => (code === 1000 || code === 1005 || code === HELD_CLOSE ? arm.close(1000) : arm.close(LOST_CLOSE, "the driver's connection was lost")));
736774 let answered = true;
737775 browser.on("pong", () => (answered = true));
738776 const heartbeat = setInterval(() => {
@@ -762,6 +800,7 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
762800 if (refused) return wss.handleUpgrade(req, socket, head, (browser) => browser.close(4000 + refused.status, refused.error));
763801 driver = who!.id;
764802 }
803 if (gate.held) return wss.handleUpgrade(req, socket, head, (browser) => browser.close(HELD_CLOSE, `stopped by ${gate.held!.by}: resume to drive`));
765804 void (async () => {
766805 let arm: WebSocket;
767806 let base: string;
@@ -793,7 +832,7 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
793832 const turnEnded = lobby?.onTurnEnd((who) => {
794833 for (const [browser, id] of streams) if (id === who.id) browser.close(1000, "your turn ended");
795834 const base = remote && onArm() ? remote.http : simulation;
796 fetch(`${base}/stop`, { method: "POST", signal: AbortSignal.timeout(readTimeoutMs) }).then(
835 fetch(`${base}/stop`, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ by: who.name, hold: false }), signal: AbortSignal.timeout(readTimeoutMs) }).then(
797836 (r) => r.body?.cancel(),
798837 (err: unknown) => console.error(`stop after ${who.name}'s turn: ${reason(err)}`),
799838 );
Software/so101/src/server.test.ts +17 −7
@@ -192,11 +192,13 @@ describe("viewer page", () => {
192192 expect(page).toMatch(/\.stick \{[^}]*touch-action: none/);
193193 expect(page).toMatch(/#grips button \{[^}]*min-height: (7[2-9]|[89]\d)px/);
194194 expect(page).toMatch(/#chips button small \{[^}]*white-space: nowrap/); // a long label never wraps and shifts the row
195 // One "Stop": the E-stop. The other button starts hidden, comes up only while a move, a playback, a recording or a park
196 // runs (which the phone had no other way to end), and is named for what it ends.
197 expect(page).toContain('<button id="stop" class="danger" hidden');
198 expect(page).toContain('const ENDS_LABEL = { move: "End move", play: "End playback", record: "End recording", park: "End park" };');
199 expect(page).toContain('$("stop").hidden = !label || !(mayDrive() || isOwner());');
195 // One STOP in every layout: one stop control in the markup, and no layout hides it. Resume is its own control.
196 expect(page.match(/<button[^>]*data-stop/g)?.length).toBe(1);
197 expect(page).not.toMatch(/#stop \{[^}]*display: none/);
198 expect(page.match(/api\("\/stop"/g)?.length).toBe(1);
199 expect(page).not.toContain('api("/estop", ');
200 expect(page).toContain('api("/resume", {})');
201 expect(page).toContain("Resume driving");
200202 expect(page.match(/>Stop</g)?.length).toBe(1);
201203 });
202204 });
@@ -239,8 +241,13 @@ describe("moves over HTTP", () => {
239241 expect((await post("/grip", { position: 2000, speed: "fast" })).status).toBe(400);
240242 });
241243
242 test("/stop with nothing running is harmless", async () => {
243 expect(await (await post("/stop", {})).json()).toEqual({ stopped: null });
244 test("/stop with nothing running is harmless, and holds until /resume", async () => {
245 expect(await (await post("/stop", { by: "ann" })).json()).toEqual({ stopped: null });
246 const held = (await (await fetch(`${base}/status`)).json()) as { held: { by: string } | null };
247 expect(held.held?.by).toBe("ann");
248 expect((await post("/move", { targets: { "1": 2100 } })).status).toBe(423);
249 expect(await (await post("/resume", {})).json()).toEqual({ resumed: true, held: null });
250 expect(((await (await fetch(`${base}/status`)).json()) as { held: unknown }).held).toBeNull();
244251 });
245252 });
246253
@@ -311,6 +318,7 @@ describe("recordings over HTTP", () => {
311318 const stopped = (await (await post("/stop", {})).json()) as { stopped: string; saved: string };
312319 expect(stopped.stopped).toBe("record");
313320 expect(stopped.saved).toBe("short");
321 await post("/resume", {});
314322 });
315323 });
316324
@@ -339,6 +347,7 @@ describe("/stop on a stream", () => {
339347 await sleep(30);
340348 status = (await (await fetch(`${base}/status`)).json()) as { driving: string | null };
341349 expect(status.driving).toBeNull();
350 await post("/resume", {});
342351 });
343352 });
344353
@@ -397,6 +406,7 @@ describe("a bus that fails mid-session", () => {
397406 expect([a.status, b.status].sort()).toEqual([200, 409]);
398407 const list = (await (await fetch(`${url}/recordings`)).json()) as Record<string, unknown>;
399408 expect(Object.keys(list)).toEqual(["race"]);
409 await call("/resume", {});
400410 });
401411
402412 test("/grip owns the arm while it writes, and the next supervised gripper move puts the gripper's force limit back", async () => {
Software/so101/src/server.ts +77 −16
@@ -12,7 +12,10 @@
1212 * POST /move {"targets": {"3": 2048}, "speed"?, "torque"?, "release"?} supervised, any joints
1313 * POST /grip {"position": 2000, "torque": 30, "speed"?: 300} close the gripper toward a position at a low torque cap and hold
1414 * POST /torque {"on": true|false}
15 * POST /stop end whatever drives the arm: a move, a playback, a recording, or a stream; the arm holds
15 * POST /stop {"by"?: "name", "hold"?: true} end whatever drives the arm: a move, a playback, a recording,
16 * or a stream; the arm holds where it is, and stays held (no stream, no move) until /resume.
17 * "hold": false only ends what drives (a turn that ends)
18 * POST /resume {"by"?: "name"} end the hold; nothing moves until a command sent after it arrives
1619 * POST /estop {"by"?: "name"} emergency stop: halt everything, hold with torque, refuse every move until cleared
1720 * POST /estop/clear {"by"?: "name"}
1821 * GET /recordings the saved recordings with their length
@@ -31,7 +34,8 @@
3134 * GET /model/<file> the URDF and its meshes; GET /vendor/<pkg>/<file> three.js and the URDF loader
3235 * GET /app/<file> the modules the page imports itself (the leader arm over Web Serial), as JavaScript
3336 * GET /printables the sheets to print for the vision quest (board, cube markers, gripper marker); GET /printables/<file>
34 * WS /follow client sends {"1": ticks, "2": ticks, ...} frames; the server drives the arm
37 * WS /follow client sends {"1": ticks, "2": ticks, ...} frames; the server drives the arm. A frame may carry
38 * "seq" and "t" (src/command-gate.ts): then a stale, replayed or pre-resume one is dropped
3539 * WS /watch read-only: the server sends the /status body every watchEveryMs, one bus read shared by every watcher;
3640 * the hosted app's relay keeps one open, so viewers see the arm without each of them asking the Pi.
3741 * Between them, {"machine": [entries], "run": "<id>"}: every register written on the bus, every
@@ -53,6 +57,7 @@ import { createHash, randomUUID } from "node:crypto";
5357 import { fileURLToPath } from "node:url";
5458 import { WebSocketServer, type WebSocket } from "ws";
5559 import { browserModule } from "./browser-modules.ts";
60 import { Clock, CommandGate, HELD_CLOSE } from "./command-gate.ts";
5661 import type { FeetechBus } from "./bus.ts";
5762 import { ArmModel, gripperTicksToRadians, ticksToRadians, type Violation } from "./kinematics.ts";
5863 import { clamp, type Frame } from "./mapping.ts";
@@ -62,7 +67,7 @@ import { MotionError, moveToPose, playTrajectory, recordTrajectory, STALL_SPAN_T
6267 import { assertPoseName, parsePoseFile, serializePoseFile, type PoseFile } from "./poses.ts";
6368 import { JOINTS, registerName, STS3215, ticksToDegrees } from "./sts3215.ts";
6469 import { largestStep, parseTrajectory, serializeTrajectory, type Trajectory } from "./trajectory.ts";
65 import { accelRegister, bound, DEFAULT_SAFETY, Governor, jointBounds, motionLimits, parseFrame, SAFETY_KEYS, SafetyError, TICKS_PER_DEGREE, validateSafety, type SafetySettings } from "./safety.ts";
70 import { accelRegister, bound, DEFAULT_SAFETY, Governor, jointBounds, motionLimits, SAFETY_KEYS, SafetyError, TICKS_PER_DEGREE, validateSafety, type SafetySettings } from "./safety.ts";
6671
6772 export interface ServerOptions {
6873 host?: string;
@@ -262,6 +267,7 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
262267 let current: AbortController | null = null;
263268 const claim = (kind: Driver): AbortController => {
264269 if (estop) throw new HttpError(423, stopped(estop));
270 if (gate.held) throw new HttpError(423, holding(gate.held));
265271 if (driver) throw new HttpError(409, `${describe(driver)} is driving the arm`);
266272 driver = kind;
267273 current = new AbortController();
@@ -295,6 +301,9 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
295301 } catch (err) {
296302 if ((err as NodeJS.ErrnoException).code !== "ENOENT") estop = { by: "an unreadable record", at: new Date().toISOString() };
297303 }
304 /** STOP's hold: every frame is dropped and every move refused until /resume (src/command-gate.ts). */
305 const gate = new CommandGate();
306 const holding = (h: { by: string; at: string }) => `stopped (${h.by}, ${h.at.slice(0, 19).replace("T", " ")} UTC): the arm holds until someone resumes`;
298307 const stopped = (e: EStop) => `emergency stop (${e.by}, ${e.at.slice(0, 19).replace("T", " ")} UTC): the arm holds until an admin clears it`;
299308 /** Torque on, goal where each joint is: the arm stays put and resists. */
300309 const holdHere = async () => {
@@ -312,7 +321,7 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
312321 machine.note("state", "emergency stop", { by });
313322 }
314323 const was = driver;
315 if (was === "stream") for (const client of wss.clients) client.close(ESTOP_CLOSE, "emergency stop"); // its close handler holds, never parks
324 if (was === "stream") for (const client of wss.clients) end(client, ESTOP_CLOSE, "emergency stop"); // its close handler holds, never parks
316325 else if (was === "record" && recording) {
317326 const { done } = recording;
318327 recording = null; // nothing is saved: the recording ends where it was cut
@@ -340,7 +349,7 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
340349 * of every arm joint's range), under supervision, unless the arm is stopped, busy, or the way there is in the zone.
341350 */
342351 const park = async (why: string) => {
343 if (estop || driver) return;
352 if (estop || gate.held || driver) return;
344353 const controller = claim("park");
345354 try {
346355 const pose = loadPoses()[settings.parkPose];
@@ -560,8 +569,10 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
560569 blocked,
561570 jammed: [...jams.keys()],
562571 stream: streamStats,
563 held,
572 /** The stream's dead-man hold (no frame lately); `held` is the STOP's, which only /resume ends. */
573 deadman: held,
564574 estop,
575 held: gate.held,
565576 machine: { run, seq: machine.seq },
566577 };
567578 };
@@ -812,6 +823,7 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
812823 const { on } = await readBody(req);
813824 if (typeof on !== "boolean") throw new HttpError(400, "on must be true or false");
814825 if (estop) throw new HttpError(423, stopped(estop));
826 if (gate.held && !on) throw new HttpError(423, holding(gate.held));
815827 if (driver) throw new HttpError(409, `${describe(driver)} is driving the arm`);
816828 if (on) {
817829 const positions = await bus.syncRead(ids, STS3215.Present_Position);
@@ -821,14 +833,23 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
821833 return json(res, 200, { torque: on });
822834 }
823835 case "POST /stop": {
836 const body = await readBody(req);
837 // Held first, before anything awaits: a frame already on its way finds the gate shut.
838 if (body.hold !== false && gate.hold(actor(body))) metrics.event("hold", { by: actor(body) });
824839 const was = driver;
825 if (was === "stream") for (const client of wss.clients) client.close(1000, "stopped from the viewer"); // its close handler holds the arm
840 if (was === "stream") for (const client of wss.clients) end(client, 1000, "stopped from the viewer"); // its close handler holds the arm
826841 else if (was === "record") {
827842 const summary = await finishRecording();
828843 return json(res, 200, { stopped: was, ...summary });
829844 } else current?.abort(); // the supervisor stops the move where it is
830845 return json(res, 200, { stopped: was });
831846 }
847 case "POST /resume": {
848 const by = actor(await readBody(req));
849 const resumed = gate.resume();
850 if (resumed) metrics.event("resume", { by });
851 return json(res, 200, { resumed, held: null });
852 }
832853 case "POST /estop": {
833854 const by = actor(await readBody(req));
834855 const was = driver;
@@ -936,18 +957,41 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
936957 // and handed to the governor, which moves the goals toward it within the speed and acceleration limits.
937958 // Silence makes the arm hold (the dead-man); a connection that is lost rather than closed makes it park.
938959 const wss = new WebSocketServer({ noServer: true, maxPayload: 4096 });
939 wss.on("connection", (ws: WebSocket) => {
960 /** Each stream's halt: what it has queued or under way stops at once, not when the close handshake comes back. */
961 const halts = new Map<WebSocket, () => void>();
962 const end = (ws: WebSocket, code: number, why: string) => {
963 halts.get(ws)?.();
964 ws.close(code, why);
965 };
966 /** Each sender's clock, by address: one device keeps one clock, so a burst it flushes on a new stream is still known as old. */
967 const clocks = new Map<string, Clock>();
968 const clock = (req: IncomingMessage | undefined) => {
969 const key = req?.socket.remoteAddress ?? "";
970 if (!clocks.has(key) && clocks.size >= 64) clocks.delete(clocks.keys().next().value!);
971 const c = clocks.get(key) ?? new Clock();
972 clocks.set(key, c);
973 return c;
974 };
975 wss.on("connection", (ws: WebSocket, req?: IncomingMessage) => {
940976 if (estop) {
941977 ws.close(ESTOP_CLOSE, stopped(estop));
942978 return;
943979 }
980 if (gate.held) {
981 ws.close(HELD_CLOSE, holding(gate.held));
982 return;
983 }
944984 if (driver) {
945985 ws.close(1013, `${describe(driver)} is driving the arm`);
946986 return;
947987 }
948988 claim("stream");
949989 const startedAt = Date.now();
950 const stats = { frames: 0, dropped: 0, refused: 0, merged: 0, clamped: 0 };
990 const stats = { frames: 0, dropped: 0, refused: 0, merged: 0, clamped: 0, gated: 0 };
991 /** Seq, age and the hold, frame by frame; stamps are optional here (the CLI leader sends none). */
992 const admission = gate.open(clock(req), false);
993 /** Set the moment the stream ends: whatever is queued or under way gives up, and nothing more reaches the bus. */
994 let closed = false;
951995 streamStats = stats;
952996 const holdAfterMs = options.watchdogMs ?? settings.holdAfterMs;
953997 const period = 1000 / settings.rateHz;
@@ -967,6 +1011,7 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
9671011
9681012 const hold = () =>
9691013 run(async () => {
1014 if (closed) return;
9701015 const positions = await bus.syncRead(ids, STS3215.Present_Position);
9711016 await bus.syncWrite(STS3215.Goal_Position, ids.filter((id) => positions.has(id)).map((id) => [id, positions.get(id)!] as const));
9721017 governor.reset(positions);
@@ -976,6 +1021,7 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
9761021 });
9771022 const arm = () =>
9781023 run(async () => {
1024 if (closed) return;
9791025 const positions = await bus.syncRead(ids, STS3215.Present_Position);
9801026 present = new Map(positions);
9811027 const ready = ids.filter((id) => positions.has(id));
@@ -1013,7 +1059,7 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
10131059 const watch = setInterval(
10141060 () =>
10151061 void run(async () => {
1016 if (!armed || last === null) return;
1062 if (closed || !armed || last === null) return;
10171063 const positions = await bus.syncRead(ids, STS3215.Present_Position);
10181064 const loads = await bus.syncRead(ids, STS3215.Present_Load);
10191065 for (const id of ids) {
@@ -1035,6 +1081,7 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
10351081 ticking = true;
10361082 void run(async () => {
10371083 try {
1084 if (closed) return;
10381085 const step = governor.step(dt);
10391086 if (step.length) await bus.syncWrite(STS3215.Goal_Position, step);
10401087 } finally {
@@ -1054,6 +1101,7 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
10541101
10551102 const follow = (frame: ReadonlyMap<number, number>) =>
10561103 run(async () => {
1104 if (closed) return; // queued before a stop: dropped, never replayed
10571105 const b = bounds();
10581106 const targets: Array<readonly [number, number]> = [];
10591107 let ease = last === null; // after a hold, everything eases
@@ -1115,7 +1163,7 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
11151163 await bus.syncWrite(STS3215.Goal_Position, targets);
11161164 const deadline = Date.now() + 4000;
11171165 const wanted = new Map(targets);
1118 while (Date.now() < deadline) {
1166 while (Date.now() < deadline && !closed) {
11191167 const positions = await bus.syncRead([...wanted.keys()], STS3215.Present_Position);
11201168 const loads = await bus.syncRead([...wanted.keys()], STS3215.Present_Load);
11211169 let worst = 0;
@@ -1128,7 +1176,7 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
11281176 if (worst <= 25) break;
11291177 await new Promise((r) => setTimeout(r, 50));
11301178 }
1131 await bus.syncWrite(STS3215.Goal_Velocity, targets.map(([id]) => [id, speedLimits().speed] as const));
1179 if (!closed) await bus.syncWrite(STS3215.Goal_Velocity, targets.map(([id]) => [id, speedLimits().speed] as const));
11321180 } finally {
11331181 easing = false;
11341182 }
@@ -1149,7 +1197,7 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
11491197 let lastTaken = 0;
11501198 const take = () => {
11511199 pacer = undefined;
1152 if (!pending) return;
1200 if (!pending || closed) return;
11531201 // While the arm catches up, the frame waits (later ones merge into it) and is taken once it has: never dropped,
11541202 // or a leader that sends one last frame and stops would leave the arm short of it.
11551203 if (easing) return void (pacer = setTimeout(take, period));
@@ -1159,8 +1207,13 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
11591207 follow(frame);
11601208 };
11611209 ws.on("message", (data) => {
1162 const frame = parseFrame(String(data));
1163 if (!frame) return;
1210 if (closed) return;
1211 const verdict = admission.admit(String(data));
1212 if (!verdict.ok) {
1213 if (verdict.drop !== "malformed") stats.gated++;
1214 return;
1215 }
1216 const frame = verdict.joints;
11641217 clearTimeout(timer);
11651218 timer = setTimeout(() => void hold(), holdAfterMs);
11661219 if (!arming) {
@@ -1171,11 +1224,19 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
11711224 pending = new Map([...(pending ?? []), ...frame]);
11721225 pacer ??= setTimeout(take, Math.max(0, lastTaken + period - Date.now()));
11731226 });
1174 ws.on("close", (code) => {
1227 const halt = () => {
1228 closed = true;
1229 pending = null;
11751230 clearTimeout(timer);
11761231 clearTimeout(pacer);
11771232 clearInterval(watch);
11781233 clearInterval(ticker);
1234 };
1235 halts.set(ws, halt);
1236 ws.on("close", (code) => {
1237 halt();
1238 halts.delete(ws);
1239 admission.close();
11791240 clearInterval(heartbeat);
11801241 const lost = !CLEAN_CLOSE.has(code) && code !== ESTOP_CLOSE;
11811242 void run(async () => {
Software/so101/src/stop.test.ts added +372
@@ -0,0 +1,372 @@
1 /**
2 * STOP never makes the arm move (issue #66). A fake Pi stands where `so101 serve` would, behind the real relay and
3 * lobby, and records every target that reaches it. It is the worst Pi there is: it holds nothing itself and keeps its
4 * streams open through a stop, so whatever the relay lets through, it records.
5 *
6 * With STOP_LOG=1 the recorded streams are printed: the page's commands and what reached the Pi, on one clock.
7 */
8 import { afterAll, afterEach, beforeAll, describe, expect, test } from "bun:test";
9 import { mkdtempSync } from "node:fs";
10 import { createServer, type Server } from "node:http";
11 import { tmpdir } from "node:os";
12 import { join } from "node:path";
13 import WebSocket, { WebSocketServer } from "ws";
14 import { FeetechBus } from "./bus.ts";
15 import { HELD_CLOSE } from "./command-gate.ts";
16 import { FakeBus } from "./fake-bus.ts";
17 import { sign } from "./identity.ts";
18 import { Lobby } from "./lobby.ts";
19 import { parseRemote, startRelay, type RunningRelay } from "./relay.ts";
20 import { startServer, type RunningServer } from "./server.ts";
21
22 const KEY = "test-identity-key";
23 const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
24 const shut = (s: { close(): Promise<void> } | undefined) => (s ? Promise.race([s.close(), sleep(500)]) : undefined);
25 const person = (id: string) => {
26 const identity = `u-${id}|${id}|user|${Date.now() + 3_600_000}`;
27 return { "x-forge-identity": identity, "x-forge-signature": sign(identity, KEY) };
28 };
29 const FAST = { backoffMs: [5, 10, 20, 40], staleMs: 200, offlineAfterMs: 5000, connectTimeoutMs: 200, pollEveryMs: 10 };
30 const LOG = process.env.STOP_LOG === "1";
31
32 /** One line of the recorded stream: what the page did, and what reached the Pi. */
33 interface Entry {
34 at: number;
35 who: "page" | "pi";
36 what: string;
37 n?: number;
38 }
39
40 /** The fake Pi: a status on /watch, and a /follow that records every frame it gets, and /stop and /resume. */
41 class FakePi {
42 readonly got: Array<{ at: number; n: number }> = [];
43 readonly http: Server;
44 port = 0;
45 #pump: ReturnType<typeof setInterval> | undefined;
46 private readonly timeline: Entry[];
47 constructor(timeline: Entry[]) {
48 this.timeline = timeline;
49 const follow = new WebSocketServer({ noServer: true });
50 const watch = new WebSocketServer({ noServer: true });
51 this.http = createServer((req, res) => {
52 req.resume();
53 req.on("end", () => {
54 if (req.method === "POST") this.timeline.push({ at: Date.now(), who: "pi", what: `POST ${req.url}` });
55 res.writeHead(200, { "content-type": "application/json" });
56 res.end(req.url === "/status" ? JSON.stringify(this.status) : "{}");
57 });
58 });
59 this.http.on("upgrade", (req, socket, head) => {
60 const to = req.url === "/follow" ? follow : req.url === "/watch" ? watch : null;
61 if (!to) return socket.destroy();
62 to.handleUpgrade(req, socket, head, (ws) => {
63 if (to !== follow) return;
64 ws.on("message", (d) => {
65 const n = Number(JSON.parse(String(d))["1"]) - 1000;
66 this.got.push({ at: Date.now(), n });
67 this.timeline.push({ at: Date.now(), who: "pi", what: `target #${n}`, n });
68 });
69 });
70 });
71 this.#pump = setInterval(() => watch.clients.forEach((c) => c.send(JSON.stringify(this.status))), 10);
72 }
73 get status() {
74 return { joints: [{ id: 1, position: 2048 }, { id: 2, position: 2048 }], driving: null, estop: null, held: null };
75 }
76 async listen() {
77 await new Promise<void>((r) => this.http.listen(0, "127.0.0.1", r));
78 this.port = (this.http.address() as { port: number }).port;
79 return this.port;
80 }
81 close() {
82 clearInterval(this.#pump);
83 this.http.closeAllConnections();
84 this.http.close();
85 }
86 }
87
88 /** A page: its own stamps (seq for the life of the page, t when the hand made the frame), and one /follow socket at a time. */
89 class Page {
90 ws: WebSocket | null = null;
91 seq = 0;
92 closedWith: string | null = null;
93 readonly name: string;
94 readonly headers: Record<string, string>;
95 private readonly port: () => number;
96 private readonly timeline: Entry[];
97 constructor(name: string, headers: Record<string, string>, port: () => number, timeline: Entry[]) {
98 this.name = name;
99 this.headers = headers;
100 this.port = port;
101 this.timeline = timeline;
102 }
103 /** A frame made now, numbered n: joint 1 at 1000 + n, so what reaches the Pi says which frame it was. */
104 make(n: number, at = Date.now()) {
105 return { n, text: JSON.stringify({ 1: 1000 + n, seq: ++this.seq, t: at }), at };
106 }
107 async open(): Promise<boolean> {
108 if (this.ws?.readyState === WebSocket.OPEN) return true;
109 const ws = new WebSocket(`ws://127.0.0.1:${this.port()}/follow`, { headers: this.headers });
110 this.ws = ws;
111 ws.on("close", (code, why) => {
112 this.closedWith = `${code} ${why}`;
113 if (this.ws === ws) this.ws = null;
114 });
115 ws.on("error", () => {});
116 const opened = await Promise.race([new Promise<boolean>((r) => ws.once("open", () => r(true))), new Promise<boolean>((r) => ws.once("close", () => r(false))), sleep(500).then(() => false)]);
117 // A refused stream is accepted and closed at once: give its close a moment to come.
118 await sleep(5);
119 return opened && ws.readyState === WebSocket.OPEN;
120 }
121 /** On whatever socket is there, open or not: a page that buffers would do exactly this. */
122 send(frame: { n: number; text: string }, what = "send") {
123 this.timeline.push({ at: Date.now(), who: "page", what: `${this.name} ${what} #${frame.n}`, n: frame.n });
124 if (this.ws?.readyState === WebSocket.OPEN) this.ws.send(frame.text);
125 }
126 async post(path: string, port: number) {
127 const r = await fetch(`http://127.0.0.1:${port}${path}`, { method: "POST", headers: { "content-type": "application/json", ...this.headers }, body: "{}" });
128 this.timeline.push({ at: Date.now(), who: "page", what: `${this.name} ${path.slice(1).toUpperCase()} → ${r.status}` });
129 return r;
130 }
131 drop() {
132 this.ws?.terminate();
133 this.ws = null;
134 }
135 }
136
137 function print(title: string, timeline: Entry[]) {
138 if (!LOG || !timeline.length) return;
139 const t0 = timeline[0]!.at;
140 console.log(`\n# ${title}\n${timeline.map((e) => `+${String(e.at - t0).padStart(5)} ms ${e.who.padEnd(4)} ${e.what}`).join("\n")}`);
141 }
142
143 /** Deterministic randomness, so a failing interleaving can be run again from its seed. */
144 function mulberry32(seed: number) {
145 return () => {
146 seed |= 0;
147 seed = (seed + 0x6d2b79f5) | 0;
148 let t = Math.imul(seed ^ (seed >>> 15), 1 | seed);
149 t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t;
150 return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
151 };
152 }
153
154 describe("STOP through the relay, to a Pi that holds nothing itself", () => {
155 let simulation: RunningServer;
156 let pi: FakePi;
157 let relay: RunningRelay | undefined;
158 let lobby: Lobby | undefined;
159 let timeline: Entry[];
160 beforeAll(async () => {
161 simulation = await startServer(new FeetechBus(new FakeBus([1, 2, 6])), { host: "127.0.0.1", port: 0, ids: [1, 2, 6], posesFile: join(mkdtempSync(join(tmpdir(), "so101-stop-")), "poses.json") });
162 });
163 afterAll(() => shut(simulation));
164 afterEach(async () => {
165 await shut(relay);
166 lobby?.close();
167 pi?.close();
168 relay = lobby = undefined;
169 });
170 const start = async () => {
171 timeline = [];
172 pi = new FakePi(timeline);
173 const piPort = await pi.listen();
174 lobby = new Lobby({ identityKey: KEY });
175 relay = await startRelay({ port: 0, simulation: `http://127.0.0.1:${simulation.port}`, remote: parseRemote({ ARM_SERVER: `ws://127.0.0.1:${piPort}` })!, link: FAST, lobby });
176 for (let i = 0; i < 100 && relay.link().mode !== "live"; i++) await sleep(10);
177 expect(relay.link().mode).toBe("live");
178 return relay.port;
179 };
180
181 test("(b) 100 commands sent while held, then STOP and resume: the Pi receives none of them, then the one new command", async () => {
182 const port = await start();
183 lobby!.turns.join({ id: "u-alice", name: "alice", owner: false });
184 const page = new Page("alice", person("alice"), () => port, timeline);
185 expect(await page.open()).toBe(true);
186 page.send(page.make(1));
187 page.send(page.make(2));
188 await sleep(50);
189 expect(pi.got.map((g) => g.n)).toEqual([1, 2]);
190
191 expect((await page.post("/stop", port)).status).toBe(200);
192 const held: Array<{ n: number; text: string }> = [];
193 // 100 commands while held: some on the socket the stop is closing, some kept by a page that buffers them
194 for (let n = 100; n < 200; n++) {
195 const f = page.make(n);
196 held.push(f);
197 if (n % 2) page.send(f, "send (held)");
198 }
199 expect(await page.open()).toBe(false); // a new stream is refused while held
200 expect(page.closedWith).toStartWith(`${HELD_CLOSE} stopped by alice`);
201 expect((await page.post("/stop", port)).status).toBe(200); // pressed again: still held
202 await sleep(50);
203 expect((await page.post("/resume", port)).status).toBe(200);
204 // The buffer flushes on the new stream after the resume, in order, quickly, and every frame is dropped.
205 expect(await page.open()).toBe(true);
206 for (const f of held) page.send(f, "flush (buffered)");
207 await sleep(80);
208 expect(pi.got.filter((g) => g.n >= 100)).toEqual([]);
209 // After resume the arm stays put until a new command, and that one goes.
210 page.send(page.make(500));
211 await sleep(50);
212 expect(pi.got.map((g) => g.n)).toEqual([1, 2, 500]);
213 print("(b) replay: 100 commands while held, STOP, resume", timeline);
214 });
215
216 test("(a) any interleaving of drag, slider, STOP, resume, disconnect and turn change: nothing reaches the Pi between a STOP and a resume plus a new command", async () => {
217 const SEEDS = 8, STEPS = 40;
218 for (let seed = 1; seed <= SEEDS; seed++) {
219 const port = await start();
220 const random = mulberry32(seed);
221 const pages = [new Page("alice", person("alice"), () => port, timeline), new Page("bob", person("bob"), () => port, timeline)];
222 lobby!.turns.join({ id: "u-alice", name: "alice", owner: false });
223 lobby!.turns.join({ id: "u-bob", name: "bob", owner: false });
224 const driving = () => pages.find((p) => lobby!.turns.isDriver(`u-${p.name}`))!;
225 /** Every frame made, with when; when each STOP took effect and each resume was answered. */
226 const made = new Map<number, number>();
227 const stops: number[] = [];
228 const resumes: number[] = [];
229 let held = false;
230 const stash: Array<{ n: number; text: string }> = [];
231 let n = 0;
232 for (let step = 0; step < STEPS; step++) {
233 const page = driving();
234 const r = random();
235 if (r < 0.45) {
236 // a drag on the model, or a slider: a fresh frame on the stream, opened if need be
237 await page.open();
238 const f = page.make(++n);
239 made.set(f.n, f.at);
240 page.send(f, r < 0.25 ? "drag" : "slider");
241 } else if (r < 0.55) {
242 // made but kept back, like a socket that was still connecting
243 const f = page.make(++n);
244 made.set(f.n, f.at);
245 stash.push(f);
246 timeline.push({ at: Date.now(), who: "page", what: `${page.name} buffer #${f.n}`, n: f.n });
247 } else if (r < 0.62) {
248 await page.open();
249 for (const f of stash.splice(0)) page.send(f, "flush (buffered)");
250 } else if (r < 0.74) {
251 const sent = Date.now();
252 expect((await page.post("/stop", port)).status).toBe(200);
253 stops.push(sent);
254 held = true;
255 } else if (r < 0.86) {
256 const answer = await page.post("/resume", port);
257 if (answer.status === 200) {
258 resumes.push(Date.now());
259 held = false;
260 }
261 } else if (r < 0.93) {
262 page.drop();
263 timeline.push({ at: Date.now(), who: "page", what: `${page.name} disconnect` });
264 } else {
265 const was = page.name;
266 lobby!.turns.end(`u-${was}`);
267 lobby!.turns.join({ id: `u-${was}`, name: was, owner: false });
268 timeline.push({ at: Date.now(), who: "page", what: `turn: ${was} → ${driving().name}` });
269 }
270 await sleep(random() < 0.1 ? 250 : Math.floor(random() * 25));
271 }
272 await sleep(80);
273 // Every frame that reached the Pi was made while the arm was not held, and reached it before any later STOP took effect.
274 const heldAt = (t: number) => {
275 const stop = stops.filter((s) => s <= t).at(-1);
276 return stop !== undefined && !resumes.some((r) => r > stop && r <= t);
277 };
278 for (const { at, n: k } of pi.got) {
279 const madeAt = made.get(k)!;
280 expect({ seed, frame: k, heldWhenMade: heldAt(madeAt) }).toEqual({ seed, frame: k, heldWhenMade: false });
281 const stopAfter = stops.find((s) => s >= madeAt);
282 expect({ seed, frame: k, afterStop: stopAfter !== undefined && at > stopAfter }).toEqual({ seed, frame: k, afterStop: false });
283 }
284 // Not vacuous: each run ends with a resume and a new command, and that command arrives.
285 const page = driving();
286 expect((await page.post("/resume", port)).status).toBe(200);
287 expect(await page.open()).toBe(true);
288 const last = page.make(++n);
289 page.send(last, "drag");
290 await sleep(50);
291 if (pi.got.at(-1)?.n !== last.n) print(`seed ${seed} FAILED`, timeline);
292 expect({ seed, last: pi.got.at(-1)?.n }).toEqual({ seed, last: last.n });
293 if (seed === 3) print(`(a) property run, seed ${seed}: ${pi.got.length} of ${n} frames reached the Pi; ${stops.length} stops`, timeline);
294 void held;
295 await shut(relay);
296 lobby!.close();
297 pi.close();
298 relay = lobby = undefined;
299 }
300 }, 120_000);
301 });
302
303 describe("STOP on the arm server itself", () => {
304 let fake: FakeBus;
305 let server: RunningServer;
306 let base: string;
307 beforeAll(async () => {
308 fake = new FakeBus([1, 2, 6], { stuck: new Set([2]) });
309 server = await startServer(new FeetechBus(fake), { host: "127.0.0.1", port: 0, ids: [1, 2, 6], posesFile: join(mkdtempSync(join(tmpdir(), "so101-stop-arm-")), "poses.json"), watchdogMs: 2000, jamSampleMs: 1000 });
310 base = `http://127.0.0.1:${server.port}`;
311 });
312 afterAll(() => shut(server));
313 const post = (path: string, body: unknown = {}) => fetch(`${base}${path}`, { method: "POST", body: JSON.stringify(body) });
314 const stream = async () => {
315 const ws = new WebSocket(`ws://127.0.0.1:${server.port}/follow`);
316 await new Promise((r) => ws.once("open", r));
317 return ws;
318 };
319 /** Goal_Position (42) and Goal_Velocity (46) writes since `from`. */
320 const goals = (from: number) => fake.writes.slice(from).filter((w) => w.address === 42 || w.address === 46);
321
322 test("STOP during the catch-up ease ends it at once: no more of the ease, and the arm holds where it is", async () => {
323 const ws = await stream();
324 // joint 2 is stuck: the ease toward 2600 would wait out its 4 s, then speed the servos back up
325 ws.send(JSON.stringify({ 1: 2100, 2: 2600, seq: 1, t: Date.now() }));
326 await sleep(150);
327 const from = fake.writes.length;
328 expect(await (await post("/stop", { by: "ann" })).json()).toEqual({ stopped: "stream" });
329 await sleep(150);
330 const after = goals(from);
331 expect(after.filter((w) => w.address === 46)).toEqual([]); // the ease did not finish by speeding up again
332 const holds = after.filter((w) => w.address === 42).map((w) => [w.id, w.bytes[0]! | (w.bytes[1]! << 8)]);
333 expect(holds).toContainEqual([2, 2048]); // held where it is, not driven on to 2600
334 expect(holds).not.toContainEqual([2, 2600]);
335 expect(((await (await fetch(`${base}/status`)).json()) as { held: { by: string } }).held.by).toBe("ann");
336 });
337
338 test("while held, a stream is refused and nothing it sends moves the arm; after resume, only a new command does", async () => {
339 const from = fake.writes.length;
340 const ws = new WebSocket(`ws://127.0.0.1:${server.port}/follow`);
341 const closed = await new Promise<string>((r) => ws.on("close", (code, why) => r(`${code} ${why}`)));
342 expect(closed).toStartWith(`${HELD_CLOSE} stopped (ann`);
343 expect((await post("/move", { targets: { 1: 2200 } })).status).toBe(423);
344 expect((await post("/torque", { on: false })).status).toBe(423);
345 expect(goals(from)).toEqual([]);
346
347 const here = ((await (await fetch(`${base}/status`)).json()) as { joints: Array<{ id: number; position: number }> }).joints.find((j) => j.id === 1)!.position;
348 const t0 = Date.now();
349 expect(await (await post("/resume", { by: "ann" })).json()).toEqual({ resumed: true, held: null });
350 const s = await stream();
351 // frames made before the resume, flushed after it, in order: none of them (this device's clock is known from the stream before the stop)
352 for (let i = 0; i < 100; i++) s.send(JSON.stringify({ 1: 2300 + i, seq: i + 1, t: t0 - 150 + i }));
353 await sleep(100);
354 expect(goals(from).filter((w) => w.address === 42 && w.id === 1)).toEqual([]);
355 s.send(JSON.stringify({ 1: 2150, seq: 500, t: Date.now() }));
356 await sleep(100);
357 const moved = goals(from).filter((w) => w.address === 42 && w.id === 1).map((w) => w.bytes[0]! | (w.bytes[1]! << 8));
358 expect(moved.length).toBeGreaterThan(0);
359 expect(new Set(moved)).toEqual(new Set([here, 2150])); // armed where it was, then the new command
360 s.close();
361 await sleep(50);
362 });
363
364 test("a turn that ends stops what drives without holding", async () => {
365 const ws = await stream();
366 ws.send(JSON.stringify({ 1: 2100 }));
367 await sleep(50);
368 expect(await (await post("/stop", { hold: false })).json()).toEqual({ stopped: "stream" });
369 expect(((await (await fetch(`${base}/status`)).json()) as { held: unknown }).held).toBeNull();
370 await sleep(50);
371 });
372 });
Software/so101/src/teleop.test.ts +4 −1
@@ -87,7 +87,10 @@ describe("teleop", () => {
8787 const frames = sent();
8888 expect(frames.length).toBeGreaterThanOrEqual(10);
8989 expect(frames.length).toBeLessThanOrEqual(17); // 500 ms at 30 Hz, never the 60 readings it was handed
90 expect(frames.at(-1)).toEqual({ 1: last, 6: 2500 }); // nothing older is left behind to go out later
90 const { seq, t, ...joints } = frames.at(-1)!;
91 expect(joints).toEqual({ 1: last, 6: 2500 }); // nothing older is left behind to go out later
92 expect(frames.map((f) => f.seq)).toEqual(frames.map((_, i) => i + 1)); // stamped in order, for the server to drop any replay
93 expect(Math.abs(Date.now() - t!)).toBeLessThan(1000);
9194 expect(new Set(frames.map((f) => f[1])).size).toBe(frames.length); // a reading is never sent twice
9295 expect(sockets.length).toBe(1);
9396 });
Software/so101/src/teleop.ts +3 −1
@@ -111,6 +111,8 @@ export class Teleop {
111111 #ws: SocketLike | null = null;
112112 #latest: Sent | null = null;
113113 #lastSentAt = -Infinity;
114 /** Each frame's place in this page's stream of commands: the server drops one that is not newer than the last. */
115 #seq = 0;
114116 #lastReadingSent = -Infinity;
115117 #timer: ReturnType<typeof setTimeout> | undefined;
116118 #retry: ReturnType<typeof setTimeout> | undefined;
@@ -248,7 +250,7 @@ export class Teleop {
248250 const latest = this.#latest;
249251 if (!ws || ws.readyState !== OPEN || this.#state !== "live" || !latest || latest.at <= this.#lastReadingSent) return;
250252 if (!Object.keys(latest.frame).length) return;
251 ws.send(JSON.stringify(latest.frame));
253 ws.send(JSON.stringify({ ...latest.frame, seq: ++this.#seq, t: Date.now() })); // stamped: never applied late or twice
252254 this.frames++;
253255 this.#lastSentAt = this.#now();
254256 this.#lastReadingSent = latest.at;
Software/so101/src/terminal.test.ts +52 −3
@@ -12,7 +12,7 @@ import { identify, sign } from "./identity.ts";
1212 import { Lobby } from "./lobby.ts";
1313 import { startRelay, type RunningRelay } from "./relay.ts";
1414 import { startServer, type RunningServer } from "./server.ts";
15 import { Door, emptyRoom, hear, liveUrl, machineEntry, machineLines, machineUrl, plain, terminalHeaders, typed } from "./terminal.ts";
15 import { Door, emptyRoom, hear, liveUrl, machineEntry, machineLines, machineUrl, plain, poster, terminalHeaders, typed } from "./terminal.ts";
1616 import { jointRows } from "./ui/machine.js";
1717 import { Turns } from "./turns.ts";
1818
@@ -93,7 +93,7 @@ describe("terminal lines", () => {
9393
9494 test("the machine in the page's words, and nothing from the arm reaches the terminal as an escape", () => {
9595 expect(machineUrl("ws://127.0.0.1:4123/live")).toBe("ws://127.0.0.1:4123/machine");
96 const status = { joints: [{ id: 2, joint: "shoulder_lift", position: 1234, degrees: 108.5, load: -12.3, temperature: 41, torque: true, target: null, min: 818, max: 3487, lo: 848, hi: 3457 }], driving: null, held: false, estop: null };
96 const status = { joints: [{ id: 2, joint: "shoulder_lift", position: 1234, degrees: 108.5, load: -12.3, temperature: 41, torque: true, target: null, min: 818, max: 3487, lo: 848, hi: 3457 }], driving: null, held: null, deadman: false, estop: null };
9797 const lines = machineLines({ status, link: { mode: "live", host: "pi:8101", rttMs: 17, fps: 4, reconnects: 0, problem: null }, room: { driver: "alice", paused: false, speedCapPercent: 100 }, skew: 0 });
9898 expect(lines).toEqual([
9999 "2 shoulder_lift · 1234 ticks 108.5° · target — · load -12.3% · 41 °C · torque on · limits 818–3487 · bounds 848–3457",
@@ -106,6 +106,24 @@ describe("terminal lines", () => {
106106 });
107107
108108 /** Both doors on one room: the page's WebSocket and the terminal's, gated by the same server. */
109 describe("terminal STOP", () => {
110 test("/stop and /resume are the app's own routes; the status says STOPPED and resumed once each, not on every push", () => {
111 const room = emptyRoom();
112 expect(typed(room, "/stop")).toEqual({ post: "/stop" });
113 expect(typed(room, "/RESUME")).toEqual({ post: "/resume" });
114 const status = (held: unknown) => ({ type: "status", status: { joints: [], held } });
115 expect(hear(room, status(null))).toEqual([]); // the first status, not stopped: nothing to say
116 const held = { by: "alice\u001b[31m", at: "2026-10-20T22:05:07.000Z" };
117 expect(hear(room, status(held))).toEqual(["STOPPED by alice[31m at 22:05:07 UTC: the arm holds until the driver or an admin types /resume"]);
118 expect(hear(room, status(held))).toEqual([]);
119 expect(hear(room, { type: "status", status: null, error: "reconnecting" })).toEqual([]);
120 expect(hear(room, status(null))).toEqual(["resumed: the arm may move again, from a new command"]);
121 expect(hear(room, status(null))).toEqual([]);
122 // A terminal that comes in while the arm is stopped is told at once.
123 expect(hear(emptyRoom(), status(held))).toHaveLength(1);
124 });
125 });
126
109127 describe("terminal door", () => {
110128 let simulation: RunningServer;
111129 let relay: RunningRelay | undefined;
@@ -130,7 +148,7 @@ describe("terminal door", () => {
130148 /** A terminal as the forge would start it for this person. */
131149 const terminal = async (port: number, env: Record<string, string>) => {
132150 const printed: string[] = [];
133 const door = new Door(liveUrl({ ...env, PORT: String(port) }), terminalHeaders(env), (line) => printed.push(line));
151 const door = new Door(liveUrl({ ...env, PORT: String(port) }), terminalHeaders(env), (line) => printed.push(line), poster({ ...env, PORT: String(port) }));
134152 await door.open();
135153 open.push(door);
136154 const printedLine = (match: RegExp) => waitFor(() => printed.find((l) => match.test(l)), `line matching ${match}; printed ${JSON.stringify(printed)}`);
@@ -295,4 +313,35 @@ describe("terminal door", () => {
295313 await owner.printedLine(/alice's turn ended .*bob has control/);
296314 expect(lobby!.admin.log.some((e) => /ended alice's turn/.test(JSON.stringify(e)))).toBe(true);
297315 });
316
317 test("STOP and resume: only the driver or an owner, from the terminal or the page, and every terminal hears the change", async () => {
318 const port = await start();
319 const alice = await terminal(port, ttyEnv("u-alice", "alice", "user"));
320 alice.door.input("/join");
321 await waitFor(() => (lobby!.turns.driver?.id === "u-alice" ? true : undefined), "alice driving");
322 const carol = await terminal(port, ttyEnv("u-carol", "carol", "user"));
323 carol.door.input("/stop");
324 await carol.printedLine(/^error: alice has control/);
325 expect(relay!.link().mode).toBe("simulated");
326 const page = await viewer(port, web("u-alice", "alice"));
327
328 alice.door.input("/stop");
329 await page.heardOne((m) => m.type === "status" && m.status?.held?.by === "alice", "the hold on the page");
330 await alice.printedLine(/^STOPPED by alice at \d\d:\d\d:\d\d UTC/);
331 await carol.printedLine(/^STOPPED by alice/);
332 carol.door.input("/resume");
333 await waitFor(() => (carol.printed.filter((l) => /^error: alice has control/.test(l)).length === 2 ? true : undefined), "carol refused resume");
334
335 // Resumed from the page: the terminals hear it; stopped again from an owner's terminal, the page hears that.
336 const r = await fetch(`http://127.0.0.1:${port}/resume`, { method: "POST", headers: { "content-type": "application/json", ...web("u-alice", "alice") }, body: "{}" });
337 expect(r.status).toBe(200);
338 await alice.printedLine(/^resumed: the arm may move again/);
339 await carol.printedLine(/^resumed/);
340 const owner = await terminal(port, ttyEnv("u-owner", "trevato", "admin"));
341 owner.door.input("/stop");
342 await page.heardOne((m) => m.type === "status" && m.status?.held?.by === "trevato", "the owner's stop on the page");
343 owner.door.input("/resume");
344 await alice.printedLine(/^STOPPED by trevato/);
345 await waitFor(() => (alice.printed.filter((l) => /^resumed/.test(l)).length === 2 ? true : undefined), "the owner's resume on alice's terminal");
346 });
298347 });
Software/so101/src/terminal.ts +65 −7
@@ -1,17 +1,22 @@
11 /**
2 * The room's terminal door: chat, the line and the owner's mute and end-turn, typed instead of tapped.
2 * The room's terminal door: chat, the line, STOP and resume, and the owner's mute and end-turn, typed instead of tapped.
33 *
44 * The forge runs forge.json's `tty` on a pty when someone `ssh`es to the app (FORGE_APP_SSH says how). That program
55 * is handed the app's own environment — PORT, FORGE_APP_IDENTITY_KEY — plus who is at the keyboard:
66 * FORGE_APP_USER, FORGE_APP_USER_ID and FORGE_APP_USER_ROLE (`user`, `admin` for whoever may write the repository,
77 * empty for a guest). Its cell may reach the running app on http://127.0.0.1:$PORT, so the door is simply another
88 * client of /live: it signs that identity the way the forge signs a browser's (src/identity.ts) and the Lobby gates
9 * it exactly as it gates the page. Nothing here decides who may do what; the server does, and says why not.
9 * it exactly as it gates the page. STOP and resume are the page's own POST /stop and /resume on the same app, signed the
10 * same way, so the relay lets the driver and the admins do them and nobody else. The arm's status comes over /live
11 * too: when it is stopped, or resumed, from here or from a page, every terminal says so.
12 * Nothing here decides who may do what; the server does, and says why not.
1013 *
1114 * Typed lines:
1215 * <text> say it
1316 * /join /leave ask to drive (the owner takes over), or hand back / leave the line
1417 * /ready /extend confirm you are there when called (15 s), or add 30 s to your turn
18 * /stop the driver or an admin: halt the arm and hold it, whatever drives it, until /resume
19 * /resume the driver or an admin: end the hold; the arm stays put until a new command
1520 * /end [who] owner: end that turn (none: whoever drives), or take them out of the line
1621 * /mute <who> /unmute <who> owner
1722 * /machine [off] the Machine panel (/machine, src/machine-feed.ts): the joints, the link and the server as they are,
@@ -44,10 +49,34 @@ export function terminalHeaders(env: Env, now = Date.now()): Record<string, stri
4449
4550 /** The running app's lobby, beside the terminal: PORT as the forge sets it, 8102 as bench/fake-serve.ts defaults. */
4651 export function liveUrl(env: Env): string {
52 return `ws://127.0.0.1:${appPort(env)}/live`;
53 }
54
55 /** The same app over HTTP, for /stop and /resume. */
56 export function appUrl(env: Env): string {
57 return `http://127.0.0.1:${appPort(env)}`;
58 }
59
60 function appPort(env: Env): number {
4761 const raw = env.PORT ?? "8102";
4862 const port = /^\d{1,5}$/.test(raw) ? Number(raw) : NaN;
4963 if (!(port >= 1 && port <= 65535)) throw new Error(`PORT is not a port: ${JSON.stringify(raw.slice(0, 20))}`);
50 return `ws://127.0.0.1:${port}/live`;
64 return port;
65 }
66
67 export type Post = (path: "/stop" | "/resume") => Promise<{ status: number; error: string | null }>;
68
69 /** POST to the app as the page does, signed afresh for each request (a signature is good for a minute). */
70 export function poster(env: Env, fetcher: typeof fetch = fetch): Post {
71 return async (path) => {
72 try {
73 const r = await fetcher(`${appUrl(env)}${path}`, { method: "POST", headers: { "content-type": "application/json", ...terminalHeaders(env) }, body: "{}", signal: AbortSignal.timeout(5000) });
74 const body = (await r.json().catch(() => ({}))) as { error?: unknown };
75 return { status: r.status, error: r.ok ? null : plain(body.error ?? `status ${r.status}`) };
76 } catch (err) {
77 return { status: 0, error: plain((err as Error).message) };
78 }
79 };
5180 }
5281
5382 /** Text from the room, safe to write to a terminal: no escapes, no control characters, no line breaks. */
@@ -67,13 +96,15 @@ export interface Room {
6796 driver: Someone | null;
6897 queue: Someone[];
6998 paused: boolean;
99 /** The arm's STOP: who and when, as a line; null while it may move, undefined until the first status. */
100 held: string | null | undefined;
70101 /** Whether the call to confirm, and the "you are next" cue, were already printed (each is said once). */
71102 told: { called: boolean; next: boolean };
72103 /** Everyone seen by name, most recent last: to turn a typed name into an id. */
73104 seen: Map<string, Someone>;
74105 }
75106
76 export const emptyRoom = (): Room => ({ you: null, muted: false, banned: false, driver: null, queue: [], paused: false, told: { called: false, next: false }, seen: new Map() });
107 export const emptyRoom = (): Room => ({ you: null, muted: false, banned: false, driver: null, queue: [], paused: false, held: undefined, told: { called: false, next: false }, seen: new Map() });
77108
78109 const someone = (x: unknown): Someone | null => {
79110 if (typeof x !== "object" || x === null) return null;
@@ -93,7 +124,7 @@ const said = (m: unknown): string => {
93124 return from ? `${plain(from.name)}: ${plain(message?.text)}` : `· ${plain(message?.text)}`;
94125 };
95126
96 export const HELP = "type to chat · /join /leave /ready /extend · /end [who] /mute <who> /unmute <who> (owner) · /machine [off] · /who · /quit";
127 export const HELP = "type to chat · /join /leave /ready /extend · /stop /resume (driver, owner) · /end [who] /mute <who> /unmute <who> (owner) · /machine [off] · /who · /quit";
97128
98129 /** The Machine feed's socket beside the lobby's: same app, same port. */
99130 export const machineUrl = (live: string): string => live.replace(/\/live$/, "/machine");
@@ -171,6 +202,17 @@ export function hear(room: Room, frame: Frame): string[] {
171202 case "said":
172203 see(room, someone((frame.message as { from?: unknown } | null)?.from));
173204 return [said(frame.message)];
205 case "status": {
206 // Said when it changes, not on every push: a stop made here or on a page reaches every terminal once.
207 const h = (frame.status as { held?: unknown } | null)?.held as { by?: unknown; at?: unknown } | null | undefined;
208 if (frame.status === null || h === undefined) return [];
209 const at = typeof h?.at === "string" ? h.at.slice(11, 19) : "";
210 const now = h ? `STOPPED by ${plain(h.by)}${at ? ` at ${at} UTC` : ""}: the arm holds until the driver or an admin types /resume` : null;
211 const was = room.held;
212 room.held = now;
213 if (now === was || (was === undefined && now === null)) return [];
214 return [now ?? "resumed: the arm may move again, from a new command"];
215 }
174216 case "unsaid":
175217 return ["· a line was removed"];
176218 case "error":
@@ -192,7 +234,7 @@ function target(room: Room, word: string): string | null {
192234 return ID.test(word) ? word : null;
193235 }
194236
195 export type Typed = { send: Frame } | { print: string } | { machine: boolean } | { quit: true } | null;
237 export type Typed = { send: Frame } | { post: "/stop" | "/resume" } | { print: string } | { machine: boolean } | { quit: true } | null;
196238
197239 /** What a typed line asks for. Validation the server repeats anyway; this only saves a round trip on a typo. */
198240 export function typed(room: Room, input: string): Typed {
@@ -218,6 +260,9 @@ export function typed(room: Room, input: string): Typed {
218260 const id = target(room, arg);
219261 return id ? { send: { type: "mute", id, on: verb.toLowerCase() === "mute" } } : { print: `error: who is ${plain(arg)}?` };
220262 }
263 case "stop":
264 case "resume":
265 return { post: `/${verb.toLowerCase()}` as "/stop" | "/resume" };
221266 case "machine":
222267 if (arg && arg.toLowerCase() !== "off" && arg.toLowerCase() !== "on") return { print: "usage: /machine [off]" };
223268 return { machine: arg.toLowerCase() !== "off" };
@@ -242,11 +287,14 @@ export class Door {
242287 private readonly url: string;
243288 private readonly headers: Record<string, string>;
244289 private readonly print: (line: string) => void;
290 private readonly post: Post | null;
245291
246 constructor(url: string, headers: Record<string, string>, print: (line: string) => void) {
292 /** `post` reaches the app's /stop and /resume; without it, the door says it cannot. */
293 constructor(url: string, headers: Record<string, string>, print: (line: string) => void, post: Post | null = null) {
247294 this.url = url;
248295 this.headers = headers;
249296 this.print = print;
297 this.post = post;
250298 }
251299
252300 /** Connect; resolves once seated, rejects when the app does not answer. `onClose` hears the app going away. */
@@ -280,11 +328,21 @@ export class Door {
280328 if (!t) return true;
281329 if ("quit" in t) return false;
282330 if ("print" in t) this.print(t.print);
331 else if ("post" in t) void this.ask(t.post);
283332 else if ("machine" in t) this.watchMachine(t.machine);
284333 else this.send(t.send);
285334 return true;
286335 }
287336
337 /** The answer is only printed when it is a refusal: the change itself comes back to every terminal with the status. */
338 private async ask(path: "/stop" | "/resume"): Promise<void> {
339 if (!this.post) return this.print(`error: ${path.slice(1)} is not available here`);
340 const { error } = await this.post(path);
341 if (error) this.print(`error: ${error}`);
342 else if (path === "/stop" && this.room.held) this.print(this.room.held);
343 else if (path === "/resume" && this.room.held === null) this.print("the arm was not stopped");
344 }
345
288346 close(): void {
289347 this.ws?.close();
290348 this.machineWs?.close();
Software/so101/src/tty.ts +2 −2
@@ -4,7 +4,7 @@
44 */
55
66 import { createInterface, type Interface } from "node:readline";
7 import { Door, HELP, liveUrl, terminalHeaders } from "./terminal.ts";
7 import { Door, HELP, liveUrl, poster, terminalHeaders } from "./terminal.ts";
88
99 const env = process.env;
1010 const where = env.FORGE_APP_ENV === "preview" ? `preview of #${env.FORGE_APP_PULL ?? "?"}` : (env.FORGE_APP_ENV ?? "local");
@@ -17,7 +17,7 @@ const print = (line: string) => {
1717 rl?.prompt(true);
1818 };
1919
20 const door = new Door(liveUrl(env), terminalHeaders(env), print);
20 const door = new Door(liveUrl(env), terminalHeaders(env), print, poster(env));
2121 print(`\x1b[1mSO-101 arm\x1b[0m · ${where} · ${HELP}`);
2222 try {
2323 await door.open(() => {
Software/so101/src/ui/drive.js +3 −3
@@ -29,10 +29,10 @@ export function shape(v, dead = 0.12) {
2929 * Whether a finger may drive now: it is this person's turn, the arm is not stopped and not reconnecting, and its
3030 * limits are known. The page asks before a press starts and on every status poll, letting go of whatever is held
3131 * the moment the answer turns false.
32 * @param {{ mayDrive: boolean, estop?: unknown, reconnecting?: unknown, limits?: Limits | null }} state
32 * @param {{ mayDrive: boolean, estop?: unknown, held?: unknown, reconnecting?: unknown, limits?: Limits | null }} state
3333 */
34 export function canDrive({ mayDrive, estop, reconnecting, limits }) {
35 return Boolean(mayDrive) && !estop && !reconnecting && Object.keys(limits ?? {}).length > 0;
34 export function canDrive({ mayDrive, estop, held, reconnecting, limits }) {
35 return Boolean(mayDrive) && !estop && !held && !reconnecting && Object.keys(limits ?? {}).length > 0;
3636 }
3737
3838 /**
Software/so101/src/ui/drive.test.ts +1
@@ -285,6 +285,7 @@ describe("canDrive", () => {
285285 test("refuses when it is not your turn, the arm is stopped, the arm is reconnecting, or no limits have come", () => {
286286 expect(canDrive({ ...ok, mayDrive: false })).toBe(false);
287287 expect(canDrive({ ...ok, estop: true })).toBe(false);
288 expect(canDrive({ ...ok, held: { by: "ann", at: "2026-10-20T22:05:00Z" } })).toBe(false); // STOP, until someone resumes
288289 expect(canDrive({ ...ok, reconnecting: true })).toBe(false);
289290 expect(canDrive({ ...ok, limits: {} })).toBe(false);
290291 expect(canDrive({ ...ok, limits: null })).toBe(false);
Software/so101/src/ui/frames.js +15 −6
@@ -1,20 +1,23 @@
11 // @ts-check
22 /**
3 * Frames on their way to /follow: `send` coalesces them into one message per slot, joint by joint, the newest target
4 * winning; `flush` puts what is pending and the given frame on the wire now, because it is where a finger let go.
5 * `write` gets the JSON; whether a socket is open is the page's business.
3 * Frames on their way to /follow: `send` coalesces them into one per slot, the newest target winning; `flush` sends
4 * now (a finger let go). Each is stamped with `seq` and `t`, when its newest target was given (src/command-gate.ts).
65 * @typedef {Record<number, number>} Frame
76 */
87 export class Frames {
98 /**
109 * @param {(data: string) => void} write
11 * @param {{ wait?: number, later?: (fn: () => void, ms: number) => unknown, cancel?: (handle: unknown) => void }} [options]
10 * @param {{ wait?: number, later?: (fn: () => void, ms: number) => unknown, cancel?: (handle: unknown) => void, now?: () => number }} [options]
1211 */
1312 constructor(write, options = {}) {
1413 this.write = write;
1514 this.wait = options.wait ?? 33;
1615 this.later = options.later ?? ((fn, ms) => setTimeout(fn, ms));
1716 this.cancel = options.cancel ?? ((h) => clearTimeout(/** @type {any} */ (h)));
17 this.now = options.now ?? (() => Date.now());
18 /** The last seq written. */
19 this.seq = 0;
20 this.at = 0;
1821 /** @type {Frame | null} */
1922 this.pending = null;
2023 /** @type {unknown} */
@@ -24,11 +27,12 @@ export class Frames {
2427 /** @param {Frame} frame */
2528 send(frame) {
2629 this.pending = { ...(this.pending ?? {}), ...frame };
30 this.at = this.now();
2731 if (this.timer !== null) return;
2832 this.timer = this.later(() => {
2933 this.timer = null;
3034 const data = this.pending; this.pending = null;
31 if (data) this.write(JSON.stringify(data));
35 if (data) this.stamped(data, this.at);
3236 }, this.wait);
3337 }
3438
@@ -36,7 +40,12 @@ export class Frames {
3640 flush(frame) {
3741 const data = { ...(this.pending ?? {}), ...frame };
3842 this.drop();
39 this.write(JSON.stringify(data));
43 this.stamped(data, this.now());
44 }
45
46 /** @param {Frame} data @param {number} t */
47 stamped(data, t) {
48 this.write(JSON.stringify({ ...data, seq: ++this.seq, t }));
4049 }
4150
4251 /** Forget what is pending: the stream ended. */
Software/so101/src/ui/frames.test.ts +25 −2
@@ -4,10 +4,19 @@ import { Frames } from "./frames.js";
44 /** Frames on a hand-run timer, writing to an array. */
55 function rig() {
66 const wire: Array<Record<string, number>> = [];
7 const stamps: Array<{ seq: number; t: number }> = [];
78 let due: (() => void) | null = null;
8 const frames = new Frames((data) => wire.push(JSON.parse(data)), { later: (fn) => ((due = fn), 1), cancel: () => (due = null) });
9 let clock = 1000;
10 const frames = new Frames(
11 (data) => {
12 const { seq, t, ...joints } = JSON.parse(data);
13 stamps.push({ seq, t });
14 wire.push(joints);
15 },
16 { later: (fn) => ((due = fn), 1), cancel: () => (due = null), now: () => clock },
17 );
918 const slot = () => { const fn = due; due = null; fn?.(); };
10 return { frames, wire, slot, waiting: () => due !== null };
19 return { frames, wire, stamps, slot, waiting: () => due !== null, tick: (ms: number) => (clock += ms) };
1120 }
1221
1322 describe("Frames", () => {
@@ -45,4 +54,18 @@ describe("Frames", () => {
4554 slot();
4655 expect(wire).toEqual([]);
4756 });
57
58 test("every frame is stamped: seq counts up, t is when its newest target was given", () => {
59 const { frames, stamps, slot, tick } = rig();
60 frames.send({ 1: 2000 });
61 tick(10);
62 frames.send({ 1: 2010 });
63 tick(20);
64 slot();
65 frames.flush({ 1: 2020 });
66 tick(5);
67 frames.send({ 2: 2000 });
68 slot();
69 expect(stamps).toEqual([{ seq: 1, t: 1010 }, { seq: 2, t: 1030 }, { seq: 3, t: 1035 }]);
70 });
4871 });
Software/so101/src/ui/machine.js +2 −2
@@ -53,7 +53,7 @@ export function linkRows(link) {
5353
5454 /**
5555 * Who and what holds the arm, label and value.
56 * @param {{ driving?: string | null, held?: boolean, estop?: { by: string, at: string } | null, jammed?: number[], blocked?: { what: string, link: string } | null, zone?: string } | null | undefined} status
56 * @param {{ driving?: string | null, held?: { by: string, at: string } | null, deadman?: boolean, estop?: { by: string, at: string } | null, jammed?: number[], blocked?: { what: string, link: string } | null, zone?: string } | null | undefined} status
5757 * @param {{ driver?: string | null, paused?: boolean, speedCapPercent?: number } | null | undefined} room
5858 * @returns {Array<[string, string]>}
5959 */
@@ -61,7 +61,7 @@ export function stateRows(status, room) {
6161 return [
6262 ["turn", room?.driver ?? "nobody"],
6363 ["driving", status ? status.driving ?? "nothing (idle)" : DASH],
64 ["held", status ? (status.held ? "yes: dead-man hold, no frame lately" : "no") : DASH],
64 ["held", status ? (status.held ? `STOPPED · ${status.held.by} · ${status.held.at}` : status.deadman ? "yes: dead-man hold, no frame lately" : "no") : DASH],
6565 ["E-stop", status?.estop ? `ON · ${status.estop.by} · ${status.estop.at}` : status ? "off" : DASH],
6666 ["zone", status?.zone ?? DASH],
6767 ["blocked", status?.blocked ? `${status.blocked.link} · ${status.blocked.what}` : "no"],
Software/so101/src/ui/machine.test.ts +3 −1
@@ -14,8 +14,10 @@ describe("the Machine panel's text", () => {
1414 expect(linkRows({ mode: "reconnecting", host: "pi:8101", problem: "pi:8101 unreachable (ECONNREFUSED)", rttMs: 17.4, fps: 4, reconnects: 2 })).toEqual([
1515 ["arm", "reconnecting · pi:8101"], ["round trip", "17.4 ms"], ["status frames", "4/s"], ["reconnects", "2"], ["problem", "pi:8101 unreachable (ECONNREFUSED)"],
1616 ]);
17 const rows = Object.fromEntries(stateRows({ driving: "stream", held: true, estop: { by: "ana", at: "2026-10-20T22:00:00.000Z" }, jammed: [2], zone: "enforce" }, { driver: "ana", paused: true, speedCapPercent: 40 }));
17 const rows = Object.fromEntries(stateRows({ driving: "stream", deadman: true, estop: { by: "ana", at: "2026-10-20T22:00:00.000Z" }, jammed: [2], zone: "enforce" }, { driver: "ana", paused: true, speedCapPercent: 40 }));
1818 expect(rows).toEqual({ turn: "ana", driving: "stream", held: "yes: dead-man hold, no frame lately", "E-stop": "ON · ana · 2026-10-20T22:00:00.000Z", zone: "enforce", blocked: "no", jammed: "2", "speed cap": "40%", queue: "paused" });
19 // A STOP outranks the dead-man hold: it is the one only /resume ends.
20 expect(Object.fromEntries(stateRows({ held: { by: "ana", at: "2026-10-20T22:00:00.000Z" }, deadman: true }, null)).held).toBe("STOPPED · ana · 2026-10-20T22:00:00.000Z");
1921 });
2022
2123 test("log lines: arrow, what, detail, age; copy is one tab-separated line each, in UTC", () => {
Software/so101/src/viewer.html +61 −34
@@ -282,7 +282,7 @@
282282 #pad { width: min(30vh, 150px); }
283283 }
284284 @media (max-width: 640px) and (orientation: portrait) { #touch { display: grid; } #reach, #joints { display: none !important; } }
285 /* One Stop: the E-stop. The other button appears only while a move, a playback, a recording or a park runs, and says which it ends. */
285 /* One STOP in every layout; it holds until Resume. */
286286 @media (orientation: landscape) and (max-height: 500px) {
287287 #shell.driving.bare #sticks { display: block; }
288288 #shell.driving.bare #top { top: 6px; bottom: auto; left: auto; right: calc(6px + var(--safe-r)); max-width: 42%; justify-content: flex-end; }
@@ -321,8 +321,7 @@
321321 <div class="who"><div id="driverLine" class="none">Connecting…</div><div id="clock" aria-live="off"></div><div id="capLine"></div></div>
322322 <div id="turnActions"></div>
323323 <button id="sheetToggle" hidden aria-controls="sheet" aria-expanded="true">Hide</button>
324 <button id="stop" class="danger" hidden title="End what the arm is doing and hold (space)">End move</button>
325 <button id="estop" class="estop" title="Emergency stop (Esc): halt, hold with torque, until an admin clears it" aria-label="Emergency stop">Stop</button>
324 <button id="stop" class="estop" data-stop title="Stop (space or Esc): halt everything, hold with torque, until someone resumes" aria-label="Stop the arm">Stop</button>
326325 </section>
327326
328327 <section id="sheet" aria-label="Chat, queue and controls">
@@ -786,20 +785,36 @@
786785
787786 // ---------------------------------------------------------------- the stream: the page as a leader
788787 // Opened while a slider, a jog button, the joystick or a model joint is held; frames go at most every 33 ms; closed shortly after letting go.
788 // Stamped targets (src/ui/frames.js): while the socket opens only the newest waits, and goes only if still fresh.
789789 const stream = {
790 ws: null, closeTimer: null, halted: false,
791 // one message per slot to the socket that is open, or the one about to be
790 ws: null, closeTimer: null, halted: false, waiting: null,
792791 frames: new Frames((data) => {
793792 const ws = stream.ws;
794793 if (ws?.readyState === 1) ws.send(data);
795 else if (ws?.readyState === 0) ws.addEventListener("open", () => ws.send(data), { once: true });
794 else if (ws?.readyState === 0) stream.waiting = { data, at: Date.now() };
796795 }),
796 /** STOP: drop what is pending, close, send nothing until a release or a resume. */
797 halt() {
798 clearTimeout(this.closeTimer);
799 this.frames.drop();
800 this.waiting = null;
801 const ws = this.ws;
802 this.ws = null;
803 this.halted = true;
804 ws?.close();
805 },
797806 open() {
798807 if (!mayDrive()) return; // the server would refuse it: only the person whose turn it is streams
799808 clearTimeout(this.closeTimer);
800809 if (this.ws && this.ws.readyState <= 1) return;
801810 if (this.halted) return; // the server ended this hold (Stop, or the arm is busy): nothing more until the next press
802811 const ws = new WebSocket(`${location.protocol === "https:" ? "wss" : "ws"}://${location.host}/follow`);
812 this.waiting = null;
813 ws.onopen = () => {
814 const w = this.waiting;
815 this.waiting = null;
816 if (this.ws === ws && w && Date.now() - w.at <= 200) ws.send(w.data);
817 };
803818 ws.onclose = (e) => {
804819 if (this.ws !== ws) return;
805820 this.ws = null;
@@ -858,34 +873,37 @@
858873 const r = await api("/pose/go", { name });
859874 msg(r.reached ? `reached ${name}${r.warning ? ` — ${r.warning}` : ""}` : `${r.error}`, !r.reached);
860875 }
876 // STOP: drop what this page would still send, then the server halts and holds the arm until Resume.
861877 async function stop() {
878 stream.halt();
879 driver.releaseAll();
862880 reachEnd();
881 stopLeaderDrive("stopped");
863882 const r = await api("/stop", {});
864 msg(r.stopped ? `stopped ${r.stopped}${r.saved ? `; saved recording ${r.saved}` : ""}; the arm holds` : r.ok ? "stopped: nothing was driving" : r.error, !r.ok);
883 msg(r.ok ? `STOPPED${r.stopped ? ` ${r.stopped}` : ""}${r.saved ? `; saved recording ${r.saved}` : ""}: the arm holds until you resume` : r.error, true);
865884 if (r.saved) loadRecordings();
866885 }
867 $("stop").onclick = stop;
868 /** The second button, only while there is something besides the E-stop to end, named for what it ends. */
869 const ENDS_LABEL = { move: "End move", play: "End playback", record: "End recording", park: "End park" };
870 function showStop() {
871 const label = ENDS_LABEL[driving()];
872 $("stop").hidden = !label || !(mayDrive() || isOwner());
873 if (label) $("stop").textContent = label;
874 }
875 // The emergency stop: the server halts whatever drives the arm and holds it with torque on until an admin clears it.
876 async function eStop() {
877 driver.releaseAll();
878 const r = await api("/estop", {});
879 msg(r.ok ? "EMERGENCY STOP: the arm holds; an admin must clear it" : r.error, true);
880 if (stream.ws) stream.ws.close();
881 stopLeaderDrive("emergency stop");
886 // On the press, not the release; a keyboard press is a click with no detail.
887 $("stop").addEventListener("pointerdown", (e) => { if (e.button === 0 && !$("stop").disabled) { e.preventDefault(); stop(); } });
888 $("stop").addEventListener("click", (e) => { if (e.detail === 0) stop(); });
889 // Resume takes a second tap.
890 let resumeArmed = null;
891 async function resume(b) {
892 if (!resumeArmed) {
893 b.textContent = "Tap again to resume";
894 resumeArmed = setTimeout(() => { resumeArmed = null; b.textContent = "Resume driving"; }, 3000);
895 return;
896 }
897 clearTimeout(resumeArmed); resumeArmed = null;
898 const r = await api("/resume", {});
899 if (r.ok) stream.halted = false;
900 msg(r.ok ? "resumed: the arm stays where it is until you move it" : r.error, !r.ok);
882901 }
883902 async function clearEStop() {
884903 if (!confirm("Clear the emergency stop? The arm stays where it is; the driver can move it again.")) return;
885904 const r = await api("/estop/clear", {});
886905 msg(r.ok ? "the emergency stop is cleared" : r.error, !r.ok);
887906 }
888 $("estop").onclick = eStop;
889907 $("hold").onclick = async () => { const r = await api("/torque", { on: true }); msg(r.ok ? "holding: torque on" : r.error, !r.ok); };
890908 $("releaseBtn").onclick = async () => {
891909 if (!confirm("Release torque? Hold the arm so it does not fall.")) return;
@@ -1352,7 +1370,7 @@
13521370 flush: (f) => stream.flush(f),
13531371 end: () => stream.end(),
13541372 begin: () => {
1355 if (!canDrive({ mayDrive: mayDrive(), estop: status?.estop, reconnecting: status?.reconnecting, limits })) return false;
1373 if (!canDrive({ mayDrive: mayDrive(), estop: status?.estop, held: status?.held, reconnecting: status?.reconnecting, limits })) return false;
13561374 stream.open();
13571375 return true;
13581376 },
@@ -1475,7 +1493,7 @@
14751493 addEventListener("keydown", (e) => {
14761494 if (e.target.closest("input, select, textarea, dialog")) return;
14771495 if (e.key === "f" || e.key === "F") fullscreen();
1478 if (e.key === "Escape" && (mayDrive() || isOwner())) { e.preventDefault(); eStop(); }
1496 if (e.key === "Escape" && (mayDrive() || isOwner())) { e.preventDefault(); stop(); }
14791497 if (e.key === " " && (mayDrive() || isOwner())) { e.preventDefault(); if (e.target instanceof HTMLButtonElement) e.target.blur(); stop(); }
14801498 });
14811499 for (const b of document.querySelectorAll("[data-close]")) b.onclick = () => { $(b.dataset.close).hidden = true; };
@@ -1553,7 +1571,7 @@
15531571 const say = (m) => { if (lobby.ws?.readyState === 1) lobby.ws.send(JSON.stringify(m)); };
15541572 let drove = false;
15551573 function applyLock() {
1556 const can = mayDrive() && !status?.estop, owner = isOwner();
1574 const can = mayDrive() && !status?.estop && !status?.held, owner = isOwner();
15571575 $("t-controls").hidden = !mayDrive();
15581576 $("t-admin").hidden = !owner;
15591577 $("t-chat").hidden = $("t-queue").hidden = !relayed;
@@ -1561,9 +1579,8 @@
15611579 shell.classList.toggle("driving", can);
15621580 if (!can) driver.releaseAll();
15631581 wake(can);
1564 showStop();
1565 $("estop").disabled = !(mayDrive() || owner);
1566 $("estop").title = $("estop").disabled ? "Emergency stop: the driver and the admins can press it" : "Emergency stop (Esc): halt, hold with torque, until an admin clears it";
1582 $("stop").disabled = !(mayDrive() || owner);
1583 $("stop").title = $("stop").disabled ? "Stop: the driver and the admins can press it" : "Stop (space or Esc): halt everything, hold with torque, until someone resumes";
15671584 // The server takes settings, the safety limits among them, from admins only.
15681585 for (const id of ["zoneRow", "trims", "safety", "jointLimits"]) $(id).inert = !owner;
15691586 viewer.toggleAttribute("disable-dragging", !can);
@@ -1893,11 +1910,22 @@
18931910 return `${part} is ${(-beyond).toFixed(0)} mm ${v.what === "table" ? "above the table" : "from the base"}, inside the ${marginMm} mm margin`;
18941911 };
18951912 if (d.estop) {
1913 b.dataset.held = "";
18961914 b.className = "stop";
18971915 b.replaceChildren(`EMERGENCY STOP by ${d.estop.by} at ${new Date(d.estop.at).toLocaleTimeString()}. The arm holds with torque on until an admin clears it.`);
18981916 if (isOwner()) b.append(el("button", { textContent: "Clear the stop", className: "danger", onclick: clearEStop }));
18991917 return;
19001918 }
1919 // Built once per hold: a button replaced under a finger is never clicked.
1920 const key = d.held ? `${d.held.by} ${d.held.at} ${mayDrive() || isOwner()}` : "";
1921 if (key && b.dataset.held === key) return;
1922 b.dataset.held = key;
1923 if (d.held) {
1924 b.className = "stop";
1925 b.replaceChildren(`STOPPED by ${d.held.by} at ${new Date(d.held.at).toLocaleTimeString()}. The arm holds where it is; nothing sent before a resume will move it.`);
1926 if (mayDrive() || isOwner()) b.append(el("button", { id: "resume", textContent: resumeArmed ? "Tap again to resume" : "Resume driving", className: "go", onclick: (e) => resume(e.currentTarget) }));
1927 return;
1928 }
19011929 if (d.reconnecting) { b.className = "warn"; b.textContent = `Reconnecting to the arm… showing where it was at ${new Date(d.seenAt).toLocaleTimeString()}. Driving is paused, and the arm holds where it is.`; return; }
19021930 if (typeof d.source === "string" && d.source.startsWith("simulated: offline")) { b.className = "bad"; b.textContent = "The arm has been unreachable for over a minute. This is the SIMULATION, not the real arm."; return; }
19031931 if (d.jammed?.length) { b.className = "bad"; b.textContent = `jam: ${d.jammed.map((id) => NAMES[id] ?? id).join(", ")} pushed against something and stopped — back the leader off`; }
@@ -1912,12 +1940,12 @@
19121940 function applyStatus(d) {
19131941 if (!started) return void (early = d);
19141942 {
1915 const wasStopped = Boolean(status?.estop);
1943 const wasStopped = Boolean(status?.estop || status?.held);
19161944 status = d;
19171945 driver.seed(d.joints.filter((j) => j.position != null).map((j) => [j.id, j.position]));
1918 if (Boolean(d.estop) !== wasStopped) applyLock();
1946 if (Boolean(d.estop || d.held) !== wasStopped) applyLock();
19191947 // Stopped, reconnecting or no longer this person's turn: a held thumb lets go, and its zero goes out now.
1920 if (!canDrive({ mayDrive: mayDrive(), estop: d.estop, reconnecting: d.reconnecting, limits })) driver.releaseAll();
1948 if (!canDrive({ mayDrive: mayDrive(), estop: d.estop, held: d.held, reconnecting: d.reconnecting, limits })) driver.releaseAll();
19211949 // The round trip to the arm, which every command also makes: shown once it is long enough to feel.
19221950 const lag = [...rtts].sort((a, b) => a - b)[rtts.length >> 1] ?? 0;
19231951 $("lag").hidden = rtts.length < 3 || lag <= 250; $("lag").textContent = `· ${Math.round(lag)} ms lag`;
@@ -1928,8 +1956,7 @@
19281956 teleop.observe(Object.fromEntries(d.joints.filter((j) => typeof j.position === "number").map((j) => [j.id, j.position])), (started + performance.now()) / 2);
19291957 renderTracks(); renderLag();
19301958 }
1931 $("estop").classList.toggle("on", Boolean(d.estop)); $("estop").textContent = d.estop ? "Stopped" : "Stop";
1932 showStop();
1959 $("stop").classList.toggle("on", Boolean(d.estop || d.held)); $("stop").textContent = d.estop || d.held ? "Stopped" : "Stop";
19331960 $("record").textContent = d.driving === "record" ? "■ Stop recording" : "● Record";
19341961 for (const id of ["hold", "releaseBtn", "play"]) $(id).disabled = d.driving !== null;
19351962 if (tab === "controls") { render(d); renderTouch(d); }

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs