Public: anyone can read this; the crew can change it.

Migrated from github.com/Trevato/SO-ARM100

workbenchthis machine

Watch anonymously, sign in to drive: one driver at a time, a visible queue, and a chat beside the arm #6

Merged opened this in answer to

Check · Review · Land done · Live — Landed; nothing deploys from this branch, so this is the end of its path.

Merged

quill/arm:quill/5-watch-anonymously-sign-in-to-drive-one-d → main 2 commits fork

  • Merged. quill/5-watch-anonymously-sign-in-to-drive-one-d was merged into main as cfdb4ea.

Review

Sign in to review this change.
13 files changed +1244 −9
Software/README.md +30
@@ -188,6 +188,36 @@ error; it is not replayed on the simulation. Changes (`POST`, `PUT`, the
188188 stream) are only accepted from the app's own page, so another site can't
189189 drive the arm through it.
190190
191 **Watch, take turns, chat.** Anyone with the link can watch: the 3D arm,
192 its live pose and the cameras. Driving needs a forge account and a turn.
193
194 - **Take control** (after **Sign in to take control**) joins the queue in
195 the panel beside the arm. When nobody is driving, you drive at once.
196 - A turn lasts 3 minutes, with a countdown. It ends early when you press
197 **Hand back**, after 30 s with no command, or 10 s after your connection
198 drops. The next person in the queue then drives. Someone in the queue
199 whose connection drops for 10 s leaves it.
200 - Everyone sees the queue live: who drives, who is next, each place, and
201 the longest you might wait. Your own place is highlighted.
202 - The server decides who drives. Any command (a move, a pose, a recording,
203 the torque, the settings, the `/follow` stream) from anyone else is
204 refused before it reaches the arm. When a turn ends, its stream is closed
205 and whatever it started is stopped.
206 - Chat: everyone reads. Signed-in people post one message every 2 s, up to
207 500 characters. The last 200 messages are kept in
208 `$FORGE_APP_DATA/so101/chat.json`. Driver changes appear as grey lines.
209 - The owner (anyone who may write this repository) can take control at
210 any time (this skips the queue), end anyone's turn, take people out of
211 the queue, delete messages and mute people. The owner can also press
212 **Stop** during anyone's turn.
213
214 The queue, the countdown and the chat all come over one WebSocket (`/live`):
215 no polling. The forge says who someone is with a signed `x-forge-identity`
216 header. The app checks that signature with `FORGE_APP_IDENTITY_KEY` and
217 trusts nothing else (`src/identity.ts`). Run locally (no `FORGE_APP_ENV`,
218 no key), nobody signs in and every visitor drives, as before. Anonymous
219 viewing needs the app set to public on the forge.
220
191221 **The safe zone.** The server loads `Simulation/SO101/so101_new_calib.urdf`
192222 and its meshes (`--model <dir>`, or the repo layout, or `model/` beside the
193223 package) and runs every target through the arm's geometry: a pose that puts
Software/so101/bench/fake-serve.ts +8 −2
@@ -5,12 +5,15 @@
55 * PORT, HOST and FORGE_APP_DATA override the port, the 127.0.0.1 bind and the temp dir (how the forge runs it).
66 * ARM_SERVER=ws://<host>:8101 puts a real arm's `so101 serve` behind the same page: the relay in src/relay.ts
77 * sends the arm API, /follow and the cameras there while it answers, and to the simulation when it does not.
8 * Anyone may watch; only the signed-in person whose turn it is may drive (src/lobby.ts). The forge signs who that is
9 * with FORGE_APP_IDENTITY_KEY. Run on one's own machine, away from the forge and with no key, every visitor is the owner.
810 */
911 import { mkdirSync, mkdtempSync } from "node:fs";
1012 import { tmpdir } from "node:os";
1113 import { join } from "node:path";
1214 import { FeetechBus } from "../src/bus.ts";
1315 import { FakeBus } from "../src/fake-bus.ts";
16 import { Lobby } from "../src/lobby.ts";
1417 import { parseRemote, startRelay, type Remote } from "../src/relay.ts";
1518 import { startServer } from "../src/server.ts";
1619
@@ -34,6 +37,9 @@ try {
3437 problem = (err as Error).message;
3538 console.error(`${problem}; simulating`);
3639 }
37 const relay = await startRelay({ host, port, simulation: `http://127.0.0.1:${simulation.port}`, remote, problem });
40 const identityKey = process.env.FORGE_APP_IDENTITY_KEY ?? "";
41 const local = !identityKey && !process.env.FORGE_APP_ENV;
42 const lobby = new Lobby({ identityKey, local, chatFile: join(dir, "chat.json") });
43 const relay = await startRelay({ host, port, simulation: `http://127.0.0.1:${simulation.port}`, remote, problem, lobby });
3844 const { mode, host: armHost } = relay.link();
39 console.log(`arm on http://${host}:${relay.port} (${mode === "live" ? `live arm ${armHost}` : "simulated"}; data in ${dir})`);
45 console.log(`arm on http://${host}:${relay.port} (${mode === "live" ? `live arm ${armHost}` : "simulated"}; data in ${dir}${local ? "; local: every visitor drives as the owner" : ""})`);
Software/so101/src/chat.test.ts added +56
@@ -0,0 +1,56 @@
1 import { describe, expect, test } from "bun:test";
2 import { mkdtempSync } from "node:fs";
3 import { tmpdir } from "node:os";
4 import { join } from "node:path";
5 import { Chat, KEEP, MAX_CHARS } from "./chat.ts";
6
7 const alice = { id: "a", name: "alice", owner: false };
8 const bob = { id: "b", name: "bob", owner: false };
9
10 describe("chat", () => {
11 test("one message per 2 s per person", () => {
12 let clock = 0;
13 const chat = new Chat({ now: () => clock });
14 chat.post(alice, "hi");
15 expect(() => chat.post(alice, "again")).toThrow(/one message every 2 s/);
16 chat.post(bob, "hello"); // someone else is not held up
17 clock += 1999;
18 expect(() => chat.post(alice, "again")).toThrow(/one message/);
19 clock += 1;
20 expect(chat.post(alice, "again").text).toBe("again");
21 });
22
23 test("500 characters, no empty lines, no control characters", () => {
24 let clock = 0;
25 const chat = new Chat({ now: () => (clock += 5000) });
26 expect(chat.post(alice, "é".repeat(MAX_CHARS)).text.length).toBe(MAX_CHARS);
27 expect(() => chat.post(alice, "x".repeat(MAX_CHARS + 1))).toThrow(/500/);
28 expect(() => chat.post(alice, " \n ")).toThrow(/say something/);
29 expect(() => chat.post(alice, 42)).toThrow(/text/);
30 expect(chat.post(alice, "a\u0000b\nc\u202Ed").text).toBe("ab cd");
31 });
32
33 test("the last 200 lines are kept, and survive a restart with the mutes", () => {
34 const file = join(mkdtempSync(join(tmpdir(), "so101-chat-")), "chat.json");
35 let clock = 0;
36 const chat = new Chat({ file, now: () => (clock += 5000) });
37 for (let i = 0; i < KEEP + 5; i++) chat.post(alice, `line ${i}`);
38 chat.mute("b", "bob", true);
39 expect(chat.messages.length).toBe(KEEP);
40 const again = new Chat({ file });
41 expect(again.messages.length).toBe(KEEP);
42 expect(again.messages[0]!.text).toBe("line 5");
43 expect(again.muted.get("b")).toBe("bob");
44 expect(again.system("x").id).toBe(chat.messages.at(-1)!.id + 1);
45 });
46
47 test("a muted person cannot post; a deleted line is gone", () => {
48 const chat = new Chat();
49 const line = chat.post(alice, "spam");
50 chat.mute("a", "alice", true);
51 expect(() => chat.post(alice, "more")).toThrow(/muted/);
52 expect(chat.delete(line.id)).toBe(true);
53 expect(chat.delete(line.id)).toBe(false);
54 expect(chat.messages).toEqual([]);
55 });
56 });
Software/so101/src/chat.ts added +118
@@ -0,0 +1,118 @@
1 /**
2 * The chat beside the arm: everyone reads, signed-in people post, the owner deletes lines and mutes people.
3 * One line per `rateMs` per person, `MAX_CHARS` characters, the last `KEEP` lines kept; driver changes are
4 * system lines. Text is stored as typed (minus control characters) and only ever shown as text.
5 * Kept in one JSON file beside the arm's data, rewritten whole on each change (it is a few dozen KB at most).
6 */
7
8 import { mkdirSync, readFileSync, renameSync, writeFileSync } from "node:fs";
9 import { dirname } from "node:path";
10 import type { Person } from "./identity.ts";
11
12 export const MAX_CHARS = 500;
13 export const KEEP = 200;
14
15 export interface Message {
16 id: number;
17 at: number;
18 /** null for a system line. */
19 from: { id: string; name: string } | null;
20 text: string;
21 }
22
23 export class ChatError extends Error {}
24
25 export interface ChatOptions {
26 /** Where the log and the mutes live; none keeps them in memory. */
27 file?: string;
28 rateMs?: number;
29 now?: () => number;
30 }
31
32 /** Control characters and the bidi overrides that could turn a line around on screen. */
33 const UNPRINTABLE = /[\u0000-\u0008\u000b-\u001f\u007f-\u009f\u200e\u200f\u202a-\u202e\u2066-\u2069]/g;
34
35 export function cleanText(text: unknown): string {
36 if (typeof text !== "string") throw new ChatError("a message is text");
37 const clean = text.replace(/\r\n?|\n|\t/g, " ").replace(UNPRINTABLE, "").trim();
38 if (!clean) throw new ChatError("say something");
39 if ([...clean].length > MAX_CHARS) throw new ChatError(`at most ${MAX_CHARS} characters`);
40 return clean;
41 }
42
43 export class Chat {
44 messages: Message[] = [];
45 /** id → name of everyone muted. */
46 readonly muted = new Map<string, string>();
47 private next = 1;
48 private readonly lastPost = new Map<string, number>();
49 private readonly file: string | undefined;
50 private readonly rateMs: number;
51 private readonly now: () => number;
52
53 constructor(options: ChatOptions = {}) {
54 this.file = options.file;
55 this.rateMs = options.rateMs ?? 2000;
56 this.now = options.now ?? Date.now;
57 if (this.file) this.load(this.file);
58 }
59
60 post(who: Person, text: unknown): Message {
61 if (this.muted.has(who.id)) throw new ChatError("you are muted");
62 const clean = cleanText(text);
63 const now = this.now();
64 const last = this.lastPost.get(who.id);
65 if (last !== undefined && now - last < this.rateMs) throw new ChatError(`one message every ${this.rateMs / 1000} s`);
66 this.lastPost.set(who.id, now);
67 return this.add({ id: who.id, name: who.name }, clean);
68 }
69
70 system(text: string): Message {
71 return this.add(null, text);
72 }
73
74 delete(id: number): boolean {
75 const before = this.messages.length;
76 this.messages = this.messages.filter((m) => m.id !== id);
77 if (this.messages.length === before) return false;
78 this.save();
79 return true;
80 }
81
82 mute(id: string, name: string, on: boolean): void {
83 if (on) this.muted.set(id, name);
84 else this.muted.delete(id);
85 this.save();
86 }
87
88 private add(from: Message["from"], text: string): Message {
89 const message = { id: this.next++, at: this.now(), from, text };
90 this.messages.push(message);
91 if (this.messages.length > KEEP) this.messages.splice(0, this.messages.length - KEEP);
92 this.save();
93 return message;
94 }
95
96 private load(file: string): void {
97 let stored: { messages?: unknown; muted?: unknown };
98 try {
99 stored = JSON.parse(readFileSync(file, "utf8"));
100 } catch {
101 return; // none yet, or unreadable: start over rather than refuse to start
102 }
103 if (Array.isArray(stored.messages)) {
104 this.messages = stored.messages
105 .filter((m): m is Message => typeof m?.id === "number" && typeof m.at === "number" && typeof m.text === "string" && (m.from === null || (typeof m.from?.id === "string" && typeof m.from?.name === "string")))
106 .slice(-KEEP);
107 }
108 if (stored.muted && typeof stored.muted === "object") for (const [id, name] of Object.entries(stored.muted)) if (typeof name === "string") this.muted.set(id, name);
109 this.next = Math.max(0, ...this.messages.map((m) => m.id)) + 1;
110 }
111
112 private save(): void {
113 if (!this.file) return;
114 mkdirSync(dirname(this.file), { recursive: true });
115 writeFileSync(`${this.file}.tmp`, JSON.stringify({ messages: this.messages, muted: Object.fromEntries(this.muted) }));
116 renameSync(`${this.file}.tmp`, this.file);
117 }
118 }
Software/so101/src/fake-serve.test.ts +11 −1
@@ -30,7 +30,7 @@ async function start(env: Record<string, string>): Promise<{ host: string; port:
3030
3131 test("fake-serve listens on PORT and HOST and keeps its data under FORGE_APP_DATA", async () => {
3232 dir = mkdtempSync(join(tmpdir(), "so101-appdata-"));
33 const { host, port, line } = await start({ PORT: "0", HOST: "127.0.0.1", FORGE_APP_DATA: dir });
33 const { host, port, line } = await start({ PORT: "0", HOST: "127.0.0.1", FORGE_APP_DATA: dir, FORGE_APP_ENV: "", FORGE_APP_IDENTITY_KEY: "" });
3434 expect(host).toBe("127.0.0.1");
3535 expect(port).toBeGreaterThan(0);
3636 expect(line).toContain(join(dir, "so101"));
@@ -41,3 +41,13 @@ test("fake-serve listens on PORT and HOST and keeps its data under FORGE_APP_DAT
4141 expect(saved.status).toBe(200);
4242 expect(existsSync(join(dir, "so101", "poses.json"))).toBe(true);
4343 }, 20_000);
44
45 test("on the forge, an anonymous visitor watches but cannot command the arm", async () => {
46 dir = mkdtempSync(join(tmpdir(), "so101-appdata-"));
47 const { port, line } = await start({ PORT: "0", HOST: "127.0.0.1", FORGE_APP_DATA: dir, FORGE_APP_ENV: "preview", FORGE_APP_IDENTITY_KEY: "a key" });
48 expect(line).not.toContain("local");
49 expect((await fetch(`http://127.0.0.1:${port}/status`)).status).toBe(200);
50 const refused = await fetch(`http://127.0.0.1:${port}/pose/save`, { method: "POST", body: JSON.stringify({ name: "here" }) });
51 expect(refused.status).toBe(401);
52 expect(existsSync(join(dir, "so101", "poses.json"))).toBe(false);
53 }, 20_000);
Software/so101/src/identity.test.ts added +33
@@ -0,0 +1,33 @@
1 import { describe, expect, test } from "bun:test";
2 import { identify, sign } from "./identity.ts";
3
4 const KEY = "k".repeat(64);
5 const later = Date.now() + 60_000;
6 const signed = (identity: string, key = KEY) => ({ "x-forge-identity": identity, "x-forge-signature": sign(identity, key) });
7
8 describe("identify", () => {
9 test("a header the forge signed names the person; admin is the owner", () => {
10 expect(identify(signed(`u1|alice|user|${later}`), KEY)).toEqual({ id: "u1", name: "alice", owner: false });
11 expect(identify(signed(`u2|trevato|admin|${later}`), KEY)).toEqual({ id: "u2", name: "trevato", owner: true });
12 });
13
14 test("no headers, no key, or a header alone is nobody", () => {
15 expect(identify({}, KEY)).toBeNull();
16 expect(identify(signed(`u1|alice|user|${later}`), "")).toBeNull();
17 expect(identify({ "x-forge-identity": `u1|alice|admin|${later}` }, KEY)).toBeNull();
18 });
19
20 test("a forged or altered identity is nobody", () => {
21 expect(identify(signed(`u1|alice|admin|${later}`, "another key"), KEY)).toBeNull();
22 const real = signed(`u1|alice|user|${later}`);
23 expect(identify({ ...real, "x-forge-identity": `u1|alice|admin|${later}` }, KEY)).toBeNull();
24 expect(identify({ ...real, "x-forge-signature": "nope" }, KEY)).toBeNull();
25 });
26
27 test("an expired stamp, an unknown role or an odd field is nobody", () => {
28 expect(identify(signed(`u1|alice|user|${Date.now() - 1}`), KEY)).toBeNull();
29 expect(identify(signed(`u1|alice|owner|${later}`), KEY)).toBeNull();
30 expect(identify(signed(`u1|<b>|user|${later}`), KEY)).toBeNull();
31 expect(identify(signed(`u1|alice|user|${later}|x`), KEY)).toBeNull();
32 });
33 });
Software/so101/src/identity.ts added +51
@@ -0,0 +1,51 @@
1 /**
2 * Who is asking. The forge signs every request it proxies to the app, the WebSocket handshake included:
3 *
4 * x-forge-identity: <userId>|<username>|<role>|<expiresMs>
5 * x-forge-signature: base64url(HMAC-SHA256(FORGE_APP_IDENTITY_KEY, that string))
6 *
7 * `role` is `user` (signed in) or `admin` (may write the app's repository: the app's owner here). The forge
8 * strips both headers from whatever a client sends, so a signature that verifies is the only trust there is:
9 * no headers, no key, a bad signature, an odd field or a stamp in the past all mean nobody.
10 */
11
12 import { createHmac, timingSafeEqual } from "node:crypto";
13 import type { IncomingHttpHeaders } from "node:http";
14
15 export interface Person {
16 id: string;
17 name: string;
18 /** May override the queue, end turns, delete chat lines and mute people. */
19 owner: boolean;
20 }
21
22 /** A signed-in person, or null for an anonymous visitor. */
23 export type Who = Person | null;
24
25 /** Everyone at the keyboard of a local, forge-less run: there is nobody to sign in with, and nobody else. */
26 export const LOCAL: Person = { id: "local", name: "local", owner: true };
27
28 const NAME = /^[A-Za-z0-9][A-Za-z0-9_.-]{0,63}$/;
29 const ID = /^[A-Za-z0-9_-]{1,64}$/;
30
31 /** The signature the forge computes. Exported so a test can be the forge. */
32 export const sign = (identity: string, key: string): string => createHmac("sha256", key).update(identity).digest("base64url");
33
34 const same = (a: string, b: string): boolean => {
35 const x = Buffer.from(a);
36 const y = Buffer.from(b);
37 return x.length === y.length && timingSafeEqual(x, y);
38 };
39
40 export function identify(headers: IncomingHttpHeaders, key: string, now = Date.now()): Who {
41 const identity = headers["x-forge-identity"];
42 const signature = headers["x-forge-signature"];
43 if (!key || typeof identity !== "string" || typeof signature !== "string") return null;
44 if (!same(sign(identity, key), signature)) return null;
45 const parts = identity.split("|");
46 if (parts.length !== 4) return null;
47 const [id, name, role, expires] = parts as [string, string, string, string];
48 if (role !== "user" && role !== "admin") return null;
49 if (!ID.test(id) || !NAME.test(name) || !/^\d{1,16}$/.test(expires) || Number(expires) <= now) return null;
50 return { id, name, owner: role === "admin" };
51 }
Software/so101/src/lobby.test.ts added +206
@@ -0,0 +1,206 @@
1 import { afterAll, afterEach, beforeAll, describe, expect, test } from "bun:test";
2 import { mkdtempSync } from "node:fs";
3 import { tmpdir } from "node:os";
4 import { join } from "node:path";
5 import WebSocket from "ws";
6 import { FeetechBus } from "./bus.ts";
7 import { Chat } from "./chat.ts";
8 import { FakeBus } from "./fake-bus.ts";
9 import { sign } from "./identity.ts";
10 import { Lobby } from "./lobby.ts";
11 import { startRelay, type RunningRelay } from "./relay.ts";
12 import { startServer, type RunningServer } from "./server.ts";
13 import { Turns } from "./turns.ts";
14
15 const KEY = "test-identity-key";
16 const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
17 const shut = (s: { close(): Promise<void> } | undefined) => (s ? Promise.race([s.close(), sleep(500)]) : undefined);
18 const as = (id: string, name: string, role = "user") => {
19 const identity = `${id}|${name}|${role}|${Date.now() + 3_600_000}`;
20 return { "x-forge-identity": identity, "x-forge-signature": sign(identity, KEY) };
21 };
22 const ALICE = as("u-alice", "alice");
23 const BOB = as("u-bob", "bob");
24 const OWNER = as("u-owner", "trevato", "admin");
25
26 type Msg = Record<string, any>;
27 /** A /live client that keeps everything it heard and can wait for the next thing that matches. */
28 async function client(port: number, headers: Record<string, string> = {}) {
29 const ws = new WebSocket(`ws://127.0.0.1:${port}/live`, { headers });
30 const heard: Msg[] = [];
31 const waiters: Array<{ match: (m: Msg) => boolean; resolve: (m: Msg) => void }> = [];
32 ws.on("message", (data) => {
33 const m = JSON.parse(String(data)) as Msg;
34 heard.push(m);
35 for (const w of [...waiters]) if (w.match(m)) waiters.splice(waiters.indexOf(w), 1), w.resolve(m);
36 });
37 await new Promise((r, j) => (ws.once("open", r), ws.once("error", j)));
38 const next = (match: (m: Msg) => boolean, ms = 2000) =>
39 new Promise<Msg>((resolve, reject) => {
40 waiters.push({ match, resolve });
41 setTimeout(() => reject(new Error(`nothing matched in ${ms} ms; heard ${JSON.stringify(heard.slice(-5))}`)), ms);
42 });
43 const send = (m: Msg) => ws.send(JSON.stringify(m));
44 const last = (type: string) => heard.filter((m) => m.type === type).at(-1);
45 /** What the server said on arrival, already heard or still coming. */
46 const first = (match: (m: Msg) => boolean) => Promise.resolve(heard.find(match) ?? next(match));
47 return { ws, heard, next, first, send, last };
48 }
49 const drivingIs = (name: string | null) => (m: Msg) => m.type === "state" && (m.driver?.name ?? null) === name;
50
51 let simulation: RunningServer;
52 beforeAll(async () => {
53 simulation = await startServer(new FeetechBus(new FakeBus([1, 2, 6])), { host: "127.0.0.1", port: 0, ids: [1, 2, 6], posesFile: join(mkdtempSync(join(tmpdir(), "so101-lobby-")), "poses.json"), watchdogMs: 60 });
54 });
55 afterAll(() => shut(simulation));
56
57 describe("lobby", () => {
58 let relay: RunningRelay | undefined;
59 let lobby: Lobby | undefined;
60 const sockets: WebSocket[] = [];
61 afterEach(async () => {
62 for (const s of sockets.splice(0)) s.terminate();
63 await shut(relay);
64 lobby?.close();
65 relay = lobby = undefined;
66 });
67 const start = async (turns: Turns = new Turns(), chat = new Chat()) => {
68 lobby = new Lobby({ identityKey: KEY, turns, chat });
69 relay = await startRelay({ port: 0, simulation: `http://127.0.0.1:${simulation.port}`, lobby });
70 return relay.port;
71 };
72 const join = async (port: number, headers: Record<string, string> = {}) => {
73 const c = await client(port, headers);
74 sockets.push(c.ws);
75 return c;
76 };
77 const torque = (port: number, headers: Record<string, string> = {}) =>
78 fetch(`http://127.0.0.1:${port}/torque`, { method: "POST", headers: { "content-type": "application/json", ...headers }, body: '{"on":true}' });
79 /** How a /follow stream goes: "open", or the close code and reason it is refused with. */
80 const follow = (port: number, headers: Record<string, string> = {}) =>
81 new Promise<string>((resolve) => {
82 const ws = new WebSocket(`ws://127.0.0.1:${port}/follow`, { headers });
83 sockets.push(ws);
84 const timer = setTimeout(() => resolve("open"), 150); // a refused stream is closed at once
85 ws.once("close", (code, why) => (clearTimeout(timer), resolve(`${code} ${why}`)));
86 ws.once("error", () => (clearTimeout(timer), resolve("error")));
87 });
88
89 test("anyone watches; reads pass, commands from anonymous visitors are refused", async () => {
90 const port = await start();
91 const anon = await join(port);
92 expect((await anon.first((m) => m.type === "hello")).you).toBeNull();
93 expect(anon.last("state")).toMatchObject({ driver: null, queue: [] });
94 expect((await fetch(`http://127.0.0.1:${port}/status`)).status).toBe(200);
95 const refused = await torque(port);
96 expect(refused.status).toBe(401);
97 expect(((await refused.json()) as { error: string }).error).toContain("sign in");
98 expect(await follow(port)).toBe("4401 sign in to take control");
99 anon.send({ type: "join" });
100 expect((await anon.next((m) => m.type === "error")).error).toContain("sign in");
101 });
102
103 test("a forged identity header is nobody", async () => {
104 const port = await start();
105 const forged = { "x-forge-identity": `u-owner|trevato|admin|${Date.now() + 60_000}`, "x-forge-signature": sign(`u-owner|trevato|admin|${Date.now() + 60_000}`, "not the key") };
106 expect((await torque(port, forged)).status).toBe(401);
107 expect((await torque(port, { "x-forge-identity": ALICE["x-forge-identity"] })).status).toBe(401);
108 const c = await join(port, forged);
109 expect((await c.first((m) => m.type === "hello")).you).toBeNull();
110 });
111
112 test("two signed in: one drives, the other queues, and takes over on hand-back", async () => {
113 const port = await start();
114 const alice = await join(port, ALICE);
115 const bob = await join(port, BOB);
116 alice.send({ type: "join" });
117 await bob.next(drivingIs("alice"));
118 bob.send({ type: "join" });
119 const queued = await alice.next((m) => m.type === "state" && m.queue.length === 1);
120 expect(queued.queue[0]).toMatchObject({ name: "bob", place: 1 });
121 expect(queued.queue[0].waitMs).toBeGreaterThan(170_000);
122
123 expect((await torque(port, ALICE)).status).toBe(200);
124 const refused = await torque(port, BOB);
125 expect(refused.status).toBe(403);
126 expect(((await refused.json()) as { error: string }).error).toContain("alice has control");
127 expect(await follow(port, BOB)).toStartWith("4403 alice has control");
128 expect(await follow(port, ALICE)).toBe("open");
129
130 alice.send({ type: "leave" });
131 await bob.next(drivingIs("bob"));
132 expect((await torque(port, BOB)).status).toBe(200);
133 expect((await torque(port, ALICE)).status).toBe(403);
134 expect(bob.heard.some((m) => m.type === "said" && m.message.from === null && /alice's turn ended \(handed back\); bob has control/.test(m.message.text))).toBe(true);
135 });
136
137 test("an idle driver loses the turn to the next in line, and their stream is closed", async () => {
138 const port = await start(new Turns({ idleMs: 300 }));
139 const alice = await join(port, ALICE);
140 const bob = await join(port, BOB);
141 alice.send({ type: "join" });
142 await bob.next(drivingIs("alice"));
143 bob.send({ type: "join" });
144 await alice.next((m) => m.type === "state" && m.queue.length === 1);
145 const stream = new WebSocket(`ws://127.0.0.1:${port}/follow`, { headers: ALICE });
146 sockets.push(stream);
147 await new Promise((r) => stream.once("open", r));
148 const closed = new Promise<string>((r) => stream.once("close", (_code, why) => r(why.toString())));
149 await bob.next(drivingIs("bob"), 2000);
150 expect(await closed).toBe("your turn ended");
151 expect(bob.heard.some((m) => m.type === "said" && /\(idle\)/.test(m.message.text))).toBe(true);
152 });
153
154 test("a dropped driver's turn passes on after the grace period", async () => {
155 const port = await start(new Turns({ graceMs: 200 }));
156 const alice = await join(port, ALICE);
157 const bob = await join(port, BOB);
158 alice.send({ type: "join" });
159 bob.send({ type: "join" });
160 await bob.next((m) => m.type === "state" && m.queue.length === 1);
161 alice.ws.terminate();
162 await bob.next(drivingIs("bob"), 2000);
163 });
164
165 test("the owner jumps the queue and ends turns", async () => {
166 const port = await start();
167 const alice = await join(port, ALICE);
168 const owner = await join(port, OWNER);
169 alice.send({ type: "join" });
170 await owner.next(drivingIs("alice"));
171 alice.send({ type: "end", id: "u-owner" });
172 expect((await alice.next((m) => m.type === "error")).error).toContain("only the owner");
173 owner.send({ type: "join" });
174 await alice.next(drivingIs("trevato"));
175 expect((await torque(port, OWNER)).status).toBe(200);
176 alice.send({ type: "join" });
177 await owner.next((m) => m.type === "state" && m.queue.length === 1);
178 owner.send({ type: "end", id: "u-owner" });
179 await alice.next(drivingIs("alice"));
180 owner.send({ type: "end", id: "u-alice" });
181 await alice.next(drivingIs(null));
182 });
183
184 test("chat: everyone reads, signed-in people post once per 2 s, the owner deletes and mutes", async () => {
185 const port = await start();
186 const anon = await join(port);
187 const alice = await join(port, ALICE);
188 const owner = await join(port, OWNER);
189 alice.send({ type: "post", text: "hello <b>arm</b>" });
190 const said = await anon.next((m) => m.type === "said");
191 expect(said.message).toMatchObject({ from: { id: "u-alice", name: "alice" }, text: "hello <b>arm</b>" });
192 alice.send({ type: "post", text: "again" });
193 expect((await alice.next((m) => m.type === "error")).error).toContain("one message every 2 s");
194 anon.send({ type: "post", text: "me too" });
195 expect((await anon.next((m) => m.type === "error")).error).toContain("sign in");
196
197 owner.send({ type: "delete", id: said.message.id });
198 expect((await anon.next((m) => m.type === "unsaid")).id).toBe(said.message.id);
199 owner.send({ type: "mute", id: "u-alice", on: true });
200 expect((await alice.next((m) => m.type === "hello")).muted).toBe(true);
201 expect((await owner.next((m) => m.type === "muted")).muted).toEqual([{ id: "u-alice", name: "alice" }]);
202
203 const late = await join(port);
204 expect((await late.first((m) => m.type === "chat")).messages).toEqual([]);
205 });
206 });
Software/so101/src/lobby.ts added +226
@@ -0,0 +1,226 @@
1 /**
2 * The room around the arm, over one WebSocket (/live): who drives, who waits, and the chat. Everyone may
3 * watch; a signed-in person may ask to drive and may post; the owner may override. The relay asks the lobby
4 * who is behind each request (`who`) and whether they may command the arm (`refusal`).
5 *
6 * Client → server, one JSON object per frame:
7 * {"type": "join"} ask to drive (the owner takes over at once)
8 * {"type": "leave"} hand back the turn, or leave the line
9 * {"type": "post", "text": "…"} say something
10 * {"type": "end", "id": "<user id>"} owner: end that turn, or take them out of the line
11 * {"type": "delete", "id": 12} owner: delete a chat line
12 * {"type": "mute", "id": "<user id>", "on": true|false} owner
13 * {"type": "pong"} the answer to {"type": "ping"}
14 * Server → client: hello (who you are), state (the turn and the line), chat (the log), said, unsaid, muted
15 * (owners only), ping, error.
16 */
17
18 import type { IncomingMessage } from "node:http";
19 import type { Duplex } from "node:stream";
20 import WebSocket, { WebSocketServer } from "ws";
21 import { Chat, ChatError } from "./chat.ts";
22 import { identify, LOCAL, type Person, type Who } from "./identity.ts";
23 import { Turns, type TurnEvent } from "./turns.ts";
24
25 export interface LobbyOptions {
26 /** FORGE_APP_IDENTITY_KEY. Without it nobody is signed in, unless `local`. */
27 identityKey?: string;
28 /** A forge-less run on one machine: every visitor is the owner and may command the arm (there is no one to sign in as). */
29 local?: boolean;
30 /** The chat's file; none keeps it in memory. */
31 chatFile?: string;
32 turns?: Turns;
33 chat?: Chat;
34 /** How often each socket is pinged, and how long it may stay silent before it counts as dropped. */
35 pingMs?: number;
36 silentMs?: number;
37 }
38
39 interface Seat {
40 ws: WebSocket;
41 who: Who;
42 heard: number;
43 }
44
45 const MAX_FRAME = 4096;
46 const USER_ID = /^[A-Za-z0-9_-]{1,64}$/;
47
48 export class Lobby {
49 readonly turns: Turns;
50 readonly chat: Chat;
51 readonly local: boolean;
52 private readonly key: string;
53 private readonly wss = new WebSocketServer({ noServer: true, maxPayload: MAX_FRAME });
54 private readonly seats = new Set<Seat>();
55 private readonly endListeners = new Set<(who: Person) => void>();
56 private readonly pinger: ReturnType<typeof setInterval>;
57 private readonly silentMs: number;
58 private lastActivityState = 0;
59
60 constructor(options: LobbyOptions = {}) {
61 this.key = options.identityKey ?? "";
62 this.local = Boolean(options.local);
63 this.turns = options.turns ?? new Turns();
64 this.chat = options.chat ?? new Chat({ file: options.chatFile });
65 this.silentMs = options.silentMs ?? 15_000;
66 this.turns.on((event) => this.turnChanged(event));
67 this.pinger = setInterval(() => this.ping(), options.pingMs ?? 5000);
68 this.pinger.unref?.();
69 }
70
71 /** Who is behind this request: a verified forge identity, the local owner, or nobody. */
72 who(req: IncomingMessage): Who {
73 if (this.local) return LOCAL;
74 return identify(req.headers, this.key);
75 }
76
77 /** Why this person may not command the arm right now, or null when they may (and then it counts as activity). */
78 refusal(who: Who): { status: number; error: string } | null {
79 if (this.local) return null;
80 if (!who) return { status: 401, error: "sign in to take control" };
81 if (!this.turns.isDriver(who.id)) {
82 const d = this.turns.driver;
83 return { status: 403, error: d ? `${d.name} has control; take control to join the queue` : "take control first" };
84 }
85 this.activity(who.id);
86 return null;
87 }
88
89 /** The driver did something: the idle clock starts over; the room hears about it at most every 2 s. */
90 activity(id: string): void {
91 this.turns.activity(id);
92 const now = Date.now();
93 if (this.turns.isDriver(id) && now - this.lastActivityState > 2000) {
94 this.lastActivityState = now;
95 this.broadcastState();
96 }
97 }
98
99 /** Called with whoever's turn just ended, for whatever reason: their streams and moves must stop. */
100 onTurnEnd(listener: (who: Person) => void): () => void {
101 this.endListeners.add(listener);
102 return () => this.endListeners.delete(listener);
103 }
104
105 upgrade(req: IncomingMessage, socket: Duplex, head: Buffer): void {
106 const who = this.who(req);
107 this.wss.handleUpgrade(req, socket, head, (ws) => this.seat(ws, who));
108 }
109
110 close(): void {
111 clearInterval(this.pinger);
112 for (const s of this.seats) s.ws.terminate();
113 this.wss.close();
114 this.turns.dispose();
115 }
116
117 // ------------------------------------------------------------------ sockets
118 private seat(ws: WebSocket, who: Who): void {
119 const seat: Seat = { ws, who, heard: Date.now() };
120 this.seats.add(seat);
121 if (who) this.turns.connect(who.id);
122 this.send(seat, { type: "hello", you: who, signin: !this.local, muted: who ? this.chat.muted.has(who.id) : false });
123 this.send(seat, { type: "state", ...this.turns.snapshot() });
124 this.send(seat, { type: "chat", messages: this.chat.messages });
125 if (who?.owner) this.send(seat, this.mutedList());
126 ws.on("message", (data, binary) => {
127 seat.heard = Date.now();
128 if (binary) return;
129 let message: unknown;
130 try {
131 message = JSON.parse(String(data));
132 } catch {
133 return this.send(seat, { type: "error", error: "frames are JSON" });
134 }
135 try {
136 this.handle(seat, message);
137 } catch (err) {
138 this.send(seat, { type: "error", error: err instanceof ChatError ? err.message : "that did not work" });
139 if (!(err instanceof ChatError)) console.error(`lobby: ${err instanceof Error ? err.message : String(err)}`);
140 }
141 });
142 ws.on("close", () => {
143 this.seats.delete(seat);
144 if (who) this.turns.disconnect(who.id);
145 });
146 ws.on("error", () => ws.terminate());
147 }
148
149 private handle(seat: Seat, message: unknown): void {
150 if (typeof message !== "object" || message === null || Array.isArray(message)) return this.send(seat, { type: "error", error: "frames are JSON objects" });
151 const m = message as Record<string, unknown>;
152 const who = seat.who;
153 if (m.type === "pong") return;
154 if (!who) return this.send(seat, { type: "error", error: "sign in to take part" });
155 switch (m.type) {
156 case "join":
157 this.turns.join(who);
158 return;
159 case "leave":
160 this.turns.leave(who.id);
161 return;
162 case "post": {
163 const said = this.chat.post(who, m.text);
164 return this.broadcast({ type: "said", message: said });
165 }
166 }
167 if (!who.owner) return this.send(seat, { type: "error", error: "only the owner can do that" });
168 switch (m.type) {
169 case "end":
170 if (typeof m.id !== "string" || !USER_ID.test(m.id)) return this.send(seat, { type: "error", error: "end whom?" });
171 this.turns.end(m.id);
172 return;
173 case "delete":
174 if (!Number.isSafeInteger(m.id)) return this.send(seat, { type: "error", error: "delete which line?" });
175 if (this.chat.delete(m.id as number)) this.broadcast({ type: "unsaid", id: m.id });
176 return;
177 case "mute": {
178 if (typeof m.id !== "string" || !USER_ID.test(m.id) || typeof m.on !== "boolean") return this.send(seat, { type: "error", error: "mute whom?" });
179 const id = m.id;
180 const name = [...this.seats].find((s) => s.who?.id === id)?.who?.name ?? [...this.chat.messages].reverse().find((x) => x.from?.id === id)?.from?.name ?? this.chat.muted.get(id) ?? id;
181 this.chat.mute(id, name, m.on);
182 for (const s of this.seats) {
183 if (s.who?.id === id) this.send(s, { type: "hello", you: s.who, signin: !this.local, muted: m.on });
184 if (s.who?.owner) this.send(s, this.mutedList());
185 }
186 return;
187 }
188 }
189 this.send(seat, { type: "error", error: "unknown request" });
190 }
191
192 private turnChanged(event: TurnEvent): void {
193 if (event.kind === "turn") {
194 const { ended, started } = event;
195 if (ended) for (const listener of this.endListeners) listener(ended.who);
196 const parts = [ended ? `${ended.who.name}'s turn ended (${ended.reason})` : null, started ? `${started.name} has control` : ended ? "nobody has control" : null].filter(Boolean);
197 if (parts.length) this.broadcast({ type: "said", message: this.chat.system(parts.join("; ")) });
198 }
199 this.broadcastState();
200 }
201
202 private broadcastState(): void {
203 this.broadcast({ type: "state", ...this.turns.snapshot() });
204 }
205
206 private mutedList() {
207 return { type: "muted", muted: [...this.chat.muted].map(([id, name]) => ({ id, name })) };
208 }
209
210 private ping(): void {
211 const now = Date.now();
212 for (const s of this.seats) {
213 if (now - s.heard > this.silentMs) s.ws.terminate(); // its close handler starts the grace period
214 else this.send(s, { type: "ping" });
215 }
216 }
217
218 private send(seat: Seat, body: unknown): void {
219 if (seat.ws.readyState === WebSocket.OPEN) seat.ws.send(JSON.stringify(body));
220 }
221
222 private broadcast(body: unknown): void {
223 const text = JSON.stringify(body);
224 for (const s of this.seats) if (s.ws.readyState === WebSocket.OPEN) s.ws.send(text);
225 }
226 }
Software/so101/src/relay.ts +49 −4
@@ -6,6 +6,11 @@
66 * not, the relay says so on /link and answers from the simulation instead, so the page never hangs on a dead link.
77 * The page itself, the 3D model and the vendored scripts always come from here.
88 *
9 * With a lobby (src/lobby.ts), anyone may watch but only the person whose turn it is may command the arm: every
10 * request that is not a read, and the /follow stream, is refused for anyone else before it reaches an arm. When a
11 * turn ends, that person's streams are closed and whatever they started is stopped.
12 *
13 * WS /live the lobby: the turn, the queue, the chat
914 * GET /link {"mode": "live"|"simulated", "host": "<host:port>"|null, "problem": "…"|null}
1015 * GET /camera/<1|2>/<stream|state|snapshot> the arm machine's camera (µStreamer on :8080 and :8081 by default)
1116 */
@@ -15,6 +20,7 @@ import type { Duplex } from "node:stream";
1520 import { Readable } from "node:stream";
1621 import { pipeline } from "node:stream/promises";
1722 import WebSocket, { WebSocketServer } from "ws";
23 import type { Lobby } from "./lobby.ts";
1824
1925 export interface Remote {
2026 /** The arm server's HTTP base, no trailing slash. */
@@ -49,6 +55,8 @@ export interface RelayOptions {
4955 probeEveryMs?: number;
5056 /** How long a read through to the arm server may wait for its answer before the simulation answers instead. */
5157 readTimeoutMs?: number;
58 /** Who may command the arm; without one, anyone who reaches the relay may (a run on one's own machine). */
59 lobby?: Lobby | null;
5260 }
5361
5462 export interface RunningRelay {
@@ -120,6 +128,7 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
120128 const probeTimeoutMs = options.probeTimeoutMs ?? 1500;
121129 const probeEveryMs = options.probeEveryMs ?? 5000;
122130 const readTimeoutMs = options.readTimeoutMs ?? 2500;
131 const lobby = options.lobby ?? null;
123132
124133 // ------------------------------------------------------------------ the link
125134 let live = false;
@@ -216,6 +225,12 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
216225 const path = url.pathname + url.search;
217226 const reading = req.method === "GET" || req.method === "HEAD";
218227 if (!reading && !sameOrigin(req)) return json(res, 403, { error: "only this page may change the arm" });
228 if (!reading && lobby) {
229 const who = lobby.who(req);
230 // Stop is the owner's at any time; everything else is the driver's alone.
231 const refused = url.pathname === "/stop" && who?.owner ? null : lobby.refusal(who);
232 if (refused) return json(res, refused.status, { error: refused.error });
233 }
219234 if (req.method === "GET" && url.pathname === "/link") return json(res, 200, link());
220235
221236 const camera = CAMERA.exec(url.pathname);
@@ -267,9 +282,18 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
267282 });
268283 /** Close codes a server may send on; the rest (1005, 1006, 1015) only describe a connection that died. */
269284 const sendable = (code: number) => (code >= 1000 && code <= 1003) || (code >= 1007 && code <= 1014) || (code >= 3000 && code <= 4999);
270 const bridge = (browser: WebSocket, arm: WebSocket) => {
285 /** Each open /follow stream and whose it is, so a turn that ends can close it. */
286 const streams = new Map<WebSocket, string>();
287 const bridge = (browser: WebSocket, arm: WebSocket, driver: string | null) => {
288 if (driver !== null) {
289 streams.set(browser, driver);
290 browser.on("close", () => streams.delete(browser));
291 }
271292 arm.on("message", (data, binary) => browser.readyState === WebSocket.OPEN && browser.send(data, { binary }));
272 browser.on("message", (data, binary) => arm.readyState === WebSocket.OPEN && arm.send(data, { binary }));
293 browser.on("message", (data, binary) => {
294 if (driver !== null) lobby?.activity(driver);
295 if (arm.readyState === WebSocket.OPEN) arm.send(data, { binary });
296 });
273297 arm.on("close", (code, why) => browser.close(sendable(code) ? code : 1011, why.length ? why.toString() : "the arm server went away"));
274298 browser.on("close", () => arm.close());
275299 arm.on("error", () => browser.terminate());
@@ -280,9 +304,18 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
280304 };
281305 http.on("upgrade", (req: IncomingMessage, socket: Duplex, head: Buffer) => {
282306 const url = new URL(req.url ?? "/", "http://x");
283 if (url.pathname !== "/follow") return refuse(socket, "404 Not Found");
307 if (url.pathname !== "/follow" && !(url.pathname === "/live" && lobby)) return refuse(socket, "404 Not Found");
284308 if (!sameOrigin(req)) return refuse(socket, "403 Forbidden");
285309 socket.on("error", () => socket.destroy());
310 if (url.pathname === "/live") return lobby!.upgrade(req, socket, head);
311 let driver: string | null = null;
312 if (lobby) {
313 const who = lobby.who(req);
314 const refused = lobby.refusal(who);
315 // Accepted and closed at once: a page cannot read a refused handshake's status, but it can read a close reason.
316 if (refused) return wss.handleUpgrade(req, socket, head, (browser) => browser.close(4000 + refused.status, refused.error));
317 driver = who!.id;
318 }
286319 void (async () => {
287320 let arm: WebSocket;
288321 try {
@@ -297,10 +330,21 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
297330 }
298331 }
299332 if (socket.destroyed) return arm.close();
300 wss.handleUpgrade(req, socket, head, (browser) => bridge(browser, arm));
333 wss.handleUpgrade(req, socket, head, (browser) => bridge(browser, arm, driver));
301334 })();
302335 });
303336
337 // ------------------------------------------------------------------ a turn that ends
338 // Its streams close (the arm server holds the arm when a stream ends) and a move, playback or recording stops.
339 const turnEnded = lobby?.onTurnEnd((who) => {
340 for (const [browser, id] of streams) if (id === who.id) browser.close(1000, "your turn ended");
341 const base = remote && live ? remote.http : simulation;
342 fetch(`${base}/stop`, { method: "POST", signal: AbortSignal.timeout(readTimeoutMs) }).then(
343 (r) => r.body?.cancel(),
344 (err: unknown) => console.error(`stop after ${who.name}'s turn: ${reason(err)}`),
345 );
346 });
347
304348 // ------------------------------------------------------------------ start
305349 if (remote) await probe(); // bounded by the probe timeout: the first page already knows which arm it has
306350 const timer = remote ? setInterval(() => void probe(), probeEveryMs) : undefined;
@@ -313,6 +357,7 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
313357 close: () =>
314358 new Promise<void>((resolve) => {
315359 clearInterval(timer);
360 turnEnded?.();
316361 for (const c of inFlight) c.abort();
317362 for (const client of wss.clients) client.terminate();
318363 wss.close();
Software/so101/src/turns.test.ts added +125
@@ -0,0 +1,125 @@
1 import { afterEach, describe, expect, test } from "bun:test";
2 import { Turns, type TurnEvent } from "./turns.ts";
3
4 const alice = { id: "a", name: "alice", owner: false };
5 const bob = { id: "b", name: "bob", owner: false };
6 const carol = { id: "c", name: "carol", owner: false };
7 const owner = { id: "o", name: "trevato", owner: true };
8
9 let clock = 1_000_000;
10 let turns: Turns;
11 const make = () => {
12 clock = 1_000_000;
13 turns = new Turns({ turnMs: 180_000, idleMs: 30_000, graceMs: 10_000, now: () => clock });
14 const events: TurnEvent[] = [];
15 turns.on((e) => events.push(e));
16 return events;
17 };
18 const advance = (ms: number) => {
19 clock += ms;
20 turns.check();
21 };
22 afterEach(() => turns.dispose());
23
24 describe("turns", () => {
25 test("the first to ask drives; the rest wait in order, with an estimated wait", () => {
26 make();
27 expect(turns.join(alice)).toBe("driving");
28 expect(turns.join(bob)).toBe("queued");
29 expect(turns.join(carol)).toBe("queued");
30 expect(turns.join(bob)).toBe("queued"); // asking twice keeps the place
31 for (let i = 0; i < 3; i++) {
32 advance(20_000);
33 turns.activity("a");
34 }
35 const s = turns.snapshot();
36 expect(s.driver).toMatchObject({ id: "a", endsAt: 1_000_000 + 180_000 });
37 expect(s.queue).toEqual([
38 { id: "b", name: "bob", place: 1, waitMs: 120_000 },
39 { id: "c", name: "carol", place: 2, waitMs: 300_000 },
40 ]);
41 });
42
43 test("handing back gives the turn to the next in line, with a fresh clock", () => {
44 const events = make();
45 turns.join(alice);
46 turns.join(bob);
47 advance(5_000);
48 turns.leave("a");
49 expect(turns.driver).toMatchObject({ id: "b", startedAt: 1_005_000, endsAt: 1_185_000 });
50 expect(events.at(-1)).toEqual({ kind: "turn", ended: { who: alice, reason: "handed back" }, started: bob });
51 });
52
53 test("30 s without a command ends the turn; commands keep it", () => {
54 make();
55 turns.join(alice);
56 turns.join(bob);
57 for (let i = 0; i < 5; i++) {
58 advance(20_000);
59 turns.activity("a");
60 }
61 expect(turns.driver?.id).toBe("a");
62 turns.activity("b"); // not the driver: no effect on anything
63 advance(30_000);
64 expect(turns.driver?.id).toBe("b");
65 });
66
67 test("the turn ends when its time is up, however busy", () => {
68 make();
69 turns.join(alice);
70 for (let i = 0; i < 9; i++) {
71 advance(20_000);
72 turns.activity("a");
73 }
74 expect(turns.driver).toBeNull();
75 });
76
77 test("a dropped connection ends the turn after 10 s; coming back in time keeps it", () => {
78 make();
79 turns.join(alice);
80 turns.join(bob);
81 turns.connect("a");
82 turns.connect("b");
83 turns.disconnect("a");
84 advance(9_000);
85 turns.activity("a");
86 turns.connect("a");
87 advance(5_000);
88 expect(turns.driver?.id).toBe("a");
89 turns.disconnect("a");
90 advance(10_000);
91 expect(turns.driver?.id).toBe("b");
92 });
93
94 test("someone in line who drops for 10 s leaves the line; a second tab keeps them in", () => {
95 make();
96 turns.join(alice);
97 turns.join(bob);
98 turns.join(carol);
99 turns.connect("b");
100 turns.connect("b");
101 turns.connect("c");
102 turns.disconnect("b"); // one of two tabs
103 turns.disconnect("c");
104 advance(10_000);
105 turns.activity("a");
106 expect(turns.queue.map((p) => p.id)).toEqual(["b"]);
107 });
108
109 test("leaving the line, and the owner taking over or ending a turn", () => {
110 const events = make();
111 turns.join(alice);
112 turns.join(bob);
113 turns.join(carol);
114 turns.leave("b");
115 expect(turns.queue.map((p) => p.id)).toEqual(["c"]);
116 expect(turns.join(owner)).toBe("driving");
117 expect(events.at(-1)).toEqual({ kind: "turn", ended: { who: alice, reason: "taken over by the owner" }, started: owner });
118 expect(turns.queue.map((p) => p.id)).toEqual(["c"]);
119 turns.end("o");
120 expect(turns.driver?.id).toBe("c");
121 turns.join(bob);
122 turns.end("b"); // out of the line
123 expect(turns.queue).toEqual([]);
124 });
125 });
Software/so101/src/turns.ts added +180
@@ -0,0 +1,180 @@
1 /**
2 * Whose turn it is. One driver at a time, decided here and nowhere else; everyone else who asked waits in line.
3 *
4 * A turn lasts `turnMs`. It ends early when the driver hands back, sends nothing for `idleMs`, or has had no
5 * connection for `graceMs`; someone waiting in line who loses their connection for `graceMs` leaves the line.
6 * The owner jumps the line: asking to drive ends whoever drives now. The owner can also end anyone's turn.
7 * Plain state and deadlines; one timer, set to the nearest deadline, runs `check()`.
8 */
9
10 import type { Person } from "./identity.ts";
11
12 export interface Turn extends Person {
13 startedAt: number;
14 endsAt: number;
15 lastActive: number;
16 }
17
18 export type EndReason = "handed back" | "time is up" | "idle" | "disconnected" | "ended by the owner" | "taken over by the owner";
19
20 export type TurnEvent = { kind: "turn"; ended: { who: Person; reason: EndReason } | null; started: Person | null } | { kind: "queue" };
21
22 export interface Snapshot {
23 now: number;
24 turnMs: number;
25 idleMs: number;
26 driver: { id: string; name: string; owner: boolean; endsAt: number; idleEndsAt: number } | null;
27 queue: Array<{ id: string; name: string; place: number; waitMs: number }>;
28 }
29
30 export interface TurnsOptions {
31 turnMs?: number;
32 idleMs?: number;
33 graceMs?: number;
34 now?: () => number;
35 }
36
37 export class Turns {
38 readonly turnMs: number;
39 readonly idleMs: number;
40 readonly graceMs: number;
41 private readonly now: () => number;
42 driver: Turn | null = null;
43 readonly queue: Person[] = [];
44 /** Open connections per person, and since when someone with none has been gone. */
45 private readonly connections = new Map<string, number>();
46 private readonly gone = new Map<string, number>();
47 private readonly listeners = new Set<(event: TurnEvent) => void>();
48 private timer: ReturnType<typeof setTimeout> | undefined;
49
50 constructor(options: TurnsOptions = {}) {
51 this.turnMs = options.turnMs ?? 180_000;
52 this.idleMs = options.idleMs ?? 30_000;
53 this.graceMs = options.graceMs ?? 10_000;
54 this.now = options.now ?? Date.now;
55 }
56
57 on(listener: (event: TurnEvent) => void): () => void {
58 this.listeners.add(listener);
59 return () => this.listeners.delete(listener);
60 }
61
62 isDriver(id: string): boolean {
63 return this.driver?.id === id;
64 }
65
66 /** Ask to drive: drive now when nobody does, the owner takes over, everyone else gets in line. */
67 join(who: Person): "driving" | "queued" {
68 if (this.isDriver(who.id)) return "driving";
69 if (who.owner && this.driver) {
70 this.remove(who.id);
71 this.handOver("taken over by the owner", who);
72 return "driving";
73 }
74 if (!this.queue.some((p) => p.id === who.id)) this.queue.push({ ...who });
75 if (!this.driver) this.handOver(null);
76 else this.emit({ kind: "queue" });
77 return this.isDriver(who.id) ? "driving" : "queued";
78 }
79
80 /** Hand back the turn, or leave the line. */
81 leave(id: string): void {
82 if (this.isDriver(id)) return this.handOver("handed back");
83 if (this.remove(id)) this.emit({ kind: "queue" });
84 }
85
86 /** The owner ends someone's turn, or takes them out of the line. */
87 end(id: string): void {
88 if (this.isDriver(id)) return this.handOver("ended by the owner");
89 if (this.remove(id)) this.emit({ kind: "queue" });
90 }
91
92 /** The driver did something: the idle clock starts over. */
93 activity(id: string): void {
94 if (this.driver?.id === id) this.driver.lastActive = this.now();
95 }
96
97 connect(id: string): void {
98 this.connections.set(id, (this.connections.get(id) ?? 0) + 1);
99 this.gone.delete(id);
100 this.schedule();
101 }
102
103 disconnect(id: string): void {
104 const n = (this.connections.get(id) ?? 1) - 1;
105 if (n > 0) return void this.connections.set(id, n);
106 this.connections.delete(id);
107 this.gone.set(id, this.now());
108 this.schedule();
109 }
110
111 /** Apply every deadline that has passed. */
112 check(): void {
113 const now = this.now();
114 let queueChanged = false;
115 for (const [id, since] of this.gone) {
116 if (now - since < this.graceMs) continue;
117 this.gone.delete(id);
118 if (this.isDriver(id)) this.handOver("disconnected");
119 else if (this.remove(id)) queueChanged = true;
120 }
121 const d = this.driver;
122 if (d && now >= d.endsAt) this.handOver("time is up");
123 else if (d && now - d.lastActive >= this.idleMs) this.handOver("idle");
124 if (queueChanged) this.emit({ kind: "queue" });
125 this.schedule();
126 }
127
128 snapshot(): Snapshot {
129 const now = this.now();
130 const d = this.driver;
131 const left = d ? Math.max(0, d.endsAt - now) : 0;
132 return {
133 now,
134 turnMs: this.turnMs,
135 idleMs: this.idleMs,
136 driver: d ? { id: d.id, name: d.name, owner: d.owner, endsAt: d.endsAt, idleEndsAt: Math.min(d.endsAt, d.lastActive + this.idleMs) } : null,
137 // At most: a turn can end early, never late.
138 queue: this.queue.map((p, i) => ({ id: p.id, name: p.name, place: i + 1, waitMs: left + i * this.turnMs })),
139 };
140 }
141
142 dispose(): void {
143 clearTimeout(this.timer);
144 this.listeners.clear();
145 }
146
147 private remove(id: string): boolean {
148 const i = this.queue.findIndex((p) => p.id === id);
149 if (i < 0) return false;
150 this.queue.splice(i, 1);
151 return true;
152 }
153
154 /** End the current turn (if a reason is given) and start the next: `next`, or the head of the line. */
155 private handOver(reason: EndReason | null, next?: Person): void {
156 const ended = this.driver && reason ? { who: { id: this.driver.id, name: this.driver.name, owner: this.driver.owner }, reason } : null;
157 if (reason) this.driver = null;
158 if (this.driver) return;
159 const who = next ?? this.queue.shift();
160 const now = this.now();
161 this.driver = who ? { ...who, startedAt: now, endsAt: now + this.turnMs, lastActive: now } : null;
162 this.emit({ kind: "turn", ended, started: who ?? null });
163 this.schedule();
164 }
165
166 private emit(event: TurnEvent): void {
167 for (const listener of this.listeners) listener(event);
168 }
169
170 private schedule(): void {
171 clearTimeout(this.timer);
172 const deadlines = [...this.gone.values()].map((since) => since + this.graceMs);
173 if (this.driver) deadlines.push(this.driver.endsAt, this.driver.lastActive + this.idleMs);
174 if (!deadlines.length) return;
175 // The idle deadline moves with every command; waking early just reschedules.
176 const wait = Math.max(0, Math.min(...deadlines) - this.now());
177 this.timer = setTimeout(() => this.check(), wait + 5);
178 this.timer.unref?.();
179 }
180 }
Software/so101/src/viewer.html +151 −2
@@ -76,9 +76,36 @@
7676 .events { font-size: 12.5px; font-family: ui-monospace, SFMono-Regular, Menlo, monospace; } .events div { padding: 2px 0; border-bottom: 1px solid #1b2129; }
7777 .trim { display: grid; grid-template-columns: max-content auto; gap: 6px 12px; align-items: center; max-width: 360px; }
7878 :fullscreen #hud { opacity: .35; transition: opacity .3s; } :fullscreen #hud:hover { opacity: 1; }
79 [inert] { opacity: .5; }
80 /* the room: who drives, who waits, and the chat */
81 #shell { display: flex; width: 100%; height: 100%; }
82 #shell > #app { flex: 1; min-width: 0; }
83 #side { width: 320px; flex: none; display: flex; flex-direction: column; min-height: 0; background: #0d1117; border-left: 1px solid var(--line); }
84 #side[hidden] { display: none; }
85 @media (max-width: 760px) { #shell { flex-direction: column; } #side { width: auto; height: 40%; border-left: 0; border-top: 1px solid var(--line); } }
86 #turn { padding: 12px; border-bottom: 1px solid var(--line); display: grid; gap: 8px; }
87 #driverLine { font-size: 15px; font-weight: 600; } #driverLine.none { color: var(--dim); font-weight: 400; }
88 #clock { color: var(--dim); font-size: 12px; font-variant-numeric: tabular-nums; min-height: 1.4em; } #clock.idle { color: var(--warn); }
89 #turnActions:empty { display: none; }
90 a.btn { display: inline-flex; align-items: center; text-decoration: none; background: var(--blue); border: 1px solid var(--blue); color: #fff; border-radius: 6px; padding: 7px 12px; min-height: 34px; }
91 #queue { list-style: none; margin: 0; padding: 0; display: grid; gap: 2px; font-size: 13px; }
92 #queue li { display: flex; gap: 8px; align-items: center; padding: 3px 6px; border-radius: 6px; }
93 #queue li.me { background: rgba(56,139,253,.18); } #queue .place { color: var(--dim); font-variant-numeric: tabular-nums; min-width: 1.6em; }
94 #queue .wait { margin-left: auto; color: var(--dim); font-variant-numeric: tabular-nums; }
95 #mutedLine { color: var(--dim); font-size: 12px; } #mutedLine:empty { display: none; }
96 #chat { flex: 1; min-height: 0; overflow: auto; padding: 8px 12px; display: flex; flex-direction: column; gap: 3px; font-size: 13px; overflow-wrap: anywhere; }
97 #chat .line { line-height: 1.45; } #chat .line b { color: #c9d1d9; font-weight: 600; margin-right: 6px; } #chat .line.me b { color: #58a6ff; }
98 #chat .sys { color: var(--dim); font-style: italic; }
99 #chat .mod { display: none; margin-left: 6px; } #chat .line:hover .mod, #chat .line:focus-within .mod { display: inline-flex; gap: 4px; }
100 #chat .mod button { min-height: 22px; padding: 0 6px; font-size: 11px; }
101 #say { display: flex; gap: 6px; padding: 8px 12px calc(8px + env(safe-area-inset-bottom)); border-top: 1px solid var(--line); }
102 #say input { flex: 1; min-width: 0; font: inherit; padding: 6px 10px; border-radius: 6px; border: 1px solid var(--line); background: var(--bg); color: var(--fg); }
103 #sayNote { color: var(--dim); font-size: 12px; padding: 0 12px 8px; } #sayNote:empty { display: none; }
104 @media (prefers-reduced-motion: reduce) { * { transition: none !important; scroll-behavior: auto !important; } }
79105 </style>
80106 </head>
81107 <body>
108 <div id="shell">
82109 <div id="app">
83110 <div id="stage">
84111 <div class="pane on" id="pane1"><img id="cam1" alt="camera 1"><span class="tag" id="tag1">camera 1</span></div>
@@ -96,12 +123,14 @@
96123 <div id="joints"></div>
97124 <div class="row" id="poses"></div>
98125 <div class="row">
126 <span class="row" id="drive">
99127 <label class="inline">jog <select id="rate"><option value="100">slow</option><option value="300" selected>medium</option><option value="800">fast</option></select></label>
100128 <button id="hold" title="torque on, stay where it is">Hold</button>
101129 <button id="releaseBtn" class="danger" title="torque off">Release</button>
102130 <button id="record" class="rec">● Record</button>
103131 <button id="play">▶ Play…</button>
104132 <button id="liveBtn" class="go" hidden title="read the leader arm through this computer's USB port and stream it (Chrome)">Live from leader</button>
133 </span>
105134 <button id="metricsBtn">Metrics</button>
106135 <button id="settingsBtn">Settings</button>
107136 <button id="calib">3D calibration</button>
@@ -194,6 +223,19 @@
194223 <div class="actions"><label class="inline">rate <select id="playRate"><option value="0.5">0.5×</option><option value="1" selected>1×</option><option value="1.5">1.5×</option></select></label><button id="playClose">Close</button></div>
195224 </dialog>
196225 </div>
226 <aside id="side" hidden aria-label="Control and chat">
227 <section id="turn" aria-live="polite">
228 <div id="driverLine" class="none">Connecting…</div>
229 <div id="clock"></div>
230 <div class="row" id="turnActions"></div>
231 <ol id="queue" aria-label="Queue"></ol>
232 <div id="mutedLine"></div>
233 </section>
234 <div id="chat" role="log" aria-label="Chat"></div>
235 <form id="say" hidden><input id="sayText" maxlength="500" autocomplete="off" placeholder="Say something" aria-label="Message"><button class="go">Send</button></form>
236 <div id="sayNote"></div>
237 </aside>
238 </div>
197239 <script type="module">
198240 import * as THREE from "three";
199241 import { STLLoader } from "three/examples/jsm/loaders/STLLoader.js";
@@ -208,6 +250,7 @@
208250 const data = await r.json().catch(() => ({}));
209251 return Object.defineProperties(data, { ok: { value: r.ok }, status: { value: r.status } });
210252 };
253 const el = (tag, props = {}, ...children) => { const e = Object.assign(document.createElement(tag), props); e.append(...children); return e; };
211254 const NAMES = { 1: "shoulder_pan", 2: "shoulder_lift", 3: "elbow_flex", 4: "wrist_flex", 5: "wrist_roll", 6: "gripper" };
212255 const IDS = Object.fromEntries(Object.entries(NAMES).map(([id, n]) => [n, Number(id)]));
213256
@@ -277,6 +320,7 @@
277320 const stream = {
278321 ws: null, pending: null, timer: null, closeTimer: null, halted: false,
279322 open() {
323 if (!mayDrive()) return; // the server would refuse it: only the person whose turn it is streams
280324 clearTimeout(this.closeTimer);
281325 if (this.ws && this.ws.readyState <= 1) return;
282326 if (this.halted) return; // the server ended this hold (Stop, or the arm is busy): nothing more until the next press
@@ -284,7 +328,7 @@
284328 ws.onclose = (e) => {
285329 if (this.ws !== ws) return;
286330 this.ws = null;
287 if (e.code === 1013 || e.code === 1000) { this.halted = true; msg(e.reason || "the arm is busy", true); }
331 if (e.code === 1013 || e.code === 1000 || e.code >= 4000) { this.halted = true; msg(e.reason || "the arm is busy", true); }
288332 };
289333 this.ws = ws;
290334 },
@@ -596,7 +640,7 @@
596640 addEventListener("keydown", (e) => {
597641 if (e.target.closest("input, select, textarea, dialog")) return;
598642 if (e.key === "f" || e.key === "F") fullscreen();
599 if (e.key === " ") { e.preventDefault(); if (e.target instanceof HTMLButtonElement) e.target.blur(); stop(); }
643 if (e.key === " " && (mayDrive() || isOwner())) { e.preventDefault(); if (e.target instanceof HTMLButtonElement) e.target.blur(); stop(); }
600644 });
601645 for (const b of document.querySelectorAll("[data-close]")) b.onclick = () => { $(b.dataset.close).hidden = true; };
602646
@@ -642,6 +686,110 @@
642686 $("metricsBtn").onclick = showMetrics;
643687 setInterval(() => { if (!$("metricsPanel").hidden) showMetrics(); }, 5000);
644688
689 // ---------------------------------------------------------------- the room: the turn, the queue and the chat over /live
690 // Behind the hosted app's relay only the person whose turn it is may drive, and the server refuses everyone else.
691 // The page follows suit: the controls are inert until it is your turn. Everything here arrives on one socket.
692 const lobby = { ws: null, hello: null, state: null, offset: 0, retry: 500, mutedList: [] };
693 function mayDrive() { return !relayed || (lobby.hello !== null && (!lobby.hello.signin || (lobby.hello.you !== null && lobby.state?.driver?.id === lobby.hello.you.id))); }
694 function isOwner() { return !relayed || Boolean(lobby.hello?.you?.owner); }
695 const say = (m) => { if (lobby.ws?.readyState === 1) lobby.ws.send(JSON.stringify(m)); };
696 function applyLock() {
697 const can = mayDrive();
698 for (const id of ["joints", "poses", "drive"]) $(id).inert = !can;
699 $("stop").inert = !(can || isOwner());
700 for (const part of $("settingsPanel").querySelectorAll(".row, .trim")) part.inert = !can;
701 viewer.disableDragging = !can;
702 }
703 const serverNow = () => Date.now() + lobby.offset;
704 const minutes = (ms) => { const t = Math.max(0, Math.ceil(ms / 1000)); return `${Math.floor(t / 60)}:${String(t % 60).padStart(2, "0")}`; };
705 function tickClock() {
706 const d = lobby.state?.driver, c = $("clock");
707 if (!lobby.state || !lobby.hello?.signin) return void (c.textContent = lobby.hello && !lobby.hello.signin ? "Local: no sign-in, every visitor drives." : "");
708 if (!d) { c.className = ""; c.textContent = "Free: take control to drive."; return; }
709 const now = serverNow(), idle = d.idleEndsAt - now;
710 const idleSoon = idle < 15000 && d.idleEndsAt < d.endsAt;
711 c.className = idleSoon ? "idle" : "";
712 c.textContent = idleSoon ? `Idle: the turn ends in ${Math.max(0, Math.ceil(idle / 1000))} s unless the arm moves` : `${minutes(d.endsAt - now)} left in this turn`;
713 }
714 setInterval(tickClock, 250);
715 function renderTurn() {
716 const st = lobby.state, me = lobby.hello?.you ?? null;
717 if (!st || !lobby.hello) return;
718 const d = st.driver;
719 $("driverLine").className = d ? "" : "none";
720 $("driverLine").textContent = d ? (me && d.id === me.id ? "You have control" : `${d.name} has control`) : "Nobody has control";
721 const button = (textContent, className, onclick) => el("button", { textContent, className, onclick });
722 const queued = me && st.queue.some((q) => q.id === me.id);
723 const actions = [];
724 if (!lobby.hello.signin) { /* a run on one's own machine: everyone drives, nobody queues */ }
725 else if (!me) actions.push(el("a", { className: "btn", href: `/__forge/signin?to=${encodeURIComponent(location.pathname + location.search)}`, textContent: "Sign in to take control" }));
726 else if (d?.id === me.id) actions.push(button("Hand back", "", () => say({ type: "leave" })));
727 else if (queued) actions.push(button("Leave the queue", "", () => say({ type: "leave" })));
728 else if (me) actions.push(button(me.owner && d ? "Take control now" : "Take control", "go", () => say({ type: "join" })));
729 if (me?.owner && d && d.id !== me.id) actions.push(button(`End ${d.name}'s turn`, "danger", () => say({ type: "end", id: d.id })));
730 $("turnActions").replaceChildren(...actions);
731 $("queue").replaceChildren(...st.queue.map((q) => {
732 const mine = me && q.id === me.id;
733 const li = el("li", { className: mine ? "me" : "" }, el("span", { className: "place", textContent: `${q.place}.` }), el("span", { textContent: mine ? `${q.name} (you)` : q.name }), el("span", { className: "wait", textContent: `≤ ${Math.max(1, Math.ceil(q.waitMs / 60000))} min` }));
734 if (me?.owner) li.append(button("×", "x", () => say({ type: "end", id: q.id })));
735 return li;
736 }));
737 $("mutedLine").replaceChildren(...(me?.owner && lobby.mutedList.length ? ["Muted: ", ...lobby.mutedList.flatMap((m) => [el("span", { textContent: `${m.name} ` }), button("unmute", "x", () => say({ type: "mute", id: m.id, on: false }))])] : []));
738 $("say").hidden = !me || lobby.hello.muted;
739 $("sayNote").textContent = !lobby.hello.signin ? "" : !me ? "Sign in to chat." : lobby.hello.muted ? "The owner has muted you." : "";
740 tickClock();
741 applyLock();
742 }
743 const chatBox = $("chat");
744 function chatLine(m) {
745 const me = lobby.hello?.you;
746 const line = el("div", { className: m.from ? `line${me && m.from.id === me.id ? " me" : ""}` : "line sys", title: new Date(m.at).toLocaleTimeString() });
747 line.dataset.id = m.id;
748 if (m.from) line.append(el("b", { textContent: m.from.name }));
749 line.append(el("span", { textContent: m.text }));
750 if (m.from && me?.owner) line.append(el("span", { className: "mod" }, el("button", { textContent: "delete", onclick: () => say({ type: "delete", id: m.id }) }), el("button", { textContent: "mute", onclick: () => say({ type: "mute", id: m.from.id, on: true }) })));
751 return line;
752 }
753 function addChat(lines, replace = false) {
754 const atEnd = chatBox.scrollHeight - chatBox.scrollTop - chatBox.clientHeight < 40;
755 if (replace) chatBox.replaceChildren();
756 chatBox.append(...lines.map(chatLine));
757 while (chatBox.children.length > 200) chatBox.firstChild.remove();
758 if (atEnd || replace) chatBox.scrollTop = chatBox.scrollHeight;
759 }
760 function lobbyConnect() {
761 const ws = new WebSocket(`${location.protocol === "https:" ? "wss" : "ws"}://${location.host}/live`);
762 lobby.ws = ws;
763 ws.onopen = () => { lobby.retry = 500; };
764 ws.onmessage = (e) => {
765 const m = JSON.parse(e.data);
766 if (m.type === "ping") return say({ type: "pong" });
767 if (m.type === "hello") lobby.hello = m;
768 else if (m.type === "state") { lobby.offset = m.now - Date.now(); lobby.state = m; }
769 else if (m.type === "chat") return addChat(m.messages, true);
770 else if (m.type === "said") return addChat([m.message]);
771 else if (m.type === "unsaid") return chatBox.querySelector(`[data-id="${Number(m.id)}"]`)?.remove();
772 else if (m.type === "muted") lobby.mutedList = m.muted;
773 else if (m.type === "error") return msg(m.error, true);
774 renderTurn();
775 };
776 ws.onclose = () => {
777 if (lobby.ws !== ws) return;
778 lobby.ws = null; lobby.hello = null;
779 applyLock();
780 $("driverLine").className = "none"; $("driverLine").textContent = "Reconnecting…";
781 setTimeout(lobbyConnect, lobby.retry);
782 lobby.retry = Math.min(lobby.retry * 2, 8000);
783 };
784 }
785 $("say").onsubmit = (e) => {
786 e.preventDefault();
787 const text = $("sayText").value.trim();
788 if (!text) return;
789 say({ type: "post", text });
790 $("sayText").value = "";
791 };
792
645793 // ---------------------------------------------------------------- 3D calibration
646794 $("calib").onclick = async () => {
647795 $("panel").hidden = false;
@@ -757,6 +905,7 @@
757905 setTimeout(poll, 200);
758906 }
759907 (async () => {
908 if (relayed) { $("side").hidden = false; applyLock(); lobbyConnect(); }
760909 limits = await api("/limits");
761910 await loadSettings();
762911 await loadPoses();

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs