Public: anyone can read this; the crew can change it.

Migrated from github.com/Trevato/SO-ARM100

workbenchthis machine

src/facts.ts keeps 90 days of turns, events and per-minute samples in facts.sqlite under FORGE_APP_DATA, written by the relay (no API, no page) #539

Merged opened this

Check · Review · Land done · Live — Landed; nothing deploys from this branch, so this is the end of its path.

Merged

smith/arm:smith/528-src-facts-ts-keeps-90-days-of-turns-even → main 3 commits fork

  • Merged. smith/528-src-facts-ts-keeps-90-days-of-turns-even was merged into main as 5f2aecf.

Review

Sign in to review this change.
7 files changed +463 −2
Software/so101/README.md +10
@@ -49,6 +49,16 @@ The owner can test a joint's P (the position gain, 1–64) on the real arm, with
4949
5050 Nobody can drive while it runs, and it will not start while someone does (409; 423 on an E-stop or hold). Only one run at a time. Progress shows in the Servos section (reload the page) and with `/tune` alone, which also gives the last result (`GET /tuning`). Every result is stored with the deployed commit and listed newest first in the Precision panel (joint, P from→to, error and overshoot before→after, kept or put back) and as `tunings` in `GET /precision`. All of it is the owner's: anyone else gets 404.
5151
52 ## Facts kept for the Data view
53
54 The relay (`src/relay.ts`, through `src/facts.ts`) keeps 90 days of what happened in `facts.sqlite` under `FORGE_APP_DATA` (`so101/facts.sqlite`), every row tagged with the app's commit. Older rows are dropped. Nothing reads it yet: there is no API or page.
55
56 - **turns**: who, start, length, why it ended, wait in line; and, when the arm's `/precision` answers about a second after the turn, releases, snap-backs by cause, jams and arrive p95. That card is the arm's current session if it still has one, otherwise its pooled last ten minutes, so after a turn it usually covers the last ten minutes of driving, not that turn alone. Read those columns as a trend, not per turn.
57 - **events**: visit (one per browser an hour, at most 600 an hour), name set, joined line, chat (counted, never the text), STOP (only when accepted), jam, block, reconnect, app start.
58 - **samples**: once a minute, watchers, drivers, queue length, link live, and fps per H.264 camera.
59
60 "Who" is an account id or a 16-hex hash of the browser id, never a name or address.
61
5262 ## Faster limits
5363
5464 A person runs this at the arm while nobody drives: `bun bench/limits-search.ts http://<arm>:8101` (add `--joint 2` for one joint; with no URL it runs on the fake bus). From today's 90 °/s and 360 °/s² it steps each joint's speed and acceleration up within the safety ranges (the gripper is skipped), sweeps at each step, and keeps the last step that did not make p95 error or overshoot worse, writing them through `PUT /settings`. It prints its command line, then a table of limits and p95 error, overshoot and time to arrive, before → after. It refuses to start while anyone drives or a STOP or E-stop is in force, and if one appears mid-run it stops and puts the starting limits back.
Software/so101/bench/fake-serve.ts +3 −1
@@ -24,6 +24,7 @@ import { Lobby } from "../src/lobby.ts";
2424 import { Moderator, modelFromEnv } from "../src/moderator.ts";
2525 import { parseRemote, parseVideo, startRelay, type Remote } from "../src/relay.ts";
2626 import { piSyncFromEnv } from "../src/pi-deploy.ts";
27 import { Facts } from "../src/facts.ts";
2728 import { startServer } from "../src/server.ts";
2829
2930 const fake = new FakeBus([1, 2, 3, 4, 5, 6]);
@@ -69,7 +70,8 @@ try {
6970 } catch (err) {
7071 console.error(`${(err as Error).message}; cameras stay MJPEG`);
7172 }
73 const facts = new Facts({ file: join(dir, "facts.sqlite") });
7274 const piSync = piSyncFromEnv(process.env, remote?.http ?? null, remote?.host ?? null, () => lobby.turns.snapshot().driver === null, dir);
73 const relay = await startRelay({ piSync: piSync ?? undefined, host, port, simulation: `http://127.0.0.1:${simulation.port}`, remote, problem, lobby, cameras, leaders, video, summaryEveryMs: 60_000, holdFile: join(dir, "hold.json"), baselineFile: join(dir, "servo-factory.json") });
75 const relay = await startRelay({ facts, piSync: piSync ?? undefined, host, port, simulation: `http://127.0.0.1:${simulation.port}`, remote, problem, lobby, cameras, leaders, video, summaryEveryMs: 60_000, holdFile: join(dir, "hold.json"), baselineFile: join(dir, "servo-factory.json") });
7476 const { mode, host: armHost } = relay.link();
7577 console.log(`arm on http://${host}:${relay.port} (${mode === "simulated" ? "simulated" : `arm ${armHost}: ${mode}`}; data in ${dir}${local ? "; local: every visitor drives as the owner" : ""})`);
Software/so101/src/arm-edges.ts added +24
@@ -0,0 +1,24 @@
1 /**
2 * The arm's jam and block, as moments rather than states: the arm's /status says which joints are jammed and whether
3 * a zone blocks it for as long as that lasts, at 25 Hz. A fact is the moment it began, so this remembers the last
4 * status and answers only with what is new. Plain data in; no clock, no storage.
5 */
6
7 export type Edge = "jam" | "block";
8
9 export class ArmEdges {
10 #jammed = new Set<number>();
11 #blocked = false;
12
13 /** The edges this status brings: "jam" once per joint that has just jammed, "block" once when a block has just begun. */
14 observe(status: { jammed?: unknown; blocked?: unknown } | null | undefined): Edge[] {
15 const edges: Edge[] = [];
16 const jammed = new Set(Array.isArray(status?.jammed) ? status.jammed.filter((id): id is number => Number.isSafeInteger(id)) : []);
17 for (const id of jammed) if (!this.#jammed.has(id)) edges.push("jam");
18 this.#jammed = jammed;
19 const blocked = status?.blocked != null && status.blocked !== false;
20 if (blocked && !this.#blocked) edges.push("block");
21 this.#blocked = blocked;
22 return edges;
23 }
24 }
Software/so101/src/facts.test.ts added +163
@@ -0,0 +1,163 @@
1 import { expect, test } from "bun:test";
2 import { mkdtempSync } from "node:fs";
3 import { tmpdir } from "node:os";
4 import { join } from "node:path";
5 import { ArmEdges } from "./arm-edges.ts";
6 import { Facts, KEEP_MS, MAX_VISITS_PER_HOUR, parseCard, whoIs } from "./facts.ts";
7 import type { Snapshot } from "./turns.ts";
8
9 const person = (id: string, name = id) => ({ id, name, owner: false });
10 const snap = (queue: Array<{ id: string; waitMs?: number }> = []): Snapshot =>
11 ({ now: 0, turnMs: 0, idleMs: 0, paused: false, maxQueue: 9, driver: null, queue: queue.map((q, i) => ({ id: q.id, name: "x", place: i + 1, waitMs: q.waitMs ?? 0 })) }) as Snapshot;
12
13 test("who is an account id or 16 hex, never a name", () => {
14 expect(whoIs("a-abc123")).toBe("a-abc123");
15 expect(whoIs("0123456789abcdef0123456789abcdef")).toMatch(/^[0-9a-f]{16}$/);
16 expect(whoIs("alice")).toMatch(/^[0-9a-f]{16}$/);
17 expect(whoIs(null)).toBeNull();
18 });
19
20 test("events and samples are kept with the commit; chat is counted without text", () => {
21 const facts = new Facts({ commit: "abc123", now: () => 1000 });
22 facts.event("visit", "0123456789abcdef0123456789abcdef");
23 facts.event("chat", "a-xyz");
24 facts.sample({ watchers: 3, drivers: 1, queue: 2, linkLive: true, fps: { "1": 14.5 } });
25 const events = facts.events();
26 expect(events.map((e) => [e.kind, e.commit])).toEqual([["visit", "abc123"], ["chat", "abc123"]]);
27 expect(events[0]!.who).toMatch(/^[0-9a-f]{16}$/);
28 expect(facts.samples()).toEqual([{ at: 1000, commit: "abc123", watchers: 3, drivers: 1, queue: 2, linkLive: true, fps: { "1": 14.5 } }]);
29 });
30
31 test("a turn is written when it ends, with its wait in line, and takes its card later", () => {
32 let t = 0;
33 const facts = new Facts({ commit: "c", now: () => t });
34 t = 1000;
35 facts.turn({ kind: "queue" }, snap([{ id: "bob", waitMs: 0 }]));
36 expect(facts.events().map((e) => e.kind)).toEqual(["joined line"]);
37 t = 31_000;
38 expect(facts.turn({ kind: "turn", ended: null, started: person("bob") }, snap())).toBeNull();
39 t = 91_000;
40 const id = facts.turn({ kind: "turn", ended: { who: person("bob"), reason: "time is up" }, started: null }, snap());
41 expect(id).not.toBeNull();
42 facts.card(id!, parseCard({ session: { releases: 4, arriveP95Ms: 120, snapBacks: { total: 3, byCause: { jam: 2, STOP: 1, zone: 0 } } } })!);
43 const [row] = facts.turns();
44 expect(row).toMatchObject({ seconds: 60, waitSeconds: 30, ended: "time is up", releases: 4, snapBacks: { jam: 2, STOP: 1 }, jams: 2, arriveP95Ms: 120, commit: "c" });
45 expect(row!.who).toMatch(/^[0-9a-f]{16}$/);
46 });
47
48 test("a turn nobody saw begin is not invented", () => {
49 const facts = new Facts({ now: () => 5 });
50 expect(facts.turn({ kind: "turn", ended: { who: person("x"), reason: "idle" }, started: null }, snap())).toBeNull();
51 expect(facts.turns()).toEqual([]);
52 });
53
54 test("card parsing takes the session, else the window, and ignores what is not a card", () => {
55 expect(parseCard({ session: null, window: { releases: 2, arriveP95Ms: null, snapBacks: { byCause: { cap: 1, bogus: 9 } } } })).toEqual({ releases: 2, snapBacks: { cap: 1 }, jams: 0, arriveP95Ms: null });
56 expect(parseCard({ session: null, window: null })).toBeNull();
57 expect(parseCard("nope")).toBeNull();
58 expect(parseCard({ session: { releases: -1, arriveP95Ms: "x" } })).toEqual({ releases: 0, snapBacks: {}, jams: 0, arriveP95Ms: null });
59 });
60
61 test("rows older than 90 days are dropped, on open and with the samples", () => {
62 const dir = mkdtempSync(join(tmpdir(), "facts-"));
63 const file = join(dir, "facts.sqlite");
64 let t = 0;
65 const old = new Facts({ file, now: () => t });
66 old.event("visit", "a");
67 old.sample({ watchers: 0, drivers: 0, queue: 0, linkLive: false, fps: {} });
68 t = KEEP_MS - 1;
69 old.event("STOP", "a");
70 old.close();
71 t = KEEP_MS + 1;
72 const facts = new Facts({ file, now: () => t });
73 expect(facts.events().map((e) => e.kind)).toEqual(["STOP"]);
74 expect(facts.samples()).toEqual([]);
75 t = KEEP_MS * 2;
76 facts.sample({ watchers: 1, drivers: 0, queue: 0, linkLive: true, fps: {} });
77 expect(facts.events()).toEqual([]);
78 expect(facts.samples()).toHaveLength(1);
79 });
80
81 test("jam and block are edges: one per beginning, not one per status", () => {
82 const edges = new ArmEdges();
83 expect(edges.observe({ jammed: [], blocked: null })).toEqual([]);
84 expect(edges.observe({ jammed: [2], blocked: null })).toEqual(["jam"]);
85 expect(edges.observe({ jammed: [2], blocked: null })).toEqual([]);
86 expect(edges.observe({ jammed: [2, 3], blocked: { what: "zone" } })).toEqual(["jam", "block"]);
87 expect(edges.observe({ jammed: [2, 3], blocked: { what: "zone" } })).toEqual([]);
88 expect(edges.observe({ jammed: [], blocked: null })).toEqual([]);
89 expect(edges.observe({ jammed: [2], blocked: { what: "zone" } })).toEqual(["jam", "block"]);
90 expect(edges.observe(null)).toEqual([]);
91 });
92
93 test("a browser's visits are one row an hour, and all visits are capped", () => {
94 let t = 0;
95 const facts = new Facts({ now: () => t });
96 for (let i = 0; i < 10; i++) facts.visit("0123456789abcdef0123456789abcdef");
97 expect(facts.events()).toHaveLength(1);
98 t = 3_600_001;
99 facts.visit("0123456789abcdef0123456789abcdef");
100 expect(facts.events()).toHaveLength(2);
101 for (let i = 0; i < MAX_VISITS_PER_HOUR + 50; i++) facts.visit(`b${i}`);
102 expect(facts.events()).toHaveLength(1 + MAX_VISITS_PER_HOUR);
103 });
104
105 async function relayWith(facts: Facts, card = { session: { releases: 2, snapBacks: { byCause: { STOP: 1 } } } }) {
106 const { Lobby } = await import("./lobby.ts");
107 const { startRelay } = await import("./relay.ts");
108 const { sign } = await import("./identity.ts");
109 const key = "k".repeat(64);
110 const identity = `id-ann|ann|admin|${Date.now() + 600_000}`;
111 const headers = { "x-forge-identity": identity, "x-forge-signature": sign(identity, key) };
112 const sim = Bun.serve({ port: 0, hostname: "127.0.0.1", fetch: (req) => (new URL(req.url).pathname === "/status" ? Response.json({ joints: [], held: null }) : Response.json(card)) });
113 const lobby = new Lobby({ identityKey: key });
114 const relay = await startRelay({ port: 0, simulation: `http://127.0.0.1:${sim.port}`, lobby, facts });
115 const stop = async () => {
116 await relay.close();
117 lobby.close();
118 sim.stop(true);
119 };
120 return { relay, lobby, headers, stop };
121 }
122
123 test("ten loads of / by one browser write one visit", async () => {
124 const facts = new Facts();
125 const { relay, stop } = await relayWith(facts);
126 try {
127 const cookie = "arm_browser=0123456789abcdef0123456789abcdef";
128 for (let i = 0; i < 10; i++) await (await fetch(`http://127.0.0.1:${relay.port}/`, { headers: { cookie } })).text();
129 expect(facts.events().filter((e) => e.kind === "visit")).toHaveLength(1);
130 } finally {
131 await stop();
132 }
133 });
134
135 test("a refused /stop writes no row; an accepted one writes one", async () => {
136 const facts = new Facts();
137 const { relay, headers, stop } = await relayWith(facts);
138 try {
139 const refused = await fetch(`http://127.0.0.1:${relay.port}/stop`, { method: "POST" });
140 expect(refused.status).toBeGreaterThanOrEqual(400);
141 await fetch(`http://127.0.0.1:${relay.port}/estop`, { method: "POST" });
142 expect(facts.events().filter((e) => e.kind === "STOP")).toHaveLength(0);
143 expect((await fetch(`http://127.0.0.1:${relay.port}/stop`, { method: "POST", headers })).status).toBe(200);
144 expect(facts.events().filter((e) => e.kind === "STOP")).toHaveLength(1);
145 } finally {
146 await stop();
147 }
148 });
149
150 test("the relay writes app start and a turn's row with its card", async () => {
151 const facts = new Facts({ commit: "t" });
152 const { lobby, stop } = await relayWith(facts);
153 try {
154 lobby.turns.join({ id: "id-ann", name: "ann", owner: false });
155 lobby.turns.end("id-ann");
156 for (let i = 0; i < 100 && facts.turns()[0]?.releases == null; i++) await Bun.sleep(50);
157 expect(facts.events().map((e) => e.kind)).toEqual(["app start"]);
158 expect(facts.turns()[0]).toMatchObject({ ended: "ended by the owner", releases: 2, snapBacks: { STOP: 1 } });
159 expect(JSON.stringify(facts.events())).not.toContain("ann");
160 } finally {
161 await stop();
162 }
163 });
Software/so101/src/facts.ts added +214
@@ -0,0 +1,214 @@
1 /**
2 * What happened, kept 90 days: turns, events and per-minute samples in facts.sqlite under FORGE_APP_DATA, every row
3 * tagged with the app's commit. The relay writes them (src/relay.ts) and only when something happens or once a minute,
4 * never per frame. Nobody is named: a person is an account id or a 16-hex hash of the browser id, chat is counted and
5 * never kept, and no address is stored. A failing write is logged and dropped: facts never get in the arm's way.
6 */
7
8 import { createHash } from "node:crypto";
9 import { openDb, type Db } from "./sqlite.ts";
10 import { deployedCommit } from "./precision-history.ts";
11 import { CAUSES, type Cause } from "./precision.ts";
12 import type { Snapshot, TurnEvent } from "./turns.ts";
13
14 const HOUR_MS = 3_600_000;
15 export const MAX_VISITS_PER_HOUR = 600;
16 export const KEEP_MS = 90 * 24 * 3_600_000;
17 export const EVENTS = ["visit", "name set", "joined line", "chat", "STOP", "jam", "block", "reconnect", "app start"] as const;
18 export type EventKind = (typeof EVENTS)[number];
19
20 /** Who a row is about: an account's own id (`a-…`, random), else 16 hex of the hash of whatever id the person has. Never a name or address. */
21 export function whoIs(id: string | null | undefined): string | null {
22 if (!id) return null;
23 if (/^a-[A-Za-z0-9_-]{1,32}$/.test(id)) return id;
24 return createHash("sha256").update(id).digest("hex").slice(0, 16);
25 }
26
27 /** What /precision tells of a turn once it has answered. */
28 export interface TurnCard {
29 releases: number;
30 /** Snap-backs by cause, the causes that happened only. */
31 snapBacks: Partial<Record<Cause, number>>;
32 jams: number;
33 arriveP95Ms: number | null;
34 }
35
36 const count = (v: unknown): number => (typeof v === "number" && Number.isSafeInteger(v) && v >= 0 ? v : 0);
37
38 /** The numbers kept from the arm's GET /precision: the session being driven, else the last ten minutes; null when it has no card. */
39 export function parseCard(body: unknown): TurnCard | null {
40 const root = body && typeof body === "object" ? (body as Record<string, unknown>) : null;
41 const card = [root?.session, root?.window].find((c) => c && typeof c === "object") as Record<string, unknown> | undefined;
42 if (!card) return null;
43 const by = (card.snapBacks as { byCause?: Record<string, unknown> } | null)?.byCause ?? {};
44 const snapBacks: Partial<Record<Cause, number>> = {};
45 for (const cause of CAUSES) if (count(by[cause]) > 0) snapBacks[cause] = count(by[cause]);
46 const p95 = card.arriveP95Ms;
47 return { releases: count(card.releases), snapBacks, jams: snapBacks.jam ?? 0, arriveP95Ms: typeof p95 === "number" && Number.isFinite(p95) ? p95 : null };
48 }
49
50 export interface Sample {
51 watchers: number;
52 drivers: number;
53 queue: number;
54 linkLive: boolean;
55 /** Decoded frames a second by camera id. */
56 fps: Record<string, number>;
57 }
58
59 export interface TurnRow {
60 id: number;
61 at: string;
62 commit: string;
63 who: string;
64 startedAt: string;
65 seconds: number;
66 ended: string;
67 waitSeconds: number;
68 releases: number | null;
69 snapBacks: Partial<Record<Cause, number>> | null;
70 jams: number | null;
71 arriveP95Ms: number | null;
72 }
73
74 export class Facts {
75 readonly commit: string;
76 readonly #db: Db;
77 readonly #now: () => number;
78 /** When each person got in line, and when the driver began. */
79 readonly #queued = new Map<string, number>();
80 #driver: { id: string; since: number; waitMs: number } | null = null;
81 #pruned = 0;
82
83 constructor(options: { file?: string; commit?: string; now?: () => number } = {}) {
84 this.commit = options.commit ?? deployedCommit();
85 this.#now = options.now ?? Date.now;
86 this.#db = openDb(options.file ?? ":memory:");
87 this.#db.run("PRAGMA journal_mode = WAL");
88 this.#db.run(`CREATE TABLE IF NOT EXISTS turns (id INTEGER PRIMARY KEY, at INTEGER NOT NULL, commit_id TEXT NOT NULL, who TEXT NOT NULL, started INTEGER NOT NULL,
89 seconds REAL NOT NULL, ended TEXT NOT NULL, wait_seconds REAL NOT NULL, releases INTEGER, snap_backs TEXT, jams INTEGER, arrive_p95_ms REAL)`);
90 this.#db.run("CREATE TABLE IF NOT EXISTS events (at INTEGER NOT NULL, commit_id TEXT NOT NULL, kind TEXT NOT NULL, who TEXT)");
91 this.#db.run(`CREATE TABLE IF NOT EXISTS samples (at INTEGER NOT NULL, commit_id TEXT NOT NULL, watchers INTEGER NOT NULL, drivers INTEGER NOT NULL, queue INTEGER NOT NULL,
92 link_live INTEGER NOT NULL, fps TEXT NOT NULL)`);
93 for (const t of ["turns", "events", "samples"]) this.#db.run(`CREATE INDEX IF NOT EXISTS ${t}_at ON ${t} (at)`);
94 this.#attempt(() => this.prune());
95 }
96
97 /** Write unless it fails; a failure is said once on the console and costs nothing else. */
98 #attempt<T>(write: () => T): T | null {
99 try {
100 return write();
101 } catch (err) {
102 console.error(`facts: ${err instanceof Error ? err.message : err}`);
103 return null;
104 }
105 }
106
107 /** `who` is a raw id; it is hashed here, so a caller cannot keep a name by mistake. */
108 event(kind: EventKind, who?: string | null): void {
109 this.#attempt(() => void this.#db.query("INSERT INTO events (at, commit_id, kind, who) VALUES (?, ?, ?, ?)").run(this.#now(), this.commit, kind, whoIs(who)));
110 }
111
112 /** A page view: one row per browser an hour, and at most MAX_VISITS_PER_HOUR in all, so looping on / cannot fill the file. */
113 visit(browser: string | null | undefined): void {
114 const now = this.#now();
115 if (now - this.#visitWindow >= HOUR_MS) {
116 this.#visitWindow = now;
117 this.#visits = 0;
118 for (const [id, at] of this.#seen) if (now - at >= HOUR_MS) this.#seen.delete(id);
119 }
120 const key = browser ?? "";
121 if (this.#visits >= MAX_VISITS_PER_HOUR || now - (this.#seen.get(key) ?? -Infinity) < HOUR_MS) return;
122 this.#seen.set(key, now);
123 this.#visits++;
124 this.event("visit", browser);
125 }
126 readonly #seen = new Map<string, number>();
127 #visits = 0;
128 #visitWindow = 0;
129
130 sample(s: Sample): void {
131 this.#attempt(() => {
132 this.#db.query("INSERT INTO samples (at, commit_id, watchers, drivers, queue, link_live, fps) VALUES (?, ?, ?, ?, ?, ?, ?)").run(this.#now(), this.commit, s.watchers, s.drivers, s.queue, s.linkLive ? 1 : 0, JSON.stringify(s.fps));
133 if (this.#now() - this.#pruned > 3_600_000) this.prune();
134 });
135 }
136
137 /** The turns changed: note who joined the line, and write the row of a turn that ended. Returns that row's id, to give its card to. */
138 turn(event: TurnEvent, snapshot: Snapshot): number | null {
139 return this.#attempt(() => {
140 const now = this.#now();
141 for (const p of snapshot.queue) {
142 if (this.#queued.has(p.id)) continue;
143 this.#queued.set(p.id, now - Math.max(0, p.waitMs));
144 this.event("joined line", p.id);
145 }
146 let id: number | null = null;
147 if (event.kind === "turn") {
148 const ended = event.ended;
149 if (ended && this.#driver?.id === ended.who.id) {
150 const d = this.#driver;
151 const result = this.#db
152 .query("INSERT INTO turns (at, commit_id, who, started, seconds, ended, wait_seconds) VALUES (?, ?, ?, ?, ?, ?, ?)")
153 .run(now, this.commit, whoIs(d.id), d.since, Number(((now - d.since) / 1000).toFixed(1)), ended.reason, Number((d.waitMs / 1000).toFixed(1))) as { lastInsertRowid: number | bigint };
154 id = Number(result.lastInsertRowid);
155 }
156 const started = event.started;
157 this.#driver = started ? { id: started.id, since: now, waitMs: Math.max(0, now - (this.#queued.get(started.id) ?? now)) } : null;
158 }
159 const inLine = new Set(snapshot.queue.map((p) => p.id));
160 for (const queued of this.#queued.keys()) if (!inLine.has(queued)) this.#queued.delete(queued);
161 return id;
162 });
163 }
164
165 /** /precision answered after the turn: the row gets the card. */
166 card(turnId: number, card: TurnCard): void {
167 this.#attempt(() => void this.#db.query("UPDATE turns SET releases = ?, snap_backs = ?, jams = ?, arrive_p95_ms = ? WHERE id = ?").run(card.releases, JSON.stringify(card.snapBacks), card.jams, card.arriveP95Ms, turnId));
168 }
169
170 /** Drop what is older than 90 days. */
171 prune(): void {
172 const before = this.#now() - KEEP_MS;
173 this.#pruned = this.#now();
174 for (const t of ["turns", "events", "samples"]) this.#db.query(`DELETE FROM ${t} WHERE at < ?`).run(before);
175 }
176
177 turns(): TurnRow[] {
178 const rows = this.#db.query("SELECT * FROM turns ORDER BY id").all() as Array<Record<string, unknown>>;
179 return rows.map((r) => ({
180 id: r.id as number,
181 at: new Date(r.at as number).toISOString(),
182 commit: r.commit_id as string,
183 who: r.who as string,
184 startedAt: new Date(r.started as number).toISOString(),
185 seconds: r.seconds as number,
186 ended: r.ended as string,
187 waitSeconds: r.wait_seconds as number,
188 releases: r.releases as number | null,
189 snapBacks: r.snap_backs ? (JSON.parse(r.snap_backs as string) as Partial<Record<Cause, number>>) : null,
190 jams: r.jams as number | null,
191 arriveP95Ms: r.arrive_p95_ms as number | null,
192 }));
193 }
194
195 events(): Array<{ at: number; commit: string; kind: EventKind; who: string | null }> {
196 return (this.#db.query("SELECT at, commit_id, kind, who FROM events ORDER BY at, rowid").all() as Array<{ at: number; commit_id: string; kind: EventKind; who: string | null }>).map((r) => ({ at: r.at, commit: r.commit_id, kind: r.kind, who: r.who }));
197 }
198
199 samples(): Array<{ at: number; commit: string; watchers: number; drivers: number; queue: number; linkLive: boolean; fps: Record<string, number> }> {
200 return (this.#db.query("SELECT * FROM samples ORDER BY at, rowid").all() as Array<Record<string, unknown>>).map((r) => ({
201 at: r.at as number,
202 commit: r.commit_id as string,
203 watchers: r.watchers as number,
204 drivers: r.drivers as number,
205 queue: r.queue as number,
206 linkLive: r.link_live === 1,
207 fps: JSON.parse(r.fps as string) as Record<string, number>,
208 }));
209 }
210
211 close(): void {
212 this.#db.close();
213 }
214 }
Software/so101/src/lobby.ts +8
@@ -148,6 +148,7 @@ export class Lobby {
148148 private readonly seats = new Set<Seat>();
149149 private readonly changeListeners = new Set<() => void>();
150150 private readonly endListeners = new Set<(who: Person) => void>();
151 private readonly chatListeners = new Set<(who: Person) => void>();
151152 private readonly readBackListeners = new Set<(ms: number[]) => void>();
152153 private readonly arriveListeners = new Set<(tell: (text: string) => void) => void>();
153154 private readonly pinger: ReturnType<typeof setInterval>;
@@ -327,6 +328,12 @@ export class Lobby {
327328 return () => this.endListeners.delete(listener);
328329 }
329330
331 /** Called with the author of each chat line that was posted (never the text). */
332 onChat(listener: (who: Person) => void): () => void {
333 this.chatListeners.add(listener);
334 return () => this.chatListeners.delete(listener);
335 }
336
330337 /** Called as each page opens the room, with a way to tell that page alone something (JSON text): what was published before it came. */
331338 onArrive(listener: (tell: (text: string) => void) => void): () => void {
332339 this.arriveListeners.add(listener);
@@ -488,6 +495,7 @@ export class Lobby {
488495 if (held > 0) throw new ChatError(`slow mode: wait ${Math.ceil(held / 1000)}s`);
489496 const said = this.chat.post(who, m.text);
490497 this.lastPosted.set(who.id, this.now());
498 for (const listener of this.chatListeners) listener(who);
491499 this.broadcast({ type: "said", message: this.marked(said) });
492500 void this.moderator?.check(said);
493501 return;
Software/so101/src/relay.ts +41 −1
@@ -63,13 +63,15 @@ import WebSocket, { WebSocketServer } from "ws";
6363 import { LiveMetrics } from "./live-metrics.ts";
6464 import { wallMs } from "./clock.ts";
6565 import { ArmLink, type LinkTimings } from "./arm-link.ts";
66 import { ArmEdges } from "./arm-edges.ts";
67 import { parseCard, type Facts } from "./facts.ts";
6668 import { MachineFeed } from "./machine-feed.ts";
6769 import { refusalBatch } from "./refusals.ts";
6870 import { BROWSER_HOP, PAGE_HOP, parseSnapBack } from "./precision.ts";
6971 import { perServo } from "./machine-log.ts";
7072 import { parseFrame } from "./safety.ts";
7173 import { acceptsGzip, GzipCache, worthCompressing } from "./compress.ts";
72 import { browserCookie } from "./browser-id.ts";
74 import { browserCookie, browserId } from "./browser-id.ts";
7375 import { AccountError, sessionCookie, visitor } from "./accounts.ts";
7476 import { CameraSocket } from "./camera-socket.ts";
7577 import { OFFLINE_FRAME } from "./offline-frame.ts";
@@ -140,6 +142,10 @@ export interface RelayOptions {
140142 camera?: FeedOptions;
141143 /** The Pi's H.264 streams per camera id (src/video-feed.ts); a camera without one stays MJPEG. */
142144 video?: Record<number, VideoSources>;
145 /** Keep what happened for 90 days (src/facts.ts): written on events and once a minute, never per frame; without one nothing is kept. */
146 facts?: Facts;
147 /** How often the samples are taken, ms (a minute). */
148 sampleEveryMs?: number;
143149 /** Keep STOP's hold here, and restore it on start; without one it is in memory. */
144150 holdFile?: string;
145151 /** Keep the arm's first complete /servo-params read here (servo-factory.json); without one nothing is recorded. */
@@ -258,6 +264,8 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
258264 const readTimeoutMs = options.readTimeoutMs ?? 2500;
259265 const lobby = options.lobby ?? null;
260266 const heartbeatMs = options.heartbeatMs ?? 1000;
267 const facts = options.facts ?? null;
268 const edges = new ArmEdges();
261269 const metrics = new LiveMetrics();
262270 const cameras = options.cameras ?? new CameraHub({ feed: { ...options.camera, metrics }, seed: remote ? [{ name: "Camera 1", source: remote.cameras[0]! }] : [] });
263271 const leaders = options.leaders ?? new LeaderCalibrations();
@@ -300,6 +308,7 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
300308 metrics,
301309 onStatus: (status) => {
302310 if (!onArm()) return;
311 if (facts) for (const edge of edges.observe(status)) facts.event(edge); // a write only at the moment a jam or block begins
303312 machine.status(status);
304313 void push(); // on to the pages as it comes, at the arm's 25 Hz: no wait for the next tick
305314 },
@@ -312,6 +321,7 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
312321 onChange: (state, previous) => {
313322 if (state === "live") {
314323 console.log(`arm server ${host}: live`);
324 if (previous === "reconnecting") facts?.event("reconnect");
315325 tried = false;
316326 if (!baseline.recorded()) void recordBaseline();
317327 }
@@ -626,6 +636,7 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
626636 const headers: Record<string, string> = { "content-type": response.headers.get("content-type") ?? "text/html; charset=utf-8", "cache-control": "no-store", vary: "accept-encoding" };
627637 const browser = browserCookie(req.headers.cookie);
628638 if (browser) headers["set-cookie"] = browser;
639 if (response.status === 200) facts?.visit(browserId(req.headers.cookie) ?? browser?.split(";")[0]?.split("=")[1]);
629640 if (!acceptsGzip(req.headers)) {
630641 res.writeHead(response.status, headers);
631642 return res.end(body);
@@ -702,6 +713,7 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
702713 try {
703714 const { person, token } = lobby.accounts.create((body as { name?: unknown } | null)?.name, visitor(req));
704715 res.setHeader("set-cookie", sessionCookie(token));
716 facts?.event("name set", person.id);
705717 return json(res, 200, { you: person });
706718 } catch (err) {
707719 if (err instanceof AccountError) return json(res, err.status, { error: err.message });
@@ -802,6 +814,7 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
802814
803815 if (req.method === "GET" && PAGE.test(url.pathname)) return await page(req, res, await ask(simulation + path, req, undefined, null, false));
804816 if (req.method === "GET" && STATIC.test(url.pathname) && acceptsGzip(req.headers)) return await packed(res, await ask(simulation + path, req, undefined, null, false), path);
817 if (route === "POST /stop" || route === "POST /estop") facts?.event("STOP", lobby ? lobby.who(req)?.id : null);
805818 if (route === "POST /stop") {
806819 // Shut before anything is asked of an arm: a frame already on its way finds the gate closed.
807820 gate.hold(by);
@@ -1092,6 +1105,30 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
10921105 );
10931106 });
10941107
1108 // ------------------------------------------------------------------ facts
1109 // A turn's row is written when it ends; the arm's card for it is added when /precision answers.
1110 const turnFacts = facts && lobby ? lobby.turns.on((event) => {
1111 const row = facts.turn(event, lobby.turns.snapshot());
1112 if (row === null) return;
1113 setTimeout(() => {
1114 armPrecision(null)
1115 .then(async (answer) => {
1116 const card = answer.ok ? parseCard(await answer.json()) : null;
1117 if (card) facts.card(row, card);
1118 })
1119 .catch(() => undefined); // the row stays without its card
1120 }, 1000).unref?.();
1121 }) : undefined;
1122 const chatFacts = facts ? lobby?.onChat((who) => facts.event("chat", who.id)) : undefined;
1123 const sampleTimer = facts ? setInterval(() => {
1124 const snap = lobby?.turns.snapshot();
1125 const fps: Record<string, number> = {};
1126 for (const c of videoCards()) if (c.fpsP50 !== null) fps[String(c.camera)] = c.fpsP50;
1127 facts.sample({ watchers: lobby?.watching ?? 0, drivers: snap?.driver ? 1 : 0, queue: snap?.queue.length ?? 0, linkLive: mode() === "live", fps });
1128 }, options.sampleEveryMs ?? 60_000) : undefined;
1129 sampleTimer?.unref?.();
1130 facts?.event("app start");
1131
10951132 // ------------------------------------------------------------------ start
10961133 if (lobby) follow();
10971134 if (!armLink) connect();
@@ -1119,6 +1156,9 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
11191156 clearInterval(summary);
11201157 settingsChanged?.();
11211158 turnEnded?.();
1159 turnFacts?.();
1160 chatFacts?.();
1161 clearInterval(sampleTimer);
11221162 readBacks?.();
11231163 arrived?.();
11241164 armLink?.close();

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs