Public: anyone can read this; the crew can change it.

Migrated from github.com/Trevato/SO-ARM100

workbenchthis machine

The relay holds one LiveMetrics, reports the Pi round trip from arm-link, and serves it at GET /metrics.prom #261

Merged opened this

Check · Review · Land done · Live — Landed; nothing deploys from this branch, so this is the end of its path.

Merged

smith/arm:smith/245-the-relay-holds-one-livemetrics-reports → main 2 commits fork 2 threads · 2 unresolved

  • Merged. smith/245-the-relay-holds-one-livemetrics-reports was merged into main as 0f8ed54.

Review

Sign in to review this change.
4 files changed +60 −2
Software/README.md +15
@@ -732,6 +732,21 @@ The vision kit in `Software/vision` works against this server and stream:
732732 `SO101_ARM=http://<pi>:8101 SO101_CAMERA=http://<pi>:8080/stream`, then the
733733 `just vision-*` recipes. See `Software/vision/QUEST.md`.
734734
735 ## Metrics
736
737 `GET /metrics.prom` on the relay answers Prometheus text (`text/plain; version=0.0.4`, `no-store`, with the usual
738 security headers). It carries the round trip to the arm server, which `src/arm-link.ts` measures on every ping (or
739 `GET /status` when polling) and reports as `pi_rtt_ms`: the series `pi_rtt_ms{quantile="0.5"}`,
740 `pi_rtt_ms{quantile="0.95"}`, `pi_rtt_ms_sum` and `pi_rtt_ms_count`, over the last 256 samples. It is empty until the
741 first round trip is measured.
742
743 ```
744 curl https://arm--forge-trevato-apps.open-platform.sh/metrics.prom
745 ```
746
747 The endpoint is unauthenticated, like `/link` and `/status`: it exposes only timing aggregates, no names, tokens or
748 commands, and cannot move the arm. `GET /metrics` is unchanged and is still the arm server's own, proxied.
749
735750 ## Launch check
736751
737752 `so101/bench/launch-check.ts` checks a deployed page as an anonymous visitor and
Software/so101/src/relay.test.ts +27
@@ -418,6 +418,33 @@ describe("relay", () => {
418418 }
419419 });
420420
421 test("a fake arm that waits 50 ms shows in the round-trip p95 on /metrics.prom, and /metrics stays the arm's JSON", async () => {
422 const wss = new WebSocketServer({ noServer: true, autoPong: false });
423 camera = createServer((_req, res) => res.writeHead(404).end());
424 camera.on("upgrade", (req, socket, head) =>
425 wss.handleUpgrade(req, socket, head, (ws) => {
426 ws.on("ping", (data) => void setTimeout(() => ws.readyState === WebSocket.OPEN && ws.pong(data), 50));
427 }),
428 );
429 const pump = setInterval(() => wss.clients.forEach((c) => c.send('{"joints":[]}')), 10);
430 try {
431 relay = await startRelay({ port: 0, simulation: simBase, remote: remoteAt(await listening(camera)), link: { ...FAST, staleMs: 500, pingEveryMs: 100 } });
432 await sleep(700);
433 const res = await fetch(`http://127.0.0.1:${relay.port}/metrics.prom`);
434 expect(res.status).toBe(200);
435 expect(res.headers.get("content-type")).toStartWith("text/plain; version=0.0.4");
436 expect(res.headers.get("x-content-type-options")).toBe("nosniff");
437 const text = await res.text();
438 const p95 = Number(/^pi_rtt_ms\{quantile="0.95"\} (\S+)$/m.exec(text)?.[1]);
439 expect(p95).toBeGreaterThanOrEqual(45);
440 expect(p95).toBeLessThan(500);
441 expect((await fetch(`http://127.0.0.1:${relay.port}/metrics`)).status).toBe(404); // still the arm server's, which this fake does not have
442 } finally {
443 clearInterval(pump);
444 for (const c of wss.clients) c.terminate();
445 }
446 });
447
421448 test("a bad setting is reported and the arm is simulated", async () => {
422449 relay = await startRelay({ port: 0, simulation: simBase, problem: 'ARM_SERVER "ftp://x" must start with http://' });
423450 expect(await link(`http://127.0.0.1:${relay.port}`)).toEqual({ mode: "simulated", host: null, problem: 'ARM_SERVER "ftp://x" must start with http://' });
Software/so101/src/relay.ts +8
@@ -56,6 +56,7 @@ import type { Duplex } from "node:stream";
5656 import { Readable } from "node:stream";
5757 import { pipeline } from "node:stream/promises";
5858 import WebSocket, { WebSocketServer } from "ws";
59 import { LiveMetrics } from "./live-metrics.ts";
5960 import { ArmLink, type LinkTimings } from "./arm-link.ts";
6061 import { MachineFeed } from "./machine-feed.ts";
6162 import { perServo } from "./machine-log.ts";
@@ -229,6 +230,7 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
229230 const leaders = options.leaders ?? new LeaderCalibrations();
230231
231232 // ------------------------------------------------------------------ the link
233 const metrics = new LiveMetrics();
232234 let armLink: ArmLink | null = null;
233235 /** The simulation's own watch, held only while someone has the Machine panel open and the simulation is the arm. */
234236 let simLink: ArmLink | null = null;
@@ -244,6 +246,7 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
244246 remote,
245247 open: (url, timeoutMs) => open(url, timeoutMs),
246248 timings: options.link,
249 metrics,
247250 onStatus: (status) => onArm() && machine.status(status),
248251 onMachine: (entries, run) => onArm() && machine.arm(entries, run),
249252 onChange: (state, previous) => {
@@ -571,6 +574,11 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
571574 if (refused) return json(res, refused.status, { error: refused.error });
572575 by = who!.name;
573576 }
577 // Public on purpose, like /link: timing aggregates only (README, "Metrics").
outdated

This sits after the lobby check, but for a GET the check is skipped when reading, so the route is public. That is fine for aggregate timings, but document it in the README and add a line saying so.

578 if (req.method === "GET" && url.pathname === "/metrics.prom") {
579 res.writeHead(200, { "content-type": "text/plain; version=0.0.4; charset=utf-8", "cache-control": "no-store" });
580 return res.end(metrics.prometheus());
581 }
574582 if (req.method === "GET" && url.pathname === "/link") return json(res, 200, link());
575583 if (req.method === "GET" && url.pathname === "/status") return await status(res, path);
576584

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs