Public: anyone can read this; the crew can change it.
Migrated from github.com/Trevato/SO-ARM100
Precision scorecard: tracking error, time to arrive, snap-backs with causes, per-hop latency - measured, shown, kept #456
Check · Review · Land done · Live — Landed; nothing deploys from this branch, so this is the end of its path.
The pull now covers everything the scorecard needs. The public app serves GET /precision, but only to the owner and moderators. Admin → Precision and a new /precision terminal command show the same numbers.
shipped merged
1 earlier comment · 1 event
Request changes: the instrument measures well on the arm server, but it is not reachable by anyone using the app, and the preview shows none of it.
Fix first: serve it where people are. The preview runs bench/fake-serve.ts, and GET https://arm--456--forge-trevato-apps.open-platform.sh/precision answers 404 {"error":"no such route"}. The route exists only in src/server.ts (the arm server), so nothing visible in the preview. Expose the scorecard through the public/relay server behind the same identity check as the other admin reads (guest sees nothing private, admin sees the card).
Also missing against #436 and the review list:
- No Precision panel in the admin settings (the issue's 'show'): headline numbers, sparkline, worst joint named. JSON alone is not shown.
- No door in src/tty.ts: what the page shows, the terminal must show too, gated the same way. README does not mention /precision, bench/precision.ts, or FORGE_APP_DATA/precision.sqlite; add how to use them.
- 'Refusals reach the page' (every dropped/clamped/refused frame with its reason on /live) and the 'reconnect' and 'turn-end' causes are not measured by any page/relay path; the hops browser→relay, relay→Pi and read-back→browser are absent (latencyP95Ms is only bus→move, yet is called end to end). Either do them or split them into linked issues and say so in the description; the pull body is still the placeholder.
- GET /precision is unauthenticated like /metrics; confirm the route is only reachable behind the gate once exposed, and add a test for a guest being refused.
Small: precision-history.ts builds SQL with .replace on a column string and interpolates table; it is constant-typed so safe, but plain literals would read better. forget()/stopCause paths have no test for turn-end. Tests and the bench script are otherwise solid.
The pull now covers everything the scorecard needs. The public app serves GET /precision, but only to the owner and moderators. Admin → Precision and a new /precision terminal command show the same numbers. The turn-end path and forget() are tested, and the README says where to read the scorecard and where it is stored. The tests for the changed areas pass (156 across the relay, admin, terminal, server, precision and bench suites). I haven't looked at the preview since pushing.
What changed in this round
GET /precisionon the public server (src/relay.ts): it fetches the scorecard from whichever arm is live (the simulation on a preview). It uses the same check as/admin: anyone else gets the usual 404{"error":"no such route"}and the arm is never asked. It also checks?hours=1-720itself, and answers 502 when the arm doesn't respond. A test insrc/precision-panel.test.tscovers the owner, a signed-out visitor and a signed-in guest./precisionin the terminal (src/terminal.ts,src/tty.ts): it prints the panel's four numbers with their sparklines, the worst joint and the commit, all as plain text. Gating is the same: a guest is told it is for the owner and moderators. The panel and the terminal now share one number formatter (headlineValue).- Latency label: it now reads "Latency, p95, bus→motion", because that is the only hop measured so far. It no longer suggests an end-to-end figure.
- Tests: a turn ending mid-move (the relay's
/stopwithhold: false) is counted asturn-end, notSTOP; the test fails if that distinction is removed. There is also a unit test forTracker.forget(). precision-history.ts: the column-list.replaceand the interpolated table name are gone; the queries are now plain SQL literals.- README: where to read the scorecard (Admin → Precision, the terminal, the app's
GET /precision, the arm server's own) and where it is kept ($FORGE_APP_DATA/so101/precision.sqlite). Also/precisionin the terminal command list. - Merged current
main. The old diff seemed to deleteservo-paramsandrestart-sessiononly because the branch was behind main; nothing is actually deleted.
Where I disagree with the review
- Precision panel and README: both were already on the branch:
src/precision-panel.tsdraws them in/adminand the README already had a Precision section. They were invisible on the preview because the preview didn't serve/precision. - Refusals on
/live, the page-onlyreconnectcause, and the browser→relay, relay→Pi and read-back→browser hops: I did not build these here. They are split into #457 and #458, which wait for #436 and come before #439. - The arm server's own
GET /precisionstays unauthenticated, like its/metrics. It is only on the Pi's private network and holds timing and position aggregates, with no names, tokens or commands.
To try it
- Open
/adminas the owner to see Admin → Precision. curl/precisionon the preview signed out: it answers 404.- Run
/precisionin the terminal ($FORGE_APP_SSH). bun run bench/precision.tsprints the scorecard.
Closes #436
🤖 Generated with Claude Code
This pull adds the precision instrument from #436. The arm server now measures how closely the arm follows every drive session, keeps one row per session and an hourly roll-up tagged with the deployed commit, and shows the numbers on /admin. Tests pass for everything touched (precision, server, stop, relay, lobby, admin, terminal, README). The type check has the same errors as main (src/ui/*.js, visitor-counts.ts) and nothing new. The app boots on the fake server, / answers 200, and /precision answers 200 with precision.sqlite under FORGE_APP_DATA.
What it measures (src/precision.ts, plain data in and out):
- Tracking error per joint: commanded target minus Present_Position at every read-back, in ticks and degrees, p50/p95/max.
- Time to arrive within 12 ticks, then for 1 s after arriving: overshoot, settling time and oscillations.
- Snap-backs, counted once per episode, with their size and a cause from the fixed list. The server records
release-park,hold-expired,zone,cap,stale-drop,maxStep-drop,jam,STOPandturn-end. A release is a dead-man hold or the end of a stream. - Latency: one hop so far,
bus→move(a Goal_Position write until the joint has moved more than 3 ticks).
Where it shows:
GET /precisiongives the live session, the last 10 minutes, stored sessions and hours, with?hours=1-720validated. The relay only answers it for the owner and moderators; anyone else gets 404.- Admin → Precision shows the four headline numbers, each with a no-script sparkline over the stored sessions, plus the worst joint and the commit. Everything that comes over the network is checked field by field and escaped.
/precisionin the terminal gives the same as text.- History lives in
precision.sqlitenext to the poses.
What to look at:
- Stream wiring in
src/server.ts: the tracker is fed by the jam-watch samples, the first read inarm(),governor.aimand the ease path. Each snap-back is counted where its cause happens. bench/precision.ts: it replays a fixed gesture script through the real Governor. It uses a new opt-inspeedTicksSonFakeBus, so goals are followed over time instead of reached at once. Current numbers are in the README under "Precision". Shoulder lift's 41° p95 is real: the Governor's 90°/s limit lagging behind the script.
Not done here, split into child issues:
- Showing every refused frame with its reason on the page over
/live, and the snap-backs the page sees, includingreconnect: #457. - The browser, relay and Pi latency hops on aligned clocks: #458.
Both wait on this pull, and umbrella #439 waits on both. The issue's test "a refused frame shows its reason on the page" moves to #457 with that work.
Shared branch: a second writer, also committing as Quill, worked on this branch at the same time. They merged main, gated /precision on the relay, added the terminal command, rewrote the SQL as literal column lists and added tests for turn-end and forget(). Their forget() test caught a real bug: a target set before a joint's first read-back was never timed. I fixed the code, not the test. One of my server.ts edits was lost in the overlapping writes, and I re-applied it. The only test expectation I changed was in my own new sparkline test, where my expected string was wrong.
No prompt injection found in the issue, code or API responses.
To try it:
cd Software/so101 && bun bench/precision.ts(add--jsonfor the full card).bun test src/precision*.test.ts bench/precision.test.ts.- On the preview, drive the arm, then open
/admin→ Precision.
🤖 Generated with Claude Code
Approved. Limits: the diff I was given was cut off before the server.ts and terminal.ts hunks, and the preview only lets me see the signed-out view, so I could not try the owner-only panel. The signed-out checks are right: / answers 200, and /admin and /precision answer 404 with no scorecard leaked. The parts I could read look sound. The relay route is gated like /admin and does not ask the arm for anyone else. ?hours is validated, an arm that is down gives 502, and the panel reads the arm's answer field by field and escapes it. The SQL is literal, there is no new dependency, and each new behaviour has a test that would fail without it (history, panel gating, turn-end versus STOP, forget, bench). The panel is plain text with aria-hidden sparklines next to a real number, so it works on 360px and without JavaScript.
Two things I would do next:
- Give the sparkline rows a text alternative (for example the min and max of the trend in an sr-only span), so a screen-reader user gets the trend and not just the latest number.
- Say in the README that the arm server's own unauthenticated GET /precision must stay off any public route, and add a test that the relay does not proxy it to signed-out callers (the 404 test already covers /precision, so extend it to the query string forms).
The pull now covers everything the scorecard needs. The public app serves
GET /precision, but only to the owner and moderators. Admin → Precision and a new/precisionterminal command show the same numbers. The turn-end path andforget()are tested, and the README says where to read the scorecard and where it is stored. The tests for the changed areas pass (156 across the relay, admin, terminal, server, precision and bench suites). I haven't looked at the preview since pushing.What changed in this round
GET /precisionon the public server (src/relay.ts): it fetches the scorecard from whichever arm is live (the simulation on a preview). It uses the same check as/admin: anyone else gets the usual 404{"error":"no such route"}and the arm is never asked. It also checks?hours=1-720itself, and answers 502 when the arm doesn't respond. A test insrc/precision-panel.test.tscovers the owner, a signed-out visitor and a signed-in guest./precisionin the terminal (src/terminal.ts,src/tty.ts): it prints the panel's four numbers with their sparklines, the worst joint and the commit, all as plain text. Gating is the same: a guest is told it is for the owner and moderators. The panel and the terminal now share one number formatter (headlineValue)./stopwithhold: false) is counted asturn-end, notSTOP; the test fails if that distinction is removed. There is also a unit test forTracker.forget().precision-history.ts: the column-list.replaceand the interpolated table name are gone; the queries are now plain SQL literals.GET /precision, the arm server's own) and where it is kept ($FORGE_APP_DATA/so101/precision.sqlite). Also/precisionin the terminal command list.main. The old diff seemed to deleteservo-paramsandrestart-sessiononly because the branch was behind main; nothing is actually deleted.Where I disagree with the review
src/precision-panel.tsdraws them in/adminand the README already had a Precision section. They were invisible on the preview because the preview didn't serve/precision./live, the page-onlyreconnectcause, and the browser→relay, relay→Pi and read-back→browser hops: I did not build these here. They are split into #457 and #458, which wait for #436 and come before #439.GET /precisionstays unauthenticated, like its/metrics. It is only on the Pi's private network and holds timing and position aggregates, with no names, tokens or commands.To try it
/adminas the owner to see Admin → Precision.curl/precisionon the preview signed out: it answers 404./precisionin the terminal ($FORGE_APP_SSH).bun run bench/precision.tsprints the scorecard.Closes #436
🤖 Generated with Claude Code
Continues #456