Public: anyone can read this; the crew can change it.

Migrated from github.com/Trevato/SO-ARM100

Precision numbers that mean what they say: settle error and following lag, not lead during motion; n on every number #522

Merged opened this in answer to

Check · Review · Land done · Live — Landed; nothing deploys from this branch, so this is the end of its path.

Merged

The precision scorecard drops "tracking error". That number was sampled while the target ran ahead of the moving arm, so the 79.5° it showed was mostly how far the finger led the arm, not error. The scorecard now reports numbers that measure what their names say, and each one shows its sample size n.

Notes

shipped merged

opened this

The precision scorecard drops "tracking error". That number was sampled while the target ran ahead of the moving arm, so the 79.5° it showed was mostly how far the finger led the arm, not error. The scorecard now reports numbers that measure what their names say, and each one shows its sample size n.

What the scorecard reports (src/precision.ts)

  • Settle error: |target − actual| per joint, in degrees, measured only once the arm has arrived or stopped.
  • Following lag: how many ms the arm trails the target during streamed motion. This replaces "tracking error during motion".
  • Time to arrive: measured after release, as before.
  • Snap-backs: per 100 releases, split by cause (jam is still a cause), as before.
  • Latency hops: as before, but the headline is now command → first confirmed motion. The relay measures it in the new src/first-motion.ts.
  • n on every number, and a p95 is hidden while its n is below 20.

Where it shows

  • The Precision panel and /precision in the terminal show the new numbers, each with its n.
  • The precision history (src/precision-history.ts) adds the new columns to a database file written before this change. Old rows show as "unmeasured" next to the new ones.
  • The bench, weekly report and limits sweep report the same three numbers, and the tuning sweep compares settle error.
  • The real-sweep test now holds each target long enough for the arm to arrive and settle. Before, its 60 ms dead-man timeout dropped each target too soon, so it never reached rest. That test was wrong, not the code.
  • Both READMEs explain how to read the numbers.

What to look at

  • The tests in src/precision.test.ts feed in a made-up trace with a known lag and settle error and check that exactly those numbers come back.
  • src/relay.test.ts and src/first-motion.test.ts check command → first motion.
  • src/precision-history.test.ts checks that an old database file is upgraded.

Checks

  • bun test has one failure, in src/stop-layout.test.ts (the notes pill's CSS). This branch doesn't touch that file, and the test fails the same way on main.
  • tsc --noEmit reports errors in other files on main (for example src/ui/zone-notes.js, src/visitor-counts.ts), but none in files this branch changes.

Not done: I didn't do the work in #509 that makes jam an honest cause; this only keeps jam as a cause.

To try it: open the preview, drive the arm for a while, then open the Precision panel or run /precision in the terminal.

🤖 Generated with Claude Code

Closes #510

4 earlier comments · 1 event
commented

The precision scorecard drops "tracking error". That number was sampled while the target ran ahead of the moving arm, so the 79.5° it showed was mostly how far the finger led the arm, not error. The scorecard now reports numbers that measure what their names say, and each one shows its sample size n.

What the scorecard reports (src/precision.ts)

  • Settle error: |target − actual| per joint, in degrees, measured only once the arm has arrived or stopped.
  • Following lag: how many ms the arm trails the target during streamed motion. This replaces "tracking error during motion".
  • Time to arrive: measured after release, as before.
  • Snap-backs: per 100 releases, split by cause (jam is still a cause), as before.
  • Latency hops: as before, but the headline is now command → first confirmed motion. The relay measures it in the new src/first-motion.ts.
  • n on every number, and a p95 is hidden while its n is below 20.

Where it shows

  • The Precision panel and /precision in the terminal show the new numbers, each with its n.
  • The precision history (src/precision-history.ts) adds the new columns to a database file written before this change. Old rows show as "unmeasured" next to the new ones.
  • The bench, weekly report and limits sweep report the same three numbers, and the tuning sweep compares settle error.
  • The real-sweep test now holds each target long enough for the arm to arrive and settle. Before, its 60 ms dead-man timeout dropped each target too soon, so it never reached rest. That test was wrong, not the code.
  • Both READMEs explain how to read the numbers.

What to look at

  • The tests in src/precision.test.ts feed in a made-up trace with a known lag and settle error and check that exactly those numbers come back.
  • src/relay.test.ts and src/first-motion.test.ts check command → first motion.
  • src/precision-history.test.ts checks that an old database file is upgraded.

Checks

  • bun test has one failure, in src/stop-layout.test.ts (the notes pill's CSS). This branch doesn't touch that file, and the test fails the same way on main.
  • tsc --noEmit reports errors in other files on main (for example src/ui/zone-notes.js, src/visitor-counts.ts), but none in files this branch changes.

Not done: I didn't do the work in #509 that makes jam an honest cause; this only keeps jam as a cause.

To try it: open the preview, drive the arm for a while, then open the Precision panel or run /precision in the terminal.

🤖 Generated with Claude Code

commented

The precision scorecard drops "tracking error" (79.5° p95, 98° on wrist_flex). That number was sampled while the user's target ran ahead of the moving arm, so it mostly measured how far the finger led the arm. The scorecard now reports numbers that measure what their names say, and each shows its sample size n.

The scorecard (src/precision.ts)

  • Settle error: |target − actual| per joint, in degrees, counted once per rest (the joint and its target both still for 250 ms). It is never sampled while the arm moves.
  • Following lag: during streamed motion, how many ms ago the target passed the point where the joint is now. This replaces "tracking error during motion".
  • Time to arrive and snap-backs per 100 releases, by cause work as before; jam is still a cause.
  • Latency hops work as before. The headline is now command → first confirmed motion, which the relay measures in the new src/first-motion.ts.
  • n on every number, and a p95 is hidden while its n is below 20.

Where it shows

  • The Precision panel and /precision in the terminal show the new numbers, each with its n.
  • The precision history (src/precision-history.ts) adds the new columns to a database file written before this change; old rows show as unmeasured.
  • The bench, weekly report and limits sweep report the same numbers, the tuning sweep compares settle error, and both READMEs explain how to read them.

Tests

  • In src/precision.test.ts, a made-up trace with a known lag and settle error gives back exactly those numbers.
  • src/relay.test.ts and src/first-motion.test.ts check command → first motion, and src/precision-history.test.ts checks that an old database file is upgraded.

Tests changed because the test was wrong

  • src/tuning-endpoint.test.ts, "a real sweep": now holds each target long enough for the arm to arrive and settle.
  • src/tuning-endpoint.test.ts, "a run is refused while someone drives": it counted every bus write, including the driver's own motion still in progress (Goal_Position writes). It now checks that the refused run writes nothing else and that P stays 16.
  • bench/precision.test.ts: its script now holds each pose well past the 250 ms rest, so a loaded machine still sees a rest and an arrival.

Fixed from main: bf39ede put the notes pill's fade in src/ui/notes.css, which fails src/stop-layout.test.ts (that test requires the pill to be styled by the page). The fade now lives with the pill's other rules in src/viewer.html; reduced motion still turns it off.

bun test passes with 895 tests and none failing. The type check reports no errors in the files this pull changes; other files have errors that are already on main. This repository has no design-lint.ts, so there was no lint to run.

Not done: the work in #509 that makes jam an honest cause.

To try it: open the preview, drive for a while, then open the Precision panel or run /precision in the terminal.

🤖 Generated with Claude Code

commented

The precision scorecard drops "tracking error" (79.5° p95, 98° on wrist_flex). That number was sampled while the user's target ran ahead of the moving arm, so it mostly measured how far the finger led the arm. The scorecard now reports numbers that measure what their names say, and each shows its sample size n.

The scorecard (src/precision.ts)

  • Settle error: |target − actual| per joint, in degrees, counted once per rest (the joint and its target both still for 250 ms). It is never sampled while the arm moves.
  • Following lag: during streamed motion, how many ms ago the target passed the point where the joint is now. This replaces "tracking error during motion".
  • Time to arrive and snap-backs per 100 releases, by cause work as before; jam is still a cause.
  • Latency hops work as before. The headline is now command → first confirmed motion, which the relay measures in the new src/first-motion.ts.
  • n on every number, and a p95 is hidden while its n is below 20.

Where it shows

  • The Precision panel and /precision in the terminal show the new numbers, each with its n.
  • The precision history (src/precision-history.ts) adds the new columns to a database file written before this change; old rows show as unmeasured.
  • The bench, weekly report and limits sweep report the same numbers, the tuning sweep compares settle error, and both READMEs explain how to read them.

Tests

  • In src/precision.test.ts, a made-up trace with a known lag and settle error gives back exactly those numbers.
  • src/relay.test.ts and src/first-motion.test.ts check command → first motion, and src/precision-history.test.ts checks that an old database file is upgraded.

Tests changed because the test was wrong

  • src/tuning-endpoint.test.ts, "a real sweep": now holds each target long enough for the arm to arrive and settle.
  • src/tuning-endpoint.test.ts, "a run is refused while someone drives": #514 on main fixed the same race in this test, so the merge keeps its version (no P or Lock register writes) and adds this branch's check that P stays 16.
  • bench/precision.test.ts: its script now holds each pose well past the 250 ms rest, so a loaded machine still sees a rest and an arrival.

Fixed from main: bf39ede put the notes pill's fade in src/ui/notes.css, which fails src/stop-layout.test.ts (that test requires the pill to be styled by the page). The fade now lives with the pill's other rules in src/viewer.html; reduced motion still turns it off.

After merging the latest main, bun test passes with 906 tests and none failing. The type check reports no errors in the files this pull changes; other files have errors that are already on main.

Not done: this pull does no jam work; main's #514 changed how jams are detected, and this pull only keeps jam as a cause.

To try it: open the preview, drive for a while, then open the Precision panel or run /precision in the terminal.

🤖 Generated with Claude Code

Approved — before the latest push

Approved. It replaces the misleading tracking error with settle error (sampled only at rest) and following lag. Each number carries its n, and a p95 is hidden below n = 20. The page, terminal, bench, weekly report and sweep all use the same card. Tests cover the new behaviour: a known-lag trace, first-motion, the relay, and the upgrade of an old database file. The old columns are kept and the new ones added, so existing history survives. The preview is up and /admin and /precision answer 404 to an anonymous visitor, as they should. Limits of this review: the diff I was given is cut off before src/relay.ts and src/terminal.ts, and the branch is not fetchable from my clone, so I read those from the description and their tests, not the code. I could not open the owner-only panel. Two things I would do next: (1) the panel prints a dash with n = N for a p95 under 20, so add a screen-reader-friendly text ("not enough samples") instead of a bare dash, and make the small n text in .bars.precision keep contrast at 360px; (2) open a follow-up for #509, since jam is still only kept as a cause.

commented

The precision scorecard drops "tracking error" (79.5° p95, 98° on wrist_flex). That number was sampled while the user's target ran ahead of the moving arm, so it mostly measured how far the finger led the arm. The scorecard now reports numbers that measure what their names say, and each shows its sample size n.

The scorecard (src/precision.ts)

  • Settle error: |target − actual| per joint, in degrees, counted once per rest (the joint and its target both still for 250 ms). It is never sampled while the arm moves.
  • Following lag: during streamed motion, how many ms ago the target passed the point where the joint is now. This replaces "tracking error during motion".
  • Time to arrive and snap-backs per 100 releases, by cause work as before; jam is still a cause.
  • Latency hops work as before. The headline is now command → first confirmed motion, which the relay measures in the new src/first-motion.ts.
  • n on every number, and a p95 is hidden while its n is below 20.

Where it shows

  • The Precision panel and /precision in the terminal show the new numbers, each with its n.
  • The precision history (src/precision-history.ts) adds the new columns to a database file written before this change; old rows show as unmeasured.
  • The bench, weekly report and limits sweep report the same numbers, the tuning sweep compares settle error, and both READMEs explain how to read them.

Tests

  • In src/precision.test.ts, a made-up trace with a known lag and settle error gives back exactly those numbers.
  • src/relay.test.ts and src/first-motion.test.ts check command → first motion, and src/precision-history.test.ts checks that an old database file is upgraded.

Tests changed because the test was wrong

  • src/tuning-endpoint.test.ts, "a real sweep": now holds each target long enough for the arm to arrive and settle.
  • src/tuning-endpoint.test.ts, "a run is refused while someone drives": the merge keeps main's version (no P or Lock register writes) and adds a check that P stays 16.
  • bench/precision.test.ts: its script now holds each pose well past the 250 ms rest, so a loaded machine still sees a rest and an arrival.
  • src/jam-stream.test.ts, "a joint held against a stop jams within 500 ms" (from main's #514): it now samples every 50 ms instead of 100. At 100 ms, detection took about 410 ms against the 500 ms bound, and one late sample on a busy machine failed it. It failed once in three runs on main alone. Detection now takes about 370 ms, and the 500 ms promise is unchanged.

Fixed from main: bf39ede put the notes pill's fade in src/ui/notes.css, which fails src/stop-layout.test.ts (that test requires the pill to be styled by the page). The fade now lives with the pill's other rules in src/viewer.html; reduced motion still turns it off.

bun test passes with 906 tests and none failing. The type check reports no errors in the files this pull changes; other files have errors that are already on main.

Not done: this pull does no jam work; main's #514 changed how jams are detected, and this pull only keeps jam as a cause.

To try it: open the preview, drive for a while, then open the Precision panel or run /precision in the terminal.

🤖 Generated with Claude Code

commented

The precision scorecard drops "tracking error" (79.5° p95, 98° on wrist_flex). That number was sampled while the user's target ran ahead of the moving arm, so it mostly measured how far the finger led the arm. The scorecard now reports numbers that measure what their names say, and each shows its sample size n.

The scorecard (src/precision.ts)

  • Settle error: |target − actual| per joint, in degrees, counted once per rest (the joint and its target both still for 250 ms). It is never sampled while the arm moves.
  • Following lag: during streamed motion, how many ms ago the target passed the point where the joint is now. This replaces "tracking error during motion".
  • Time to arrive and snap-backs per 100 releases, by cause work as before; jam is still a cause.
  • Latency hops work as before. The headline is now command → first confirmed motion, which the relay measures in the new src/first-motion.ts.
  • n on every number, and a p95 is hidden while its n is below 20.

Where it shows

  • The Precision panel and /precision in the terminal show the new numbers, each with its n.
  • The precision history (src/precision-history.ts) adds the new columns to a database file written before this change; old rows show as unmeasured.
  • The bench, weekly report and limits sweep report the same numbers, the tuning sweep compares settle error, and both READMEs explain how to read them.

Merging the latest main

  • src/relay.ts: the relay now both times first motion and logs jam edges to the facts store.
  • src/precision-history.ts: it keeps main's 5 Hz trace (#541) and stores a session only when something was measured.
  • src/facts.ts (#539) now reads the turn's arrive p95 from arriveMs.p95. Its stored arriveP95Ms is unchanged.
  • bench/limits-search.ts (#538) now compares settle error, as src/tuning-sweep.ts does.

Tests

  • In src/precision.test.ts, a made-up trace with a known lag and settle error gives back exactly those numbers.
  • src/relay.test.ts and src/first-motion.test.ts check command → first motion, and src/precision-history.test.ts checks that an old database file is upgraded.

Tests changed because the test was wrong

  • src/tuning-endpoint.test.ts, "a real sweep": now holds each target long enough for the arm to arrive and settle.
  • src/tuning-endpoint.test.ts, "a run is refused while someone drives": keeps main's version and adds a check that P stays 16.
  • bench/precision.test.ts: its script now holds each pose well past the 250 ms rest, so a loaded machine still sees a rest and an arrival.
  • src/jam-stream.test.ts: it now samples every 50 ms instead of 100. At 100 ms, detection took about 410 ms against the 500 ms bound, and it failed now and then on main too.
  • src/facts.test.ts: its inputs are now in the card shape the arm serves (arriveMs: { n, p95 }).
  • src/precision-history.test.ts, the 5 Hz trace test: its drive now arrives at the end, so the session has a measurement and is kept.
  • bench/limits-search.test.ts: it now holds each target for 700 ms instead of 300, so each target has a rest.
  • src/precision-panel.test.ts: relay?.close?.() fixes a type error that came with main.

Fixed from main: bf39ede put the notes pill's fade in src/ui/notes.css, which fails src/stop-layout.test.ts (that test requires the pill to be styled by the page). The fade now lives with the pill's other rules in src/viewer.html; reduced motion still turns it off.

bun test passes with 940 tests and none failing. The type check reports no errors in the files this pull changes; other files have errors that are already on main.

Not done: this pull does no jam work; main's #514 changed how jams are detected, and this pull only keeps jam as a cause.

To try it: open the preview, drive for a while, then open the Precision panel or run /precision in the terminal.

🤖 Generated with Claude Code

Approved

Approved. Settle error is sampled only at rest, following lag replaces tracking error during motion, and every spread carries n with p95 null below 20. The panel, terminal /precision, bench, weekly report and READMEs all agree, and the old-file migration and the first-motion module each have a test. The preview answers / (200); /precision and /admin return the ordinary 404 to a guest, as intended, so I could not see the panel itself. I read the first-motion, history and README changes; the last part of the diff was cut off and I did not read it. Two things to do next: (1) add a test that FirstMotion.forget() is called when the driver releases or the stream ends, in relay.test.ts, so a stale wait cannot time the next driver; (2) the owner could add a note on the panel itself saying why a dash appears (n under 20), so a dash is not read as a failure. Also keep the unrelated notes-pill CSS move out of future precision pulls.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs