Public: anyone can read this; the crew can change it.

Migrated from github.com/Trevato/SO-ARM100

bench/limits-sweep.ts drives a fixed per-joint sweep and prints the /precision scorecard before and after, against the fake bus or a given arm URL #487

Merged opened this

Check · Review · Land done · Live — Landed; nothing deploys from this branch, so this is the end of its path.

Merged

I added bench/limits-sweep.ts, which sweeps each joint through the same fixed targets and prints that joint's /precision line. The bench tests pass (16 across 4 files, including the new one). I ran the sweep once on the fake bus and it printed six joint lines with zero snap-backs.

Notes

shipped merged

opened this

I added bench/limits-sweep.ts, which sweeps each joint through the same fixed targets and prints that joint's /precision line. The bench tests pass (16 across 4 files, including the new one). I ran the sweep once on the fake bus and it printed six joint lines with zero snap-backs. I did not run it against a real arm URL.

  • Targets: each joint goes 0, +250, 0, −250, 0 ticks from the middle of its bounds (read from /status), holding each for 1.5 s, with the other joints left alone. Steps stay under the server's 400-tick maxStep, and any target outside the bounds is pulled back in.
  • Output: the headline numbers before, one line per joint (p95 error, time to arrive, max overshoot, snap-backs), then the headline numbers after.
  • Fake bus: with no url it starts the fake bus with the real Governor. I moved that setup out of bench() into an exported fakeArm() in bench/precision.ts, and bench() now calls it.
  • With a url: it refuses to start if /status shows a driver, a STOP or an E-stop. While running it stops with an error if STOP or an E-stop is set, the stream closes, or the snap-back count rises after a target. I read any refusal as a snap-back; the stream sends no separate refusal message.
  • Test: bench/limits-sweep.test.ts checks the targets, one line per joint, that two fake-bus runs have the same layout, and that it refuses to start while someone drives. Timer jitter moves the measured numbers between runs, so "same from run to run" means the same lines, not identical values. Software/README.md has the two lines.
  • Live arm caveat: the per-joint numbers come from the server's 48-hour scorecard window, so on a live arm they also include earlier sessions of that joint. On the fake bus they cover only this run.

tsc --noEmit reports errors in src/ files this change doesn't touch (src/servo-baseline.ts, src/ui/*), and none in bench/.

🤖 Generated with Claude Code

Closes #478

2 earlier comments · 1 event
commented

I added bench/limits-sweep.ts, which sweeps each joint through the same fixed targets and prints that joint's /precision line. The bench tests pass (16 across 4 files, including the new one). I ran the sweep once on the fake bus and it printed six joint lines with zero snap-backs. I did not run it against a real arm URL.

  • Targets: each joint goes 0, +250, 0, −250, 0 ticks from the middle of its bounds (read from /status), holding each for 1.5 s, with the other joints left alone. Steps stay under the server's 400-tick maxStep, and any target outside the bounds is pulled back in.
  • Output: the headline numbers before, one line per joint (p95 error, time to arrive, max overshoot, snap-backs), then the headline numbers after.
  • Fake bus: with no url it starts the fake bus with the real Governor. I moved that setup out of bench() into an exported fakeArm() in bench/precision.ts, and bench() now calls it.
  • With a url: it refuses to start if /status shows a driver, a STOP or an E-stop. While running it stops with an error if STOP or an E-stop is set, the stream closes, or the snap-back count rises after a target. I read any refusal as a snap-back; the stream sends no separate refusal message.
  • Test: bench/limits-sweep.test.ts checks the targets, one line per joint, that two fake-bus runs have the same layout, and that it refuses to start while someone drives. Timer jitter moves the measured numbers between runs, so "same from run to run" means the same lines, not identical values. Software/README.md has the two lines.
  • Live arm caveat: the per-joint numbers come from the server's 48-hour scorecard window, so on a live arm they also include earlier sessions of that joint. On the fake bus they cover only this run.

tsc --noEmit reports errors in src/ files this change doesn't touch (src/servo-baseline.ts, src/ui/*), and none in bench/.

🤖 Generated with Claude Code

Approved — before the latest push

Approved. A bench-only change: no new route, no new dependency (ws is already used), and the sweep refuses to start while anyone is driving, STOP or an E-stop is in force, and stops on a closed stream or a snap-back. The fakeArm() extraction keeps bench() behaviour, and the new tests would fail without the change (targets clamp, one line per joint in a stable shape, refusal while driving). I read the diff and did not run the sweep or the tests; the preview serves the app, not the bench, so there is nothing to click. Two things I would do next: 1) Run it once against a real arm and note the result, since the first target (the middle) may be farther than the 400-tick maxStep from where the joint rests, and the sweep would then abort as a snap-back; if so, approach the middle in steps. 2) Make the per-joint line cover only this run on a live arm, for example by printing the delta against the before card, so it is not mixed with the 48 h window.

commented

I merged main into this branch and pushed it. The merge had one conflict, in Software/README.md. It was my two limits-sweep lines against the new bench/precision-report.ts paragraph and its "Open experiment" line from main. I kept both, in that order, and changed no wording. The bench tests pass (19 across 5 files); I ran only the bench tests, not the full check.

The first fetch from the forge returned a 503, so I retried after a short wait and it went through.

The pull still adds bench/limits-sweep.ts, which drives each joint through a fixed sweep and prints that joint's /precision line. It runs on the fake bus with no url, or against an arm URL, where it won't start while someone is driving and stops on STOP or any refusal. bench/precision.ts now exports fakeArm() for it. The new test is bench/limits-sweep.test.ts. The per-joint numbers on a live arm come from the server's 48-hour window, so they include earlier sessions of that joint.

🤖 Generated with Claude Code

commented

I fixed the one failing test by changing the test, not the code under test. The test is flaky under load. I pushed the fix, but the forge has not yet re-run the full check.

The failing test was "a release lands on the target > a 40° drag released at once" in Software/so101/src/release.test.ts. It passes on its own, and it failed only in the full 827-test run, which suggests timing under load. I did not find which assertion failed. I assumed it was the one expecting the arm to still be mid-move at release. The test sends a 40° drag in 200 ms to a fake servo moving at 2000 ticks/s. If the drag runs slow on a busy machine, the arm has already arrived by release, and the assertion that some goals were still short of the target fails.

The fix makes the fake servo move at 800 ticks/s in that test only. The arm is then still on its way when the release lands, even if the drag is slow. The post-release wait goes from 1.5 s to 2.5 s so the slower arm can arrive before the checks. The speed goes back to 2000 after the checks. If the test fails midway, 800 stays in place for the tests that follow.

release.test.ts passes on its own (5 pass). bun x tsc --noEmit reports errors in files I didn't touch, such as zone-notes.test.ts and visitor-counts.ts, and none in release.test.ts.

🤖 Generated with Claude Code

Approved

Approved. bench/limits-sweep.ts does what the issue asks: a fixed per-joint sweep that prints the /precision line per joint, before and after, on the fake bus or against a URL. It refuses to start while someone drives or STOP/E-stop is set, and it stops on any refusal. fakeArm() is a clean extraction from bench(). The tests cover targets, one line per joint with a stable shape, and the refusal. It adds no dependency and no page or tty surface, so the web and terminal checks do not apply. The README says how to run it.

Two things to do next:

  1. In src/release.test.ts, speed is a module-level variable that is reset only at the end of the test body. If an assertion fails, 800 leaks into the later tests. Reset it in afterEach, or pass it per test.
  2. The release.test.ts flake fix is unrelated to the sweep. Next time put it in its own pull. Also, the per-joint numbers on a live arm come from the 48 h window, so the first run on a real arm will include earlier sessions. Consider printing a note when the before-card already holds data for the joint.
Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs