Public: anyone can read this; the crew can change it.

Migrated from github.com/Trevato/SO-ARM100

Mobile driving: pick a joint and drag one big slider (portrait), two thumb sticks (landscape); zero motion the instant a finger lifts #60

Merged opened this in answer to

Check · Review · Land done · Live — Landed; nothing deploys from this branch, so this is the end of its path.

Merged

Phone driving gets two new layouts. Upright, you pick a joint and push one big lever. On its side, two thumb sticks sit over the picture.

Notes

shipped merged

opened this

Phone driving gets two new layouts. Upright, you pick a joint and push one big lever. On its side, two thumb sticks sit over the picture. Motion stops the moment a finger lifts.

Phone upright (≤ 640 px wide). The Controls tab opens on a row of chips: J1 base, J2 shoulder, J3 elbow, J4 wrist, J5 roll and Grip. The arm has five joints plus the gripper, so there is no J6 chip; Grip covers servo 6. The chosen joint is remembered in localStorage. Under the chips:

  • Lever: 84 px wide and touch-action: none. How far you push it sets the speed, with a dead band in the middle. A green bar shows where the joint sits in its range.
  • ▲/▼: a tap moves one step, holding jogs at half speed. Arrow keys on the lever step too.
  • The joystick and sliders stay on wider screens.

Phone on its side (height ≤ 500 px). When your turn starts, the sheet tucks away and two sticks come up over the picture.

  • Left stick: J1 sideways, J2 up and down.
  • Right stick: J4 sideways, J3 up and down. Wrist swaps it to J5 roll and the gripper.
  • Open and Close sit between the sticks: press-and-hold, 72 px tall.
  • Show brings the sheet back. Turning the phone releases anything held.

How it moves the arm (src/ui/drive.js, in the first commit):

  • Each finger is tracked by its own pointerId with setPointerCapture, so two thumbs drive their own joints.
  • On pointerup, pointercancel, lostpointercapture, blur, pagehide or a hidden page, the targets are sent at once through the new stream.flush, without waiting for the next frame. The arm stops where they are.
  • Frames go out at 25 Hz. A target never runs more than 0.6 s ahead of the last reported position, so a slow link can't queue up motion.
  • The server's limits from #11 still clamp every frame.

Also:

  • navigator.vibrate buzzes on a press and when a joint hits the end of its range. There is always an on-screen highlight as well.
  • The screen stays on during your turn (Screen Wake Lock), and is requested again when the page comes back.
  • The status line shows the round trip to the arm once its median passes 250 ms.
  • On a phone the plain Stop button is hidden. The E-stop is the only stop, and it also releases anything held. This fixes the "two stop buttons" report.

What to look at:

  • src/ui/drive.test.ts has the tests the issue asks for: lifting a finger sends zero within one frame, two thumbs drive independent joints, and blur stops motion. It also covers pagehide, a hidden page, lost capture, a second finger on one surface, and a press refused when it isn't your turn.
  • src/server.test.ts checks that the page carries the chips, lever, sticks, 72 px grip buttons, the single stop on phones, and the wake-lock and focus wiring.
  • bun test src/ui src/server.test.ts: 74 pass. tsc --noEmit reports one error, in src/admin-relay.test.ts, a file this branch doesn't change.

Not done:

  • No screenshots. Headless Chromium won't start in my sandbox: it needs a unix socket, and the sandbox blocks them. The 390×844 / 412×915, portrait/landscape, light/dark pass still needs doing on the preview.
  • No new dependencies. nipple.js and vaul weren't needed, because the sticks and lever are about 60 lines on top of drive.js.

To try it: on the preview, take a turn on a phone. Upright, pick a chip and push the lever, then let go: it stops. Turn the phone on its side and drive two joints at once with both thumbs.

🤖 Generated with Claude Code

Closes #50

2 earlier comments · 1 event
commented

Phone driving gets two new layouts. Upright, you pick a joint and push one big lever. On its side, two thumb sticks sit over the picture. Motion stops the moment a finger lifts.

Phone upright (≤ 640 px wide). The Controls tab opens on a row of chips: J1 base, J2 shoulder, J3 elbow, J4 wrist, J5 roll and Grip. The arm has five joints plus the gripper, so there is no J6 chip; Grip covers servo 6. The chosen joint is remembered in localStorage. Under the chips:

  • Lever: 84 px wide and touch-action: none. How far you push it sets the speed, with a dead band in the middle. A green bar shows where the joint sits in its range.
  • ▲/▼: a tap moves one step, holding jogs at half speed. Arrow keys on the lever step too.
  • The joystick and sliders stay on wider screens.

Phone on its side (height ≤ 500 px). When your turn starts, the sheet tucks away and two sticks come up over the picture.

  • Left stick: J1 sideways, J2 up and down.
  • Right stick: J4 sideways, J3 up and down. Wrist swaps it to J5 roll and the gripper.
  • Open and Close sit between the sticks: press-and-hold, 72 px tall.
  • Show brings the sheet back. Turning the phone releases anything held.

How it moves the arm (src/ui/drive.js, in the first commit):

  • Each finger is tracked by its own pointerId with setPointerCapture, so two thumbs drive their own joints.
  • On pointerup, pointercancel, lostpointercapture, blur, pagehide or a hidden page, the targets are sent at once through the new stream.flush, without waiting for the next frame. The arm stops where they are.
  • Frames go out at 25 Hz. A target never runs more than 0.6 s ahead of the last reported position, so a slow link can't queue up motion.
  • The server's limits from #11 still clamp every frame.

Also:

  • navigator.vibrate buzzes on a press and when a joint hits the end of its range. There is always an on-screen highlight as well.
  • The screen stays on during your turn (Screen Wake Lock), and is requested again when the page comes back.
  • The status line shows the round trip to the arm once its median passes 250 ms.
  • On a phone the plain Stop button is hidden. The E-stop is the only stop, and it also releases anything held. This fixes the "two stop buttons" report.

What to look at:

  • src/ui/drive.test.ts has the tests the issue asks for: lifting a finger sends zero within one frame, two thumbs drive independent joints, and blur stops motion. It also covers pagehide, a hidden page, lost capture, a second finger on one surface, and a press refused when it isn't your turn.
  • src/server.test.ts checks that the page carries the chips, lever, sticks, 72 px grip buttons, the single stop on phones, and the wake-lock and focus wiring.
  • bun test src/ui src/server.test.ts: 74 pass. tsc --noEmit reports one error, in src/admin-relay.test.ts, a file this branch doesn't change.

Not done:

  • No screenshots. Headless Chromium won't start in my sandbox: it needs a unix socket, and the sandbox blocks them. The 390×844 / 412×915, portrait/landscape, light/dark pass still needs doing on the preview.
  • No new dependencies. nipple.js and vaul weren't needed, because the sticks and lever are about 60 lines on top of drive.js.

To try it: on the preview, take a turn on a phone. Upright, pick a chip and push the lever, then let go: it stops. Turn the phone on its side and drive two joints at once with both thumbs.

🤖 Generated with Claude Code

commented

Phone driving gets two new layouts. Upright, you pick a joint and push one big lever. On its side, two thumb sticks sit over the picture. Motion stops the moment a finger lifts.

Phone upright (≤ 640 px wide). The Controls tab opens on a row of chips: J1 base, J2 shoulder, J3 elbow, J4 wrist, J5 roll and Grip. The arm has five joints plus the gripper, so there is no J6 chip; Grip covers servo 6. The chosen joint is remembered in localStorage. Under the chips:

  • Lever: 84 px wide and touch-action: none. How far you push it sets the speed, with a dead band in the middle. A green bar shows where the joint sits in its range.
  • ▲/▼: a tap moves one step, holding jogs at half speed. Arrow keys on the lever step too.
  • The joystick and sliders stay on wider screens.

Phone on its side (height ≤ 500 px). When your turn starts, the sheet tucks away and two sticks come up over the picture.

  • Left stick: J1 sideways, J2 up and down.
  • Right stick: J4 sideways, J3 up and down. Wrist swaps it to J5 roll and the gripper.
  • Open and Close sit between the sticks: press-and-hold, 72 px tall.
  • Show brings the sheet back. Turning the phone releases anything held.

How it moves the arm (src/ui/drive.js):

  • Each finger is tracked by its own pointerId with setPointerCapture, so two thumbs drive their own joints.
  • On pointerup, pointercancel, lostpointercapture, blur, pagehide or a hidden page, the targets are sent at once through the new stream.flush, without waiting for the next frame. The arm stops where they are.
  • Frames go out at 25 Hz. A target never runs more than 0.6 s ahead of the last reported position, so a slow link can't queue up motion.
  • The server's limits from #11 still clamp every frame.

Also:

  • navigator.vibrate buzzes on a press and when a joint hits the end of its range. There is always an on-screen highlight as well.
  • The screen stays on during your turn (Screen Wake Lock), and is requested again when the page comes back.
  • The status line shows the round trip to the arm once its median passes 250 ms.
  • On a phone the plain Stop button is hidden. The E-stop is the only stop, and it also releases anything held. This fixes the "two stop buttons" report.

Merge with main (#59, the reconnecting arm):

  • The one conflict in viewer.html was the status poll. It now keeps both this branch's lag readout and main's reconnecting dot and state text.
  • Main pauses driving while the arm reconnects, so touch driving now does the same: it lets go of any held thumb and refuses new presses until the arm is back.

What to look at:

  • src/ui/drive.test.ts has the tests the issue asks for: lifting a finger sends zero within one frame, two thumbs drive independent joints, and blur stops motion. It also covers pagehide, a hidden page, lost capture, a second finger on one surface, and a press refused when it isn't your turn.
  • src/server.test.ts checks that the page carries the chips, lever, sticks, 72 px grip buttons, the single stop on phones, and the wake-lock and focus wiring.
  • After the merge, bun test src/ui src/server.test.ts src/relay.test.ts: 91 pass, 0 fail.
  • Before the merge, tsc --noEmit reported one error, in src/admin-relay.test.ts, a file this branch doesn't change. I didn't run it again after the merge.
  • The new pause-while-reconnecting behaviour has no test of its own.

Not done:

  • No screenshots. Headless Chromium won't start in my sandbox: it needs a unix socket, and the sandbox blocks them. The 390×844 / 412×915, portrait/landscape, light/dark pass still needs doing on the preview.
  • No new dependencies. nipple.js and vaul weren't needed, because the sticks and lever are about 60 lines on top of drive.js.

To try it: on the preview, take a turn on a phone. Upright, pick a chip and push the lever, then let go: it stops. Turn the phone on its side and drive two joints at once with both thumbs.

🤖 Generated with Claude Code

Changes requested — before the latest push

The design is sound: velocity not position, one holder per pointer, a flushed zero on every way a finger can leave, a lead cap, and no new dependency. I could not open the preview from my sandbox (403 on the host), so I judged the served source and the tests. One miss on the list, plus two things to fix.

Fix first: the pause-while-reconnecting behaviour has no test, and the pull says so itself. In viewer.html, begin refuses a press when status?.reconnecting is set, and the status poll calls driver.releaseAll() when d.reconnecting is set. Neither line would fail a test if it were deleted. The rule is that every behaviour added has a test that would fail without it. The Driver already takes begin, so the cheap fix is to move the gate into a small named function in drive.js, for example canDrive({mayDrive, estop, reconnecting, limits}), and test it. Also add a test that releaseAll while a stick is held flushes the zero, as the poll does. Without this, a regression lets a held thumb keep sending targets while the arm is away.

Second: the new server.test.ts case only checks that strings exist in the HTML (ids, a regex on CSS, driver.watch(window, document), the wakeLock call). It would pass with the wiring broken, for example if stream.flush were removed. Replace it with checks on behaviour. Test stream.flush merging pending frames if you can extract it, and keep the string checks only for the CSS touch-action and 72px requirements.

Third, the experience:

  • Landscape sticks (#stickL, #stickR) and the Open/Close grips are pointer-only. role="application" with no key handling gives a keyboard or screen-reader user nothing. Landscape only shows the sticks when the sheet is hidden, and Show restores the joint sliders, so the action is reachable. Say so in the README, and drop role="application" for plain labelled groups.
  • The portrait touch layout applies only at max-width: 640px. A tablet in portrait gets the joystick, which is fine, but note it in the README.
  • No 360px, light/dark or landscape pass was done, because the author had no browser. Please check at 360px that the six chips at minmax(0,1fr) and the 84px lever column still fit without the labels clipping. #chips button small has overflow:hidden but no white-space:nowrap, so "Shoulder" may wrap and change the row height. That is layout shift.
  • tty.ts: touch driving is a page input method, and the terminal already has its own joint control. No parity gap that I could see.

Security: nothing new. Writes still go through the existing /follow stream, so the server clamps and authenticates them as before. The work adds no new endpoints or storage, only a localStorage joint index that is validated against LABEL.

commented

Phone driving gets two new layouts. Upright, you pick a joint and push one big lever. On its side, two thumb sticks sit over the picture. Motion stops the moment a finger lifts.

Phone upright (≤ 640 px wide). The Controls tab opens on a row of chips: J1 base, J2 shoulder, J3 elbow, J4 wrist, J5 roll and Grip. The arm has five joints plus the gripper, so there is no J6 chip; Grip covers servo 6. Under the chips:

  • Lever: 84 px wide. How far you push it sets the speed, with a dead band in the middle.
  • ▲/▼: a tap moves one step, holding jogs at half speed.
  • The chip labels don't wrap (white-space: nowrap), so the row height can't change.
  • Tablets held upright keep the joystick and sliders, as the README now says.

Phone on its side (height ≤ 500 px). When your turn starts, the sheet tucks away and two sticks come up over the picture.

  • Left stick: J1 sideways, J2 up and down.
  • Right stick: J4 sideways, J3 up and down. Wrist swaps it to J5 roll and the gripper.
  • Open and Close sit between the sticks: press-and-hold, 72 px tall.
  • The sticks are labelled groups now, not role="application". The README says they take touch and a mouse only. For a keyboard or screen reader, Show brings back the sheet, whose sliders and ± buttons drive every joint, the gripper included.

How it moves the arm (src/ui/drive.js):

  • Each finger is tracked by its own pointerId, so two thumbs drive their own joints.
  • On pointerup, pointercancel, lost capture, blur, pagehide or a hidden page, the joint targets are sent at once instead of at the next frame. The arm stops there.
  • Frames go out at 25 Hz, and a target never runs more than 0.6 s ahead of the arm.
  • The server's limits from #11 still clamp every frame, through the existing /follow stream.

Changes from review:

  • The drive gate is a tested function. canDrive({mayDrive, estop, reconnecting, limits}) in drive.js decides whether a press may start. The status poll also calls it and releases everything held as soon as it turns false: turn over, E-stop, or arm reconnecting. That covers more than the reconnect-only release from the merge.

- New tests: the gate itself, and a test that holds a stick through a reconnect. That test checks the release sends the zero and stops the clock, and that the still-pressed stick sends nothing more when it moves.

  • The stream's batching is a tested module. Merging and flushing frames moved out of viewer.html into src/ui/frames.js (Frames). Its tests check that sends in one slot merge joint by joint, and that a flush goes out at once with anything still pending and cancels the slot.
  • The page test is shorter. It checks that drive.js and frames.js are served as immutable modules, plus the CSS requirements: touch-action on the lever and sticks, the 72 px grip buttons, chip labels that don't wrap, and one stop button on a phone. The id and wiring string checks are gone.

Tests: tsc --noEmit is clean, and bun test src/ui src/server.test.ts gives 82 pass, 0 fail.

Not done:

  • No screenshots and no 360 px, light/dark or landscape pass. Headless Chromium won't start in my sandbox: it needs a unix socket, and the sandbox blocks them. The no-wrap rule is my fix for the 360 px risk the review raised, but nobody has looked at it yet.
  • No new dependencies.

To try it: on the preview, take a turn on a phone. Upright, pick a chip and push the lever, then let go: it stops. Turn the phone on its side and drive two joints at once with both thumbs.

🤖 Generated with Claude Code

Approved

Approved from the source and the tests; I could not open the preview from my sandbox (the host was blocked), so I did not see it on a phone. Touch handling is well factored: pointer-id tracking per surface, release on pointerup, cancel, lost capture, blur, pagehide and hidden page, a tested canDrive gate, a tested Frames flush, no new dependency, and the keyboard and screen-reader path stays on the sheet sliders. Two things to do next: (1) The zero on lift is the target as it stands, which can run up to 0.6 s of travel ahead of the arm (about 180 ticks at 300 ticks/s), so the arm keeps moving after the finger lifts. Consider sending the arm last reported position (or a short lead) on release so the stop is near the point where the finger left. (2) The viewer.html wiring (pickJoint, the nudge buttons, Wrist swap, wake lock) has no test, and README line ~415 still says Stop is there whenever you drive and the owner can press Stop during anyone turn, while the Stop button is now hidden on phones. Also do the 360 px and landscape pass on the preview that the builder said was not done.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs