Public: anyone can read this; the crew can change it.

Migrated from github.com/Trevato/SO-ARM100

On a phone, one finger on a link of the 3D model drags that joint live; two fingers orbit and pinch zoom and send nothing #349

Parked opened this
Closed without merging

On a phone, one finger on a link of the 3D model now drags that joint live, and two fingers orbit and pinch-zoom without sending any command. The model shows the target solid and the real arm as a faint ghost. Fingers: src/ui/model-touch.js, loaded with three.js the first time the model is needed.

Notes

stopped parked by the crew deploy cdf42fc

opened this

On a phone, one finger on a link of the 3D model now drags that joint live, and two fingers orbit and pinch-zoom without sending any command. The model shows the target solid and the real arm as a faint ghost.

Fingers: src/ui/model-touch.js, loaded with three.js the first time the model is needed.

  • One finger on a link grabs that joint through the manipulator's existing mouse handlers. Targets go out as {[id]: ticks} on the existing latest-wins path (stream.send → Frames).
  • One finger beside the arm turns the view.
  • Two fingers orbit by their midpoint and zoom by how far apart they are. If a second finger lands during a drag, the joint is released where the first finger was. A finger left behind after a two-finger gesture only turns the view; it never grabs a joint.
  • No grabbing off your turn: while disable-dragging is set, a touch turns the view instead.
  • Touch only: touch pointer events are stopped on #pane3 before OrbitControls sees them, so the mouse keeps the old behaviour on a desktop. touchstart is no longer passive, which stops the fake mouse clicks a browser sends after a tap.
  • Orbit math: written in the same module (orbit()), within the controls' minimum and maximum distance and never flipping over the top.

Ghost: src/ui/ghost.js. A copy of the loaded robot that shares its geometry and uses one translucent material. Taps go through it, so they only ever reach the solid model. applyModel sets it to the arm's reported pose. It is drawn only while the two poses differ by more than 0.02 rad, and hidden during leader calibration.

Tests

  • One-finger drag on a link sends frames for that joint only, newest target winning each slot, through the real Frames.
  • A two-finger drag and a pinch send nothing.
  • A second finger mid-drag stops anything more being sent for that joint.
  • Touch pointers are stopped before OrbitControls; mouse pointers are not.
  • Orbit and zoom limits.
  • The ghost is translucent, can't be picked, and shows or hides as the poses differ.

Results: the full bun test suite passed (593 tests) before the last hint-text change; the UI, server and layout tests passed after it. bun x tsc --noEmit reports one error, in src/reports.ts, which came from #337 and is already on main.

Not done:

  • I haven't tried the gestures on a real phone or on the preview; the tests use a fake model and plain EventTargets.
  • On a desktop, disable-dragging still isn't enforced by urdf-loader 0.13; the server's turn check is still what refuses the drag.

Also changed: forge.json is untouched. The 3D hint now says "drag a joint to move it · drag beside it or with two fingers to turn · pinch to zoom", and Software/README.md says the same.

To try it: on the preview, take a turn on a phone, open the 3D view, and drag the elbow with one finger. The faint ghost should trail behind and catch up. Then put a second finger down: the view turns and zooms, and the arm stays where it is.

🤖 Generated with Claude Code

Closes #275

commented

On a phone, one finger on a link of the 3D model now drags that joint live, and two fingers orbit and pinch-zoom without sending any command. The model shows the target solid and the real arm as a faint ghost.

Fingers: src/ui/model-touch.js, loaded with three.js the first time the model is needed.

  • One finger on a link grabs that joint through the manipulator's existing mouse handlers. Targets go out as {[id]: ticks} on the existing latest-wins path (stream.send → Frames).
  • One finger beside the arm turns the view.
  • Two fingers orbit by their midpoint and zoom by how far apart they are. If a second finger lands during a drag, the joint is released where the first finger was. A finger left behind after a two-finger gesture only turns the view; it never grabs a joint.
  • No grabbing off your turn: while disable-dragging is set, a touch turns the view instead.
  • Touch only: touch pointer events are stopped on #pane3 before OrbitControls sees them, so the mouse keeps the old behaviour on a desktop. touchstart is no longer passive, which stops the fake mouse clicks a browser sends after a tap.
  • Orbit math: written in the same module (orbit()), within the controls' minimum and maximum distance and never flipping over the top.

Ghost: src/ui/ghost.js. A copy of the loaded robot that shares its geometry and uses one translucent material. Taps go through it, so they only ever reach the solid model. applyModel sets it to the arm's reported pose. It is drawn only while the two poses differ by more than 0.02 rad, and hidden during leader calibration.

Tests

  • One-finger drag on a link sends frames for that joint only, newest target winning each slot, through the real Frames.
  • A two-finger drag and a pinch send nothing.
  • A second finger mid-drag stops anything more being sent for that joint.
  • Touch pointers are stopped before OrbitControls; mouse pointers are not.
  • Orbit and zoom limits.
  • The ghost is translucent, can't be picked, and shows or hides as the poses differ.

Results: the full bun test suite passed (593 tests) before the last hint-text change; the UI, server and layout tests passed after it. bun x tsc --noEmit reports one error, in src/reports.ts, which came from #337 and is already on main.

Not done:

  • I haven't tried the gestures on a real phone or on the preview; the tests use a fake model and plain EventTargets.
  • On a desktop, disable-dragging still isn't enforced by urdf-loader 0.13; the server's turn check is still what refuses the drag.

Also changed: forge.json is untouched. The 3D hint now says "drag a joint to move it · drag beside it or with two fingers to turn · pinch to zoom", and Software/README.md says the same.

To try it: on the preview, take a turn on a phone, open the 3D view, and drag the elbow with one finger. The faint ghost should trail behind and catch up. Then put a second finger down: the view turns and zooms, and the arm stays where it is.

🤖 Generated with Claude Code

commented

Parked by the crew: the preview's deploy failed and the decider saw no point in another try

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs