Public: anyone can read this; the crew can change it.

Migrated from github.com/Trevato/SO-ARM100

Mobile driving: pick a joint and drag one big slider (portrait), two thumb sticks (landscape); zero motion the instant a finger lifts #81

Parked opened this in answer to
Closed without merging

Phone portrait now shows the camera on top at 16:9 and the 3D model right under it, both always on screen, and the model is the main way to drive. I couldn't run the page in a browser here: headless Chromium can't start in this sandbox because it can't create a unix socket.

Notes

stopped parked by the crew deploy 3e6f446

opened this

Phone portrait now shows the camera on top at 16:9 and the 3D model right under it, both always on screen, and the model is the main way to drive. I couldn't run the page in a browser here: headless Chromium can't start in this sandbox because it can't create a unix socket. So there are no screenshots in this pull, and the new browser layout test skipped locally. Nothing here has been tried by touch on a real phone.

Driving by touching the model (new src/ui/grab.js)

  • A finger (or the mouse) on a part of the arm turns that part's joint. Only that joint is sent, live, through the existing 33 ms latest-wins stream.
  • A finger on empty space orbits the view, and two fingers orbit and pinch-zoom. Neither sends anything.
  • When the finger lifts, the joint's last target goes out at once. Pointercancel, a lost capture, a hidden page and a window losing focus do the same.
  • The model shows your target. A faint ghost of the real arm appears when the arm lags behind by more than 15 ticks.
  • The URDF element's disable-dragging attribute did nothing, so anyone could turn a joint on the model even when it wasn't their turn. Now all real mouse events stop before they reach the element. A joint can only be taken by someone allowed to drive, and otherwise touching the model just orbits it.

Sliders

  • Every joint has its own slider again, all at once, one compact row each (label · slider · value) with the target and actual marks from #79.
  • They sit in a tray under the model. The grabber on top of the tray, or tapping the open tab again, folds it away so the model gets the space. The tray opens by itself when your turn starts.

Notes

  • The stop and E-stop banner, the arm's warnings, the owner's announcement and the page's replies no longer cover the camera or the model.
  • They become one pill in the status strip ("2 notes", red for a stop or an error). Tapping it opens a sheet, which also holds Resume driving and Clear the stop.

Removed

  • The thumb sticks, the one-joint lever and its chips, the desktop gripper joystick, and their modules drive.js and reach.js with their tests.
  • The page's first visit is now about 40.5 KB gzipped, against the 50 KB limit (it was about 49.1 KB).

Tests

  • src/ui/grab.test.ts checks that a touch-drag on a link streams targets for that joint only, and that orbit and pinch send nothing. It also checks that a lift, blur, pagehide or hidden page sends the last target at once, and that a second finger neither drives nor orbits.
  • server.test.ts now checks that no joystick, lever or chip markup remains, that the notes live in the sheet and not over the views, and that the portrait layout stacks the camera above the model.
  • Changed tests: in relay.test.ts and server.test.ts, the checks that named reach.js now name grab.js, because reach.js was deleted.
  • src/layout.test.ts loads the page in headless Chromium at 390×844 and 412×915, light and dark. It checks the boxes: camera on top at 16:9, model under it, all six sliders visible in the tray, and no note over either view after a Stop. It skips when no browser can start, as it did here. If the forge's check machine can start one, this is the first run.
  • The browser driver is src/browser.ts, which talks to the machine's own Chromium over the DevTools protocol, so Playwright isn't added as a dependency.
  • bun x tsc --noEmit is clean, and the server, relay and UI suites pass.

Not done

  • Screenshots: bun bench/shots.ts [dir] takes them on a machine where Chromium starts: both phones, upright and on their side, light and dark, with the tray open, folded, stopped, and with the notes sheet open.
  • Phone on its side: it keeps the floating strip and sheet, now over the camera and model side by side. The issue only specified the upright layout, so nothing more was done there.
  • Saved view choice: the stored choice of views moved to a new storage key, so an old one-view-at-a-time choice doesn't hide the model.

To try it: run bun run Software/so101/bench/fake-serve.ts and open it on a phone, or in a browser's device mode at 390×844. Drag a link of the model, pinch on empty space, fold the tray, then press Stop and tap the notes pill.

🤖 Generated with Claude Code

Closes #50

commented

Phone portrait now shows the camera on top at 16:9 and the 3D model right under it, both always on screen, and the model is the main way to drive. I couldn't run the page in a browser here: headless Chromium can't start in this sandbox because it can't create a unix socket. So there are no screenshots in this pull, and the new browser layout test skipped locally. Nothing here has been tried by touch on a real phone.

Driving by touching the model (new src/ui/grab.js)

  • A finger (or the mouse) on a part of the arm turns that part's joint. Only that joint is sent, live, through the existing 33 ms latest-wins stream.
  • A finger on empty space orbits the view, and two fingers orbit and pinch-zoom. Neither sends anything.
  • When the finger lifts, the joint's last target goes out at once. Pointercancel, a lost capture, a hidden page and a window losing focus do the same.
  • The model shows your target. A faint ghost of the real arm appears when the arm lags behind by more than 15 ticks.
  • The URDF element's disable-dragging attribute did nothing, so anyone could turn a joint on the model even when it wasn't their turn. Now all real mouse events stop before they reach the element. A joint can only be taken by someone allowed to drive, and otherwise touching the model just orbits it.

Sliders

  • Every joint has its own slider again, all at once, one compact row each (label · slider · value) with the target and actual marks from #79.
  • They sit in a tray under the model. The grabber on top of the tray, or tapping the open tab again, folds it away so the model gets the space. The tray opens by itself when your turn starts.

Notes

  • The stop and E-stop banner, the arm's warnings, the owner's announcement and the page's replies no longer cover the camera or the model.
  • They become one pill in the status strip ("2 notes", red for a stop or an error). Tapping it opens a sheet, which also holds Resume driving and Clear the stop.

Removed

  • The thumb sticks, the one-joint lever and its chips, the desktop gripper joystick, and their modules drive.js and reach.js with their tests.
  • The page's first visit is now about 40.5 KB gzipped, against the 50 KB limit (it was about 49.1 KB).

Tests

  • src/ui/grab.test.ts checks that a touch-drag on a link streams targets for that joint only, and that orbit and pinch send nothing. It also checks that a lift, blur, pagehide or hidden page sends the last target at once, and that a second finger neither drives nor orbits.
  • server.test.ts now checks that no joystick, lever or chip markup remains, that the notes live in the sheet and not over the views, and that the portrait layout stacks the camera above the model.
  • Changed tests: in relay.test.ts and server.test.ts, the checks that named reach.js now name grab.js, because reach.js was deleted.
  • src/layout.test.ts loads the page in headless Chromium at 390×844 and 412×915, light and dark. It checks the boxes: camera on top at 16:9, model under it, all six sliders visible in the tray, and no note over either view after a Stop. It skips when no browser can start, as it did here. If the forge's check machine can start one, this is the first run.
  • The browser driver is src/browser.ts, which talks to the machine's own Chromium over the DevTools protocol, so Playwright isn't added as a dependency.
  • bun x tsc --noEmit is clean, and the server, relay and UI suites pass.

Not done

  • Screenshots: bun bench/shots.ts [dir] takes them on a machine where Chromium starts: both phones, upright and on their side, light and dark, with the tray open, folded, stopped, and with the notes sheet open.
  • Phone on its side: it keeps the floating strip and sheet, now over the camera and model side by side. The issue only specified the upright layout, so nothing more was done there.
  • Saved view choice: the stored choice of views moved to a new storage key, so an old one-view-at-a-time choice doesn't hide the model.

To try it: run bun run Software/so101/bench/fake-serve.ts and open it on a phone, or in a browser's device mode at 390×844. Drag a link of the model, pinch on empty space, fold the tray, then press Stop and tap the notes pill.

🤖 Generated with Claude Code

commented

Parked by the crew: the preview's deploy failed in the place it ran, not in the change, and it could not be run again

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs