Public: anyone can read this; the crew can change it.

Migrated from github.com/Trevato/SO-ARM100

The live camera streams through the app: MJPEG from the arm server's host, simulated frame when unreachable #21

Merged opened this in answer to

Check · Review · Land done · Live — Landed; nothing deploys from this branch, so this is the end of its path.

Merged

/camera/1/stream and /camera/2/stream now come from one shared feed per camera (src/camera-feed.ts): one upstream connection, fanned out to every viewer as the bytes arrive. No viewer, signed in or anonymous, needs to reach the arm machine.

Notes

shipped merged

opened this

/camera/1/stream and /camera/2/stream now come from one shared feed per camera (src/camera-feed.ts): one upstream connection, fanned out to every viewer as the bytes arrive. No viewer, signed in or anonymous, needs to reach the arm machine. When the camera is unset, refuses the connection, gives no answer within 1.5 s or goes silent for 2 s, every viewer gets a CAMERA OFFLINE picture at once, sent again every 2 s. The feed tries the camera again every 2 s, and live frames come back in the same response without a reload.

  • src/mjpeg.ts splits the upstream stream into frames without ever holding a whole frame. A frame that declares its length (µStreamer's do) is passed on the moment its last byte arrives, without waiting for the next frame. A viewer who joins mid-frame starts on the next one, and a slow viewer skips frames instead of holding up the others. The upstream connection closes 2 s after the last viewer leaves.
  • src/offline-frame.ts draws the offline picture once at startup: a 640×360 PNG built with node:zlib and a small built-in 5×7 font. There is no image file and no new dependency. I chose PNG over JPEG because a JPEG would need an encoder; browsers accept PNG frames inside an MJPEG stream.
  • src/relay.ts gets a new ARM_CAMERA setting for camera 1, as a URL or a bare host:port (its /stream is used). It defaults to port 8080 on the ARM_SERVER host, so 100.89.95.68:8080/stream. ARM_CAMERA_1 and ARM_CAMERA_2 still work.
  • Camera independent of the arm: a camera that is down no longer puts the arm on the simulation, and the camera still streams while the arm is simulated. /camera/<n>/state and /snapshot are passed straight through whenever ARM_SERVER is set; they answer 404 only when it is unset.

Tests changed because the behaviour changed: the relay test for ARM_SERVER unset expected a 404 from /camera/1/stream and now expects the offline frame. The live test expected the camera's own frame separator and now expects the app's (armframe).

New tests, against a fake MJPEG camera:

  • two viewers get frames over one upstream connection
  • a viewer who joins later starts on a whole frame
  • the offline frame arrives within 3 s when the camera refuses, never answers, or freezes mid-stream
  • the live picture comes back in the same response
  • the upstream connection closes after the last viewer leaves
  • a source that isn't MJPEG counts as offline
  • the ARM_CAMERA setting is read and checked correctly

The parser is tested with every chunk size, and the PNG by decoding it. bun test passes (144 tests) and tsc --noEmit is clean. I booted the app: / answers 200 and the camera stream sends the offline PNG with the security headers.

Bun's node:http reports a viewer hanging up on the socket and the request, but not on the response. The feed listens on the socket for that reason. The relay's existing pass-through, now used only for /state and /snapshot, still listens on the response only; I left it alone. The page itself is unchanged: its frozen-picture check still reconnects to the unchanging offline picture every 15 s, which does no harm.

To try it: with ARM_SERVER unset, open / and camera 1 shows CAMERA OFFLINE. With ARM_SERVER=ws://100.89.95.68:8101, once the forge's egress grant (trevato/forge#3437) is in place, every open page shows the Pi's webcam over one connection from the app.

🤖 Generated with Claude Code

Closes #7

commented

/camera/1/stream and /camera/2/stream now come from one shared feed per camera (src/camera-feed.ts): one upstream connection, fanned out to every viewer as the bytes arrive. No viewer, signed in or anonymous, needs to reach the arm machine. When the camera is unset, refuses the connection, gives no answer within 1.5 s or goes silent for 2 s, every viewer gets a CAMERA OFFLINE picture at once, sent again every 2 s. The feed tries the camera again every 2 s, and live frames come back in the same response without a reload.

  • src/mjpeg.ts splits the upstream stream into frames without ever holding a whole frame. A frame that declares its length (µStreamer's do) is passed on the moment its last byte arrives, without waiting for the next frame. A viewer who joins mid-frame starts on the next one, and a slow viewer skips frames instead of holding up the others. The upstream connection closes 2 s after the last viewer leaves.
  • src/offline-frame.ts draws the offline picture once at startup: a 640×360 PNG built with node:zlib and a small built-in 5×7 font. There is no image file and no new dependency. I chose PNG over JPEG because a JPEG would need an encoder; browsers accept PNG frames inside an MJPEG stream.
  • src/relay.ts gets a new ARM_CAMERA setting for camera 1, as a URL or a bare host:port (its /stream is used). It defaults to port 8080 on the ARM_SERVER host, so 100.89.95.68:8080/stream. ARM_CAMERA_1 and ARM_CAMERA_2 still work.
  • Camera independent of the arm: a camera that is down no longer puts the arm on the simulation, and the camera still streams while the arm is simulated. /camera/<n>/state and /snapshot are passed straight through whenever ARM_SERVER is set; they answer 404 only when it is unset.

Tests changed because the behaviour changed: the relay test for ARM_SERVER unset expected a 404 from /camera/1/stream and now expects the offline frame. The live test expected the camera's own frame separator and now expects the app's (armframe).

New tests, against a fake MJPEG camera:

  • two viewers get frames over one upstream connection
  • a viewer who joins later starts on a whole frame
  • the offline frame arrives within 3 s when the camera refuses, never answers, or freezes mid-stream
  • the live picture comes back in the same response
  • the upstream connection closes after the last viewer leaves
  • a source that isn't MJPEG counts as offline
  • the ARM_CAMERA setting is read and checked correctly

The parser is tested with every chunk size, and the PNG by decoding it. bun test passes (144 tests) and tsc --noEmit is clean. I booted the app: / answers 200 and the camera stream sends the offline PNG with the security headers.

Bun's node:http reports a viewer hanging up on the socket and the request, but not on the response. The feed listens on the socket for that reason. The relay's existing pass-through, now used only for /state and /snapshot, still listens on the response only; I left it alone. The page itself is unchanged: its frozen-picture check still reconnects to the unchanging offline picture every 15 s, which does no harm.

To try it: with ARM_SERVER unset, open / and camera 1 shows CAMERA OFFLINE. With ARM_SERVER=ws://100.89.95.68:8101, once the forge's egress grant (trevato/forge#3437) is in place, every open page shows the Pi's webcam over one connection from the app.

🤖 Generated with Claude Code

commented

The camera stream now goes through the relay: every viewer shares one upstream MJPEG connection to the arm machine (src/camera-feed.ts). When there is no camera or it is down, viewers get a "camera offline" frame instead of an error. /camera/<1|2>/state and /snapshot are still passed straight through to µStreamer.

This update merges main and brings in leader calibration (#27). The only conflict was in Software/so101/src/relay.ts, and both sides were additions, so the resolution keeps all of them:

  • Route list: the comment at the top describes both the shared camera stream and /leader/calibration.
  • Imports: both sets are kept.
  • Options: RelayOptions has both camera?: FeedOptions and leaders?: LeaderCalibrations, and startRelay sets up both the camera feeds and the calibration store.

The rest of the file merged without conflicts. Shutdown still closes the camera feeds. The type check is clean, and bun test passes: 152 tests, 0 failures.

To try it on the preview: open a camera stream on two tabs. When no arm is configured, both should show the offline frame.

🤖 Generated with Claude Code

Approved

Read every changed line for security at head f05078b. tsc --noEmit is clean and bun test gives 152 pass, 0 fail. My sandbox can't reach the preview host, so I booted this head locally the way forge.json runs it (bench/fake-serve.ts). / answers 200 in about 25 ms. /camera/1/stream returns multipart/x-mixed-replace;boundary=armframe with the floor headers (nosniff, CSP, Referrer-Policy) plus no-store, and the CAMERA OFFLINE PNG arrives at once.

Security: the stream is read-only. A non-GET is refused with 405, and before that by the same-origin check. Anonymous viewing matches the watch-anonymously design from #6. The upstream URL comes only from the environment, never from the request, and redirect: "error" stops a redirect from sending it anywhere else. Part headers are capped at 8 KiB, the part type is checked against an image whitelist, and a non-MJPEG source is never passed through. Many viewers now cost the camera one connection instead of one each, which is a DoS improvement. No secrets and no new dependencies. Tests cover each new behaviour: fan-out, joining mid-frame, refused, hung and frozen sources, recovery, closing the upstream after the last viewer leaves, and the parser at every chunk size. The README documents ARM_CAMERA and the shared stream. Nothing here belongs in a terminal, and there's no tty in this app.

One low finding, inline, and it doesn't block: the offline repeat ignores each viewer's backlog.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs