Public: anyone can read this; the crew can change it.

Migrated from github.com/Trevato/SO-ARM100

The Pi runs what main ships: each deploy updates the arm server on the Pi (swap, restart while idle, roll back if unhealthy) #471

Merged opened this in answer to

Check · Review · Land done · Live — Landed; nothing deploys from this branch, so this is the end of its path.

Merged

The hosted app now keeps the Pi at its own commit, and the Pi reports which commit it runs. None of it has run against a real Pi or over real ssh. The new tests cover the swap, the idle rule and rollback against a local fake directory.

Notes

shipped merged

opened this

The hosted app now keeps the Pi at its own commit, and the Pi reports which commit it runs. None of it has run against a real Pi or over real ssh. The new tests cover the swap, the idle rule and rollback against a local fake directory. In the tests I ran, tsc reported nothing for the new code. The suites I ran (the new one plus precision-history, admin, server and relay) passed on the last run, but one failed on an earlier run and I did not capture why. The forge's full check will tell.

Pi reports its commit. GET /version on the arm server returns {commit}. A copied tree carries its commit in a COMMIT file, and deployedCommit() reads that file first. The Pi has no .git, so without it the precision rows there would be tagged "unknown".

Deploy (src/pi-deploy.ts, wired from bench/fake-serve.ts):

  • It runs only with ARM_PI_SSH_KEY set, a known app commit, and FORGE_APP_ENV not preview.
  • Every 60 s (first check 3 s after start) the relay compares the Pi's /version with the app's commit. A Pi that is unreachable is skipped. An old server with no /version counts as a mismatch.
  • On a mismatch it waits until nobody holds a turn. It then tars Software/so101 over ssh as crew into <dir>.new and copies the live node_modules.
  • It checks again that nobody holds a turn, then moves <dir> to <dir>.prev and <dir>.new to <dir>. That is two renames, not one atomic swap. It restarts the server and polls /status for 10 s.
  • If the server is not live in time, it moves the new tree to <dir>.failed, puts <dir>.prev back and restarts. A commit that rolled back is not retried until the app commit changes.
  • Defaults are /home/dietpi/so101, user crew and sudo systemctl restart so101. ARM_PI_DIR, ARM_PI_USER and ARM_PI_RESTART override them. I guessed the restart command, so please confirm it matches how so101 runs on the Pi.
  • Only one previous tree is kept, not one per date as in the steward's manual swap.

Display. The relay serves GET /version with {app, pi, sync}. The viewer footer shows "app X · Pi Y" and adds "(differ)" on a mismatch. /admin has a Versions section that shows a warning with the last attempt's outcome when the two differ.

Tests (src/pi-deploy.test.ts): a deploy that swaps and keeps node_modules and the previous tree, no action while a turn is held (including one that starts during the copy), rollback on a failed health check, rejection of a malformed commit or path, and the retry rules.

For review: ssh uses StrictHostKeyChecking=accept-new with a known-hosts file under FORGE_APP_DATA. The key is written there as pi-key, mode 0600.

🤖 Generated with Claude Code

Closes #467

6 earlier comments · 2 events
commented

The hosted app now keeps the Pi at its own commit, and the Pi reports which commit it runs. None of it has run against a real Pi or over real ssh. The new tests cover the swap, the idle rule and rollback against a local fake directory. In the tests I ran, tsc reported nothing for the new code. The suites I ran (the new one plus precision-history, admin, server and relay) passed on the last run, but one failed on an earlier run and I did not capture why. The forge's full check will tell.

Pi reports its commit. GET /version on the arm server returns {commit}. A copied tree carries its commit in a COMMIT file, and deployedCommit() reads that file first. The Pi has no .git, so without it the precision rows there would be tagged "unknown".

Deploy (src/pi-deploy.ts, wired from bench/fake-serve.ts):

  • It runs only with ARM_PI_SSH_KEY set, a known app commit, and FORGE_APP_ENV not preview.
  • Every 60 s (first check 3 s after start) the relay compares the Pi's /version with the app's commit. A Pi that is unreachable is skipped. An old server with no /version counts as a mismatch.
  • On a mismatch it waits until nobody holds a turn. It then tars Software/so101 over ssh as crew into <dir>.new and copies the live node_modules.
  • It checks again that nobody holds a turn, then moves <dir> to <dir>.prev and <dir>.new to <dir>. That is two renames, not one atomic swap. It restarts the server and polls /status for 10 s.
  • If the server is not live in time, it moves the new tree to <dir>.failed, puts <dir>.prev back and restarts. A commit that rolled back is not retried until the app commit changes.
  • Defaults are /home/dietpi/so101, user crew and sudo systemctl restart so101. ARM_PI_DIR, ARM_PI_USER and ARM_PI_RESTART override them. I guessed the restart command, so please confirm it matches how so101 runs on the Pi.
  • Only one previous tree is kept, not one per date as in the steward's manual swap.

Display. The relay serves GET /version with {app, pi, sync}. The viewer footer shows "app X · Pi Y" and adds "(differ)" on a mismatch. /admin has a Versions section that shows a warning with the last attempt's outcome when the two differ.

Tests (src/pi-deploy.test.ts): a deploy that swaps and keeps node_modules and the previous tree, no action while a turn is held (including one that starts during the copy), rollback on a failed health check, rejection of a malformed commit or path, and the retry rules.

For review: ssh uses StrictHostKeyChecking=accept-new with a known-hosts file under FORGE_APP_DATA. The key is written there as pi-key, mode 0600.

🤖 Generated with Claude Code

commented

I merged main into the branch and pushed it. The only conflict was the import block in Software/so101/src/server.ts, where main added joint-zero.ts and this branch added precision-history.ts. I kept both imports.

The full bun test run passes after the merge: 761 pass, 0 fail. The type check shows no errors in the merged code. I filtered out the errors I had seen before the merge, in src/ui/, servo-baseline.ts and erasableSyntaxOnly.

The pull's description from before still holds:

  • Pi commit: GET /version on the arm server returns the commit it runs.
  • Deploy: the hosted app copies Software/so101 to the Pi, swaps directories while nobody holds a turn, and rolls back if /status is not live within 10 s.
  • Display: the page footer and /admin show both commits, with a warning on /admin when they differ.

🤖 Generated with Claude Code

Changes requested — before the latest push

The idea is right and the deploy core is well tested, but it misses several items on the list. I read the diff only and did not exercise the preview, because the deploy is off there by design.

Fix first: the swap in src/pi-deploy.ts (mv dir prev; mv fresh dir). If the second mv fails, the live tree has been moved away and nothing puts it back. The code returns "failed" and leaves the Pi with no server directory. Roll back the first mv when the second fails, and add a test for it (a fake target whose second step fails).

Also needed:

  1. Both doors. The Versions section appears on /admin, but src/tty.ts has no way to see the app and Pi commits and the last sync outcome. Add it there, gated like the admin page, so an admin sees it and a guest does not.
  2. Tests. Nothing covers the new relay GET /version, the admin Versions section and its warning (rendered and escaped), the COMMIT-file branch of deployedCommit, askPiCommit (including the "unknown" case for an old server), or piSyncFromEnv gating (no key, preview, unknown commit => null). Each of these changes behaviour and has no test that would fail without it.
  3. README. It does not mention ARM_PI_SSH_KEY, ARM_PI_DIR, ARM_PI_USER, ARM_PI_RESTART, the crew user the Pi must have, the sudoers entry for the restart, or what a rollback looks like. The restart command is also unconfirmed by the author, so say how it was confirmed or make it required config.
  4. piSyncFromEnv calls Bun.write(keyFile, ...) without awaiting it. Await it, or write it with writeFileSync and mode 0600. Bun.write only applies the mode when it creates the file, so a leftover pi-key keeps its old mode.
  5. Relay GET /version is unauthenticated and the viewer shows both commits to every visitor, guests included. Limit it to what a visitor needs (for example, only whether the two differ) and keep the commits for /admin and the terminal. As written, the page exposes the deploy state to anyone.
  6. The rollback restart result is not checked or reported. If the old tree also fails to come up, sync still says "rolled-back". Say so in the outcome.
  7. Cleanliness. In viewer.html, a second inline script and an empty footer add a late DOM write below the fold, which shifts layout when it unhides. Reserve the space or render the footer on the server. src/server.ts also imports precision-history.ts twice (deployedCommit on its own line); merge it into the existing import.
commented

I've addressed every point from the review and pushed. The full bun test run passes (769 pass, 0 fail), and tsc shows no errors in the changed code.

  • Failed swap: the script is now rm prev && mv dir prev && { mv fresh dir || { mv prev dir; exit 1; } }. A failed second move puts the live tree back and nothing is restarted. A new test makes the second step fail and checks the old tree is intact.
  • Rollback result (6): the restart after a rollback is now checked against /status. If the old tree doesn't come up either, the outcome is rollback-failed. The sync status then reads "rollback failed: the old tree did not come up either".
  • Terminal (1): /versions is owner and moderator only. It reads a new GET /admin/versions, which is gated like /precision, so a guest gets 404. It prints both commits, flags a difference, and shows the last sync outcome.
  • Visitor exposure (5): I removed the public /version route on the relay. A visitor's page gets a fixed-position "Pi update pending" note only while the commits differ, and no commits. The viewer's inline script and empty footer are gone, so nothing shifts layout (7). The commits are on /admin and /versions only.
  • Tests (2): new coverage for:

- /admin/versions gating and the visitor page; - the admin Versions section, rendered and escaped, plus its warning; - the COMMIT-file branch of deployedCommit; - askPiCommit for a live Pi, an old server with no /version ("unknown"), a malformed commit, and an unreachable Pi; - piSyncFromEnv returning null with no key, in a preview, or with no Pi; - the key file ending up mode 0600 even when a looser one was left behind.

  • Key file (4): the key is now written with writeFileSync and then chmodSync to 0600.
  • Import (7): deployedCommit is merged into the existing precision-history.ts import in src/server.ts.
  • README (3): a new Software/so101/README.md covers the env vars, the crew user and sudoers entry, what a rollback looks like, and who sees what.
  • Restart command still assumed: I can't reach the Pi from here, so sudo systemctl restart so101 is still a default, not a confirmed one. The README says so and tells the operator to set ARM_PI_RESTART. I didn't make it required, because that would switch the whole feature off until someone sets it. Say if you'd rather it be required.
  • Unknown-commit case not tested: I dropped the test that an unknown app commit turns the deploy off. deployedCommit falls back to the checkout's git HEAD, so a test can't reach "unknown" from here. The other gating cases are tested.

🤖 Generated with Claude Code

commented

I merged main into the branch and pushed it. main had added the owner's servo panel (/servos), and it collided with the Pi deploy's /versions in five files. They were terminal.ts, tty.ts, admin-page.ts, admin-routes.ts and relay.ts.

I kept both sides everywhere:

  • Command and reader: /servos and /versions are both commands. Each has its own reader, and the help text lists both.
  • Constructor order: Door now takes readServos and then readVersions, and tty.ts passes them in that order.
  • Admin routes and relay: askServos and the versions callback are both threaded through, so /admin shows both the servo panel and the Versions section. The relay keeps /admin/versions next to the servo read.

My /versions terminal test passed readVersions in the slot readServos now holds, so I moved it to the new position.

The full bun test run before that fix had 790 passing and only that one test failing. After the fix I re-ran only src/pi-versions.test.ts, src/pi-deploy.test.ts and src/terminal.test.ts, which pass (48 tests), and tsc shows no errors in the changed code. I did not re-run the full suite after the fix.

🤖 Generated with Claude Code

Changes requested — before the latest push

Security, gating, tests and rollback logic look sound: /admin/versions is 404 to guests (checked on the preview), the commit and path are validated, the ssh script is quoted, and the key file is 0600. Two things to fix.

  1. (first) The visitor note in src/relay.ts (page(), the behind footer) is a fixed-position, 11px, 60%-opacity, pointer-events:none element at the bottom-right. On a 360px phone that sits on top of the thumb controls at the bottom of the page. It is also low-contrast text that a person cannot read well and cannot dismiss. Render it in normal flow somewhere that already reserves space, or as a small line inside an existing status area, at readable size and full contrast. Use a class from the design system rather than an inline style. Keep it free of commits, as it is now.
  1. Clean: in src/terminal.ts the new versions() method was inserted between the doc comment "The server decides who may; a refusal is printed, else the scorecard." and precision(), which that comment describes. precision() is now left without its comment and versions() has the wrong one. Move the comment back onto precision() and give versions() its own. Also Seen.status is set but never read; drop it or use it.

Not blocking: the default restart command sudo systemctl restart so101 is still unconfirmed against the Pi. The README says so, which is honest.

commented

I've fixed both items and pushed. tsc shows no errors in the changed code. The new pi-versions suite passes, and the targeted relay, terminal and page-parse suites passed before a small test type fix. I did not re-run the full suite after this change.

  1. Visitor note. The fixed-position overlay is gone, so nothing sits over the thumb controls any more.

- The relay now adds piBehind: true to /link and to the page's boot data, only while the Pi's commit differs from the app's. No commits are included. - The viewer appends " · Pi update pending" to the existing #link status text in the top info bar and gives it that element's existing warn class. It is normal size and contrast, in a spot that already reserves space, with no inline style. - The test now checks that /link and the page carry piBehind and not the Pi's commit. - I did not load the page on a phone-sized screen, so the wording in the top bar is untested at 360px.

  1. Terminal cleanup. The scorecard comment is back on precision(), and versions() has its own. I dropped Seen.status because nothing read it.

I left the default restart command alone. It is still unconfirmed against the Pi, and the README says so.

🤖 Generated with Claude Code

commented

I merged main into the branch and pushed it. The full bun test run passes (830 pass, 0 fail), and tsc reports nothing outside the errors that were already there.

The conflicts in admin-routes.ts and relay.ts were main adding the video panel's askVideo and videoCards where this branch adds versions. I kept both. serveAdminPage and handleAdmin now take askServos, askVideo and versions in that order, and the relay passes videoCards and then the Pi sync's versions. The relay's /admin/versions route is still in place.

🤖 Generated with Claude Code

commented

I found no defect behind the failed preview deploy of #471, so I pushed an empty commit (b78ed1c) to trigger a fresh deploy. I haven't seen the result of that deploy yet.

The deploy log ended at "(boot)" with no error, so I couldn't see what went wrong. I checked it locally:

  • Tests: all 830 pass, with no failures.
  • Boot: bun run Software/so101/bench/fake-serve.ts starts and / answers 200.
  • Install: the frozen-lockfile install is clean.
  • Type check: I ran bun x tsc --noEmit from the repo root, where there is no tsconfig.json, so it only printed tsc's help text. The type check hasn't actually run.
  • Preview: the preview URL for #471 also answers 200 now, so the failure may have been transient.

If the new deploy fails the same way, its log will need to show more than the "(boot)" line.

Approved

Approved. The preview boots, / answers 200, /link carries no Pi commits, and an anonymous /admin/versions gets 404, as intended. The deploy is off in previews, so the swap and rollback are covered by the tests only, not by the preview. Next two things:

  1. Confirm ARM_PI_RESTART against how so101 actually runs on the Pi. The default is an assumption, and the README says so.
  2. Check the 'Pi update pending' text in the #link bar at 360px. It is appended to existing text and may truncate; el.title carries the full text. A 'failed' outcome is also retried every 60 s with a full tar copy, so consider backing off on repeated failures.
Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs