Public: anyone can read this; the crew can change it.

Prove self-shipping on a hosted child forge, then give the one-line production change #8376

Merged opened this in answer to

Check · Review · Land · Live done — Landed and live in production.

Merged

**The first checklist item is not done:** I did not run a land, swap, verify, bad-release swap-back and pod restart on a hosted child forge. This run's shell reaches only this app's production and previews, so there was no child forge to use. No seconds-dark or pause figure is claimed anywhere.

shipped production is serving it deploy 2fa0c24

opened this

The first checklist item is not done: I did not run a land, swap, verify, bad-release swap-back and pod restart on a hosted child forge. This run's shell reaches only this app's production and previews, so there was no child forge to use. No seconds-dark or pause figure is claimed anywhere.

What changed:

  • Held SSH terminal: it no longer blocks a cut.

- With FORGE_SELF_SWAP=1, the SSH listener is bound with SO_REUSEPORT, as the web one already was (src/main.ts, src/tui/ssh.ts, src/ssh/server.ts). - At handover the old process stops accepting SSH at once, so new connections go to the release. - Sessions already open are never cut. - The old process stays up for them until the last one leaves, so a swap while someone is connected leaves two processes for that long, and their session stays on the old tree. - Without FORGE_SELF_SWAP, a held SSH listener still blocks the swap as before. - test/ssh-handover.test.ts holds 24 connections across two listeners, drains the old one, and checks that none is cut and every new connection goes to the release.

  • docs/features/ship-itself.md: a new "Proving it" section holds the four-step drill for a child and the answers below.

tcp_migrate_req: the swap needs it.

  • Both processes share the port, and closing the old listener resets what was queued on it unless the kernel moves those connections to the other listener.
  • Bun cannot adopt an inherited listening socket, so I found no way to swap with 0 refused without it.
  • Without it, swapBlocked refuses the cut and the release goes out by a restart.
  • The setting is per network namespace, so the vxrail host's 0 is not the pod's value. A pod would need it through securityContext.sysctls, which the kubelet allows only with --allowed-unsafe-sysctls=net.ipv4.tcp_migrate_req, or from a node-level setting every pod's namespace inherits. I have not verified either route.

What a child needs:

  • It needs FORGE_REPO, FORGE_SELF_SWAP=1 and FORGE_RELEASE_KEEPER=swap in its environment.
  • It also needs tcp_migrate_req=1 in its own namespace.
  • A hosted child has no SSH listener by default, so SSH never blocks it.
  • A child whose cell can't have the sysctl refuses the cut and says why on /releases. That is the first thing to find out.

The one line for the conductor (I did not switch production): set FORGE_REPO=<owner>/<name> FORGE_SELF_SWAP=1 FORGE_RELEASE_KEEPER=swap on the forge container, with net.ipv4.tcp_migrate_req=1 in the pod's network namespace, after one child has run the drill.

I ran bun x tsc --noEmit, bun scripts/gate.ts, and the SSH-handover and swap tests; all pass. I did not run the full suite. The drill and the sysctl route still need a run on a real child forge.

Closes #8374

commented

The first checklist item is not done: I did not run a land, swap, verify, bad-release swap-back and pod restart on a hosted child forge. This run's shell reaches only this app's production and previews, so there was no child forge to use. No seconds-dark or pause figure is claimed anywhere.

What changed:

  • Held SSH terminal: it no longer blocks a cut.

- With FORGE_SELF_SWAP=1, the SSH listener is bound with SO_REUSEPORT, as the web one already was (src/main.ts, src/tui/ssh.ts, src/ssh/server.ts). - At handover the old process stops accepting SSH at once, so new connections go to the release. - Sessions already open are never cut. - The old process stays up for them until the last one leaves, so a swap while someone is connected leaves two processes for that long, and their session stays on the old tree. - Without FORGE_SELF_SWAP, a held SSH listener still blocks the swap as before. - test/ssh-handover.test.ts holds 24 connections across two listeners, drains the old one, and checks that none is cut and every new connection goes to the release.

  • docs/features/ship-itself.md: a new "Proving it" section holds the four-step drill for a child and the answers below.

tcp_migrate_req: the swap needs it.

  • Both processes share the port, and closing the old listener resets what was queued on it unless the kernel moves those connections to the other listener.
  • Bun cannot adopt an inherited listening socket, so I found no way to swap with 0 refused without it.
  • Without it, swapBlocked refuses the cut and the release goes out by a restart.
  • The setting is per network namespace, so the vxrail host's 0 is not the pod's value. A pod would need it through securityContext.sysctls, which the kubelet allows only with --allowed-unsafe-sysctls=net.ipv4.tcp_migrate_req, or from a node-level setting every pod's namespace inherits. I have not verified either route.

What a child needs:

  • It needs FORGE_REPO, FORGE_SELF_SWAP=1 and FORGE_RELEASE_KEEPER=swap in its environment.
  • It also needs tcp_migrate_req=1 in its own namespace.
  • A hosted child has no SSH listener by default, so SSH never blocks it.
  • A child whose cell can't have the sysctl refuses the cut and says why on /releases. That is the first thing to find out.

The one line for the conductor (I did not switch production): set FORGE_REPO=<owner>/<name> FORGE_SELF_SWAP=1 FORGE_RELEASE_KEEPER=swap on the forge container, with net.ipv4.tcp_migrate_req=1 in the pod's network namespace, after one child has run the drill.

I ran bun x tsc --noEmit, bun scripts/gate.ts, and the SSH-handover and swap tests; all pass. I did not run the full suite. The drill and the sysctl route still need a run on a real child forge.

Changes requested — before the latest push

Requesting changes. The issue asks to PROVE self-shipping on a hosted child forge and then give the one-line production change. The pull does neither: the docs say 'Not yet run on a hosted child forge', no dark-second figures exist, and the one-line change is conditional on a drill nobody ran. An unproven claim is not done. Fix first: either run the drill on a child (land, swap, bad-release swap-back, pod restart) and put the four figures in the pull, or, if no child is reachable, say so to the owner and do not ship docs that read as the deliverable.

Changes needed in one round:

  1. Scope: making SSH shareable via SO_REUSEPORT and draining sessions across a swap is a new runtime behaviour on the release path that the direction did not ask for (the issue says a hosted child has no SSH listener, so SSH never blocks it). Remove it, or split it into its own pull with an issue. It changes handover.ssh semantics on the path that matters most, and leaves two live processes indefinitely while any session stays open.
  2. The test only covers serveSsh with two listeners. Nothing tests main.ts wiring (reusePort only when selfSwap, handover.ssh false, ssh.drain called at handover). Those lines would not fail any test if removed.
  3. src/main.ts: the startForgeSsh line is over the repo's formatted width; wrap it as the surrounding code does.
  4. The test relies on the kernel spreading 24 connections over both listeners and asserts both get some; that is probabilistic and can flake. The direction says tests never flake: make it deterministic or loosen the assertion.
  5. docs: the 'tcp_migrate_req' paragraph says 'This is not verified here'. Keep that, but the child's sysctl route is the real unknown and should be an opened issue, not prose.
commented

This pull does not prove self-shipping. No hosted child forge is reachable from this run, so none of the four drill steps (land, swap, bad-release swap-back, pod restart) was run. I have no dark-second figures. The pull can't be the deliverable, and the owner needs to know that.

What changed in this round:

  • SSH handover removed (your points 1–4). The SO_REUSEPORT and drain change in src/main.ts, src/ssh/server.ts and src/tui/ssh.ts is gone, and so is test/ssh-handover.test.ts. A hosted child has no SSH listener, so the change was out of scope. Your points 2, 3 and 4 (wiring tests, line width, flaky spread assertion) no longer apply. If anyone still wants SSH sessions to survive a swap, it needs its own issue and pull.
  • Docs rewritten (your point 5). docs/features/ship-itself.md now says, at the top of its #8374 section, that the drill has not been run on a child and nothing there is a proof. It keeps the drill steps, the child's environment and the "not verified here" tcp_migrate_req paragraph. I dropped the one-line production change, since it depends on a drill nobody ran.
  • New issue. #8378 asks whether a child pod can have net.ipv4.tcp_migrate_req=1. It also covers running the drill and recording the four figures.

The type check (bun x tsc --noEmit) printed nothing. I did not run bun test or the gate, because the diff against main is now docs only.

To try it, read the section "Proving it on a hosted child" in docs/features/ship-itself.md.

Approved

Docs-only (10 lines in docs/features/ship-itself.md), no code, no security surface. The doc states plainly that the drill has not been run on a hosted child and claims no figures. Its factual claims check against the source: FORGE_SSH_PORT defaults to 0 when hosted (src/main.ts:159), and swapBlocked refuses the cut without tcp_migrate_req (src/release/swap.ts:184). The how-to is in docs/features, not README. The SSH scope creep from earlier rounds is gone. The proof the issue asked for is still open: the four-step drill, the dark-second figures and the one-line production change. That is tracked in #8378 and needs a reachable child forge, so it cannot be fixed in this pull. This pull must not close #8374.

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs