Decide and write how the vxrail runs the forge with no control plane on another machine #11828

The decision is no Kubernetes for the forge on the vxrail. It runs as one process (src/release/launch.ts under a systemd unit), on one data directory (/var/lib/forge), and releases through the self-swap keeper (#8941). That is the box from docs/EXTRACT.md, and the vxrail becomes it.

Why this is the simpler choice (written as a paragraph in a new section of docs/OPERATING.md, "The vxrail without a control plane"):

• Everything the forge runs (apps, previews, checks, child forges, the crew) already runs in its own bubblewrap cells, not as pods. Kubernetes only gives it a supervisor and a volume, and systemd and a directory do the same job.
• A k3s server of the vxrail's own would be another control plane on one machine with no HA: the same single point of failure as pg-grand, just moved.
• A k3s server would also keep the image and registry path that #8942 deletes.
• The swap's kernel setting, net.ipv4.tcp_migrate_req=1, is one line in /etc/sysctl.d on a host but needs a special kubelet flag inside a pod.

The section also names what is given up: running checks as cluster Jobs, the Deployment-patch keeper, and the live kube reading behind the "What the forge stands on" table. Off Kubernetes, cells by default see more of the machine's mounts than they did in the pod, so the unit sets FORGE_CELL_MOUNTS=allow-list to keep them as narrow as before.

The steps, each a new issue with its rollback:

1. #11823 – the host is ready, taking no traffic. The unit starts launch.ts with Delegate=yes, the install script writes the sysctl and creates the forge user with the pod's uid, and a host tunnel unit is added. Proved by restoring a backup into a scratch directory.
2. #11824 – the cutover. The pod's existing data directory is bind-mounted, not copied, so rollback is just scaling the pod back to 1. Precondition: the gitops reconcile must stop managing the forge Deployment's replica count first (#8948). Otherwise it would start the pod again and two processes would write the same SQLite file.
3. #11825 – leave the cluster. The data has to be moved out of k3s storage first, because k3s-agent-uninstall.sh deletes /var/lib/rancher. Then the node is drained and the agent uninstalled.
4. #11826 – pg-grand's other tenants:

- Forgejo: a final dump of Forgejo and its database goes, encrypted, to the off-site bucket from #11805. - Tenant apps' CloudNativePG databases: the owner decides per app to bring it into the forge (SQLite) or archive it as an encrypted pg_dump. - MinIO: copied once to the same bucket. - Vxrail: none of these runs there as a cluster service.

docs/EXTRACT.md gets a short section, "The vxrail is the box", listing the three ways the vxrail differs from the standard box recipe.

Issue links: all four new issues wait on #11804. #11824 also waits for #11823, #11803, #11802 and #11805. #11825 waits for #11824, and #11826 waits for #8942, #8950 and #8951. I also made two existing issues wait for #11824: the thirty-minute proof (#11806) and the switch to self-swap releases (#8941). Most new steps need a person or the owner to run them on the machine.

The gate script, the type check and the 76 test files the fast tier picked all pass, including test/footing.test.ts and test/forge-box-unit.test.ts. I did not change the generated table in docs/OPERATING.md; its rows move in #11824 and #11825.

🤖 Generated with Claude Code

Closes #11804

The decision is no Kubernetes for the forge on the vxrail. It runs as one process (src/release/launch.ts under a systemd unit), on one data directory (/var/lib/forge), and releases through the self-swap keeper (#8941). That is the box from docs/EXTRACT.md, and the vxrail becomes it.

Why this is the simpler choice (written as a paragraph in a new section of docs/OPERATING.md, "The vxrail without a control plane"):

• Everything the forge runs (apps, previews, checks, child forges, the crew) already runs in its own bubblewrap cells, not as pods. Kubernetes only gives it a supervisor and a volume, and systemd and a directory do the same job.
• A k3s server of the vxrail's own would be another control plane on one machine with no HA: the same single point of failure as pg-grand, just moved.
• A k3s server would also keep the image and registry path that #8942 deletes.
• The swap's kernel setting, net.ipv4.tcp_migrate_req=1, is one line in /etc/sysctl.d on a host but needs a special kubelet flag inside a pod.

The section also names what is given up: running checks as cluster Jobs, the Deployment-patch keeper, and the live kube reading behind the "What the forge stands on" table. Off Kubernetes, cells by default see more of the machine's mounts than they did in the pod, so the unit sets FORGE_CELL_MOUNTS=allow-list to keep them as narrow as before.

The steps, each a new issue with its rollback:

1. #11823 – the host is ready, taking no traffic. The unit starts launch.ts with Delegate=yes, the install script writes the sysctl and creates the forge user with the pod's uid, and a host tunnel unit is added. Proved by restoring a backup into a scratch directory.
2. #11824 – the cutover. The pod's existing data directory is bind-mounted, not copied, so rollback is just scaling the pod back to 1. Precondition: the gitops reconcile must stop managing the forge Deployment's replica count first (#8948). Otherwise it would start the pod again and two processes would write the same SQLite file.
3. #11825 – leave the cluster. The data has to be moved out of k3s storage first, because k3s-agent-uninstall.sh deletes /var/lib/rancher. Then the node is drained and the agent uninstalled.
4. #11826 – pg-grand's other tenants:

- Forgejo: a final dump of Forgejo and its database goes, encrypted, to the off-site bucket from #11805. - Tenant apps' CloudNativePG databases: the owner decides per app to bring it into the forge (SQLite) or archive it as an encrypted pg_dump. - MinIO: copied once to the same bucket. - Vxrail: none of these runs there as a cluster service.

docs/EXTRACT.md gets a short section, "The vxrail is the box", listing the three ways the vxrail differs from the standard box recipe.

Issue links: all four new issues wait on #11804. #11824 also waits for #11823, #11803, #11802 and #11805. #11825 waits for #11824, and #11826 waits for #8942, #8950 and #8951. I also made two existing issues wait for #11824: the thirty-minute proof (#11806) and the switch to self-swap releases (#8941). Most new steps need a person or the owner to run them on the machine.

The gate script, the type check and the 76 test files the fast tier picked all pass, including test/footing.test.ts and test/forge-box-unit.test.ts. I did not change the generated table in docs/OPERATING.md; its rows move in #11824 and #11825.

🤖 Generated with Claude Code