Public: anyone can read this; the crew can change it.

Spin down the Kubernetes platform: everything lives in the forge #8953

Merged opened this in answer to

Check · Review · Land · Live done — Landed and live in production.

Merged

torque/forge:torque/7667-spin-down-the-kubernetes-platform-everyt → main 1 commit fork

  • Merged. torque/7667-spin-down-the-kubernetes-platform-everyt was merged into main as d4a536f.

Review

Sign in to review this change.
3 files changed +109 −0
README.md +1
@@ -560,6 +560,7 @@ This page is the front door. Everything else is a file of its own under
560560 - [`docs/CLOUD.md`](docs/CLOUD.md) — clouds are capacity, never dependency, and the cloud accounts an admin keeps.
561561 - [`docs/CLUSTER.md`](docs/CLUSTER.md) — the check lane as Kubernetes Jobs.
562562 - [`docs/EXTRACT.md`](docs/EXTRACT.md) — the forge on a machine of its own, and the platform it grew up in switched off: what a host owes it, the moves in order, and what is given up.
563 - [`docs/PLATFORM.md`](docs/PLATFORM.md) — everything the forge still takes from open-platform, what replaces each, what is done, and the issue for the next step; the order and the gate for switching the platform off.
563564 - [`docs/ops/ROLL.md`](docs/ops/ROLL.md) — rolling a release.
564565
565566 **Design and records**
docs/PLATFORM.md added +86
@@ -0,0 +1,86 @@
1 # Platform inventory — what the forge still takes from open-platform (#7667)
2
3 Owner, 2026-11-02: "this does go hand in hand with wanting everything to be
4 forge native now and getting off of the platform and the forge to forge
5 origin connections."
6
7 This page lists each thing the forge and its apps still take from the
8 platform (the Kubernetes cluster, Forgejo at git.open-platform.sh, Flux,
9 Kyverno, the Cloudflare edge, the droplet on pg-grand), what replaces it,
10 what is already done, and the issue that does the next step. It is written
11 from this repository. What lives only in the cluster or in the gitops
12 repository (`trevato-apps/gitops`) cannot be read from here and is marked
13 **unread**; #8948 reads it.
14
15 **Nothing is turned off or removed by this page or by any step it names
16 without the owner present. The forge never goes dark.** Every step has its
17 rollback written in its issue, and the platform stays up until the last
18 step (#8952).
19
20 ## What #1432 retired, and what it did not
21
22 #1432 (the forge deploys itself) is closed. What it delivered, on main:
23
24 - `releaseOnLand` checks every landed tree out to `releases/<sha>/` and
25 preflights it (`src/checks/landing.ts`); the keeper cuts, swaps in place
26 with SO_REUSEPORT, verifies under load and swaps back by name
27 (`src/keeper/`, `src/release/`; `docs/features/ship-itself.md`).
28 - A swap needs no image, no registry, no Forgejo and no Kubernetes roll.
29
30 What it did **not** retire, because production does not run it yet:
31
32 - Production is not switched to `FORGE_RELEASE_KEEPER=swap`
33 (`FORGE_REPO`, `FORGE_SELF_SWAP=1`); the swap has not run on a hosted
34 child with measured dark seconds (#8374 closed on the question, the drill
35 is #8940). Releases still go through a `v*` tag, the Forgejo release
36 workflow, an image on the platform registry, a chart, and Flux/standby
37 reconcile rolling the pod.
38 - The path is still in the tree: `.forgejo/workflows/release.yml`, the
39 Dockerfile's `VERSION`/`COMMIT` stamping, `keeper/deploy.ts`
40 `kubeDeployer`, `scripts/cut-release.ts`' image half.
41 - The droplet's `forge-release-tick` (pg-grand) and the steward's Mac
42 `tick release` still exist (**unread** here: only named in
43 `docs/features/ship-itself.md` and `release-keeper.md`).
44
45 ## The inventory
46
47 | # | The forge takes from the platform | Replaced by | Done already | Next step |
48 |---|---|---|---|---|
49 | 1 | **Image build and registry.** `release.yml` builds on the droplet runner, signs with cosign, pushes image and chart to the Forgejo registry (`forgejo-http.forgejo.svc.cluster.local:3000`). | Self-swap: the tree is the release; no image. | Code on main (#8042, #8054); runs in tests. | Drill on a child (#8940); switch production (#8941); delete the path after a week (#8942). |
50 | 2 | **Gitops and reconcile.** `trevato-apps/gitops` `apps/forge/prod/manifests.yaml`, `signup-gate.yaml`, `outsider-gate.yaml`; Flux; Kyverno; the standby reconcile on pg-grand (~15 min) which also resets the forge container's `image`. | A unit file and settings in the forge (env, limits, gates are forge settings); no reconcile. | Nothing: **unread** from here. | Read it into the forge and table every object (#8948). The reconcile stops owning `image` first (precondition of #8941). |
51 | 3 | **The pod and its supervisor.** A Deployment on local-path storage, `maxSurge 1`, restarts, a service account for the keeper's Deployment patch. | A box of the forge's own running `bun run src/main.ts` under systemd (`Restart=always`); `release/launch.ts` supervises swaps. Keeper #8374 proved the swap, not the box. | `docs/EXTRACT.md` steps 1–3, `docs/RESTORE.md`, `release/launch.ts`, `good.json`. | A box beside production taking no traffic (#8945). |
52 | 4 | **Ingress and TLS.** Cloudflare tunnel and cluster ingress carry `forge.open-platform.sh`, `*--forge.open-platform.sh` children and apps' own domains. | Caddy with a DNS-challenge wildcard and on-demand TLS asking `/__forge/domains/ask`, on the box. | Forge side done: `FORGE_TRUSTED_PROXIES`, `FORGE_CLIENT_IP_HEADER`, domains ask endpoint (`docs/DOMAINS.md`). | Staging name first, cutover runbook with rollback, the owner's go (#8946, after #8945). |
53 | 5 | **Identity.** `X-Plat-User` / `X-Plat-Edge-Token` (`src/http/edge.ts`): accounts the edge made have no password. | The forge's own password, passkey and OAuth between forges (#8902). `signin.edge` is the switch; an admin with neither is refused. | Switch exists (`docs/OPERATING.md`); `via: edge` is listed per account. | List edge-only accounts, a passkey for each, then switch off on a preview (#8947). |
54 | 6 | **Backup lanes.** The `backup` namespace's CronJobs (`FORGE_BACKUP_LANES`), `forgejo-dump` (suspended, 36 GB), the platform bucket's off-machine mirror. | The forge's own snapshot to its bucket and B2 (`docs/SNAPSHOT.md`), drilled weekly (`docs/RECOVERY.md`). | Snapshot, restore command, weekly drill, backup tile (`backup-truth.md`). | Decide the dump with the owner, a line for every lane, a restore with the platform unreachable (#8949, drill on #8945). |
55 | 7 | **Forgejo, where outside people send pulls.** grokbot's #42 to #67, `jmoran/forge-jay` (#8935), tdobbertin-espo's six (#8938) on git.open-platform.sh. | The forge: a real merge for a kept fork (#8935), a URL-to-landed-pull path for an agent (#6359), forge-to-forge origin (`docs/features/forge-origin.md`). | Origins and Bring exist (`docs/ORIGINS.md`). | Tell each person, then disable (not delete) new pulls on the Forgejo repository (#8950; waits for #6359, #8935). |
56 | 8 | **Hosted apps.** Apps the platform runs that are not on the forge. The list is in the cluster: **unread**. | Bring (paste a link) or Migrate; GitHub and Vercel go through #6007, not here. | Bring an app whole (#8919). | List with the owner's call per app (#8951). |
57 | 9 | **The droplet release tick.** `forge-release-tick` on pg-grand and the Mac's `tick release`. | The keeper (#1). | The keeper's code. | Switched off by the owner on the day #8942 records. |
58 | 10 | **Forgejo Actions jobs that are not releases.** `flake-hunt.yml`, `outside-gate.yml`, `sync-env.yml` on platform runners. | Scheduled jobs the forge runs, same verdicts. | `scripts/flake-hunt.ts`, checks and cells. | #8943. |
59 | 11 | **Previews of the forge's own repository.** `preview.yml` builds an image, signs it, pushes a chart. | A child forge in a cell (`src/spawn/placement.ts`, kind `cell`). | Apps preview from branch in a cell today. | #8944. |
60
61 ## Order and the gate
62
63 Cuts are made one at a time, each with its rollback tested before the next.
64 The map holds the order: #8941 waits for #8940, #8942 for #8941, #8946 for
65 #8945, #8950 for #6359 and #8935. Every item above, and #8951, feeds the
66 last: **#8952**, which starts nothing until all the others are done, then:
67
68 1. The owner sets a quiet period (suggested 30 days).
69 2. The platform's access logs, read at the edge, show no request from the
70 forge, its apps, its people or its agents for that period; the count is
71 posted weekly.
72 3. A restore drill passed in that period that did not touch the platform
73 (#8945, #8949).
74 4. A final snapshot of the platform's own state is in B2.
75 5. The owner says go. It is scaled to zero, not deleted, and kept a month.
76
77 ## Outside agents
78
79 Outside agents work against the forge and are pointed there (#8950). The
80 route `trevato-apps/forge` on Forgejo closes only after #6359 gives them a
81 way in, and then pulls are disabled, never deleted, with rollback in
82 Forgejo's settings.
83
84 ## Not here
85
86 Moving GitHub and Vercel in is #6007. No cut is started by this page.
test/platform-doc.test.ts added +22
@@ -0,0 +1,22 @@
1 import { expect, test } from "bun:test";
2 import { existsSync, readFileSync } from "node:fs";
3 import { join } from "node:path";
4
5 // docs/PLATFORM.md (#7667) is the inventory of what the forge still takes
6 // from the platform. It stays true to the repository: the files it says
7 // can be deleted are really there, and it names the gate.
8 const root = join(import.meta.dir, "..");
9 const doc = readFileSync(join(root, "docs/PLATFORM.md"), "utf8");
10
11 test("every file the inventory names exists", () => {
12 const paths = [...doc.matchAll(/`((?:docs|src|scripts|\.forgejo)\/[A-Za-z0-9_./-]+\.(?:md|ts|yml))`/g)].map((m) => m[1]!);
13 expect(paths.length).toBeGreaterThan(8);
14 expect(paths.filter((p) => !existsSync(join(root, p)))).toEqual([]);
15 });
16
17 test("it covers each dependency the owner listed, and says nothing is turned off", () => {
18 for (const topic of ["Image build and registry", "Gitops and reconcile", "pod and its supervisor", "Ingress and TLS", "Identity", "Backup lanes", "Forgejo, where outside people", "Hosted apps", "release tick"])
19 expect(doc).toContain(topic);
20 expect(doc).toContain("without the owner present");
21 expect(doc).toContain("#8952");
22 });

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs