7c1a849
@@ -0,0 +1,23 @@
# Design proposals
From-first-principles design proposals for open platform features, written to be
reviewed and approved **before** implementation. Each is anchored in the real
code (concrete `file:line` for every seam it proposes touching), shows the actual
manifests / tool signatures / code shapes it would add, and is honest about the
hard parts, tradeoffs, and open questions.
These cover the larger / greenfield issues where the right move is to agree on
the architecture first. (The two enhancement issues that were tractable to build
directly — observability/Grafana #6 and platform agent-work #7 — shipped as PRs
rather than proposals.)
| Proposal                                                                                     | Issue | One line                                                                                                                                      |
| -------------------------------------------------------------------------------------------- | ----- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| [self-serve-secrets-vault.md](./self-serve-secrets-vault.md)                                 | #2    | Per-app env vars backed by HashiCorp Vault, **alongside** sops (not replacing it); the hard part is single-node unseal without cloud KMS.     |
| [self-serve-custom-domains.md](./self-serve-custom-domains.md)                               | #3    | Vercel-style BYO domains: self-serve granting + DNS ownership verification + public TLS, on top of the registry/resolver that already exists. |
| [byo-identity-provider-entra-oidc-forgejo.md](./byo-identity-provider-entra-oidc-forgejo.md) | #4    | BYO upstream IdP (Entra/OIDC) at `mitosis up`, auto-syncing users + teams into Forgejo (the identity hub).                                    |
| [repo-migration-auto-port.md](./repo-migration-auto-port.md)                                 | #5    | A Forgejo repo migration triggers an agent to conform the repo onto the app-template; bulk supported.                                         |
**How to use:** read a proposal, comment on its Open questions, and approve the
approach (or steer it). Approved proposals become the build plan — each lays out
its work as phased, per-repo PRs.
@@ -0,0 +1,284 @@
> Design proposal for issue #4 — "bring your identity provider". Status: PROPOSAL for maintainer review **before** implementation. Greenfield: there is no OIDC / `auth_source` / `add-oauth` anywhere in the toolkit or component repos today (verified: `grep -rln 'add-oauth|auth_source|oauth2_client|group-team-map' gitops/ agents/src/ mcp/src/ bin/` is empty except Grafana's own unrelated OAuth in `monitoring/helmrelease.yaml`).
## Problem
Today a germinated platform has exactly one human identity: the `plat` Forgejo admin, with a freshly-minted random password (`fork_regen_secrets` in `bin/lib.sh:241-244`, echoed on the YOUR PLATFORM card at `bin/germinate:497`). Every downstream identity decision keys off Forgejo:
- **Apps** use Forgejo as their OIDC provider via better-auth `genericOAuth` (`_app-template/src/auth.ts:170-188`: `authorizationUrl = ${FORGEJO_BASE}/login/oauth/authorize`, `tokenUrl`/`userInfoUrl` in-cluster). Forgejo *is* the apps' IdP.
- **MCP authz** keys off **Forgejo org membership**: a caller may operate on `owner` iff `caller.login === owner` or `caller.orgs.includes(owner)` (`mcp/src/server.ts:844-846`), and `caller.orgs` is fetched from Forgejo `/users/<login>/orgs` (`server.ts:490-498`).
So Forgejo is the platform's **identity hub**. The ask in #4 is: let an operator point that hub at their *real* corporate IdP (Microsoft Entra ID, Okta, Keycloak, generic OIDC) at `mitosis up` time, so their **users** can log in with corporate SSO and their **teams/groups** flow through to Forgejo orgs/teams — which means they flow, unchanged, into both app login and MCP authorization.
The hard constraints this must respect:
1. **Sovereignty + sealing.** A daughter holds only its own age key. Any IdP client secret must be regenerated/sealed to the fork key and pass `fork_verify_all_sealed` (`lib.sh:325`), the hard gate at `germinate:107`. But the IdP secret is **operator-supplied**, not platform-generated — a new class of secret.
2. **One-shot germinate.** No resume (`germinate:18-20`). IdP wiring must land in the existing post-handoff window (where OAuth app registration + runner registration already happen) or be fully declarative in gitops.
3. **Identity rewrite safety.** Redirect URIs are domain-bound (`https://git.$DOMAIN/...`); they must be produced from `$DOMAIN`, like every other identity-bound value, and survive `rewrite_identity_domain` (`lib.sh:42-60`).
4. **The QA reviewer login breaks if local auth goes away.** The agents dispatcher self-provisions a *local* Forgejo user `qatester` (`dispatcher.ts:2252-2284` → `forgejo.ts:442-453` `POST /admin/users`) and the reviewer browser-drives **Forgejo's local login UI** during OAuth sign-in (`dispatcher.ts:1573-1575` feeds `QA_USER`/`QA_PASS` into the reviewer prompt). If an operator disables local login in favor of pure OIDC, the reviewer can't log in and auto-❌s every PR.
## Design (from first principles)
### What "sync users + teams" actually means in Forgejo
Two separate capabilities, and Forgejo only has one of them:
- **Forgejo as an OIDC *relying party*** to an upstream IdP — implemented and stable. You add an OAuth2 **authentication source** (type `OpenID Connect`, with an auto-discovery URL). Users who click "Sign in with <IdP>" get a Forgejo account auto-created on first login, and their **OIDC `groups` claim is mapped to Forgejo orgs/teams** via a static `group-team-map` JSON, re-evaluated **on every login** (sources: Forgejo CLI docs `forgejo admin auth add-oauth`; Gitea PR go-gitea/gitea#21441 "Map OIDC groups to Orgs/Teams"). Optional `--group-team-map-removal` strips a user from mapped teams they no longer belong to.
- **SCIM push-provisioning** (IdP proactively creates/deletes Forgejo users) — **NOT implemented** in Forgejo (open feature request codeberg.org/forgejo/forgejo#8457). So we cannot get "user created in Entra ⇒ instantly exists in Forgejo." Provisioning is **login-driven (JIT)**.
This is the pivotal architectural fact. **v1 must be login-driven (lazy / just-in-time), not push.** A user appears in Forgejo the first time they SSO in; their teams sync at that same moment and on every subsequent login.
Two load-bearing constraints of the `group-team-map`:
1. **Sync is on login, not background.** No cron, no webhook. (Acceptable: people log into git.)
2. **The org + team must already exist.** The map *references* existing orgs/teams; Forgejo does **not** auto-create them on login (verified against Gitea/Forgejo docs + PR #21441). So **someone must pre-create the orgs/teams named in the map.** This is the one piece Forgejo won't do for us, and it's where a small sync component earns its place.
### The shape of the solution
```
mitosis up  (operator sets IDP_* env)
   └─ up → germinate (threaded env, like BYO claude creds today)
        ├─ seal IDP_CLIENT_SECRET into a NEW sovereign secret  (lib.sh: fork_regen_secrets)
        ├─ [post-handoff window] forgejo admin auth add-oauth ...   ← Forgejo becomes RP
        │     --provider openidConnect --auto-discover-url <issuer>/.well-known/openid-configuration
        │     --group-claim-name groups  --group-team-map '<JSON>'  --group-team-map-removal
        └─ [post-handoff window] run the idp-team-bootstrap step:
              pre-create every org+team named in the group-team-map (Forgejo API)
              so the login-time mapping has somewhere to land.
```
Once that's in place, **nothing downstream changes**: a corporate user SSOs into Forgejo → lands in org `acme` team `engineers` → `mcp/src/server.ts:490-498` sees `orgs:["acme"]` → authz at `server.ts:844-846` passes → apps' better-auth (`auth.ts`) sees the same Forgejo identity. The identity hub does its job; the new code only points the hub upstream.
### Why imperative (germinate post-handoff) over declarative (gitea.config)
Forgejo auth sources live in the **database**, not `app.ini`. You can seed them declaratively only via the chart's `gitea.oauth` values block (which runs `forgejo migrate` + an auth-source upsert on the configure init-container). That path exists, but:
- It would put the IdP **client secret** into the forgejo HelmRelease values (sealed secret + `valuesFrom`), and the forgejo chart's oauth-values plumbing is finicky about update-vs-create semantics across chart versions.
- germinate **already** does exactly this class of work imperatively in its post-handoff window: it `kubectl exec`s into the forgejo pod to register the CI runner (`germinate:371`: `forgejo forgejo-cli actions register`), and it registers the platform OAuth *app* over the API (`fork_register_oauth`, `lib.sh:400-414`). Adding `forgejo admin auth add-oauth` there is the **same exec seam, same window**.
**Recommendation: imperative for v1** (one `kubectl exec ... forgejo admin auth add-oauth`), because it reuses a proven seam and keeps the secret out of helm values. Revisit declarative `gitea.oauth` if we later want the source to survive a DB restore without re-running germinate (see Open Questions). Note: the auth source row lives in postgres (`forgejo-pg`), which **is** backed up (cnpg-backup), so a restore keeps it; a *fresh* germinate re-creates it. That's fine.
## Implementation (per-repo, phased PRs)
### PR 1 — `plat/mitosis`: env surface + sealed sovereign secret + Forgejo RP wiring
**The env-var surface** (mirrors the BYO-claude pattern at `germinate:35`, which captures `ANTHROPIC_API_KEY` into a named local then seals it):
| env var | required | meaning | example |
|---|---|---|---|
| `IDP_ISSUER` | yes (enables feature) | OIDC issuer URL; auto-discovery is `$IDP_ISSUER/.well-known/openid-configuration` | `https://login.microsoftonline.com/<tenant>/v2.0` |
| `IDP_CLIENT_ID` | yes | OAuth2 client (application) ID | `5f3c…` |
| `IDP_CLIENT_SECRET` | yes | OAuth2 client secret | (sealed, never echoed) |
| `IDP_NAME` | no (`corp-sso`) | display name of the "Sign in with…" button + auth-source name | `Entra` |
| `IDP_PROVIDER` | no (`openidConnect`) | Forgejo provider type | `openidConnect` |
| `IDP_SCOPES` | no (`openid profile email groups`) | scopes requested | |
| `IDP_GROUP_CLAIM` | no (`groups`) | claim that carries group names | `groups` (Entra) / `roles` |
| `IDP_GROUP_TEAM_MAP` | no (`{}`) | JSON: IdP group → `{org:[teams]}` | `{"plat-admins":{"plat":["Owners"]},"app-eng":{"acme":["engineers"]}}` |
| `IDP_ADMIN_GROUP` | no | IdP group whose members become Forgejo admins | `plat-admins` |
| `IDP_GROUP_TEAM_REMOVAL` | no (`true`) | strip users from mapped teams they left | |
| `IDP_DISABLE_LOCAL_LOGIN` | no (**`false`**) | turn off Forgejo username/password login | (see QA caveat below) |
**a) Defaults + capture, `bin/germinate` (~line 35, beside `BYO_ANTHROPIC`):**
```bash
# BYO upstream IdP (default empty) — when IDP_ISSUER is set, Forgejo is wired as an
# OIDC relying party to it and the group-team-map is honored on every login. Captured
# into named locals + unset from env so no child process inherits the client secret.
BYO_IDP_ISSUER="${IDP_ISSUER:-}"
BYO_IDP_CLIENT_ID="${IDP_CLIENT_ID:-}"
BYO_IDP_CLIENT_SECRET="${IDP_CLIENT_SECRET:-}"
: "${IDP_NAME:=corp-sso}"; : "${IDP_PROVIDER:=openidConnect}"
: "${IDP_SCOPES:=openid profile email groups}"; : "${IDP_GROUP_CLAIM:=groups}"
: "${IDP_GROUP_TEAM_MAP:={}}"; : "${IDP_GROUP_TEAM_REMOVAL:=true}"
: "${IDP_ADMIN_GROUP:=}"; : "${IDP_DISABLE_LOCAL_LOGIN:=false}"
unset IDP_CLIENT_SECRET   # mirror the BYO-claude unset hygiene
```
**b) Thread through `bin/up` (~line 119)** — add to the `env … "$HERE/germinate"` handoff, same as `FORK_KEY`:
```bash
env SEED="$SEED" DOMAIN="$DOMAIN" \
  ${FORK_KEY:+FORK_KEY="$FORK_KEY"} \
  ${IDP_ISSUER:+IDP_ISSUER="$IDP_ISSUER"} \
  ${IDP_CLIENT_ID:+IDP_CLIENT_ID="$IDP_CLIENT_ID"} \
  ${IDP_CLIENT_SECRET:+IDP_CLIENT_SECRET="$IDP_CLIENT_SECRET"} \
  ${IDP_NAME:+IDP_NAME="$IDP_NAME"} ${IDP_GROUP_TEAM_MAP:+IDP_GROUP_TEAM_MAP="$IDP_GROUP_TEAM_MAP"} \
  ${IDP_GROUP_CLAIM:+IDP_GROUP_CLAIM="$IDP_GROUP_CLAIM"} ${IDP_ADMIN_GROUP:+IDP_ADMIN_GROUP="$IDP_ADMIN_GROUP"} \
  ${IDP_DISABLE_LOCAL_LOGIN:+IDP_DISABLE_LOCAL_LOGIN="$IDP_DISABLE_LOCAL_LOGIN"} \
  "$HERE/germinate"
```
Also document in `bin/mitosis` usage (`mitosis:18-20`, next to the `CLAUDE_CODE_OAUTH_TOKEN` line).
**c) Seal the IdP secret — `bin/lib.sh` `fork_regen_secrets` (after the agents block, ~line 270).** The client secret is operator-supplied (passed through env, like the claude creds at `lib.sh:266-267`). We seal it into a **new** dedicated secret in the forgejo namespace so the auth-source step can `envFrom` it without touching the admin secret:
```bash
  # BYO upstream IdP — sealed only when the operator supplied an issuer. Lives in
  # the forgejo namespace so the auth-source bootstrap Job can envFrom it. The
  # client secret is operator-supplied (threaded through env like the claude creds),
  # NOT platform-generated — but it must still be sealed to the fork key so the
  # daughter's Flux can decrypt it and fork_verify_all_sealed passes.
  if [ -n "${IDP_ISSUER:-}" ]; then
    _fork_seal "$g" "$P/forgejo/idp-oidc-secret.yaml" forgejo-idp-oidc forgejo "$_FORK_LBL_MANAGED" <<EOF
  IDP_ISSUER: "${IDP_ISSUER}"
  IDP_CLIENT_ID: "${IDP_CLIENT_ID:-}"
  IDP_CLIENT_SECRET: "${IDP_CLIENT_SECRET:-}"
  IDP_NAME: "${IDP_NAME:-corp-sso}"
  IDP_PROVIDER: "${IDP_PROVIDER:-openidConnect}"
  IDP_SCOPES: "${IDP_SCOPES:-openid profile email groups}"
  IDP_GROUP_CLAIM: "${IDP_GROUP_CLAIM:-groups}"
  IDP_GROUP_TEAM_MAP: '${IDP_GROUP_TEAM_MAP:-{}}'
  IDP_ADMIN_GROUP: "${IDP_ADMIN_GROUP:-}"
  IDP_GROUP_TEAM_REMOVAL: "${IDP_GROUP_TEAM_REMOVAL:-true}"
EOF
    git -C "$g" add -A
    git -C "$g" -c user.email=mitosis@plat -c user.name=mitosis commit -q -m "mitosis fork: seal BYO IdP OIDC secret" || true
  fi
```
The caller at `germinate:97-98` already wraps `fork_regen_secrets` in an env prefix for the claude creds; add `IDP_ISSUER`/`IDP_CLIENT_ID`/`IDP_CLIENT_SECRET`/`IDP_GROUP_TEAM_MAP`/… to that same prefix so the function sees them. Because the secret is added to `$GITOPS_DIR` and sealed before `germinate:107`, `fork_verify_all_sealed` covers it for free (it scans every `^sops:` file under `clusters/` — `lib.sh:335`). **No change to the verify gate needed.**
Add `idp-oidc-secret.yaml` to `platform/forgejo/kustomization.yaml` (PR 2) so Flux keeps it reconciled (it's harmless if the feature is off — it just won't exist, since we only seal it when `IDP_ISSUER` is set; so add it conditionally — see PR 2 note on optional resource).
**d) New `lib.sh` helper — wire Forgejo as the OIDC relying party.** Reuses the exec seam from `germinate:371`:
```bash
# fork_wire_idp — add an OIDC authentication source to the daughter Forgejo so
# corporate users SSO in and their groups map to orgs/teams. No-op unless the
# operator supplied IDP_ISSUER. Runs in germinate's post-handoff window (the real
# postgres-backed Forgejo is up). Idempotent: update the source if it already exists.
#   $1=FORGEJO_NS  $2=forgejo pod  (admin auth comes from the source's own row)
fork_wire_idp() {  # $1=ns $2=pod ; reads IDP_* from env
  local ns="$1" pod="$2"
  [ -n "${IDP_ISSUER:-}" ] || { echo "    (no IDP_ISSUER — skipping upstream IdP wiring)"; return 0; }
  local disco="${IDP_ISSUER%/}/.well-known/openid-configuration"
  local extra=()
  [ -n "${IDP_ADMIN_GROUP:-}" ] && extra+=(--admin-group "$IDP_ADMIN_GROUP")
  [ "${IDP_GROUP_TEAM_REMOVAL:-true}" = true ] && extra+=(--group-team-map-removal)
  # add-oauth creates; if a source with this name exists, update-oauth instead.
  if kubectl -n "$ns" exec "$pod" -c forgejo -- forgejo admin auth list 2>/dev/null | grep -qiw "${IDP_NAME:-corp-sso}"; then
    local id; id=$(kubectl -n "$ns" exec "$pod" -c forgejo -- forgejo admin auth list 2>/dev/null | awk -v n="${IDP_NAME:-corp-sso}" '$0 ~ n {print $1; exit}')
    kubectl -n "$ns" exec "$pod" -c forgejo -- forgejo admin auth update-oauth --id "$id" \
      --key "$IDP_CLIENT_ID" --secret "$IDP_CLIENT_SECRET" --auto-discover-url "$disco" \
      --scopes "$IDP_SCOPES" --group-claim-name "$IDP_GROUP_CLAIM" \
      --group-team-map "$IDP_GROUP_TEAM_MAP" "${extra[@]}"
  else
    kubectl -n "$ns" exec "$pod" -c forgejo -- forgejo admin auth add-oauth \
      --name "${IDP_NAME:-corp-sso}" --provider "${IDP_PROVIDER:-openidConnect}" \
      --key "$IDP_CLIENT_ID" --secret "$IDP_CLIENT_SECRET" --auto-discover-url "$disco" \
      --scopes "$IDP_SCOPES" --group-claim-name "$IDP_GROUP_CLAIM" \
      --group-team-map "$IDP_GROUP_TEAM_MAP" "${extra[@]}"
  fi
}
```
> **Secret-handling caveat (call out in review):** passing `--secret "$IDP_CLIENT_SECRET"` puts the secret on the exec'd process argv. Inside the forgejo pod that's low-risk (it's the daughter's own pod), but cleaner is to read `IDP_*` from the **sealed `forgejo-idp-oidc` secret already mounted in the pod's env** rather than from germinate's shell — i.e. run the `add-oauth` from a tiny Job that `envFrom`s the secret (this is the same Job we need anyway for team bootstrap — see PR 2). **Recommended v1: fold `fork_wire_idp` into the idp-bootstrap Job in PR 2**, and keep the germinate call site as just "wait for the Job to complete." Both shapes are shown; the Job shape is preferred for secret hygiene + idempotent reconcile.
**e) Call site — `bin/germinate`, post-handoff window** (right after `fork_register_oauth` at `germinate:305`, where the forgejo pod handle `$FGPOD`/`$fp` is already discovered around `germinate:368`):
```bash
STEP="wire upstream IdP (OIDC relying party + group-team-map)"
if [ -n "${IDP_ISSUER:-}" ]; then
  echo "==> wiring Forgejo as an OIDC relying party to ${IDP_NAME:-corp-sso}"
  # FGPOD discovered by the runner-register loop (germinate ~368); reuse it.
  fork_wire_idp "$FORGEJO_NS" "$FGPOD" \
    && echo "    upstream IdP auth source ready (group-team-map honored on login)" \
    || echo "    WARN: IdP wiring failed — operators can re-run forgejo admin auth add-oauth"
fi
```
### PR 2 — `plat/gitops`: team-bootstrap Job + redirect note + (optional) declarative path
**The one thing Forgejo won't do: pre-create the orgs/teams in the map.** Model it exactly on `platform/plat-storage/cnpg-backup-setup-job.yaml` (idempotent Job, `envFrom` a secret, runs a script). New file `platform/forgejo/idp-bootstrap-job.yaml`:
```yaml
# Idempotent bootstrap for the BYO upstream IdP: (1) ensure the OIDC auth source
# exists (add/update-oauth), (2) pre-create every org+team named in the
# group-team-map so login-time team sync has somewhere to land (Forgejo does NOT
# auto-create them). Modeled on cnpg-backup-minio-setup. Re-runnable by Flux.
apiVersion: batch/v1
kind: Job
metadata:
  name: forgejo-idp-bootstrap
  namespace: forgejo
  labels: { plat.sh/managed: "true" }
spec:
  ttlSecondsAfterFinished: 300
  backoffLimit: 6
  template:
    spec:
      restartPolicy: OnFailure
      containers:
        - name: idp-bootstrap
          image: code.forgejo.org/forgejo/forgejo:11   # ships `forgejo`, curl, jq-free shell
          envFrom:
            - secretRef: { name: forgejo-idp-oidc }     # IDP_* (sealed, fork key)
            - secretRef: { name: forgejo-admin }        # username/password for API calls
          command: ["/bin/sh","-c"]
          args:
            - |
              set -eu
              FJ=http://forgejo-http.forgejo.svc.cluster.local:3000
              # (1) auth source — add or update. (Run forgejo CLI against the same DB.)
              #     forgejo admin auth add-oauth ... (flags from IDP_* env)  [idempotent guard]
              # (2) pre-create orgs+teams from IDP_GROUP_TEAM_MAP:
              #     for each group → {org:[teams]} create org then each team via API,
              #     ignoring 409/422 already-exists.
              #     POST $FJ/api/v1/orgs                 {"username":"<org>"}
              #     POST $FJ/api/v1/orgs/<org>/teams     {"name":"<team>","permission":"write"}
              # auth: Basic <username>:<password> from forgejo-admin
```
> Keeping the *parsing* of `IDP_GROUP_TEAM_MAP` out of this stub deliberately — it needs `jq` (the forgejo image may not ship it; use the `minio/mc`-style sidecar or a `node:22-bookworm` image like the runner uses at `runner-config.yaml:13`). **Implementation note for the engineer:** prefer `image: node:22-bookworm` + a 30-line script (already the platform's CI base image, guaranteed present in the registry mirror) over hand-rolling JSON parsing in `sh`.
Wire-in: add `idp-bootstrap-job.yaml` and (when present) `idp-oidc-secret.yaml` to `platform/forgejo/kustomization.yaml` (`forgejo/kustomization.yaml:3-19`). **Subtlety:** these resources only exist when the operator supplied `IDP_ISSUER` (the secret is sealed conditionally in PR 1). Two clean options: (a) germinate `sed`-appends them to the kustomization only when sealing the secret (consistent with the cleartext-patch pattern at `germinate:85-129`); or (b) always list them and ship an empty/no-op secret when the feature is off. **Recommend (a)** — don't ship a dead Job into platforms that don't use SSO.
**Redirect URI:** Forgejo's OIDC RP callback is a *fixed* path on the Forgejo host: `https://git.$DOMAIN/user/oauth2/<source-name>/callback`. The operator registers **that** URL in Entra/Okta as the app's redirect URI. It's derived from `$DOMAIN`, so it's correct after `rewrite_identity_domain` with no special handling (it's a `git.<dom>` token, already covered by `lib.sh:42-60`). **Document this in the YOUR PLATFORM card** (see PR 1 epilogue addition) so the operator knows exactly what to paste into Entra. No app-side redirect change — apps still point at Forgejo (`auth.ts:176`), Forgejo points at Entra; apps are oblivious.
### PR 3 — `plat/agents`: keep the QA reviewer working when local login is disabled
The reviewer browser-drives Forgejo's **local** login form with `QA_USER`/`QA_PASS` (`dispatcher.ts:1573-1575`), and `qatester` is a **local** account (`forgejo.ts:442-453`). Two failure modes and their handling:
1. **Local login still enabled (default, `IDP_DISABLE_LOCAL_LOGIN=false`).** Nothing breaks: `qatester` keeps logging in via the local form, the OIDC source is *additional*. **This is the recommended v1 default** — wire the upstream IdP but **do not** disable local login. The QA path is untouched.
2. **Operator sets `IDP_DISABLE_LOCAL_LOGIN=true` (pure SSO).** Now the reviewer can't log in. Handling options, smallest first:
   - **v1 (recommended): refuse to disable local login if QA creds are sealed**, OR keep a single local-login exception. Forgejo doesn't expose a per-user "allow local login" toggle, so the clean lever is: **keep local login enabled platform-wide but hide it from the UI is not possible** — instead, *don't* set `service.DISABLE_REGISTRATION`/sign-in lockdown for v1. Document that pure-SSO lockdown is a v2 follow-up.
   - **v2: give the reviewer an OIDC identity.** Create a `qatester` group in the upstream IdP mapped to a throwaway org, and have the reviewer drive the **"Sign in with <IdP>"** button instead of the local form. This requires the operator to provision a service account in *their* IdP — out of scope for a credential we generate, so it's strictly opt-in (`QA_IDP=true` plus operator-supplied `QA_IDP_USER/PASS`). The reviewer prompt template (`dispatcher.ts:1560-1576`) would branch on which login flow to drive.
   **Minimal code change for v1:** in `germinate`, if `IDP_DISABLE_LOCAL_LOGIN=true` **and** no QA-over-OIDC creds are provided, **warn loudly and force-keep local login on** (don't ship a platform whose own reviewer is locked out). The agents repo itself needs no change for v1; the change is a germinate guard + a documented v2 path.
## Validation plan
Build on the existing depth-2 k3d cold-start harness (per MEMORY: `mitosis-prs-and-coldstart-validation.md`), 16GB Mac, colima.
1. **Unit / lib.** Add a `lib.sh` test that runs `fork_regen_secrets` with `IDP_ISSUER` set and asserts `forgejo/idp-oidc-secret.yaml` exists, decrypts with the fork key, and lists the fork recipient — then runs `fork_verify_all_sealed` and expects pass. Repeat with `IDP_ISSUER` unset and assert the file is **absent** (feature off ships nothing).
2. **Mock-IdP cold start.** Stand up a throwaway OIDC provider in-cluster (Keycloak or `dexidp/dex` with a static `groups` claim) reachable at `https://idp.<domain>`; create a client; `IDP_ISSUER=… IDP_CLIENT_ID=… IDP_CLIENT_SECRET=… IDP_GROUP_TEAM_MAP='{"app-eng":{"acme":["engineers"]}}' nix run .#up`. Assert:
   - `forgejo admin auth list` in the forgejo pod shows the source (`fork_wire_idp` landed).
   - org `acme` + team `engineers` exist (the bootstrap Job pre-created them).
   - Browser-login a Dex user in group `app-eng` → Forgejo account auto-created → `GET /users/<u>/orgs` returns `acme`.
   - **End-to-end authz:** mint a Forgejo PAT for that SSO user, call MCP `list_prs owner=acme` → passes via `server.ts:846` (orgs includes acme). Call with `owner=someone-else` → `ForbiddenError` (`server.ts:847`). This proves group→team→org→authz flows unchanged.
3. **QA-path regression.** With local login left enabled (default), confirm the dispatcher still provisions `qatester` (`dispatcher.ts:2252`) and a reviewer run logs in. Then a "Use this template" app build/review completes (the canonical mitosis smoke).
4. **Identity-rewrite safety.** Germinate the mock-IdP daughter under a *different* `DOMAIN` than the seed; assert the OIDC source's callback URL is `git.<new-domain>/user/oauth2/.../callback` and `plat-local-ca-tls` survives (`lib.sh:55-57` gate).
## Tradeoffs & alternatives considered
- **Login-driven (JIT) vs SCIM push.** SCIM would give instant create/disable, but Forgejo has no SCIM (codeberg #8457). JIT is the only real option and is fine for a git-centric platform (people log into git). *Accepted limitation:* a user who never logs into Forgejo never exists there, and **deprovisioning is not instant** — disabling someone in Entra stops *new* logins but doesn't kill an existing Forgejo session/PAT. v1 documents this; the existing `admin_revoke_user_access` MCP kill-switch (`server.ts:857`) + Forgejo admin remain the break-glass.
- **Imperative `add-oauth` vs declarative `gitea.oauth` helm values.** Declarative survives a from-scratch helm reinstall without re-running germinate, but puts the secret in helm values and fights chart-version drift. Imperative reuses the proven `kubectl exec forgejo …` seam (`germinate:371`) and the source persists in `forgejo-pg` (which is backed up). Chose imperative; revisit if "auth source must survive `helm uninstall`" becomes a requirement.
- **Map-driven team pre-creation in a Job vs in germinate shell.** germinate is one-shot and would re-create nothing on drift; a Flux-reconciled Job re-creates orgs/teams if someone deletes them and re-runs on `IDP_GROUP_TEAM_MAP` changes (operator edits the sealed secret + reconciles). Chose the Job (also fixes the secret-on-argv hygiene issue with `fork_wire_idp`).
- **Disable local login by default vs keep it.** Disabling matches "pure SSO" but breaks the reviewer (`dispatcher.ts:1573`). Chose **keep local login on by default**; pure-SSO lockdown is opt-in + a v2 follow-up that gives the reviewer an OIDC identity.
- **Generic OIDC first vs Entra-specific.** Entra's `groups` claim emits **group object IDs (GUIDs), not names**, unless the app registration is configured for group names or the operator maps GUIDs in `IDP_GROUP_TEAM_MAP`. A generic OIDC/Keycloak/Okta source with human-readable group names is simpler to validate. Chose **generic OIDC as the v1 contract**, with Entra as the headline supported config and a documented "use group names or map the GUIDs" note.
## What I'm unsure of / want maintainer input
- Whether the forgejo chart version in use (`helmrelease.yaml:12` chart `17.0.0`) ships a `forgejo` binary whose `admin auth add-oauth` accepts **all** the flags I listed (`--group-team-map`, `--group-team-map-removal`, `--group-claim-name` are confirmed in current Forgejo CLI docs, but the exact subset varies by version — needs a `forgejo admin auth add-oauth --help` in the running pod to confirm before coding).
- Entra group-claim format (GUID vs name) — affects whether `IDP_GROUP_TEAM_MAP` keys are human-readable or GUIDs. Wants a real Entra tenant to validate against.
## Open questions
- Which IdPs do we commit to for v1? I recommend generic-OIDC as the contract with Microsoft Entra as the headline tested config (Keycloak/Dex as the in-cluster test double). Okta/Google are 'should work via generic OIDC, untested.' Agree?
- Entra emits the `groups` claim as group OBJECT IDs (GUIDs), not names, unless the app registration opts into group names. Do we (a) require operators to put GUIDs in IDP_GROUP_TEAM_MAP, (b) require them to configure Entra for group names, or (c) ship a small GUID→name resolution step? (a) is least code for v1.
- Keep Forgejo LOCAL login enabled by default (so the QA reviewer's qatester local account keeps working), and make pure-SSO lockdown (IDP_DISABLE_LOCAL_LOGIN) a documented v2 that also gives the reviewer an OIDC identity? Or is breaking the reviewer under pure-SSO acceptable for v1 with just a warning?
- Imperative `forgejo admin auth add-oauth` in germinate's post-handoff window vs a Flux-reconciled bootstrap Job vs declarative `gitea.oauth` helm values. I lean toward the Job (idempotent reconcile + keeps the client secret off the exec argv, and it ALSO does the org/team pre-creation Forgejo won't). Confirm?
- Forgejo's group-team-map references EXISTING orgs/teams and does not auto-create them; sync is on-login only (no SCIM, no background job). Are you OK with login-driven JIT provisioning + a bootstrap Job that pre-creates the mapped orgs/teams, accepting that deprovisioning is not instant (handled by the existing admin_revoke_user_access kill-switch + Forgejo admin)?
- Should the IdP client secret be a NEW sovereign secret class? It's operator-SUPPLIED (not platform-generated like every other fork_regen_secrets value) yet still sealed to the fork key and gated by fork_verify_all_sealed. Confirm that's the right model vs. leaving it as a post-germinate manual `sops set` like the BYO-claude fallback path.
- Confirm the running forgejo chart (17.0.0) binary supports the full flag set (--group-team-map, --group-team-map-removal, --group-claim-name, --admin-group, --auto-discover-url) before implementation — needs a `forgejo admin auth add-oauth --help` in a live pod.
@@ -0,0 +1,527 @@
# Repo-migration auto-port: conform migrated Forgejo repos onto the app-template
> Design proposal for maintainer review — **not yet implemented**. Issue #5 (discussion):
> *"Forgejo has a nice repo migration interface. when a user triggers a migration, agents
> should automatically start migrating it to the app template. bulk migration should be
> supported."*
## Problem
Forgejo's repo-migration UI imports a foreign repo (GitHub, GitLab, a bare git URL) into a
platform org. Today the platform does **nothing** with it — worse, it actively *filters it
out*:
- The dispatcher's only "new repo" path is `route()` → `maybeBootstrap()` on the
  `repository`/`created` event (`agents/src/dispatcher.ts:448-450`). `WEBHOOK_EVENTS`
  already includes `"repository"` (`dispatcher.ts:363-371`), so the event is delivered.
- `bootstrap()` then runs a **template-shape gate**: it probes for `charts/app/Chart.yaml`
  up to 7 times over ~30s (`dispatcher.ts:538-552`) and **early-returns** if absent,
  parking the repo in `notTemplate` and re-probing one cheap time per sweep forever
  (`dispatcher.ts:545-552`). A migrated repo has no `charts/app/Chart.yaml`, so it is
  silently ignored.
So a migrated repo lands in Forgejo and just sits there. The user's intent — "make this a
platform app" — is dropped on the floor.
**Goal (v1):** when a repo is migrated into a platform org, an agent automatically
*conforms* it onto the app-template — provisions the platform substrate (OAuth + better-auth
+ optional DB) via `create_app`, then ports the foreign source onto the template's contract
(`charts/app/`, `.forgejo/workflows/`, `Dockerfile`, the Fastify/TS server shape) — and
opens a PR. **Bulk** migrations (many repos at once) are supported without blowing past the
runner's concurrency cap. "Conformed" is defined precisely (the template's own `check`
smoke gate goes green).
**Non-goal (v1):** porting arbitrary stacks. The template is Fastify 5 + TypeScript on Node
22 (`agents/prompts/builder.md:15-18`; `_app-template/.forgejo/workflows/check.yml`). v1
scopes the *automatic* port to **Node/TS-compatible source**; non-Node source (Python/Go/…)
is detected and handed to a human with a clear note rather than silently mangled (see
[Handling non-Node source](#handling-non-node-source)).
---
## Design (from first principles)
### What is the actual trigger?
Forgejo fires the **`repository`** webhook event for migrations. Empirically Forgejo's
migration completes by *creating the repository* and the `repository` event's `action` is
`"created"` (a migration is, to the events layer, a repo coming into existence with content
already present — there is no distinct `"migrated"` action in the Gitea/Forgejo 1.22 event
schema the dispatcher targets). The dispatcher already receives this event
(`dispatcher.ts:448`). **So we do not need a new webhook receiver or HMAC** — we extend the
*classification* that happens after the existing `repository`/`created` event, plus the
sweep (which catches repos migrated while the dispatcher was down, exactly like it catches
created repos today at `dispatcher.ts:2218-2221`).
> **Honest uncertainty (open question O1):** I could not confirm from the cloned code
> whether Forgejo emits `action: "created"` vs. something else (e.g. no event at all for
> some migration modes, or a `pull_request`/`push` flurry). The migration is initiated in
> the Forgejo UI, not via our API, so there's no code in `mcp/` or `agents/` that observes
> it. The **sweep is the safety net**: `listRepos()` (`forgejo.ts:118-129`) enumerates every
> repo regardless of how it arrived, so even if the webhook is silent on migration, the next
> sweep (≤60s, `SWEEP_INTERVAL_MS`) discovers it. The webhook path is an optimization;
> correctness rides on the sweep. This must be validated empirically before we trust the
> event (see Validation).
### Why a new role, not a reused builder
The builder (`spawnBuilder`, `dispatcher.ts:879`) assumes a **fresh template clone** —
`charts/app`, `src/server.ts`, `src/auth.ts` etc. already present, working out of the box
(`prompts/builder.md:13-19`). A migrated repo has the *opposite* starting condition: real
foreign source, **no** template scaffolding. The job is not "design a v1 from a name"; it is
"preserve this code's behavior while re-expressing it on the template contract." That is a
distinct task with a distinct prompt and a distinct success definition. Hence a new **`port`
role**: `prompts/port.md` + `spawnPort()` mirroring `spawnBuilder()`.
### The substrate gap in `create_app` (the load-bearing hard part)
The proposal reuses `create_app` as the onboarding mechanism (it is idempotent and already
provisions OAuth + better-auth + optional CNPG Postgres + the gitops overlay). But
`create_app` was written for the *generate-from-template* path, and on a migrated repo it
partially breaks:
- **`/repos/plat/_app-template/generate` returns 409** ("repo already existed",
  `mcp/src/server.ts:1056`) because the migrated repo pre-exists. So **the template content
  is never laid down by `create_app`** — `git_content: true` (server.ts:1047) only applies
  when the repo is *created* by generate. The migrated repo keeps its foreign tree.
  *Consequence:* the agent — not `create_app` — must bring `charts/app/`,
  `.forgejo/workflows/`, `Dockerfile`, and the server shape.
- **`waitForCI(owner, name, "v0.1.0", 5*60_000)` (server.ts:1147) will hang/fail** on a
  foreign repo: there are no `.forgejo/workflows`, so no `check`/build run ever registers
  for the v0.1.0 tag, and the wait times out. This means calling `create_app` *first* on a
  raw migrated repo throws before OAuth/gitops are provisioned.
**Two ways out** (recommend B for v1):
- **(A)** Make `create_app` tolerant of a no-workflow repo: skip/relax the `waitForCI` step
  when the repo has no `.forgejo/workflows`. Cleaner long-term but edits the most sensitive
  MCP tool and weakens a real invariant for every caller.
- **(B, recommended)** **Order the port so the template plumbing lands *before*
  `create_app`.** The port agent's *first* pushed commit adds `.forgejo/workflows/`,
  `charts/app/`, `Dockerfile`, and a minimal compiling `src/server.ts` (copied verbatim from
  `_app-template` via the MCP `read_repo` tool against `plat/_app-template`, or fetched from
  the template clone). Once that commit is on a branch, the dispatcher calls `create_app`
  (CI now has a `check` workflow to run), then the agent continues porting business logic.
  This keeps `create_app` untouched and reuses the *exact* template contract as source of
  truth. The cost: the port flow is agent-driven from commit #1, not provisioned-then-built.
I recommend **B** for v1 and propose A as a follow-up hardening (it also helps the
generate-from-template race where CI is briefly absent). See
[Tradeoffs](#tradeoffs--alternatives-considered).
### Lifecycle shape
```
repo migrated into org
        │
        │  repository/created webhook  ──OR──  sweep listRepos()
        ▼
route() → maybeBootstrap(owner, name, desc)
        │
        ├─ charts/app/Chart.yaml present?  ── yes ──► EXISTING builder/adopt path (unchanged)
        │
        └─ no, but repo is non-empty (has a default branch + commits)?
                 │  yes  ─────────────────────────────────────────────┐
                 │  no (truly empty / plain non-migrated repo)         │
                 ▼                                                     ▼
          stay in notTemplate (today's behavior)            maybePort(owner, name, desc)
                                                                       │
                                              skipRepo / already-ported / inflight guards
                                                                       │
                                                          stack-compatibility probe (Node?)
                                                             ├─ incompatible ► file issue, stop
                                                             └─ compatible
                                                                       ▼
                                                              spawnPort(...)  (port role run)
                                                                       │
                                  1. branch `port`, commit template plumbing (workflows+chart+Dockerfile+server skeleton)
                                  2. dispatcher calls create_app (now CI-capable) → OAuth+DB+gitops
                                  3. agent ports foreign code onto src/, makes `check` green
                                  4. open/ья update PR `port → main`, drop 🏗️, ship summary
                                                                       ▼
                                          EXISTING validator path (maybeReview) — unchanged
```
The crucial reuse: **once the port branch carries `charts/app/Chart.yaml` and a green
`check`, the repo is indistinguishable from a built template app**, so the existing reviewer
/ preview / auto-merge / release machinery (`maybeReview`, `applyVerdict`, `maybeRelease`)
applies verbatim. The port role only has to get the repo *to* that state. **"Conformed" ≡
the template's `check.yml` smoke gate passes** (`_app-template/.forgejo/workflows/check.yml`:
lint + typecheck + build + `/version`→200, `/`→200, `/api/me`→401, `/openapi.json` has
paths). That is the same gate the builder must pass (`prompts/builder.md:18,32`), so the
definition is already enforced platform-wide — we inherit it for free.
### Dispatch path decision: extend the dispatcher, do NOT add an MCP webhook
The prompt offers two options. **Recommendation: extend the agents dispatcher.** Reasons,
all anchored:
- The dispatcher *already* owns the Forgejo system webhook (`ensureWebhook`,
  `dispatcher.ts:2591-2616`) and the `"repository"` event (`WEBHOOK_EVENTS`,
  `dispatcher.ts:363`). The migration trigger is the same event we already consume — adding
  a net-new authenticated MCP route (`mcp/src/server.ts`) would mean a *second* webhook
  subscription, a *second* idempotency store, and a brand-new auth surface.
- **There is no inbound webhook route or HMAC verification in `mcp/` today** (confirmed: the
  MCP `/hook`-style ingress does not exist; the dispatcher's `/hook` (`dispatcher.ts:2704`)
  is unauthenticated and trusts that only Forgejo can reach `host.k3d.internal:9909` on the
  cluster network). Building HMAC verification + a `plat/workflows` enqueue for migration
  would be a large net-new subsystem for a trigger the dispatcher already receives. Out of
  scope for v1.
- Durable/async heavy lifting still lands where it belongs: the **port run itself** is a
  bounded `claude` subprocess managed by the existing `Runner` queue
  (`runner.ts:180-219`), exactly like builder/worker/reviewer. We are not introducing
  long-lived orchestration that would warrant `plat/workflows`.
> If a future requirement needs migration to be auditable/replayable across daemon restarts
> beyond what `state.json` gives us, the MCP-webhook-→-`plat/workflows` design becomes worth
> it. Noted as open question O5.
### Bulk migrations
Forgejo's bulk migration creates N repos in a burst. The design handles this for free **as
long as we respect the existing back-pressure**:
- **Concurrency:** `Runner` already caps at `maxConcurrent` (default **2**,
  `dispatcher.ts:89`; `runner.ts:209-219` `pump()`), and `port` runs go through the same
  `enqueue()`. A 50-repo bulk migration enqueues 50 port runs that drain 2-at-a-time —
  *no new concurrency guard is strictly required*. **But** port runs are
  builder-priority-class work (long, expensive); to keep a bulk migration from starving
  real builders, give `port` its **own priority below builder** (see Implementation:
  `PRIORITY`). The queue is already priority+FIFO (`runner.ts:196-198`).
- **Idempotency:** add `ported: Record<string, true>` to `State` (`state.ts:10-38`),
  mirroring `bootstrapped`. `maybePort` checks it synchronously before enqueue (just like
  `maybeBootstrap` checks `this.state.bootstrapped[key]`, `dispatcher.ts:514`) so webhook +
  sweep deliveries of the same migrated repo can't double-spawn. Set it *before* spawn (one
  port per repo ever), the same discipline as `bootstrapped` (`dispatcher.ts:708`).
- **In-flight guard:** a `port:${key}` key in the existing `inflight` set
  (`dispatcher.ts:392`, used by `boot:`/`work:`/`review:`/`release:`), registered
  synchronously before any `await`, so racing webhook+sweep deliveries can't both pass.
No special rate limiter needed beyond the above — the queue *is* the rate limiter. (If bulk
migrations of hundreds of repos prove to overwhelm Forgejo API calls during classification,
a sweep-side batch cap is a cheap follow-up; not v1.)
---
## Implementation (per-repo, phased PRs)
All code is in **plat/agents** except Phase 4 (optional MCP hardening). No gitops changes
are required — the port lands on the same overlays `create_app` already renders.
### Phase 1 — state + classification scaffolding (plat/agents)
**`src/state.ts`** — add the idempotency map (mirror `bootstrapped`):
```ts
export interface State {
  bootstrapped: Record<string, true>;
  /** "owner/repo" → true once a migrated repo was conformed onto the template (issue #5). */
  ported: Record<string, true>;            // NEW
  // …existing fields…
}
// loadState(): add `ported: parsed.ported ?? {},` to the merged defaults (state.ts:52-62)
```
**`src/dispatcher.ts`** — a non-template-but-non-empty repo is a migration candidate. Today
`bootstrap()` early-returns when `charts/app/Chart.yaml` is absent (`dispatcher.ts:544-552`).
Branch that early-return into a port decision:
```ts
// in bootstrap(), replace the `if (!shaped) { … return; }` block:
if (!shaped) {
  // Not template-shaped. Is it a MIGRATED repo (has content) rather than an
  // empty/plain repo? A migrated repo has a default branch with commits.
  const migrated = await this.fg
    .hasFile(owner, name, "")          // probe default branch contents (see forgejo note)
    .catch(() => false);
  if (migrated) {
    this.maybePort(owner, name, description);
    return;
  }
  // …existing notTemplate quiet-probe behavior (unchanged)…
  return;
}
```
> **Forgejo probe note:** `hasFile(owner, name, "charts/app/Chart.yaml")` already exists
> (`forgejo.ts:132-140`). For "repo has any content" we need a cheap signal that a default
> branch with commits exists. Cleanest is a new `Forgejo.isNonEmpty(owner, repo)` that
> GETs `/repos/{owner}/{repo}` and reads `.empty` (Forgejo's repo object exposes an `empty`
> boolean) — `!empty` ⇒ migrated/non-empty. This is one typed field add to `forgejo.ts`.
> *Do not* rely on `hasFile(..., "")` (the contents API on an empty path is ambiguous);
> the `.empty` field is the correct, documented signal.
Add the `port:` guard + classification entry point next to `maybeBootstrap`
(`dispatcher.ts:510-525`):
```ts
maybePort(owner: string, name: string, description: string): void {
  const key = `${owner}/${name}`;
  if (skipRepo(name) || this.state.ported[key] || this.portFailed.has(key)) return;
  const flight = `port:${key}`;
  if (this.inflight.has(flight)) return;       // webhook+sweep race guard
  this.inflight.add(flight);
  void this.port(owner, name, description)
    .catch((e) => log("error", "port failed", { repo: key, error: e }))
    .finally(() => this.inflight.delete(flight));
}
```
Add `private readonly portFailed = new Set<string>();` next to `bootstrapFailed`
(`dispatcher.ts:396`) — a port that exhausts its retries parks until operator restart,
exactly like `bootstrapFailed`.
**Sweep wiring** — the sweep already calls `maybeBootstrap` for every repo
(`dispatcher.ts:2218-2221`); since `maybePort` is reached *through* `bootstrap()`'s
not-shaped branch, **no extra sweep loop is needed** — a repo migrated while the daemon was
down is rediscovered by the same `listRepos()` pass that feeds `maybeBootstrap`. (Verify:
`bootstrap()` runs its shape probe on every sweep for a not-yet-bootstrapped repo, so the
port branch fires there too.)
**Tests** (`dispatcher.test.ts` conventions, pure functions only): add a
`classifyPortTarget()` pure helper (compatible-stack detection, see Phase 3) and unit-test
it the way `classifyBootstrap` is tested (`dispatcher.test.ts:13-50`).
### Phase 2 — the port run (plat/agents)
**`spawnPort()`** mirrors `spawnBuilder()` (`dispatcher.ts:879-1057`). Key differences:
```ts
private spawnPort(owner: string, name: string, description: string, portPr = 0): void {
  const key = `${owner}/${name}`;
  const prodHost = `${name}-${owner}.${this.cfg.appsDomain}`;
  const systemPrompt = assemblePrompt(this.promptTemplate("port"), {
    OWNER: owner, APP: name,
    DESCRIPTION: description || "(no description — infer from the migrated source)",
    PROD_URL: `https://${prodHost}`, PROD_HOST: prodHost,
    BRANCH: "port", PR: String(portPr),
    GIT_PUBLIC: `https://git.${this.cfg.appsDomain}`,
    TEMPLATE_REPO: "plat/_app-template",      // agent reads the contract from here
  });
  this.runner.enqueue({
    role: "port",                              // NEW role (see runner.ts change)
    owner, repo: name, item: "port",
    model: this.cfg.builderModel,              // porting is builder-class work
    maxTurns: 300, timeoutMs: 50 * 60_000,
    systemPrompt,
    userPrompt: `Conform the migrated repo "${name}" onto the platform app-template, ` +
      `preserving its behavior; build on the \`port\` branch under PR #${portPr}.`,
    clone: true, checkoutBranch: portPr ? "port" : undefined,
    onExit: async (result) => { /* mirror builder onExit: fallback summary, drop 🏗️ */ },
  });
}
```
`port()` (the async orchestrator `maybePort` kicks) does the substrate dance in order **B**:
1. **Open the construction PR immediately** (mirror `dispatcher.ts:715-747`): create branch
   `port` off `main`, seed a `PORT_PLAN.md`, open PR `🏗️ port: {{APP}}`. User sees it
   instantly. (Reuses `createBranch`/`createFile`/`createPull` verbatim — `forgejo.ts:281-324`.)
2. **Spawn the port agent** (`spawnPort`). The agent's **first** commit lands the template
   plumbing on the `port` branch (see prompt step 1 below) — this is what makes `create_app`'s
   CI wait satisfiable.
3. **Provision substrate via `create_app`** — but it must run *after* the agent's plumbing
   commit is on `main`'s lineage for CI to see a `check` workflow. **Sequencing problem:**
   `create_app` cuts `v0.1.0` off the **`main`** branch (`server.ts:1129-1147`,
   `target_commitish` = `main`'s head), and `main` on a migrated repo has **no workflows**.
   → **v1 resolution:** the port agent's first action is to land template plumbing
   (`.forgejo/workflows/`, `charts/app/`, `Dockerfile`, skeleton `src/server.ts` +
   `package.json` + configs) **directly onto `main`** via a `[skip ci]`-free commit
   *before* business-logic porting, then the dispatcher calls `create_app`. This is the
   honest cost of reusing `create_app` unchanged. (Alternative A — make `create_app`
   tolerate no-CI — removes this constraint; see Tradeoffs.)
> **This sequencing is the riskiest part of the design and the place I'm least sure of**
> (open question O2). Landing plumbing on `main` before a PR exists is unusual for this
> codebase (everything else builds on a branch). The cleanest correct ordering may actually
> be: agent lands plumbing on the `port` branch → dispatcher calls `create_app` with a
> `branch` override so v0.1.0 is cut off `port` not `main`. But `create_app` hardcodes
> `main` (`server.ts:1046,1130`). So either we (B) commit plumbing to `main` first, or (A)
> parametrize `create_app`'s branch + relax `waitForCI`. **I recommend deciding this with
> the maintainer before implementation** — it determines whether Phase 4 (MCP) is required
> or optional.
4. Agent ports business logic, iterates `check` green (`get_build`), posts ship summary,
   drops `🏗️` → hands to the **existing** validator (`maybeReview`), which auto-merges and
   `maybeRelease` ships it. No new code past the drop-prefix point.
**`onExit`/resume/idempotency** reuse the builder's patterns: a `portersLive` set mirroring
`buildersLive` (`dispatcher.ts:409`) if we want resume-on-crash; for v1 a simpler
`portFailed`-park-on-exhaust is acceptable (porting is rarer than building). Set
`this.state.ported[key] = true` before spawn (`dispatcher.ts:708` analog).
### Phase 3 — stack-compatibility gate + the `port` prompt (plat/agents)
**Compatibility probe** (pure helper, unit-tested). Before spawning, classify the migrated
source. v1 = "is this Node/TS-portable?":
```ts
export type PortClass = { kind: "node" } | { kind: "incompatible"; lang: string };
/** Decide whether a migrated repo's stack is auto-portable in v1. */
export function classifyPortTarget(signals: {
  hasPackageJson: boolean;
  topLevelFiles: string[];    // from listing the default branch root
}): PortClass {
  if (signals.hasPackageJson) return { kind: "node" };
  const goish = signals.topLevelFiles.some((f) => f === "go.mod");
  const pyish = signals.topLevelFiles.some((f) =>
    ["requirements.txt", "pyproject.toml", "Pipfile"].includes(f));
  if (goish) return { kind: "incompatible", lang: "Go" };
  if (pyish) return { kind: "incompatible", lang: "Python" };
  return { kind: "incompatible", lang: "unknown" };
}
```
On `incompatible`, `port()` files **one** issue (reuse `createLabeledIssue`,
`forgejo.ts:238-250`) — *not* labeled `agent-work* — explaining the template is
Node/TS-only today and a human should port the runtime, then marks `ported[key] = true` so
the sweep stops re-triggering. (Honest scope boundary, surfaced to the user, not silent.)
**`agents/prompts/port.md`** (new). Modeled on `builder.md` but with inverted assumptions.
Sketch of the load-bearing sections:
```md
# Role: Platform Port Agent
You are conforming a MIGRATED repository **{{OWNER}}/{{APP}}** onto the platform's
app-template. Unlike a fresh app, this repo already contains REAL foreign source whose
behavior you must PRESERVE while re-expressing it on the template contract. Build on the
`{{BRANCH}}` branch under PR #{{PR}}.
## The contract you are porting ONTO
The template is **{{TEMPLATE_REPO}}**: Fastify 5 + TypeScript (Node 22), TypeBox-schema
routes (auto-OpenAPI at /openapi.json), better-auth Forgejo SSO in `src/auth.ts` (works out
of the box — do not rewire), Postgres via the `{{APP}}-pg-app` secret, and platform plumbing
in `charts/app/`, `Dockerfile`, `.forgejo/workflows/`. Read the template via the `read_repo`
MCP tool against `{{TEMPLATE_REPO}}` — copy `charts/app/`, `.forgejo/workflows/`,
`Dockerfile`, and the skeleton `src/` files VERBATIM; never hand-author plumbing.
## Order of operations (do these in order)
1. **Land the template plumbing first** — copy `charts/app/`, `Dockerfile`,
   `.forgejo/workflows/`, and a minimal COMPILING `src/server.ts`/`package.json`/tsconfig
   from {{TEMPLATE_REPO}} so `check` can run. Commit + push. (The platform provisions OAuth +
   DB once this exists.)
2. **Port the behavior** — map the foreign app's core flows onto typed Fastify routes +
   Postgres migrations + the server-rendered UI pattern in `src/homepage.ts`. Preserve data
   shapes and user-visible behavior; do not invent features.
3. **Make `check` green** (lint + typecheck + build + smoke). Poll `get_build`. Red → fix
   forward, push, repeat. Main is branch-protected; green is non-negotiable.
4. **Ship summary + drop `🏗️`** — identical to the builder's hand-off: post a summary
   comment on PR #{{PR}}, then drop the `🏗️ ` prefix to trigger the validator, then EXIT.
   Do NOT merge or release.
## What to preserve vs. cut
- Preserve: the app's actual behavior, data model, routes a user depends on.
- Cut/replace: foreign build tooling, foreign CI, foreign auth — the template owns those.
- If a flow genuinely cannot be expressed on Node/Fastify, note it in the summary rather
  than faking it.
```
**Runner** (`runner.ts`): add `"port"` to the `Role` union (`runner.ts:22`) and
`PRIORITY` (`runner.ts:39`). Recommended priority: `{ builder: 0, port: 1, worker: 2,
reviewer: 3 }` — port is builder-class but must not starve real builders during a bulk
migration. The runner already gives `["reviewer","builder","worker"]` roles a browser
(`runner.ts:295`); **add `"port"`** there too (the port agent self-tests its preview just
like the builder). `promptTemplate`'s role type (`dispatcher.ts:431`) gains `"port"`.
### Phase 4 — (optional / deferred) make `create_app` migration-aware (plat/mcp)
Only if the maintainer picks **resolution A** (or to harden either way):
- Relax `waitForCI` (`server.ts:1147`) to **skip** when the target ref has no
  `.forgejo/workflows` (probe via the existing `read_repo`/contents call), with a `step`
  noting "CI wait skipped: no workflows yet". This makes `create_app` safe to call on a raw
  migrated repo and removes the order-B sequencing constraint.
- No `authorize()` change is needed: the port path calls **`create_app`**, already on the
  allowlist (`server.ts:819`) and gated owner==caller||org-member. The dispatcher calls it
  with the platform admin token (`mcp.ts` / `PLAT_TOKEN`), which is `is_admin` and
  short-circuits authorize (`server.ts:816`) — same as `bootstrap()` does today
  (`dispatcher.ts:664`).
**No new MCP tool, no new authorize() entry, no webhook route, no HMAC.** That is the point
of reusing `create_app`.
### Phased PR plan
| PR | Repo | Contents | Depends on |
|----|------|----------|-----------|
| 1 | plat/agents | `state.ts` `ported` map; `Forgejo.isNonEmpty`; `maybePort`/`portFailed` guards + `port:` inflight key; `classifyPortTarget` + tests. **No spawn yet** (file-an-issue-only on every candidate) so the trigger is observable in isolation. | — |
| 2 | plat/agents | `prompts/port.md`; `runner.ts` `Role`/`PRIORITY`/browser; `spawnPort` + `port()` orchestrator (order B); wire `maybePort`→`spawnPort` for `node` class. | PR 1 |
| 3 | plat/agents | Bulk hardening: resume-on-crash (`portersLive`) if needed; sweep-side batch cap if bulk load proves heavy; richer compat detection. | PR 2 |
| 4 | plat/mcp | (optional) `create_app` `waitForCI` skip-when-no-workflows (resolution A). | independent |
PR 1 is deliberately a **no-op-but-observable** slice: it lights up the trigger and logs
"would port `owner/repo` (class=node)" without running an agent, so we can confirm the
migration event/sweep classification on a real bulk migration before any expensive run
fires.
---
## Validation plan
1. **Trigger fidelity (the big unknown):** on a depth-1 germinated platform, use Forgejo's
   migration UI to import a small public Node repo into a test org. With PR-1 deployed
   (observe-only), confirm a log line `would port …` appears — and confirm **whether** it
   came from the `repository` webhook or only from the sweep (instrument both paths with a
   `source` field). This decides open question O1.
2. **Empty-repo non-regression:** create a plain empty repo (no migration) and confirm it
   stays in `notTemplate` and is **not** classified as a port target (the `.empty` probe is
   the discriminator).
3. **Single port end-to-end** (PR-2): migrate one Node repo; confirm the `port` PR opens,
   plumbing commit lands, `create_app` provisions OAuth+DB+gitops (check `list_apps` / the
   gitops overlay), `check` goes green, validator runs, auto-merge + release fire — i.e. the
   migrated repo reaches a live prod URL `https://{{name}}-{{owner}}.{{appsDomain}}`.
4. **Bulk** (PR-2/3): migrate 5–10 repos in one Forgejo bulk action. Assert: (a)
   `ported`/`inflight` prevent duplicates, (b) the runner never exceeds `maxConcurrent`
   (watch `/healthz` `activeRuns`, `dispatcher.ts:2720-2723`), (c) builders are not starved
   (enqueue a real builder mid-bulk and confirm it runs ahead of pending ports), (d) every
   repo ends `ported`.
5. **Incompatible stack:** migrate a Python repo; assert one explanatory issue is filed, no
   `agent-work` label, `ported` set, no agent spawned, no re-trigger next sweep.
6. **Idempotency across restart:** kill the dispatcher mid-bulk; on restart confirm
   `ported` (persisted via `saveState`, `state.ts:65-70`) prevents re-porting completed
   repos and the sweep resumes the rest.
7. **`create_app` 409 path:** confirm `create_app` on the now-plumbed migrated repo returns
   "repo already existed" (server.ts:1056) and still provisions OAuth+gitops without
   re-generating content.
---
## Tradeoffs & alternatives considered
- **Extend dispatcher vs. new MCP webhook + `plat/workflows`.** Chose dispatcher: the
  `repository` event is already consumed (`dispatcher.ts:363,448`), the sweep already
  enumerates all repos, and there is **no** inbound MCP webhook/HMAC today to extend cheaply.
  An MCP route would duplicate idempotency + add an auth surface for zero v1 benefit.
  Deferred unless durable cross-restart audit of migrations is required (O5).
- **New `port` role vs. reusing `builder`.** The builder assumes a working template clone
  (`prompts/builder.md:13-19`); a migrated repo has the opposite. Overloading the builder
  prompt with "sometimes there's foreign code, sometimes not" would make both jobs worse.
  A dedicated prompt + run is cleaner and keeps the builder path untouched.
- **Order B (agent lands plumbing first) vs. Order A (make `create_app` CI-tolerant).**
  B keeps `create_app` — the most sensitive MCP tool — unchanged, at the cost of an unusual
  "commit plumbing to `main` before the PR" step and a real sequencing risk (O2). A is
  cleaner conceptually but edits `create_app`'s `waitForCI` invariant for *all* callers.
  Recommend B for v1, A as follow-up hardening (PR 4). **Maintainer should pick before
  build.**
- **Concurrency: reuse the runner cap vs. a new bulk limiter.** The runner's
  `maxConcurrent`+priority queue already serializes bulk work (`runner.ts:209`); adding
  `port` as its own priority class below `builder` is enough. A dedicated bulk rate-limiter
  is gold-plating deferred to PR 3 only if real bulk load overwhelms the Forgejo API during
  classification.
- **"Conformed" definition.** Defined as the template's `check.yml` smoke gate
  (`_app-template/.forgejo/workflows/check.yml`) going green — the *same* bar the builder
  meets — so we inherit the validator/preview/auto-merge pipeline with zero new success
  criteria. Alternative (a bespoke "port-complete" check) would fragment the definition of
  "a working app."
- **Non-Node source.** v1 detects and defers (files an issue) rather than attempting a
  runtime swap. Auto-porting Python/Go would mean either a polyglot template (a much larger
  platform change) or an LLM rewriting the runtime (high risk, unbounded). Scoped out
  honestly.
---
## Open questions
(see `openQuestions` field)
@@ -0,0 +1,287 @@
# Self-serve custom domains (Vercel-style BYOD)
> Issue #3 — "users should be able to configure their own domains for apps or orgs similar to Vercel."
## Problem
The platform already has the *resolution half* of bring-your-own-domain fully built. What is missing is the *governance half*: the only way a domain enters the system today is a maintainer hand-editing `clusters/local/domains.yaml`.
What exists (verified):
- **A registry**: `clusters/local/domains.yaml` (`mitosis-work/gitops/clusters/local/domains.yaml:16-19`) maps `domain → org`. It is a plain governance file — **not** referenced by any kustomization (`clusters/local/kustomization.yaml:1-4`, `flux-kustomizations.yaml` has no entry for it); Flux never applies it. MCP reads it over the Forgejo **raw** API.
- **Pure host logic**: `mcp/src/domains.ts` — `parseDomainRegistry` (`:110`), `hostFromWebsite` (`:39`), `hostOwnedBy` (`:61`, dot-boundary safe), `isRegisteredDomain` (`:78`), `derivePreviewHost` (`:92`).
- **The I/O resolver**: `loadDomainRegistry` (`server.ts:1709`, 60s cache, fail-open to empty) + `resolveAppHost` (`server.ts:1767`) which computes the ONE prod host from `repo.website → org.website → platform default` and never throws.
- **The single-resolved-host invariant**: that one host feeds Ingress + cert dnsName + `BETTER_AUTH_URL` + OAuth `redirect_uri` + repo website, all from `resolveAppHost`'s return (`overlayManifests` `server.ts:1924`; cert block `:2320-2337`; Ingress block `:2374-2397`; redirect `redirectUrisForHost` `:906`).
- **A re-host path**: `renderGitopsOverlay` (`server.ts:2745`) already detects `currentHost !== host` (`:2811`), re-renders the whole overlay at the new host, and re-points the OAuth redirect (`:2828-2853`). So *moving an app to a new domain is already a solved problem* — once the domain is in the registry.
- **`create_app(domain:)`** (`server.ts:1063-1077`, tool schema `:4172-4179`): claims a domain at create time, but only if `hostOwnedBy(d, owner, reg)` already passes — i.e. it was *already* in the registry.
- **A CoreDNS precedent**: `coredns-custom.yaml:22` already carries `rewrite name exact app.shipits.app traefik...` — the in-cluster split-horizon rewrite for a custom domain.
- **A public-edge precedent**: the cloudflared tunnel cutover (`germinate:439-448`, `lib.sh:568-598`) routes `*.<domain> → traefik`.
**The gap, precisely:** there is no self-serve way to (a) *prove* you own a domain and (b) get the `domain → org` line written into `domains.yaml`, the per-host TLS issued, and the DNS/tunnel wired — without a maintainer. Everything downstream of "the line exists in the registry" already works.
Two sub-problems the design must solve that the platform has **never** solved before:
1. **Ownership verification.** `domains.yaml` is the ownership authority, but nothing checks that the org *actually controls* the DNS for a domain before it's written. A self-serve `add_domain` with no proof lets tenant A claim `tenantB.com`. This is the crux, and it's brand new — there is no TXT/CNAME verification code anywhere (`grep` for `resolveTxt`/`node:dns` in `mcp/src` → nothing).
2. **Public TLS.** Every cert today uses `plat-local-ca-issuer` (`pki/ca-issuer.yaml`), a private CA. A browser hitting a *real public* custom domain will not trust it. Public domains need ACME (Let's Encrypt). cert-manager is installed (`cert-manager/helmrelease.yaml`) but **no ACME `ClusterIssuer` exists** (`grep acme` over gitops → only app-name comments).
## Design (from first principles)
### The invariants that constrain everything
1. **`domains.yaml` is the sole ownership authority.** `resolveAppHost`/`hostOwnedBy` already trust it completely and fail-open to the platform default. Therefore the *only* security-critical write in this whole feature is "append a `domain → org` line." If we gate that write behind real verification, every downstream consumer is automatically safe — no other code needs a trust check.
2. **The single-resolved-host invariant must hold.** A domain is just an *input* to `resolveAppHost`. We must never let `add_domain` write Ingress/cert/redirect directly; it writes the registry line, and then the existing `create_app`/re-host path recomputes the one host. This is the key architectural decision: **`add_domain` grants capability; it does not deploy.** Deployment stays in the already-correct single-source path.
3. **Germination must not break.** The identity rewrite (`rewrite_identity_domain`, `lib.sh:42-60`) rewrites only the *seed domain* (`open-platform.sh`) → the daughter domain, dot-escaped. A tenant's custom `acme.com` line in `domains.yaml` is **not** the seed domain, so it survives the rewrite untouched — good and intended (a daughter inherits its parent's registered custom domains as governance data; whether that's desired is an open question, see below). The CA-token survival gate (`:55-57`) is unaffected.
### v1 scope decision
Ship the **grant + verify + register** loop and **public TLS via a DNS-01 ACME issuer**, org-owner gated. Defer: vanity per-app UIs, automatic cloudflared API integration, apex A-record automation, wildcard public certs. Rationale below in Tradeoffs.
### The Vercel-style flow
```
add_domain(owner, domain)
   → MCP mints a verification token, returns DNS instructions, persists a PENDING claim
   → tenant adds a TXT record at _platform-challenge.<domain>  (+ CNAME/A for routing)
verify_domain(owner, domain)
   → MCP does a live DNS TXT lookup; on match:
       - append `<domain>: <owner>` to domains.yaml via gitopsPush
       - render a per-host public Certificate (ACME DNS-01)  [if public]
       - add the CoreDNS split-horizon rewrite + (optionally) note the tunnel rule
   → now create_app(domain:) / repo.website selection resolves to it
```
This mirrors Vercel exactly: add domain → see DNS instructions → verify → it goes live. The two-call split (`add_domain` issues the challenge, `verify_domain` checks it) is deliberate — DNS propagation is async and out of our control, so the tenant calls `verify_domain` when ready (or CI polls it).
### Where verification state lives
There is no database in MCP for this. Two honest options:
- **(A) Deterministic token, no stored state.** The challenge token is `HMAC(PLAT_SECRET, owner + ":" + domain)` truncated — MCP recomputes it in `verify_domain` and never has to persist the pending claim. **This is the v1 recommendation**: it's stateless, survives pod restarts, and needs no new storage. The token leaks no secret (it's domain-scoped and only proves *we issued it*).
- (B) A `domain-claims.yaml` PENDING list in gitops. More Vercel-faithful (shows pending domains in a UI) but adds a second gitops write and a reconciliation surface. Deferred.
With (A), `add_domain` is *pure compute* (returns instructions) and only `verify_domain` writes gitops — which is the right place for the irreversible grant.
### TLS: public vs internal
The wildcard `*.<platform-domain>` cert only covers platform subdomains. A custom apex/subdomain needs its own cert. Two issuers, selected by whether the domain is the platform's own suffix:
- **Internal / platform-suffix host** → keep `plat-local-ca-issuer` (today's behavior, unchanged).
- **Real public custom domain** → a new `letsencrypt-dns01` `ClusterIssuer` using **DNS-01** (not HTTP-01).
**Why DNS-01 over HTTP-01:** HTTP-01 requires Let's Encrypt to reach `http://<domain>/.well-known/acme-challenge/...` through the public edge *before* the domain is fully cut over — a chicken-and-egg with the tunnel, and it breaks for apex + can't do wildcards. DNS-01 only needs cert-manager to create a `_acme-challenge` TXT record via the DNS provider API, which works before any traffic is routed and supports wildcards. Since the tenant is already proving ownership via DNS TXT, requiring DNS-API access (or a delegated `_acme-challenge` CNAME) is consistent.
The per-host Certificate is **still rendered by the existing overlay path** (`overlayManifests:2320`) — we only swap `issuerRef.name` based on host class. No new render path.
### Org-level vs app-level domains
`resolveAppHost` already implements both (`:1782-1796`):
- **App-level**: `repo.website = https://notes.acme.com` → that exact host (per-app override). `create_app(domain:)` sets this.
- **Org-level**: `org.website = acme.com` → every app resolves to `<name>.acme.com` automatically.
So **registering one `acme.com: acme` line enables both** — no separate org-domain vs app-domain machinery is needed. `add_domain` registers the *domain*; the org/app `website` field is the *selection*. We expose `set_app_domain` and `set_org_domain` as thin convenience wrappers that just PATCH the Forgejo `website` and trigger the re-host path. This is the cleanest possible model and it's already 90% built.
## Implementation (phased, per-repo)
### PR 1 — gitops: public ACME issuer + cert-manager egress (plat/gitops)
New component `clusters/local/platform/letsencrypt/` with a DNS-01 `ClusterIssuer`. Add to `platform/kustomization.yaml`. It carries a sops secret (DNS provider API token), so if put behind its own Flux Kustomization it needs a `decryption` block — but `platform` already has one (`flux-kustomizations.yaml:38`), so adding it under `platform/` is simplest.
`clusters/local/platform/letsencrypt/clusterissuer.yaml` (Cloudflare DNS-01 shown; provider is an open question):
```yaml
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-dns01
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: platform@<platform-domain>        # rewritten by rewrite_identity_domain
    privateKeySecretRef: { name: letsencrypt-dns01-account-key }
    solvers:
      - dns01:
          cloudflare:
            apiTokenSecretRef:
              name: cloudflare-dns-api
              key: api-token
```
`clusters/local/platform/letsencrypt/cloudflare-dns-api.yaml` — sops-sealed Secret (sops `.sops.yaml` already matches `clusters/.+\.yaml$` / `^(data|stringData)$`, so `germinate fork_regen_secrets` reseals it under the daughter's age key automatically; `fork_verify_all_sealed` gates it — `lib.sh:207-325`). The DNS-API token value is **platform-identity**, set post-Forgejo like the OAuth secrets (`germinate fork_postforgejo_secrets`), not a sovereign random.
cert-manager has **no NetworkPolicy** today (its namespace isn't default-deny), so ACME egress to `acme-v02.api.letsencrypt.org` and the DNS API works with no netpol change. **Verify this** in the validation plan (a future netpol could break it silently).
`platform/kustomization.yaml`: add `- letsencrypt`.
> Staging note: first ship `spec.acme.server: .../acme-staging-v02...` to avoid Let's Encrypt rate limits during validation, then flip to prod.
### PR 2 — mcp: `add_domain` / `verify_domain` tools + issuer selection (plat/mcp)
**2a. Pure helpers in `domains.ts`** (unit-testable, no I/O — mirrors the existing split):
```ts
// domains.ts — append a verified line, returning the new file text. Idempotent:
// if `<domain>: <owner>` already present, returns text unchanged. Refuses to
// REASSIGN a domain owned by a different org (returns {conflict: org}).
export function addDomainLine(text: string, domain: string, owner: string):
  { text: string; status: "added" | "exists" | "conflict"; conflictOrg?: string }
// The DNS label a tenant must publish. Stable, domain-scoped.
export function challengeRecordName(domain: string): string  // `_platform-challenge.${domain}`
// True iff `host` is the platform's own suffix (use internal CA) vs a real
// external domain (use ACME). platformSuffix comes from APPS_DOMAIN.
export function isPlatformHost(host: string, platformSuffix: string): boolean
```
`addDomainLine` reuses the existing line-parser shape from `parseDomainRegistry` so the file stays in the exact format that parser expects (insert under `domains:`, two-space indent).
**2b. Verification token (stateless, option A)** in `server.ts`:
```ts
// HMAC over a server secret so only the platform can mint a valid token; the
// tenant publishes it as a TXT value. No persisted claim state needed.
function domainChallengeToken(owner: string, domain: string): string {
  return "plat-verify=" + createHmac("sha256", PLAT_DOMAIN_SECRET)
    .update(`${owner}:${domain.toLowerCase()}`).digest("base64url").slice(0, 43);
}
```
`PLAT_DOMAIN_SECRET` = a new sovereign secret in `plat-mcp-secrets.yaml` (random, resealed at germination by `fork_regen_secrets` — it's a per-platform random, not an identity value, so it Just Works through the fork path).
**2c. The two tools** (`server.tool(...)` in `makeServer`, `server.ts:4138+`):
```ts
server.tool("add_domain",
  "Begin claiming a custom domain for an org (Vercel-style). Returns the DNS " +
  "records to publish. Caller must own/belong to <owner>. No effect until verify_domain.",
  { owner: z.string(), domain: z.string().describe("apex or subdomain, e.g. acme.com or app.acme.com") },
  async (args) => run("add_domain", args, () => addDomain(caller, args)));
server.tool("verify_domain",
  "Verify a pending custom-domain claim by checking its DNS TXT challenge, then " +
  "register it (writes plat/gitops domains.yaml) and trigger TLS issuance.",
  { owner: z.string(), domain: z.string() },
  async (args) => run("verify_domain", args, () => verifyDomain(caller, args)));
```
`addDomain` (pure, no gitops write):
```ts
async function addDomain(caller, {owner, domain}) {
  const d = hostFromWebsite(domain);
  if (!d) throw new Error("not a DNS-safe host");
  const token = domainChallengeToken(owner, d);
  const apex = d.split(".").slice(-2).join(".");
  return {
    domain: d,
    records: [
      { type: "TXT", name: challengeRecordName(d), value: token,
        purpose: "ownership verification" },
      { type: d === apex ? "A" : "CNAME",
        name: d, value: d === apex ? "<platform-edge-ip>" : `<platform-domain>.`,
        purpose: "route traffic to the platform" },
      { type: "CNAME", name: `_acme-challenge.${d}`,
        value: `_acme-challenge.${d}.<platform-acme-delegation>`,
        purpose: "delegate Let's Encrypt DNS-01 (optional, if not using a shared DNS-API token)" },
    ],
    next: `Publish the TXT record, then call verify_domain(${owner}, ${d}).`,
  };
}
```
`verifyDomain` (the only gitops-writing tool):
```ts
import { resolveTxt } from "node:dns/promises";
async function verifyDomain(caller, {owner, domain}) {
  const d = hostFromWebsite(domain);
  const want = domainChallengeToken(owner, d);
  let txts: string[][];
  try { txts = await resolveTxt(challengeRecordName(d)); }
  catch { throw new Error(`no TXT at ${challengeRecordName(d)} yet — DNS may still be propagating`); }
  const flat = txts.map(parts => parts.join(""));
  if (!flat.includes(want))
    throw new Error(`TXT challenge mismatch at ${challengeRecordName(d)}; expected ${want}`);
  // Verified. Append to domains.yaml via the standard gitops clone+push path.
  const work = mkdtempSync(join(tmpdir(), "plat-mcp-"));
  sh("git", ["clone","-q",GITOPS_REMOTE,work]);
  const p = join(work, "clusters/local/domains.yaml");
  const res = addDomainLine(readFileSync(p,"utf8"), d, owner);
  if (res.status === "conflict")
    throw new ForbiddenError(`'${d}' already registered to org '${res.conflictOrg}'`);
  if (res.status === "added") {
    writeFileSync(p, res.text);
    sh("git",["-C",work,"add","clusters/local/domains.yaml"]);
    sh("git",["-C",work,"-c","user.email=plat@mcp","-c","user.name=plat-mcp",
       "commit","-q","-m",`domains: register ${d} -> ${owner} (verified TXT, via MCP)\n\nCaller: ${caller.login}`]);
    gitopsPush(work);
  }
  domainRegistryCache = null;                    // bust the 60s cache (server.ts:1699)
  audit({ event: "domain.verified", owner, host: d, result: "success" });
  return { domain: d, owner, registered: res.status,
    next: `create_app(${owner}, <name>, domain: "${d}") or set the org/app website to use it.` };
}
```
**2d. Issuer selection in the overlay.** Thread an `issuer` into `EnvSpec`/`overlayManifests`, computed once where the host is resolved:
```ts
const issuer = isPlatformHost(prodHost, APPS_DOMAIN) ? "plat-local-ca-issuer" : "letsencrypt-dns01";
```
and in `overlayManifests` cert block (`server.ts:2329`) replace the hardcoded `name: plat-local-ca-issuer` with `name: ${spec.issuer}`. Because the cert is rendered by the *same single-source path*, the re-host path (`renderGitopsOverlay:2856`) recomputes it for free when a live app moves onto a public domain.
**2e. authorize() allowlist** (`server.ts:818`): add `case "add_domain":` and `case "verify_domain":` to the owner/member-gated block (`:819-853`) so they get the identical `owner === caller.login || caller.orgs.includes(owner)` check. **This is the security gate** — without it the default branch fail-closes (`:864`), but we want them explicitly owner-scoped, not admin-only.
**2f. CoreDNS split-horizon rewrite.** For in-cluster components to reach the custom host (and to mirror the `app.shipits.app` precedent at `coredns-custom.yaml:22`), `verify_domain` should also append `rewrite name exact <d> traefik.traefik.svc.cluster.local` to `platform/coredns/coredns-custom.yaml` in the same gitops commit. **Honest caveat:** that file is a static `data:` block (not templated), and a naive append per domain will grow unbounded and risks merge races with germination's identity rewrite. **v1 recommendation:** only add the rewrite for the *apex* of a newly-registered domain, and only when an app is actually deployed there (do it in the existing overlay/re-host commit, not in `verify_domain`), keeping `verify_domain`'s write surface to `domains.yaml` alone. This is an open question (below).
### PR 3 — mcp: `set_app_domain` / `set_org_domain` convenience wrappers (plat/mcp)
Thin tools that PATCH `repo.website` / `org.website` (the *selection*, validated against the now-registered domain) and then call `resolveAppHost` + `renderGitopsOverlay` — i.e. reuse the **existing re-host path** (`server.ts:2811`) verbatim. No new deploy logic. Owner-gated identically. This is what makes a *live* app move onto a freshly-verified domain in one call. Could be folded into PR 2; split for review clarity.
### PR 4 — mitosis toolkit: germination wiring (plat/mitosis)
- The new `PLAT_DOMAIN_SECRET` sovereign random + `cloudflare-dns-api` sealed secret flow through `fork_regen_secrets`/`fork_verify_all_sealed` automatically (they match the sops regex) — **verify** no new code is strictly needed, but add them to any explicit secret manifest list the fork path enumerates.
- The `letsencrypt` ClusterIssuer `email:` and any `<platform-domain>` token get rewritten by `rewrite_identity_domain` (`lib.sh:42`) since they contain the seed domain — confirm dot-escape safety holds (the `plat-local-ca-tls` survival gate at `:55` already guards the dangerous case).
- **Default issuer for daughters:** a purely-local k3d daughter has no public DNS and should keep `plat-local-ca-issuer`. Gate the ACME path behind the same `CUTOVER_TUNNEL`-style opt-in (`germinate:445`) so a local germination doesn't try to ACME-issue against Let's Encrypt and fail. This is the cleanest reuse of the existing public-edge opt-in.
## Validation plan
1. **Unit (bun test, mcp/src):** `domains.test.ts` already exists — extend it. `addDomainLine` (added/exists/conflict, format round-trips through `parseDomainRegistry`), `challengeRecordName`, `isPlatformHost` (platform suffix → true, `acme.com` → false), `domainChallengeToken` determinism. No cluster needed.
2. **Verification negative paths:** `verify_domain` with no TXT, wrong TXT, TXT for a *different* org's token → all reject; a domain already owned by another org → `ForbiddenError` conflict.
3. **End-to-end on a k3d cold-start (the depth-2 validation flow in MEMORY):**
   - Pick a real test domain with DNS-API access (or stub `resolveTxt` against a local resolver carrying a `_platform-challenge` TXT).
   - `add_domain(acme, acme-test.example)` → publish the returned TXT → `verify_domain` → assert the `domains.yaml` line appears via `gitops` and `loadDomainRegistry` (cache busted) now returns it.
   - `create_app(acme, blog, domain: "acme-test.example")` → assert overlay Ingress host = `acme-test.example`, cert `issuerRef.name = letsencrypt-dns01`, `BETTER_AUTH_URL`/redirect all = `https://acme-test.example` (the single-host invariant).
   - With staging ACME: assert the `Certificate` goes `Ready` and the served leaf chains to Let's Encrypt staging (`kubectl get certificate -n acme--blog--prod`; `openssl s_client`).
   - **Re-host:** `set_app_domain` an existing live app onto the new domain → assert `renderGitopsOverlay` took the re-host branch (`currentHost !== host`), OAuth redirect re-pointed, no second OAuth orphan.
4. **cert-manager egress:** confirm cert-manager can reach Let's Encrypt + the DNS API with the current (absent) netpol; document that adding a default-deny netpol to `cert-manager` later must include this egress.
5. **Germination survival:** germinate a daughter; assert (a) the tenant `acme-test.example` line survives the identity rewrite untouched, (b) `plat-local-ca-tls` survives (`lib.sh:55` gate), (c) the daughter without `CUTOVER_TUNNEL` defaults custom-domain apps to the internal CA and doesn't attempt ACME.
## Tradeoffs & alternatives considered
- **Stateless HMAC token (chosen) vs a `domain-claims.yaml` pending registry.** Stateless needs no storage and survives restarts; the cost is no "pending domains" listing for a UI. Acceptable for v1 — the registry file already isn't a UI.
- **DNS-01 ACME (chosen) vs HTTP-01.** DNS-01 needs DNS-API credentials but works before cutover, supports apex + wildcard, and reuses the ownership-proof channel. HTTP-01 is simpler to credential but has the routing chicken-and-egg and no wildcard. We pick DNS-01.
- **`add_domain` grants capability, not deployment (chosen).** The alternative — having `add_domain` write Ingress/cert directly — would duplicate the single-source host logic and is the exact drift the BYOD design eliminated. Keeping the write to `domains.yaml` only, and letting `resolveAppHost` recompute, preserves the invariant for free.
- **Reuse `create_app`'s re-host path (chosen) vs a new "move domain" code path.** The re-host machinery (`renderGitopsOverlay:2811`, OAuth re-point) already exists and is battle-tested; a new path would re-derive it and risk OAuth orphans.
- **TXT verification value as HMAC vs a random UUID.** HMAC needs no storage; a random UUID would. HMAC's only weakness is that anyone who learns `PLAT_DOMAIN_SECRET` can forge tokens — but that secret already guards far more, and the token alone grants nothing without also passing `authorize()`'s owner check.
- **Deferred (gold-plating):** automatic cloudflared tunnel-rule API writes (today's tunnel is dashboard-managed, `lib.sh:568`), wildcard public certs, a `list_domains`/`remove_domain` lifecycle, a pending-domains UI, apex A-record IP automation, automatic CoreDNS rewrite per domain.
## Risks / things I'm unsure of
- **Whether daughters should inherit a parent's registered custom domains.** A tenant's `acme.com` line survives germination (it's not the seed domain). For a fork that's a *different* operator, inheriting `acme.com: acme` as governance truth may be wrong — the daughter's `acme` org may be a different entity. May need a germination scrub of non-seed domain lines. Flagged as an open question.
- **CoreDNS rewrite growth/merge-race** (PR 2f) — the unbounded static append is the weakest part; v1 minimizes it.
- **cert-manager has no netpol today** — relying on that is implicit; a future hardening PR could silently break ACME.
- **MCP doing a live `resolveTxt`** adds an outbound DNS dependency to a tool call; needs a sane timeout and the mcp namespace netpol must permit DNS egress (it already resolves Forgejo, so likely fine — verify).
## Open questions
- DNS provider for the ACME DNS-01 solver: assume Cloudflare (a shared platform-managed API token), or require each tenant to delegate _acme-challenge via CNAME to a platform-controlled zone (no shared credential, but more tenant setup)? This drives PR 1's secret and PR 2's add_domain instructions.
- Should a germinated daughter inherit the parent's registered custom-domain lines in domains.yaml? They survive the seed-domain identity rewrite untouched, but a fork is often a different operator whose 'acme' org may be a different entity — do we scrub non-seed domains at germination, or treat the registry as inherited governance?
- Verification state: stateless HMAC token (v1 recommendation, no storage) vs a persisted domain-claims.yaml pending list that a future UI can show. Is a pending-domains listing a near-term requirement?
- CoreDNS split-horizon rewrite per custom domain: append to the static coredns-custom.yaml data block (grows unbounded, merge-races germination's rewrite) or only emit it from the app overlay/re-host commit for the apex when an app actually deploys there? v1 leans to the latter — confirm acceptable.
- Apex routing: do we standardize on a single platform edge IP (A record) for apexes, or require tenants to use a subdomain (CNAME-able) and reserve apex support for later? Vercel pushes users to CNAME via www + apex redirect; mirror that?
- Let's Encrypt rate limits during validation: ship the issuer pointed at acme-staging first and flip to prod after the cold-start E2E passes — agreed as the rollout order?
- Lifecycle: do we need remove_domain / list_domains in v1, or is add/verify sufficient (removal = maintainer edits domains.yaml, as today)?
@@ -0,0 +1,320 @@
## Problem
Issue #2: *"users should be able to configure their own environment variables. Use Vault (HashiCorp) as the backend."*
Today an app gets env vars from exactly three platform-minted, sops-sealed secrets that the MCP overlay renders into `clusters/local/apps/<owner>/<app>/<env>/` and wires onto the pod via the app chart's `extraEnvFrom` / `envFrom`:
- `app-auth` — OAuth + better-auth signing (`mcp/src/server.ts:2419-2431`, mounted at `_app-template/charts/app/templates/deployment.yaml:57-59`)
- `<name>-pg-app` — CNPG-generated Postgres creds, gated on `database` (`server.ts:2077-2081`)
- `<name>-s3` — MinIO creds, gated on `bucket` (`server.ts:2489-2508`, `2082-2086`)
There is **no path for a user to set their own** `STRIPE_KEY`, `OPENAI_API_KEY`, etc. The two ways they *could* do it today are both wrong: commit it to their repo (plaintext), or ask an admin to hand-edit gitops (doesn't scale, and the value would have to be sops-sealed to a key only the platform holds).
The platform's existing secret model is **sops-age**: every secret in `clusters/local/**.yaml` is encrypted under the cluster's age recipient (`.sops.yaml` `path_regex clusters/.+\.yaml$`, `encrypted_regex ^(data|stringData)$`), and germinate's sovereignty invariant is built entirely on it — fork mints a fresh age key, **reseals every platform secret** to it (`lib.sh fork_regen_secrets` ~207-317), and `fork_verify_all_sealed` (~325) is a hard gate that refuses to ship a daughter unless every sops file decrypts under the fork key.
**This proposal does NOT touch that.** Replacing sops with Vault would mean rewriting the entire fork-sealing invariant and the cold-start handoff. Instead Vault sits **alongside** sops as a *user-facing* secret store: platform-internal secrets stay sops-sealed in gitops; **user-supplied app env vars** live in Vault KV and are synced into the app namespace as an ordinary k8s Secret that the existing `extraEnvFrom` contract already consumes — so `_app-template` needs **zero changes**.
The genuinely hard part, and the bulk of this doc, is **init + unseal of a single-node, self-hosted Vault with no cloud KMS**, done in a way that survives germination and pod restarts without a human typing unseal keys.
---
## Design (from first principles)
### What "self-serve env var" must mean here
1. A user (org member, gated by the same `authorize()` check every app tool uses — `server.ts:843-852`) calls an MCP tool to set `KEY=VALUE` for their app.
2. The value is written to Vault KV v2 at a path scoped to that app: `apps/<org>/<app>`.
3. A controller in the app's namespace reads **only that path** and projects it into a k8s `Secret`.
4. The app chart's existing `extraEnvFrom` forwards that Secret onto the pod's env.
5. The plaintext value is **never** committed to gitops, never sops-sealed, never in the user's repo.
This cleanly separates **platform identity secrets** (sops, sovereign, regenerated on fork) from **user data secrets** (Vault, mutable at runtime, never in git).
### Why a sync-to-k8s-Secret seam (not direct Vault reads from the app)
The app contract is fixed and dumb on purpose: the pod reads env vars, full stop (`_app-template/charts/app/templates/deployment.yaml:57-68`). We must not require every app to embed a Vault SDK or an agent sidecar. So the platform-side job is: **Vault KV → k8s Secret in `<org>--<app>--<env>` → `extraEnvFrom`.** The app sees a normal Secret.
### VSO vs ESO — recommend Vault Secrets Operator (VSO)
| | Vault Secrets Operator (VSO) | External Secrets Operator (ESO) |
|---|---|---|
| Vendor alignment | HashiCorp-native; CRD `VaultStaticSecret` maps 1:1 to a KV path | Generic; `SecretStore` + `ExternalSecret` |
| k8s-auth | First-class `VaultAuth`/`VaultConnection` CRs | Works, but more glue |
| Validated here | chart `hashicorp/vault-secrets-operator 1.4.0`, app 1.4.0, MPL-2.0 (see Validation) | not templated |
| Footprint | one Deployment + 10 CRDs | one Deployment + CRDs |
**Recommendation: VSO.** The issue names HashiCorp explicitly, VSO is the first-party answer, and `VaultStaticSecret` is the exact primitive we need (poll a KV v2 path, write a Secret). ESO is the fallback if VSO's licensing/maintenance ever becomes a concern (it's a CNCF project).
### The unseal problem (the hard part), and the chosen strategy
Vault boots **sealed**: its storage (Raft, here) is encrypted with a master key that is itself Shamir-split into unseal keys, and Vault holds none of them at rest. On every pod start something must feed it unseal key shares. On a self-hosted single node there is **no cloud KMS** to auto-unseal against. The options:
- **Dev mode (`server.dev.enabled`)** — boots unsealed, in-memory. **Rejected:** loses all data on restart; the issue is about *durable* user secrets. Non-starter.
- **Transit auto-unseal** — Vault A auto-unseals Vault B via Transit. **Rejected:** circular (who unseals A?), and doubles the Vault footprint on a 16GB platform.
- **Cloud KMS auto-unseal (awskms/gcpckms)** — **Rejected:** the platform is sovereign and self-hosted; there is no cloud dependency by design (mirrors why germinate mints its own age key rather than using cloud KMS).
- **Shamir unseal, keys sealed into the platform's existing trust root (sops-age), replayed by an unseal Job.** **Chosen.**
**Chosen strategy — germinate-driven one-time init, sops-sealed unseal material, Job-driven unseal on every restart:**
1. **One-time init (germinate or first-boot Job):** after the Vault StatefulSet is `Running` but sealed, run `vault operator init -key-shares=1 -key-threshold=1 -format=json`. (Shares=1 is deliberate for a single-operator sovereign platform — see Tradeoffs; multi-share buys nothing when one operator holds all shares anyway.) This yields one unseal key + a root token, **printed exactly once**.
2. **Seal that material into gitops via sops**, reusing the *exact* mechanism fork already uses — write a k8s Secret manifest and `sops --encrypt --in-place` it (the `_fork_seal` helper in `lib.sh`). The unseal key + root token become an ordinary sops-sealed secret `vault-init` in the `vault` namespace. Now it is sovereign: encrypted under the cluster age key, regenerated/owned exactly like every other platform secret.
3. **Unseal on every pod start:** a small `vault-unseal` Job (or a `postStart`/init flow) reads the sops-decrypted `vault-init` Secret (Flux decrypts it; it lands as a normal k8s Secret in-cluster) and runs `vault operator unseal <key>`. Because Raft data persists on the PVC, the *same* master key always unseals it. The Job is idempotent: if Vault is already unsealed it's a no-op; a CronJob or a Flux health-check retry covers pod restarts.
This is honestly the most complex piece of the whole feature, and I'm flagging it as the #1 risk. It introduces a **stateful, must-init-exactly-once** component into an otherwise stateless, declarative platform. The mitigations: init is guarded (check seal status first; never re-init a Vault with existing Raft data — re-init would orphan the data), and the unseal material rides the existing sops trust root so germination handles it for free.
### k8s auth + per-app least privilege
VSO authenticates to Vault using Vault's **Kubernetes auth method**: it presents its ServiceAccount JWT, Vault calls TokenReview (the chart wires the `system:auth-delegator` ClusterRoleBinding — validated below), and issues a short-lived Vault token bound to a **policy**. We bind a policy that reads **only** `apps/data/<org>/<app>` (KV v2 stores data under `.../data/...`), per app, so a compromised app namespace can't read another tenant's secrets. The per-app `VaultAuth`/role is rendered by the MCP overlay alongside the `VaultStaticSecret` (concrete shapes in Implementation).
### Reserved-key collisions (must avoid)
User-set keys land in the **same pod env** as platform keys. The MCP tool MUST reject keys that collide with platform-reserved env, because `extraEnvFrom` ordering would let a user shadow (or be shadowed by) platform values unpredictably. Reserved set, read from the live overlay:
- `app-auth` keys: `FORGEJO_CLIENT_ID`, `FORGEJO_CLIENT_SECRET`, `BETTER_AUTH_SECRET`, `BETTER_AUTH_URL`, `FORGEJO_BASE` (`server.ts:2426-2430`)
- `<name>-pg-app` (CNPG discrete keys): `PGUSER`, `PGPASSWORD`, `PGHOST`, `PGPORT`, `PGDATABASE`, `DATABASE_URL`/`*_URI` (CNPG-generated)
- `<name>-s3` keys: `S3_ENDPOINT`, `S3_REGION`, `S3_BUCKET`, `S3_ACCESS_KEY`, `S3_SECRET_KEY` (`server.ts:2502-2506`)
- Anything starting with `PLAT_`, `MCP_`, `NODE_EXTRA_CA_CERTS` (chart-owned).
`set_secret` validates `^[A-Z][A-Z0-9_]*$` and rejects the reserved set.
---
## Implementation (per-repo, phased PRs)
### Phase 1 — gitops: the Vault platform component (PR to `plat/gitops`)
New directory `clusters/local/platform/vault/` mirroring the monitoring component shape (`platform/monitoring/` — namespace + helmrepository + helmrelease + ingress + cert + netpol + kustomization). Files:
**`vault/namespace.yaml`**
```yaml
apiVersion: v1
kind: Namespace
metadata:
  name: vault
  labels:
    plat.sh/managed: "true"
    plat.sh/tier: system
```
**`vault/helmrepository.yaml`**
```yaml
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata: { name: hashicorp, namespace: vault }
spec:
  interval: 1h
  url: https://helm.releases.hashicorp.com
```
**`vault/helmrelease.yaml`** — the validated values (see Validation; chart 0.33.0, image pinned to `1.18.5`):
```yaml
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata: { name: vault, namespace: vault }
spec:
  interval: 30m
  chart:
    spec:
      chart: vault
      version: ">=0.33.0 <0.34.0"
      sourceRef: { kind: HelmRepository, name: hashicorp, namespace: vault }
      interval: 30m
  install: { remediation: { retries: 3 } }
  upgrade: { remediation: { retries: 3 } }
  timeout: 8m
  values:
    global:
      enabled: true
      tlsDisable: true            # in-cluster only; Traefik terminates edge TLS
    server:
      image: { repository: hashicorp/vault, tag: "1.18.5" }  # last MPL-2.0 line
      ha:
        enabled: true
        replicas: 1
        raft:
          enabled: true
          setNodeId: true
          config: |
            ui = true
            listener "tcp" {
              address         = "[::]:8200"
              cluster_address = "[::]:8201"
              tls_disable     = 1
            }
            storage "raft" { path = "/vault/data" }
            service_registration "kubernetes" {}
      dataStorage: { enabled: true, size: 1Gi, storageClass: local-path }
      auditStorage: { enabled: false }
      affinity: ""                # single node: drop default anti-affinity
      resources:
        requests: { cpu: 50m, memory: 128Mi }
        limits:   { cpu: 500m, memory: 256Mi }
    injector: { enabled: false }  # we use VSO, not the sidecar injector
    ui: { enabled: true, serviceType: ClusterIP }
    csi: { enabled: false }
```
**`vault/ingress.yaml` + `vault/certificate.yaml`** — optional UI at `vault.open-platform.sh`, exactly mirroring `monitoring/ingress.yaml` + `monitoring/certificate.yaml` (Traefik entrypoints, `plat-local-ca-issuer` ClusterIssuer, `vault-platform-tls`). **Recommend deferring the public ingress in v1** — Vault need only be reachable in-cluster by VSO and MCP. Ship cert/ingress in a follow-up.
**`vault/netpol.yaml`** — default-deny ingress, then allow from `vault-secrets-operator-system` and `mcp` namespaces to port 8200, and intra-namespace (Raft 8201). Mirror `monitoring/netpol.yaml` structure.
**`vault/unseal-job.yaml`** — idempotent unseal (runs `vault status`; if sealed, `vault operator unseal`), reading the sops-sealed `vault-init` Secret. Plus a `vault-bootstrap` Job that, *only if Vault reports uninitialized*, enables the KV v2 mount `apps/`, enables k8s auth, and writes the VSO role/policy. Gated so it never re-inits an initialized Vault.
**`vault/vault-init.yaml`** — the sops-sealed Secret carrying `unseal_key` + `root_token`. **Placeholder committed by Phase-1; real values minted by germinate** (see Phase 3), exactly like `plat-mcp-secrets` ships with `PLACEHOLDER` values that fork fills later (`lib.sh` ~270-300).
**`vault/kustomization.yaml`** — lists the above.
**Flux wiring (`clusters/local/flux-kustomizations.yaml`):** Vault and VSO each get their **own** `Kustomization` (not folded into `platform`), because VSO ships **10 CRDs** (validated: `vaultstaticsecrets.secrets.hashicorp.com` et al.) that must exist before any app overlay references a `VaultStaticSecret`. Add, mirroring the `monitoring` block (which already carries a `decryption` block for its sops secret):
```yaml
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata: { name: vault, namespace: flux-system }
spec: { interval: 5m, path: ./clusters/local/platform/vault, prune: true, wait: true, timeout: 8m,
        decryption: { provider: sops, secretRef: { name: sops-age } },
        dependsOn: [{ name: cert-manager }], sourceRef: { kind: GitRepository, name: flux-system } }
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata: { name: vault-secrets-operator, namespace: flux-system }
spec: { interval: 5m, path: ./clusters/local/platform/vault-secrets-operator, prune: true, wait: true, timeout: 8m,
        dependsOn: [{ name: vault }], sourceRef: { kind: GitRepository, name: flux-system } }
```
And the **`apps` Kustomization must gain `dependsOn: { name: vault-secrets-operator }`** so app overlays that render a `VaultStaticSecret` don't apply before its CRD exists (today `apps` dependsOn `platform` + `pki` — `flux-kustomizations.yaml` last block).
**`clusters/local/platform/vault-secrets-operator/`** — VSO component: namespace, `helmrepository.yaml` (reuse hashicorp), `helmrelease.yaml` (chart `vault-secrets-operator >=1.4.0 <1.5.0`, with `install.crds: CreateReplace`), and a cluster-scoped default `VaultConnection` pointing at `http://vault.vault.svc.cluster.local:8200`.
### Phase 2 — mcp: the tool surface (PR to `plat/mcp`)
**New env keys** in `plat-mcp-secrets` (gitops `mcp/plat-mcp-secrets.yaml`, surfaced via `envFrom: secretRef: plat-mcp-secrets` at `mcp/charts/mcp/templates/deployment.yaml:28`):
- `VAULT_ADDR=http://vault.vault.svc.cluster.local:8200`
- `VAULT_TOKEN` — a **scoped** Vault token (policy: create/update/read/delete on `apps/data/*` + manage roles), NOT the root token. Minted by the bootstrap Job and sops-set by germinate alongside the other `plat-mcp-secrets` placeholders (`lib.sh` ~270-300 already seals this Secret with `PLACEHOLDER` slots — add three keys there).
The `mcp` namespace has **no netpol dir** (verified: `platform/mcp/` has no netpol), so MCP egress to Vault is already open.
**New file `mcp/src/tools/secrets.ts`** — thin Vault KV v2 client over `fetch` (matches the codebase's `fj()` pattern; no new SDK):
```ts
// Vault KV v2 path: secret data lives under apps/data/<org>/<app>; metadata under apps/metadata/<org>/<app>
const KV_MOUNT = "apps";
const vaultPath = (owner: string, name: string) => `${KV_MOUNT}/data/${owner}/${name}`;
const RESERVED = new Set([
  "FORGEJO_CLIENT_ID","FORGEJO_CLIENT_SECRET","BETTER_AUTH_SECRET","BETTER_AUTH_URL","FORGEJO_BASE",
  "PGUSER","PGPASSWORD","PGHOST","PGPORT","PGDATABASE","DATABASE_URL",
  "S3_ENDPOINT","S3_REGION","S3_BUCKET","S3_ACCESS_KEY","S3_SECRET_KEY",
]);
const KEY_RE = /^[A-Z][A-Z0-9_]*$/;
function validateKey(k: string) {
  if (!KEY_RE.test(k)) throw new Error(`invalid env key '${k}' (must match ${KEY_RE})`);
  if (RESERVED.has(k) || k.startsWith("PLAT_") || k.startsWith("MCP_") || k === "NODE_EXTRA_CA_CERTS")
    throw new Error(`'${k}' is a platform-reserved env var and cannot be overridden`);
}
```
**Three new `server.tool(...)` registrations** (alongside the others at `server.ts:4138+`), each calling `authorize(caller, tool, args)` first (`server.ts:4096`):
- `set_secret({ owner, name, key, value })` → validateKey; KV v2 read-modify-write of `apps/<org>/<app>` (merge, don't clobber siblings); returns `{ ok, names: [...] }`.
- `list_secrets({ owner, name })` → returns **names only**, never values: `{ names: string[] }`.
- `delete_secret({ owner, name, key })` → read-modify-write removing the key.
**`authorize()` allowlist** (`server.ts:818-867`) — add the three tool names to the owner/org-member-gated `case` block (the same one `create_app` etc. use, `server.ts:819-843`); the default branch fails closed, so they MUST be added or non-admins get `unknown tool`. They reuse the identical `owner === caller.login || caller.orgs.includes(owner)` check (`server.ts:844-852`).
**`overlayManifests()` (`server.ts:1924`)** — render a per-app `VaultStaticSecret` + a per-app `VaultAuth`, and append the synced Secret to `extraEnvFrom`. The synced Secret name is `<name>-env`. Add a gated block mirroring `dbBlock`/`bucketBlock` (`server.ts:1935`, `2055`), and extend `helmExtraEnvFrom` (`server.ts:2087-2091`):
```ts
// always rendered (a user can add secrets to any app); VSO writes <name>-env even if empty.
const vaultBlock = `---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultAuth
metadata: { name: ${name}-auth, namespace: ${ns} }
spec:
  method: kubernetes
  mount: kubernetes
  kubernetes:
    role: ${owner}-${name}            # per-app role -> policy reads only apps/data/${owner}/${name}
    serviceAccount: default
    audiences: ["vault"]
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata: { name: ${name}-env, namespace: ${ns} }
spec:
  type: kv-v2
  mount: apps
  path: ${owner}/${name}
  refreshAfter: 60s
  vaultAuthRef: ${name}-auth
  destination:
    name: ${name}-env               # the k8s Secret VSO writes; consumed by extraEnvFrom
    create: true
`;
```
```ts
const vaultEnvFrom = `      - secretRef:
          name: ${name}-env
          optional: true
`;
const helmExtraEnvFrom =
  `    extraEnvFrom:
${dbEnvFrom}${bucketEnvFrom}${vaultEnvFrom}`;   // always present now
```
`optional: true` means the pod still starts before VSO has written the Secret on first deploy. `_app-template` consumes this unchanged (`deployment.yaml:66 {{- with .Values.extraEnvFrom }}`).
The `vaultBlock` is inserted into the returned manifest string next to `${dbBlock}${bucketBlock}` (`server.ts:2319`). Since VSO writes into the **app** namespace, also add an **egress allow** from the app namespace to the `vault` namespace? — **No:** VSO (in `vault-secrets-operator-system`) reads Vault and writes the Secret via the apiserver; the **app pod never talks to Vault**. So no app-side netpol change is needed. (This is a real advantage of the sync-to-Secret seam over an agent sidecar.)
**Per-app Vault role/policy provisioning** — when `create_app` first provisions an app, MCP must also create the Vault k8s-auth role `<owner>-<name>` bound to a policy granting read on `apps/data/<owner>/<name>`, scoped to the app's ServiceAccount in `<owner>--<name>--*`. Add this to the create flow (`server.ts` step 7 region, ~1225) via the same `secrets.ts` client (`vault write auth/kubernetes/role/...` + `vault policy write`). Idempotent.
### Phase 3 — mitosis: germinate sealing + bootstrap + profile (PR to `plat/mitosis`)
- **`bin/germinate` ~25-35:** add `: "${VAULT:=0}"` (opt-in, mirroring how `MONITORING`/`full` profile is the toggle). Thread through `up`/`mitosis` usage like the existing `PROFILE` var.
- **Lean profile (`lib.sh trim_gitops_lean` ~71-129):** Vault is **opt-in**, so the default lean profile **removes** `platform/vault` + `platform/vault-secrets-operator` and drops their two `flux-kustomizations.yaml` blocks (same as it already `rm -rf`s `monitoring`/`grafana`/`pgadmin` and rewrites `flux-kustomizations.yaml`). When `VAULT=1`, skip that removal.
- **Sovereign sealing (`lib.sh fork_regen_secrets` ~207-317):** Vault's unseal material is **generated at init time, not fork time** (you can't pre-seal a key Vault hasn't minted yet). So instead of sealing it in `fork_regen_secrets`, germinate seals it **post-Forgejo**, in the same window where `fork_postforgejo_secrets`/`fork_register_oauth` run (after Flux has stood up the real platform and Vault has come up). New `fork_bootstrap_vault`:
  1. port-forward `svc/vault` (like the existing `fj_pf` Forgejo port-forward).
  2. if `vault status` says uninitialized: `vault operator init -key-shares=1 -key-threshold=1 -format=json`.
  3. `unseal` with the key.
  4. enable `apps/` KV v2 + `kubernetes` auth + write the MCP-scoped policy/token.
  5. `_fork_seal` the `vault-init` Secret (`unseal_key`,`root_token`) and `sops set` the three new `plat-mcp-secrets` keys (`VAULT_ADDR`,`VAULT_TOKEN`, and the bootstrap confirmation) — using the **exact** `_fork_seal`/`sops set` mechanics already in `lib.sh`. Then `git commit` + push so Flux reconciles a now-unsealable Vault.
  6. `fork_verify_all_sealed` (~325) automatically covers the new `vault-init.yaml` since it globs every `clusters/**` sops file — **a real win: the existing gate proves the Vault material is sovereign-sealed for free.**
This keeps the **whole** sovereignty story intact: a forked daughter mints its **own** Vault, its **own** unseal key, sealed to its **own** age key. No parent Vault state crosses the boundary.
### Phase 4 — _app-template: nothing
The `extraEnvFrom` contract already exists (`charts/app/values.yaml:74`, `deployment.yaml:66`, `worker-deployment.yaml:54`, `migrate-job.yaml:62`). **No PR needed.** This is the design paying off.
---
## Validation plan
**Already run (helm available, repo added):**
- `helm repo add hashicorp https://helm.releases.hashicorp.com` → ok. Chart versions: `vault 0.33.0` (app `2.0.2`), VSO `1.4.0` (app `1.4.0`).
- `helm template vault hashicorp/vault --version 0.33.0 -f vault-values.yaml` → **exit 0, no errors.** Rendered: `StatefulSet replicas: 1`, `image: hashicorp/vault:1.18.5`, PVC `storage: 1Gi` / `storageClassName: local-path`, Raft `storage "raft"` + `service_registration "kubernetes"` + `tls_disable = 1` present, **injector Deployment absent** (`vault-agent-injector` count = 0), `ClusterRoleBinding vault-server-binding` present (the `system:auth-delegator` binding k8s-auth needs).
- The chart's only `kind: Pod` is `templates/tests/server-test.yaml` carrying `"helm.sh/hook": test` → **Flux never applies test hooks during reconciliation**, so no values change is needed to suppress it. (Confirmed the annotation; `server.tests.enabled=false` is NOT a real chart knob — don't put it in values.)
- `helm template vso hashicorp/vault-secrets-operator --version 1.4.0` → exit 0; ships **10 `*.secrets.hashicorp.com` CRDs** incl. `vaultstaticsecrets`, justifying VSO's own `wait: true` Kustomization before `apps`.
- `helm show crds hashicorp/vault-secrets-operator --version 1.4.0` → `VaultStaticSecret` is group `secrets.hashicorp.com`, fields `mount`/`path`/`type`/`destination`/`vaultAuthRef` — matches the overlay shape above.
**To run on a live colima/k3d cluster before merge:**
1. `MONITORING`-style opt-in germinate with `VAULT=1`; assert `vault-0` pod reaches `Running`, `vault status` shows initialized+unsealed, `fork_verify_all_sealed` passes with `vault-init.yaml` in the set.
2. Kill the `vault-0` pod; assert the unseal Job re-unseals it and the `<name>-env` Secret survives (Raft persistence).
3. `create_app` an app, `set_secret(FOO=bar)`, assert (a) Vault KV has it, (b) VSO writes `<name>-env` Secret in the app ns, (c) the pod env contains `FOO=bar` after rollout, (d) `list_secrets` returns `["FOO"]` with no value.
4. Negative: `set_secret(BETTER_AUTH_SECRET=x)` → rejected; non-org-member caller → `ForbiddenError`; app B's VaultAuth role cannot read app A's path.
5. **Capstone:** fold into the depth-2 cold-start (task #10) — original→daughter, daughter germinated with `VAULT=1`, run all of (1)-(4) on the daughter to prove sovereignty.
---
## Tradeoffs & alternatives considered
- **Vault vs just more sops.** sops can already hold arbitrary secrets, and a "user-set env" could in principle be a `sops set` into a per-app secret. **Rejected** because (a) it would require a write path into gitops on every `set_secret` (commit + push + Flux reconcile latency, plus the user's plaintext momentarily in the MCP process writing a git commit), and (b) the issue explicitly says Vault. Vault gives a mutable runtime store with audit, versioned KV, and per-path policy that sops-in-git can't.
- **Shamir shares=1 vs >1.** Shares>1 (e.g. 3-of-5) is the production HA story, but on a single-operator sovereign platform all shares end up sealed under the *same* age key anyway, so >1 buys no real separation-of-duties — only more material to manage and more ways for the unseal Job to fail. v1: shares=1. A multi-operator deployment can raise it.
- **VSO vs ESO** — covered above; VSO chosen for HashiCorp-alignment, ESO is the documented fallback.
- **Image pin `1.18.5` (MPL-2.0) vs `2.0.2` (BSL).** Vault relicensed to BSL at 1.15+/2.0. For a self-replicating "grow your own platform" project, shipping a BSL binary in the seed has redistribution implications. v1 pins the **last MPL-2.0 line**. The clean long-term answer may be **OpenBao** (the LF fork of Vault, MPL-2.0, drop-in API-compatible) — VSO and the KV/k8s-auth API are identical. Flagged as an open question, not decided here.
- **Sync-to-Secret vs Vault Agent sidecar vs CSI.** Sidecar/CSI would require `_app-template` changes and app-pod→Vault egress. The VSO sync keeps the app contract untouched and the app pod Vault-unaware. Chosen.
- **Public Vault UI ingress.** Deferred from v1; in-cluster reachability is all the feature needs. Adds attack surface for no v1 benefit.
---
## Open questions
		@@ -0,0 +1,23 @@
	1	# Design proposals
	2
	3	From-first-principles design proposals for open platform features, written to be
	4	reviewed and approved before implementation. Each is anchored in the real
	5	code (concrete `file:line` for every seam it proposes touching), shows the actual
	6	manifests / tool signatures / code shapes it would add, and is honest about the
	7	hard parts, tradeoffs, and open questions.
	8
	9	These cover the larger / greenfield issues where the right move is to agree on
	10	the architecture first. (The two enhancement issues that were tractable to build
	11	directly — observability/Grafana #6 and platform agent-work #7 — shipped as PRs
	12	rather than proposals.)
	13
	14	\| Proposal \| Issue \| One line \|
	15	\| -------------------------------------------------------------------------------------------- \| ----- \| --------------------------------------------------------------------------------------------------------------------------------------------- \|
	16	\| [self-serve-secrets-vault.md](./self-serve-secrets-vault.md) \| #2 \| Per-app env vars backed by HashiCorp Vault, alongside sops (not replacing it); the hard part is single-node unseal without cloud KMS. \|
	17	\| [self-serve-custom-domains.md](./self-serve-custom-domains.md) \| #3 \| Vercel-style BYO domains: self-serve granting + DNS ownership verification + public TLS, on top of the registry/resolver that already exists. \|
	18	\| [byo-identity-provider-entra-oidc-forgejo.md](./byo-identity-provider-entra-oidc-forgejo.md) \| #4 \| BYO upstream IdP (Entra/OIDC) at `mitosis up`, auto-syncing users + teams into Forgejo (the identity hub). \|
	19	\| [repo-migration-auto-port.md](./repo-migration-auto-port.md) \| #5 \| A Forgejo repo migration triggers an agent to conform the repo onto the app-template; bulk supported. \|
	20
	21	How to use: read a proposal, comment on its Open questions, and approve the
	22	approach (or steer it). Approved proposals become the build plan — each lays out
	23	its work as phased, per-repo PRs.