Public: anyone can read this; nobody can change it.

deploy/azure: mode=vms|aks in main.bicep, HA servers carve-out, AKS germinate profile #139

Merged opened this view on origin

Check · Review · Land done · Live — Landed; nothing deploys from this branch, so this is the end of its path.

Merged

Lands the Azure work that until now lived only as uncommitted files in a local checkout (the AKS azure-pure POC + the 3-node HA expansion, both platforms since torn down — this PR is the surviving artifact).

shipped merged

opened this view on origin

Lands the Azure work that until now lived only as uncommitted files in a local checkout (the AKS azure-pure POC + the 3-node HA expansion, both platforms since torn down — this PR is the surviving artifact).

What's here

deploy/azure/main.bicep — one template, mode as a parameter

  • mode=vms (default): the existing bare-VM path, rebased onto current main — the #123/#124 PUBLIC_IP-in-node.env behavior is preserved.
  • mode=aks: managed ground — AKS (workload identity, Cilium network policy, managed Prometheus + Container Insights), ACR, public DNS zone, Key Vault with federated identities (mitosis-cert-manager → DNS Zone Contributor, mitosis-flux-sops → KV Crypto User).

deploy/azure/servers.bicep — servers-only carve-out for growing a LIVE single-node group to 3-node HA. Exists because running main.bicep ha=true against a live group replaces immutable customData and NSG rules. Previously archived only in a plat/mitosis#6 comment; param docs carry the join-token gotcha (joiners present the existing cluster's token, not a fresh GUID).

bin/germinate + bin/lib.sh — CLUSTER_PROFILE=aks: trim_gitops_aks strips k3s-specific organs (monitoring/Grafana → Azure Monitor, node-fabric, registry-node-config, coredns rewrite), repoints the runner image at ACR (az acr build replaces docker+ctr side-load), adds a Kyverno aks-image-redirect mutate policy and extends restrict-image-registries. Includes the 8 aks-ground fixes from the POC germination runs. Rebase note: runs after upstream's new strip_gitops_tenants, before identity rewrite.

Runbooks/scripts: README-aks.md (customer runbook incl. pilot hardening), mirror-images.sh (~25 az acr import), entra-forgejo-oidc.sh (Entra ID sign-in for aks-ground Forgejo), aks-cost-experiment.md (62% cost-cut runbook: CNPG PDBs block drains, B-v2 quota 0, nodepool-add CLI crash → az rest).

Review notes

  • mode=vms was verified to render byte-identical ARM vs the pre-mode template at the time of the split; it has since been rebased over #123/#124 and #132 — worth a re-render if touching the vms path.
  • Pushed via the MCP contents API (no git credential on the authoring machine), so two new scripts lack the executable bit. After merge (or on the branch): chmod +x deploy/azure/entra-forgejo-oidc.sh deploy/azure/mirror-images.sh. Both are also runnable as bash <script>.
  • History has a few superseded corrective commits (same message) from the API push; tree content is hash-verified byte-identical to the authoring branch. Squash-merge recommended.

🤖 Generated with Claude Code

commented

Pushed 5747ef0 to this branch: deploy/azure/README.md — one Azure guide for both modes, written in ASD-STE100 Simplified Technical English (trevato's ask): resources each mode creates, numbered steps, servers.bicep expansion, Entra sign-in, known limits, parameters/outputs, cost, teardown. It folds README-aks.md in (removed — one doc, no drift) and links from deploy/README.md. Also set the exec bit on mirror-images.sh / entra-forgejo-oidc.sh as the PR body asked.

Two things I changed in the runbook vs. README-aks.md — please check them against what you ran:

  • The germinate step now carries AKS_RG=<rg> — bin/germinate :?-requires it under CLUSTER_PROFILE=aks (it derives subscription + cert-manager client ID from the RG), so the old command would have stopped at the guard.
  • Mirror-before-germinate is now an explicit instruction (the aks-image-redirect comment in lib.sh says redirected pods ErrImagePull otherwise).

Review notes on the rest of the branch (read-only — no az/bicep here, so no compile; nothing below blocks merge):

  1. AKS defaults resurrect the $364/mo baseline — main.bicep:80/83/346 still pin 2× D4ds_v6 + Container Insights + managed Prometheus; your own client-readiness item 7. The experiment landed on 1× D4as_v7, monitoring off (with the 12-disk attach ceiling). Worth deciding the default before merge — the README documents what the template does today.
  2. mirror-images.sh:64-66 — the "RETRY" branch runs the identical command (only the >/dev/null is gone), so the comment over-promises; either drop it or make the retry differ. Also forgejo:15.0.5-rootless — prod runs 15.0.6; worth re-pinning against the seed vintage you cut from.
  3. entra-forgejo-oidc.sh:37 reads the Forgejo admin password into ADMIN_PW and never uses it (:54 reserved). Least privilege: drop until the REST fallback exists.
  4. servers.bicep:97/122 — for (idx, j) in serverIndexes names the item idx and the index j; it renders correctly (I traced pip/nic/vm/zones/cloudInit), but the inverted names will trip the next reader.
  5. aks-cost-experiment.md carries a subscription ID + the customer hostname, and records a torn-down platform — fine on the private forge; flagging in case you'd rather scrub the ID or move it under docs/.
  6. Commit history: four same-subject commits (b994d05/3c3d1f4/48f1c4b, de26448/45b8e14/e52a995) — squash-merge as you suggest.

If any README wording doesn't match what the aks profile actually does, say so here and I'll fix it.

— Claude (for trevato), via ci-bot

commented

Re-render check you asked for in the PR body — done with bicep 0.39.0 (via nix), head 5747ef0:

  • main.bicep and servers.bicep both compile, 0 errors. main.bicep emits the expected BCP318 "may be null" warnings on the if (mode == …) resources — benign, every access is behind the same condition or a mode ternary.
  • mode=vms ARM is byte-identical to main for all five resources (NSG, VNet, public IP, NIC, VM bodies — diffed the compiled JSON per type, ignoring the added condition). Only variable change is nodeCount gaining the mode guard; outputs are the same expressions under a mode ternary.
  • One default changed: adminSshPublicKey was required, now defaults to '' (so aks can omit it). A vms deploy that forgets the key now fails at VM creation instead of at parameter validation — acceptable, just noting it.
  • bin/lib.sh delta is one self-contained function (trim_gitops_aks, +346/−0); bin/germinate changes are gated on CLUSTER_PROFILE=aks — default k3s path untouched, and the coldstart gate is green on this head.

From my side this is good to squash-merge; the items in my earlier comment are follow-ups, not blockers.

— Claude (for trevato)

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs