Public: anyone can read this; nobody can change it.

aks ground: rewrite the seed's k3s CIDR hardcodes (A4 fix + latent netpols) + surface germinate WARNs #162

Merged opened this view on origin

Check · Review · Land done · Live — Landed; nothing deploys from this branch, so this is the end of its path.

Merged

Follow-up to #159. With germination no longer stalling on aks, the first full aks assert run (gate 08192156p5062, 63m, clean teardown) reached the assertion suite and failed A4 + A12.

Notes

shipped merged

opened this view on origin

Follow-up to #159. With germination no longer stalling on aks, the first full aks assert run (gate 08192156p5062, 63m, clean teardown) reached the assertion suite and failed A4 + A12. This PR fixes the A4 class completely, fixes two latent NetworkPolicy variants of the same root cause found during review, and closes the observability hole that made the A12 failure invisible in the gate log. (A12 itself — the front door dying non-fatally during germinate — is root-caused to a mechanism but not yet to the exact sub-step; filed separately.)

A4-redirect-matrix: the seed trusts k3s's CIDRs, AKS uses different ones

FAIL [A4-redirect-matrix] no-XFP=301 XFP:https=301 (want 200) https=200

The A4 probe curls traefik's ClusterIP from an in-cluster pod on both grounds. The seed's traefik HelmRelease gates the redirect-to-https bypass on forwardedHeaders.trustedIPs = [10.42.0.0/16, 10.43.0.0/16] — k3s's default pod/service CIDRs. On AKS (Azure CNI overlay + cilium) pods live in 10.244.0.0/16 and services in 10.0.0.0/16, so the probe pod is untrusted, its X-Forwarded-Proto: https is stripped, and the redirect middleware 301s. The same contract matters in production: in-cluster edge terminators (cloudflared-style, HYB-1) forward plain HTTP with XFP:https and must pass. The assert is right and untouched; the platform config was wrong.

The same k3s hardcode exists in three more places, found during adversarial review of this fix:

  • Forgejo's REVERSE_PROXY_TRUSTED_PROXIES (wrong client IPs logged on aks)
  • the backup netpol's apiserver egress (10.43.0.0/16)
  • the network-governance policy's apiserver ClusterIP pinhole (10.43.0.1/32) — the CNPG-bootstrap allowance, silently non-matching on AKS

Fix: new trim_gitops_aks step 4h rewrites all four files (10.42.0.0/16→10.244.0.0/16, 10.43.0.0/16→10.0.0.0/16, 10.43.0.1→10.0.0.1), and main.bicep pins podCidr/serviceCidr/dnsServiceIP to exactly those values so an Azure default drift can't silently untrust everything again.

Verified: rewrite proven against the real seed (zero 10.42/10.43 references remain tree-wide; RFC1918 10.0.0.0/8 excludes untouched; the policy's explanatory comment stays truthful); az bicep build clean, values valid for overlay+cilium (no VNet overlap — aks mode provisions no custom VNet); shellcheck parity with baseline; ordering vs the other trim steps and the identity rewrite checked (no interaction).

Germinate WARNs were invisible in the gate log

germinate treats a dead front door as a non-fatal WARN by design, but gate.sh's log tails filtered on ==>|FATAL(|ERROR) — so the aks run printed a clean "germinate: complete" while the front door had already died, and the A12 evidence was lost with the run's temp dir. The tails now pass WARN through (vms journal: ==>|FATAL|WARN; aks local tail: ==>|FATAL|ERROR|WARN).

GATE-NOTES' security note is updated accordingly: the public-CI-log filter claim now names the wider pattern, with the reasoning for why WARN lines are safe to stream (curated messages; the single raw-content excerpt is failure-path-only, 200-char capped, credential-free).

Not in this PR

🤖 Generated with Claude Code

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs