Public: anyone can read this; nobody can change it.

azure-gate: fix the aks cert-zone doubling, CF 403 preflight, and PASS-exits-1; root-cause the pkill false-FAIL (#158) #159

Merged opened this view on origin

Check · Review · Land done · Live — Landed; nothing deploys from this branch, so this is the end of its path.

Merged

Today's gate runs failed in four distinct ways — two aks, two vms. This PR fixes the two aks failure chains and the PASS-exits-1 bug that #158's success exposed, and documents the pkill one (which trevato's #158 fixed while this branch was in review — rebased onto it, no overlap remains).

Notes

shipped merged

opened this view on origin

Today's gate runs failed in four distinct ways — two aks, two vms. This PR fixes the two aks failure chains and the PASS-exits-1 bug that #158's success exposed, and documents the pkill one (which trevato's #158 fixed while this branch was in review — rebased onto it, no overlap remains). One ops item stays open (checklist at the bottom).

1. aks germination stall: the doubled hostedZoneName (the GATE-NOTES blocker)

trim_gitops_aks 4d was innocent — it writes the correct single zone into the ClusterIssuer. The doubling happens later, in rewrite_identity_domain: germinate's identity rewrite maps the seed's mother domain (open-platform.sh) to the daughter with a global sed, and a gate daughter's domain ends with the mother domain. So the already-correct hostedZoneName: gl3.gate.open-platform.sh matched on its suffix and became gl3.gate.gl3.gate.open-platform.sh → DNS-01 403 → wildcard-public-tls never issues → mcp chart build x509-stalls.

Fix: the rewrite now sentinel-shields values already equal to the daughter domain before the sweep — only in the suffix shape ($new contains $old); the reverse shape keeps the plain sweep, where shielding would silently no-op the whole rewrite.

Verified (sandbox gitops trees, BSD + GNU sed, bash 3.2 + 5.3): old code doubles, new code doesn't; email/ROOT_URL/repository: still rewrite; plat-local-ca-tls untouched; non-suffix domain shapes byte-identical to old behavior; reverse shape (old=gl3.gate.open-platform.sh new=open-platform.sh) genuinely rewrites; running twice is a byte-identical fixed point; no sentinel residue; zero new shellcheck findings.

2. aks CI run r420a1 died at 37s: Cloudflare 403 as a json traceback

First-ever CI aks run (vms never touches Cloudflare; the successful l3 run used a laptop token file). The AZURE_GATE_CF_TOKEN secret got 403 on the zone lookup, curl -f ate the body, and json.load crashed on empty stdin — after the RG had been created (create + safety-net delete for nothing).

Fix: cf_load now checks the HTTP status and dies with it plus the required scopes (Zone:Read + DNS:Edit on open-platform.sh); the cheap aks preflights (CF token, seed present, kubectl) run before az group create. down() guards ${DOMAIN:-} since the preflight makes CF_ZONE_ID exist before DOMAIN does — a death in that window must not unbound-crash the teardown trap (verified against the exact trap structure on bash 3.2/5.3: teardown runs, exit code preserved).

3. vms r425a1 false-FAIL after an all-green gate — root-caused here, fixed by #158 (docs only)

r425a1 ran with #156's "specific" pattern and still printed COLDSTART GATE PASS (63s) → Connection closed by remote host → FATAL. Root cause, proven in a Linux container: pkill -f regex-matches full argvs, and the carrier bash -c script's argv contains the pattern's own literal text — kubectl.port-forward svc/forgejo-http matches itself (its . is just characters to the regex), kills its own shell, and drops the SSH channel. The #156 pattern kills its carrier with rc=143; a self-match-broken pattern doesn't and still matches the real kubectl argv. (macOS pgrep does not reproduce this — test such patterns on Linux.)

#158 landed the fix while this was in review — [f]-bracketed reap pattern + GATE_RC read from output instead of ssh's exit + a timeout backstop — which is strictly stronger than a pattern tweak. This branch is rebased onto it and only adds the root-cause analysis + container proof to GATE-NOTES (r436a1 is #158's first live exercise).

4. vms r436a1: the first fully PASSING run still failed CI — the trailing keep-message leaks exit 1

r436a1 (first live run of #158) printed COLDSTART GATE PASS, GATE_RC=0, RESULT: PASS, tore down cleanly — and the job failed. The script's final statement is [ "$KEEP" = 1 ] && say "kept: ...", which returns 1 whenever keep=false (every normal CI run); that status becomes $? at the EXIT trap, on_exit captures it as rc, and exit $rc fails the run. It could never bite before because no CI run had ever reached the end green. Fix: explicit exit 0 on the happy path; died's exit 1 still carries real failures through the same trap.

Not in this PR

GATE-NOTES.md updated to reflect all three root causes.

🤖 Generated with Claude Code

Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs