forwardAuth breaks all app ingresses: traefik cannot resolve per-app kubernetescrd Middleware (middleware does not exist) #93

Summary

On the product-garden.com granddaughter (bare-k3s VPS), enabling forwardAuth (RBAC-2, PLAT_FORWARD_AUTH=1 + platAuth.enabled=true) makes every app ingress 404 publicly. The plat-auth service itself is healthy (healthz 200, and its /forward endpoint returns a correct 302-to-Forgejo-OAuth decision), but traefik cannot resolve the per-app forwardAuth Middleware:

ERR error="middleware \"plat--qa-smoke--prod-plat-access@kubernetescrd\" does not exist"
    entryPointName=web routerName=web-plat-qa-smoke-prod-qa-smoke-...

Traefik invalidates the whole router when a referenced middleware can't be loaded → 404 on every app hostname.

What was ruled out

• The plat-access / plat-strip Middleware CRs do exist in the app namespace (traefik.io/v1alpha1, correct names).
• traefik providers.kubernetescrd is enabled with allowCrossNamespace=true (no namespace restriction), and kubectl auth can-i list middlewares.traefik.io = yes for the traefik SA.
• A separate traefik RBAC gap (cannot list configmaps at cluster scope) was found and granted — the middleware error persisted after, so configmaps was not the cause.
• traefik was restarted twice; the error persists. These are the ONLY @kubernetescrd middlewares on the platform (the platform's own redirect-to-https is a @file-provider middleware), so the kubernetescrd middleware provider may not be loading any CRD middlewares at all on this stack/version.

Impact + current mitigation

forwardAuth cannot be enabled on an app-serving platform until this is fixed — it takes down all app HTTPS. On the granddaughter I disabled it (platAuth.enabled=false, PLAT_FORWARD_AUTH=0) so apps serve; the crew-built qa-smoke app then served 200 over the public tunnel. The forwardAuth wiring (OAuth app, sealed creds, plat-auth deployment, auth ingress) is otherwise correct and healthy.

Suspected direction

traefik's kubernetescrd Middleware informer isn't populating its registry (CR present + RBAC ok + provider watching, yet "does not exist"). Candidates: (a) traefik chart/version's kubernetescrd provider not actually watching Middleware kind despite the flag; (b) an informer-cache sync stall (the platform's traefik ClusterRole is missing configmaps, which can block WaitForCacheSync for the whole shared informer factory — granting it did not clear it here, but the traefik ClusterRole should still be audited); (c) CRD group/version mismatch between the installed traefik and the traefik.io/v1alpha1 CRs the renderer emits.

Repro

Germinate a platform, enable forwardAuth, build any app → its public URL 404s with the middleware-does-not-exist error in traefik logs. Note: this needs a real app served through traefik — the k3d coldstart gate never builds+serves an app, so it never exercises forwardAuth end-to-end (same blind spot as #89/#91).