Public: anyone can read this; nobody can change it.
Kyverno generate backlog: new namespaces wait minutes for tenant-defaults LimitRange → pods denied by require-resource-limits #220
Closed without the crew
Second writer found and removed. 35 pre-POL-2 app manifests in plat/gitops still shipped the four generic NetworkPolicies through Flux (prune: true) while plat-ns-network-governance owns the same objects with synchronize: true — the two controllers rewrote each other's resourceVersion forever, which is the conflict loop pinning the background controller and starving every queued LimitRange generate. gitops f637b377 strips git's copies (140 documents); Kyverno is now the sole owner. The regeneration window is safe: every tenant pod's own allow-egress excludes RFC1918, so nothing can reach a briefly-unpolicied sibling. Verify on the cluster with kubectl get updaterequests -A — the Pending count should drain to ~0; then this closes.
Verified on prod: kubectl get updaterequests -A = 1 (was 246 Pending). The conflict storm is gone with Kyverno as sole owner. Closing.
Observed 2026-08-25 ~20:05Z on prod while a PR preview (otavert/issue-map pr-4) came up.
Symptom: the preview namespace got its Deployment but every pod was denied:
require-resource-limits / require-requests-and-limits— because the namespace had notenant-defaults-standardLimitRange yet (pr-3, created 45 min earlier, had one). The RS sat in FailedCreate backoff.Cause: the Kyverno background controller is starved.
kubectl get updaterequests -A= 246 Pending (231 forplat-ns-network-governance, 12plat-ns-resource-governance, 3plat-ns-dos-bound), the oldest 39 min. The controller is at ~500m CPU looping onOperation cannot be fulfilled on networkpolicies … "default-deny" / "allow-ingress-traefik" / "allow-ingress-from-workflows"— resourceVersion conflicts whilesynchronize: truerules re-apply NetworkPolicies that something else also writes. Everything queued behind those (the LimitRange generate) waits.Impact: every new namespace (PR previews, new app bootstraps → v1 previews) is blocked for as long as the backlog takes; the platform looks like 'preview never comes up'.
Unblocked pr-4 by hand: copied the LimitRange from pr-3 into the namespace.
Suggested fix: find the second writer of those NetworkPolicies (org Kustomization? overlay?) and make one owner; or turn
synchronizeoff for the netpol rules; and generate the LimitRange from its own policy so a netpol storm cannot starve it. Consider a background-controller replica bump as a stopgap.🤖 Generated with Claude Code