-
Bug
-
Resolution: Done
-
Critical
-
None
-
4.13.0
-
None
-
Approved
-
False
-
The following is the description from the original bug for kube-apiserver. For openshift-apiserver we have not had any report of panic and server crashing. But the gap still exists, the panic (if it is raised for some reason) must be handled so that the openshift-apiserver does not crash
<<<<<<<<<<<<<<<<<<<<<<---------------------------------------------------------------------->>>>>>>>>>>>>>>>>>>>>>>>>>
Description of problem:
kubernetes-apiserver panics when trying to warn about pod security
Version-Release number of selected component (if applicable):
registry.ci.openshift.org/ocp/release:4.13.0-0.ci-2023-01-17-112321
How reproducible:
Often
Steps to Reproduce:
1. Create HyperShift cluster with a recent OCP release payload 2. Watch control plane operator pods
Actual results:
Expected results:
kube-apiserver crashes and restarts after it panics
Additional info:
Log from panic: E0117 13:52:17.183773 1 patch_podspecextractor.go:96] "failed to mutate object for PSA using SCC" err="pods \"pod-for-container-named-konnectivity-agent\" is forbidden: unable to validate against any security context constraint: [provider \"anyuid\": Forbidden: not usable by user or serviceaccount, provider restricted-v2: .spec.securityContext.hostNetwork: Invalid value: true: Host network is not allowed to be used, spec.containers[0].securityContext.runAsUser: Invalid value: 1000: must be in the ranges: [1000140000, 1000149999], spec.containers[0].securityContext.hostNetwork: Invalid value: true: Host network is not allowed to be used, provider \"restricted\": Forbidden: not usable by user or serviceaccount, provider \"nonroot-v2\": Forbidden: not usable by user or serviceaccount, provider \"nonroot\": Forbidden: not usable by user or serviceaccount, provider \"hostmount-anyuid\": Forbidden: not usable by user or serviceaccount, provider \"hostnetwork-v2\": Forbidden: not usable by user or serviceaccount, provider \"hostnetwork\": Forbidden: not usable by user or serviceaccount, provider \"hostaccess\": Forbidden: not usable by user or serviceaccount, provider \"privileged\": Forbidden: not usable by user or serviceaccount]" E0117 13:52:17.183796 1 patch_podspecextractor.go:97] failed to mutate object for PSA using SCC: pods "pod-for-container-named-konnectivity-agent" is forbidden: unable to validate against any security context constraint: [provider "anyuid": Forbidden: not usable by user or serviceaccount, provider restricted-v2: .spec.securityContext.hostNetwork: Invalid value: true: Host network is not allowed to be used, spec.containers[0].securityContext.runAsUser: Invalid value: 1000: must be in the ranges: [1000140000, 1000149999], spec.containers[0].securityContext.hostNetwork: Invalid value: true: Host network is not allowed to be used, provider "restricted": Forbidden: not usable by user or serviceaccount, provider "nonroot-v2": Forbidden: not usable by user or serviceaccount, provider "nonroot": Forbidden: not usable by user or serviceaccount, provider "hostmount-anyuid": Forbidden: not usable by user or serviceaccount, provider "hostnetwork-v2": Forbidden: not usable by user or serviceaccount, provider "hostnetwork": Forbidden: not usable by user or serviceaccount, provider "hostaccess": Forbidden: not usable by user or serviceaccount, provider "privileged": Forbidden: not usable by user or serviceaccount] E0117 13:52:17.183911 1 runtime.go:77] Observed a panic: Header called after Handler finished goroutine 154006 [running]: k8s.io/apimachinery/pkg/util/runtime.logPanic({0x450dce0?, 0x5cb8a40}) vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:75 +0x99 k8s.io/apimachinery/pkg/util/runtime.HandleCrash({0x0, 0x0, 0xc0209bad80?}) vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:49 +0x75 panic({0x450dce0, 0x5cb8a40}) /usr/lib/golang/src/runtime/panic.go:884 +0x212 golang.org/x/net/http2.(*responseWriter).Header(0x450dce0?) vendor/golang.org/x/net/http2/server.go:2693 +0x7a k8s.io/apiserver/pkg/endpoints/filters.(*recorder).AddWarning(0xc01ba662d0, {0x0, 0x0}, {0xc0325a3d40, 0x22d}) vendor/k8s.io/apiserver/pkg/endpoints/filters/warning.go:110 +0x43b k8s.io/apiserver/pkg/warning.AddWarning({0x5cf9038?, 0xc01ba744b0?}, {0x0, 0x0}, {0xc0325a3d40, 0x22d}) vendor/k8s.io/apiserver/pkg/warning/context.go:58 +0x82 k8s.io/kubernetes/plugin/pkg/admission/security/podsecurity.(*Plugin).Validate(0xc000e6ec30, {0x5cf9038, 0xc01ba744b0}, {0x5d15970?, 0xc0209b2c60?}, {0x5cfde80?, 0xc003263ba0?}) plugin/pkg/admission/security/podsecurity/admission.go:185 +0x555 k8s.io/apiserver/pkg/admission/metrics.pluginHandlerWithMetrics.Validate({{0x5cc4800, 0xc000e6ec30}, 0xc000b55690, {0xc000b556a0, 0x1, 0x1}}, {0x5cf9038, 0xc01ba744b0}, {0x5d15970, 0xc0209b2c60}, ...) vendor/k8s.io/apiserver/pkg/admission/metrics/metrics.go:108 +0xef github.com/openshift/library-go/pkg/apiserver/admission/admissiontimeout.pluginHandlerWithTimeout.Validate.func1() vendor/github.com/openshift/library-go/pkg/apiserver/admission/admissiontimeout/timeoutadmission.go:59 +0x8d created by github.com/openshift/library-go/pkg/apiserver/admission/admissiontimeout.pluginHandlerWithTimeout.Validate vendor/github.com/openshift/library-go/pkg/apiserver/admission/admissiontimeout/timeoutadmission.go:56 +0x22a panic: Header called after Handler finished [recovered] panic: Header called after Handler finished goroutine 154006 [running]: k8s.io/apimachinery/pkg/util/runtime.HandleCrash({0x0, 0x0, 0xc0209bad80?}) vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:56 +0xd7 panic({0x450dce0, 0x5cb8a40}) /usr/lib/golang/src/runtime/panic.go:884 +0x212 golang.org/x/net/http2.(*responseWriter).Header(0x450dce0?) vendor/golang.org/x/net/http2/server.go:2693 +0x7a k8s.io/apiserver/pkg/endpoints/filters.(*recorder).AddWarning(0xc01ba662d0, {0x0, 0x0}, {0xc0325a3d40, 0x22d}) vendor/k8s.io/apiserver/pkg/endpoints/filters/warning.go:110 +0x43b k8s.io/apiserver/pkg/warning.AddWarning({0x5cf9038?, 0xc01ba744b0?}, {0x0, 0x0}, {0xc0325a3d40, 0x22d}) vendor/k8s.io/apiserver/pkg/warning/context.go:58 +0x82 k8s.io/kubernetes/plugin/pkg/admission/security/podsecurity.(*Plugin).Validate(0xc000e6ec30, {0x5cf9038, 0xc01ba744b0}, {0x5d15970?, 0xc0209b2c60?}, {0x5cfde80?, 0xc003263ba0?}) plugin/pkg/admission/security/podsecurity/admission.go:185 +0x555 k8s.io/apiserver/pkg/admission/metrics.pluginHandlerWithMetrics.Validate({{0x5cc4800, 0xc000e6ec30}, 0xc000b55690, {0xc000b556a0, 0x1, 0x1}}, {0x5cf9038, 0xc01ba744b0}, {0x5d15970, 0xc0209b2c60}, ...) vendor/k8s.io/apiserver/pkg/admission/metrics/metrics.go:108 +0xef github.com/openshift/library-go/pkg/apiserver/admission/admissiontimeout.pluginHandlerWithTimeout.Validate.func1() vendor/github.com/openshift/library-go/pkg/apiserver/admission/admissiontimeout/timeoutadmission.go:59 +0x8d created by github.com/openshift/library-go/pkg/apiserver/admission/admissiontimeout.pluginHandlerWithTimeout.Validate vendor/github.com/openshift/library-go/pkg/apiserver/admission/admissiontimeout/timeoutadmission.go:56 +0x22a
This started happening after this PR merged:
https://github.com/openshift/kubernetes/pull/1453
- clones
-
OCPBUGS-5991 Kube APIServer panics in admission controller
- Closed
- is blocked by
-
API-1537 rebase openshift/apiserver
- Closed
- links to