-
Bug
-
Resolution: Duplicate
-
Critical
-
None
-
4.14
-
None
-
No
-
Proposed
-
False
-
This was noticed when bringing in k8s 1.27.3:
https://github.com/openshift/kubernetes/pull/1609
the exact job is:
the panic is:
E0620 22:33:59.159702 1 runtime.go:79] Observed a panic: &runtime.TypeAssertionError{_interface:(*runtime._type)(0x1dd8820), concrete:(*runtime._type)(0x1f01740), asserted:(*runtime._type)(0x20f42e0), missingMethod:""} (interface conversion: interface {} is cache.DeletedFinalStateUnknown, not *v1.Pod) goroutine 666 [running]: k8s.io/apimachinery/pkg/util/runtime.logPanic({0x1e54460?, 0xc001067b00}) /go/src/github.com/openshift/ovn-kubernetes/go-controller/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:75 +0x99 k8s.io/apimachinery/pkg/util/runtime.HandleCrash({0x0, 0x0, 0xc00005e5e0?}) /go/src/github.com/openshift/ovn-kubernetes/go-controller/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:49 +0x75 panic({0x1e54460, 0xc001067b00}) /usr/lib/golang/src/runtime/panic.go:884 +0x213 github.com/ovn-org/ovn-kubernetes/go-controller/pkg/ovn/controller/apbroute.(*ExternalGatewayMasterController).processNextPodWorkItem(0xc0004f88f0, 0xc0006feb20) /go/src/github.com/openshift/ovn-kubernetes/go-controller/pkg/ovn/controller/apbroute/master_controller.go:466 +0x586 github.com/ovn-org/ovn-kubernetes/go-controller/pkg/ovn/controller/apbroute.(*ExternalGatewayMasterController).runPodWorker(...) /go/src/github.com/openshift/ovn-kubernetes/go-controller/pkg/ovn/controller/apbroute/master_controller.go:450 github.com/ovn-org/ovn-kubernetes/go-controller/pkg/ovn/controller/apbroute.(*ExternalGatewayMasterController).Run.func2.1() /go/src/github.com/openshift/ovn-kubernetes/go-controller/pkg/ovn/controller/apbroute/master_controller.go:216 +0x22 k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0x30?) /go/src/github.com/openshift/ovn-kubernetes/go-controller/vendor/k8s.io/apimachinery/pkg/util/wait/backoff.go:226 +0x3e k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0x0?, {0x24706a0, 0xc000c9fe60}, 0x1, 0xc0004ed800) /go/src/github.com/openshift/ovn-kubernetes/go-controller/vendor/k8s.io/apimachinery/pkg/util/wait/backoff.go:227 +0xb6 k8s.io/apimachinery/pkg/util/wait.JitterUntil(0x0?, 0x3b9aca00, 0x0, 0x0?, 0x0?) /go/src/github.com/openshift/ovn-kubernetes/go-controller/vendor/k8s.io/apimachinery/pkg/util/wait/backoff.go:204 +0x89 k8s.io/apimachinery/pkg/util/wait.Until(...) /go/src/github.com/openshift/ovn-kubernetes/go-controller/vendor/k8s.io/apimachinery/pkg/util/wait/backoff.go:161 github.com/ovn-org/ovn-kubernetes/go-controller/pkg/ovn/controller/apbroute.(*ExternalGatewayMasterController).Run.func2(0xc0006724e0) /go/src/github.com/openshift/ovn-kubernetes/go-controller/pkg/ovn/controller/apbroute/master_controller.go:215 +0xcf created by github.com/ovn-org/ovn-kubernetes/go-controller/pkg/ovn/controller/apbroute.(*ExternalGatewayMasterController).Run /go/src/github.com/openshift/ovn-kubernetes/go-controller/pkg/ovn/controller/apbroute/master_controller.go:212 +0x51a panic: interface conversion: interface {} is cache.DeletedFinalStateUnknown, not *v1.Pod [recovered] panic: interface conversion: interface {} is cache.DeletedFinalStateUnknown, not *v1.Pod goroutine 666 [running]: k8s.io/apimachinery/pkg/util/runtime.HandleCrash({0x0, 0x0, 0xc00005e5e0?}) /go/src/github.com/openshift/ovn-kubernetes/go-controller/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:56 +0xd7 panic({0x1e54460, 0xc001067b00}) /usr/lib/golang/src/runtime/panic.go:884 +0x213 github.com/ovn-org/ovn-kubernetes/go-controller/pkg/ovn/controller/apbroute.(*ExternalGatewayMasterController).processNextPodWorkItem(0xc0004f88f0, 0xc0006feb20) /go/src/github.com/openshift/ovn-kubernetes/go-controller/pkg/ovn/controller/apbroute/master_controller.go:466 +0x586 github.com/ovn-org/ovn-kubernetes/go-controller/pkg/ovn/controller/apbroute.(*ExternalGatewayMasterController).runPodWorker(...) /go/src/github.com/openshift/ovn-kubernetes/go-controller/pkg/ovn/controller/apbroute/master_controller.go:450 github.com/ovn-org/ovn-kubernetes/go-controller/pkg/ovn/controller/apbroute.(*ExternalGatewayMasterController).Run.func2.1() /go/src/github.com/openshift/ovn-kubernetes/go-controller/pkg/ovn/controller/apbroute/master_controller.go:216 +0x22 k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0x30?) /go/src/github.com/openshift/ovn-kubernetes/go-controller/vendor/k8s.io/apimachinery/pkg/util/wait/backoff.go:226 +0x3e k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0x0?, {0x24706a0, 0xc000c9fe60}, 0x1, 0xc0004ed800) /go/src/github.com/openshift/ovn-kubernetes/go-controller/vendor/k8s.io/apimachinery/pkg/util/wait/backoff.go:227 +0xb6 k8s.io/apimachinery/pkg/util/wait.JitterUntil(0x0?, 0x3b9aca00, 0x0, 0x0?, 0x0?) /go/src/github.com/openshift/ovn-kubernetes/go-controller/vendor/k8s.io/apimachinery/pkg/util/wait/backoff.go:204 +0x89 k8s.io/apimachinery/pkg/util/wait.Until(...) /go/src/github.com/openshift/ovn-kubernetes/go-controller/vendor/k8s.io/apimachinery/pkg/util/wait/backoff.go:161 github.com/ovn-org/ovn-kubernetes/go-controller/pkg/ovn/controller/apbroute.(*ExternalGatewayMasterController).Run.func2(0xc0006724e0) /go/src/github.com/openshift/ovn-kubernetes/go-controller/pkg/ovn/controller/apbroute/master_controller.go:215 +0xcf created by github.com/ovn-org/ovn-kubernetes/go-controller/pkg/ovn/controller/apbroute.(*ExternalGatewayMasterController).Run /go/src/github.com/openshift/ovn-kubernetes/go-controller/pkg/ovn/controller/apbroute/master_controller.go:212 +0x51a
it only happened once, but it seems like there's a wrong handling of terminating pods.
- duplicates
-
OCPBUGS-15227 CI got "Undiagnosed panic detected in pod" in ovnkube-master because of TypeAssertionError: "interface {} is cache.DeletedFinalStateUnknown, not *v1.Pod"
- Closed