-
Bug
-
Resolution: Done-Errata
-
Undefined
-
4.16
-
None
-
Critical
-
Yes
-
Proposed
-
False
-
Description of problem:
In Azure Stack, the Azure-Disk CSI Driver node pod CrashLoopBackOff: openshift-cluster-csi-drivers azure-disk-csi-driver-node-57rxv 1/3 CrashLoopBackOff 33 (3m55s ago) 59m 10.0.1.5 ci-op-q8b6n4iv-904ed-kp5mv-worker-mtcazs-m62cj <none> <none> openshift-cluster-csi-drivers azure-disk-csi-driver-node-8wvqm 1/3 CrashLoopBackOff 35 (29s ago) 67m 10.0.0.6 ci-op-q8b6n4iv-904ed-kp5mv-master-1 <none> <none> openshift-cluster-csi-drivers azure-disk-csi-driver-node-97ww5 1/3 CrashLoopBackOff 33 (12s ago) 67m 10.0.0.7 ci-op-q8b6n4iv-904ed-kp5mv-master-2 <none> <none> openshift-cluster-csi-drivers azure-disk-csi-driver-node-9hzw9 1/3 CrashLoopBackOff 35 (108s ago) 59m 10.0.1.4 ci-op-q8b6n4iv-904ed-kp5mv-worker-mtcazs-gjqmw <none> <none> openshift-cluster-csi-drivers azure-disk-csi-driver-node-glgzr 1/3 CrashLoopBackOff 34 (69s ago) 67m 10.0.0.8 ci-op-q8b6n4iv-904ed-kp5mv-master-0 <none> <none> openshift-cluster-csi-drivers azure-disk-csi-driver-node-hktfb 2/3 CrashLoopBackOff 48 (63s ago) 60m 10.0.1.6 ci-op-q8b6n4iv-904ed-kp5mv-worker-mtcazs-kdbpf <none> <none>
The CSI-Driver container log: panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0xc8 pc=0x18ff5db] goroutine 228 [running]: sigs.k8s.io/cloud-provider-azure/pkg/provider.(*Cloud).GetZone(0xc00021ec00, {0xc0002d57d0?, 0xc00005e3e0?}) /go/src/github.com/openshift/azure-disk-csi-driver/vendor/sigs.k8s.io/cloud-provider-azure/pkg/provider/azure_zones.go:182 +0x2db sigs.k8s.io/azuredisk-csi-driver/pkg/azuredisk.(*Driver).NodeGetInfo(0xc000144000, {0x21ebbf0, 0xc0002d5470}, 0x273606a?) /go/src/github.com/openshift/azure-disk-csi-driver/pkg/azuredisk/nodeserver.go:336 +0x13b github.com/container-storage-interface/spec/lib/go/csi._Node_NodeGetInfo_Handler.func1({0x21ebbf0, 0xc0002d5470}, {0x1d71a60?, 0xc0003b0320}) /go/src/github.com/openshift/azure-disk-csi-driver/vendor/github.com/container-storage-interface/spec/lib/go/csi/csi.pb.go:7160 +0x72 sigs.k8s.io/azuredisk-csi-driver/pkg/csi-common.logGRPC({0x21ebbf0, 0xc0002d5470}, {0x1d71a60?, 0xc0003b0320?}, 0xc0003b0340, 0xc00050ae10) /go/src/github.com/openshift/azure-disk-csi-driver/pkg/csi-common/utils.go:80 +0x409 github.com/container-storage-interface/spec/lib/go/csi._Node_NodeGetInfo_Handler({0x1ec2f40?, 0xc000144000}, {0x21ebbf0, 0xc0002d5470}, 0xc000054680, 0x20167a0) /go/src/github.com/openshift/azure-disk-csi-driver/vendor/github.com/container-storage-interface/spec/lib/go/csi/csi.pb.go:7162 +0x135 google.golang.org/grpc.(*Server).processUnaryRPC(0xc000530000, {0x21ebbf0, 0xc0002d53b0}, {0x21f5f40, 0xc00057b1e0}, 0xc00011cb40, 0xc00052c810, 0x30fa1c8, 0x0) /go/src/github.com/openshift/azure-disk-csi-driver/vendor/google.golang.org/grpc/server.go:1343 +0xe03 google.golang.org/grpc.(*Server).handleStream(0xc000530000, {0x21f5f40, 0xc00057b1e0}, 0xc00011cb40) /go/src/github.com/openshift/azure-disk-csi-driver/vendor/google.golang.org/grpc/server.go:1737 +0xc4c google.golang.org/grpc.(*Server).serveStreams.func1.1() /go/src/github.com/openshift/azure-disk-csi-driver/vendor/google.golang.org/grpc/server.go:986 +0x86 created by google.golang.org/grpc.(*Server).serveStreams.func1 in goroutine 260 /go/src/github.com/openshift/azure-disk-csi-driver/vendor/google.golang.org/grpc/server.go:997 +0x145
The registrar container log: E0321 23:08:02.679727 1 main.go:103] Registration process failed with error: RegisterPlugin error -- plugin registration failed with err: rpc error: code = Unavailable desc = error reading from server: EOF, restarting registration container.
Version-Release number of selected component (if applicable):
4.16.0-0.nightly-2024-03-21-152650
How reproducible:
See it in CI profile, and manual install failed earlier.
Steps to Reproduce:
See Description
Actual results:
Azure-Disk CSI Driver node pod CrashLoopBackOff
Expected results:
Azure-Disk CSI Driver node pod should be running
Additional info:
See gather-extra and must-gather: https://gcsweb-qe-private-deck-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/qe-private-deck/logs/periodic-ci-openshift-openshift-tests-private-release-4.16-amd64-nightly-azure-stack-ipi-proxy-fips-f2/1770921405509013504/artifacts/azure-stack-ipi-proxy-fips-f2/
- links to
-
RHEA-2024:0041 OpenShift Container Platform 4.16.z bug fix update