Description of problem:
vmware-vsphere-csi-driver-operator goes in panic mode when the vcenter address is incorrect
Version-Release number of selected component (if applicable):
Vsphere IPI 4.17
How reproducible:
100%
$ oc logs vmware-vsphere-csi-driver-operator-74f65d4444-ljt78 -n openshift-cluster-csi-drivers
E1007 09:22:39.958324 1 config_yaml.go:208] Unmarshal failed: yaml: unmarshal errors:
line 1: cannot unmarshal !!seq into config.CommonConfigYAML
I1007 09:22:39.958515 1 config.go:272] ReadConfig INI succeeded. INI-based cloud-config is deprecated and will be removed in 2.0. Please use YAML based cloud-config.
I1007 09:22:39.959600 1 config.go:283] Config initialized
W1007 09:22:39.959738 1 vspherecontroller.go:910] vCenter vcenter1.vmware.gsslab.pnq2.redhat.com is missing from vCenter map
E1007 09:22:39.959815 1 runtime.go:79] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference)
goroutine 731 [running]:
k8s.io/apimachinery/pkg/util/runtime.logPanic({0x2cf0f80, 0x54f9210})
k8s.io/apimachinery@v0.30.3/pkg/util/runtime/runtime.go:75 +0x85
k8s.io/apimachinery/pkg/util/runtime.HandleCrash({0xc000b0f4e8, 0x1, 0xc001adc000?})
k8s.io/apimachinery@v0.30.3/pkg/util/runtime/runtime.go:49 +0x6b
panic({0x2cf0f80?, 0x54f9210?})
runtime/panic.go:770 +0x132
Actual results:
The storage CO goes in panic: $ oc get co storage NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE storage 4.17.0 True False True 6d21h VSphereCSIDriverOperatorCRDegraded: VMwareVSphereControllerDegraded: panic caught:...
Expected results:
The vmware-vsphere-csi-driver-operator should not go into panic even if the configuration is missing or incorrect.
Additional info:
- blocks
-
OCPBUGS-50638 vmware-vsphere-csi-driver-operator goes in panic mode
-
- Closed
-
- is cloned by
-
-
- Closed
-
- links to
-
RHEA-2024:11038
OpenShift Container Platform 4.19.z bug fix update
[OCPBUGS-43273] vmware-vsphere-csi-driver-operator goes in panic mode
Jan Safranek
added a comment - The full panic:
2024-10-07T09:22:39.959838099Z panic({0x2cf0f80?, 0x54f9210?})
2024-10-07T09:22:39.959838099Z runtime/panic.go:770 +0x132
2024-10-07T09:22:39.959838099Z github.com/openshift/vmware-vsphere-csi-driver-operator/pkg/operator/vclib.(*VSphereConfig).GetVCenterHostname(...)
2024-10-07T09:22:39.959838099Z github.com/openshift/vmware-vsphere-csi-driver-operator/pkg/operator/vclib/connection.go:75
2024-10-07T09:22:39.959838099Z github.com/openshift/vmware-vsphere-csi-driver-operator/pkg/operator/vclib.NewVSphereConnection(...)
2024-10-07T09:22:39.959838099Z github.com/openshift/vmware-vsphere-csi-driver-operator/pkg/operator/vclib/connection.go:119
2024-10-07T09:22:39.959838099Z github.com/openshift/vmware-vsphere-csi-driver-operator/pkg/operator/vspherecontroller.(*VSphereController).createVCenterConnection(0xc000b9cc88, {0xc000edbb90?, 0xc000b0f7c0?}, 0xc0001c2fc8)
2024-10-07T09:22:39.959838099Z github.com/openshift/vmware-vsphere-csi-driver-operator/pkg/operator/vspherecontroller/vspherecontroller.go:492 +0x2f0
2024-10-07T09:22:39.959838099Z github.com/openshift/vmware-vsphere-csi-driver-operator/pkg/operator/vspherecontroller.(*VSphereController).loginToVCenter(0xc000b9cc88, {0x38fe6d0, 0xc0001382d0}, 0x33756d6?)
2024-10-07T09:22:39.959838099Z github.com/openshift/vmware-vsphere-csi-driver-operator/pkg/operator/vspherecontroller/vspherecontroller.go:431 +0x5e
2024-10-07T09:22:39.959838099Z github.com/openshift/vmware-vsphere-csi-driver-operator/pkg/operator/vspherecontroller.(*VSphereController).sync(0xc000b9cc88, {0x38fe6d0, 0xc0001382d0}, {0x38ebea0, 0xc000c9a8d0})
2024-10-07T09:22:39.959838099Z github.com/openshift/vmware-vsphere-csi-driver-operator/pkg/operator/vspherecontroller/vspherecontroller.go:225 +0x7d0
2024-10-07T09:22:39.959838099Z github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile(0xc00083d8c0, {0x38fe6d0, 0xc0001382d0}, {0x38ebea0?, 0xc000c9a8d0?})
2024-10-07T09:22:39.959838099Z github.com/openshift/library-go@v0.0.0-20240715191351-e0aa70d55678/pkg/controller/factory/base_controller.go:201 +0x43
2024-10-07T09:22:39.959838099Z github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem(0xc00083d8c0, {0x38fe6d0, 0xc0001382d0})
2024-10-07T09:22:39.959838099Z github.com/openshift/library-go@v0.0.0-20240715191351-e0aa70d55678/pkg/controller/factory/base_controller.go:260 +0x1ae
2024-10-07T09:22:39.959838099Z github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1({0x38fe6d0, 0xc0001382d0})
2024-10-07T09:22:39.959838099Z github.com/openshift/library-go@v0.0.0-20240715191351-e0aa70d55678/pkg/controller/factory/base_controller.go:192 +0x89
2024-10-07T09:22:39.959838099Z k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1()
Jan Safranek
added a comment - The full panic:
2024-10-07T09:22:39.959838099Z panic({0x2cf0f80?, 0x54f9210?})
2024-10-07T09:22:39.959838099Z runtime/panic.go:770 +0x132
2024-10-07T09:22:39.959838099Z github.com/openshift/vmware-vsphere-csi-driver- operator /pkg/ operator /vclib.(*VSphereConfig).GetVCenterHostname(...)
2024-10-07T09:22:39.959838099Z github.com/openshift/vmware-vsphere-csi-driver- operator /pkg/ operator /vclib/connection.go:75
2024-10-07T09:22:39.959838099Z github.com/openshift/vmware-vsphere-csi-driver- operator /pkg/ operator /vclib.NewVSphereConnection(...)
2024-10-07T09:22:39.959838099Z github.com/openshift/vmware-vsphere-csi-driver- operator /pkg/ operator /vclib/connection.go:119
2024-10-07T09:22:39.959838099Z github.com/openshift/vmware-vsphere-csi-driver- operator /pkg/ operator /vspherecontroller.(*VSphereController).createVCenterConnection(0xc000b9cc88, {0xc000edbb90?, 0xc000b0f7c0?}, 0xc0001c2fc8)
2024-10-07T09:22:39.959838099Z github.com/openshift/vmware-vsphere-csi-driver- operator /pkg/ operator /vspherecontroller/vspherecontroller.go:492 +0x2f0
2024-10-07T09:22:39.959838099Z github.com/openshift/vmware-vsphere-csi-driver- operator /pkg/ operator /vspherecontroller.(*VSphereController).loginToVCenter(0xc000b9cc88, {0x38fe6d0, 0xc0001382d0}, 0x33756d6?)
2024-10-07T09:22:39.959838099Z github.com/openshift/vmware-vsphere-csi-driver- operator /pkg/ operator /vspherecontroller/vspherecontroller.go:431 +0x5e
2024-10-07T09:22:39.959838099Z github.com/openshift/vmware-vsphere-csi-driver- operator /pkg/ operator /vspherecontroller.(*VSphereController).sync(0xc000b9cc88, {0x38fe6d0, 0xc0001382d0}, {0x38ebea0, 0xc000c9a8d0})
2024-10-07T09:22:39.959838099Z github.com/openshift/vmware-vsphere-csi-driver- operator /pkg/ operator /vspherecontroller/vspherecontroller.go:225 +0x7d0
2024-10-07T09:22:39.959838099Z github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile(0xc00083d8c0, {0x38fe6d0, 0xc0001382d0}, {0x38ebea0?, 0xc000c9a8d0?})
2024-10-07T09:22:39.959838099Z github.com/openshift/library-go@v0.0.0-20240715191351-e0aa70d55678/pkg/controller/factory/base_controller.go:201 +0x43
2024-10-07T09:22:39.959838099Z github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem(0xc00083d8c0, {0x38fe6d0, 0xc0001382d0})
2024-10-07T09:22:39.959838099Z github.com/openshift/library-go@v0.0.0-20240715191351-e0aa70d55678/pkg/controller/factory/base_controller.go:260 +0x1ae
2024-10-07T09:22:39.959838099Z github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1({0x38fe6d0, 0xc0001382d0})
2024-10-07T09:22:39.959838099Z github.com/openshift/library-go@v0.0.0-20240715191351-e0aa70d55678/pkg/controller/factory/base_controller.go:192 +0x89
2024-10-07T09:22:39.959838099Z k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1()
PR is pre-merge verified, waiting for an accepted nightly to update the status.