-
Bug
-
Resolution: Done
-
Critical
-
4.12
-
None
-
Critical
-
No
-
3
-
WINC - Sprint 236, WINC - Sprint 237
-
2
-
False
-
Description of problem:
Running regression testing for WMCO 7.1.0 (windows-machine-config-operator-container-7.1.0-36 and windows-machine-config-operator-bundle-container-v7.1.0-35) it was found that none of the tests were passing in Azure with Windows Server 2019. When checking the WMCO status, it was in CrashLoopBackOff: $ oc get pods -n openshift-windows-machine-config-operator NAME READY STATUS RESTARTS AGE windows-machine-config-operator-f8dd9dd84-rc2kz 0/1 CrashLoopBackOff 10 (93s ago) 46m And the logs were displaying a Segmentation Violation log: Eventually, the container goes back into Running state but crashes again, ending in a never ending loop of restarts. {"level":"info","ts":"2023-05-18T12:36:28Z","msg":"Starting Controller","controller":"certificatesigningrequest","controllerGroup":"certificates.k8s.io","controllerKind":"CertificateSigningRequest"} {"level":"info","ts":"2023-05-18T12:36:28Z","msg":"Starting Controller","controller":"machine","controllerGroup":"machine.openshift.io","controllerKind":"Machine"} {"level":"info","ts":"2023-05-18T12:36:28Z","msg":"Starting EventSource","controller":"configmap","controllerGroup":"","controllerKind":"ConfigMap","source":"kind source: *v1.Node"} {"level":"info","ts":"2023-05-18T12:36:28Z","msg":"Starting EventSource","controller":"configmap","controllerGroup":"","controllerKind":"ConfigMap","source":"kind source: *v1.Node"} {"level":"info","ts":"2023-05-18T12:36:28Z","msg":"Starting Controller","controller":"configmap","controllerGroup":"","controllerKind":"ConfigMap"} {"level":"info","ts":"2023-05-18T12:36:28Z","msg":"Starting workers","controller":"certificatesigningrequest","controllerGroup":"certificates.k8s.io","controllerKind":"CertificateSigningRequest","worker count":1} {"level":"info","ts":"2023-05-18T12:36:28Z","msg":"Starting workers","controller":"configmap","controllerGroup":"","controllerKind":"ConfigMap","worker count":1} {"level":"info","ts":"2023-05-18T12:36:28Z","msg":"Starting workers","controller":"machine","controllerGroup":"machine.openshift.io","controllerKind":"Machine","worker count":1} {"level":"info","ts":"2023-05-18T12:36:28Z","msg":"Starting workers","controller":"secret","controllerGroup":"","controllerKind":"Secret","worker count":1} {"level":"info","ts":"2023-05-18T12:36:43Z","logger":"controller.windowsmachine","msg":"processing","windowsmachine":"openshift-machine-api/windows-n69vt","address":"10.0.128.7"} {"level":"info","ts":"2023-05-18T12:37:18Z","logger":"wc 10.0.128.7","msg":"configuring"} {"level":"info","ts":"2023-05-18T12:37:34Z","msg":"Observed a panic in reconciler: runtime error: invalid memory address or nil pointer dereference","controller":"machine","controllerGroup":"machine.openshift.io","controllerKind":"Machine","machine":{"name":"windows-n69vt","namespace":"openshift-machine-api"},"namespace":"openshift-machine-api","name":"windows-n69vt","reconcileID":"da85fc1b-e1c6-4221-8e46-843954ba1c8c"} panic: runtime error: invalid memory address or nil pointer dereference [recovered] panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x16b7d7a] goroutine 982 [running]: sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile.func1() /remote-source/build/windows-machine-config-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:118 +0x1f4 panic({0x1ad6200, 0x2f40420}) /usr/lib/golang/src/runtime/panic.go:884 +0x212 github.com/openshift/windows-machine-config-operator/pkg/windows.(*windows).ensureWICDSecretContent(0x2089910?, 0x0) /remote-source/build/windows-machine-config-operator/pkg/windows/windows.go:935 +0x5a github.com/openshift/windows-machine-config-operator/pkg/windows.(*windows).Bootstrap(0xc0025e4d80, {0x2066f28, 0xd}, {0xc000462190, 0x43}, {0xc00005c010, 0x29}, 0xc001ad6570?) /remote-source/build/windows-machine-config-operator/pkg/windows/windows.go:432 +0xdb github.com/openshift/windows-machine-config-operator/pkg/nodeconfig.(*nodeConfig).Configure(0xc000454780) /remote-source/build/windows-machine-config-operator/pkg/nodeconfig/nodeconfig.go:155 +0x350 github.com/openshift/windows-machine-config-operator/controllers.(*instanceReconciler).ensureInstanceIsUpToDate(0xc0005141b0, 0xc002622640, 0x1dc9485?, 0x2a?) /remote-source/build/windows-machine-config-operator/controllers/controllers.go:84 +0x5f7 github.com/openshift/windows-machine-config-operator/controllers.(*WindowsMachineReconciler).configureMachine(0xc0005141b0, {0xc0025b6c40?, 0x1d8b5f2?}, {0xc002530a8d, 0xd}, {0xc0025b6bc0?, 0x2?}, 0x2?) /remote-source/build/windows-machine-config-operator/controllers/windowsmachine_controller.go:432 +0x273 github.com/openshift/windows-machine-config-operator/controllers.(*WindowsMachineReconciler).Reconcile(0xc0005141b0, {0x2086718, 0xc00213fe60}, {{{0xc0023a0a50, 0x15}, {0xc0027bb050, 0xd}}}) /remote-source/build/windows-machine-config-operator/controllers/windowsmachine_controller.go:354 +0xea9 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x2086670?, {0x2086718?, 0xc00213fe60?}, {{{0xc0023a0a50?, 0x1c9d380?}, {0xc0027bb050?, 0x4045d4?}}}) /remote-source/build/windows-machine-config-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:121 +0xc8 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc0005a3d60, {0x2086670, 0xc000c811c0}, {0x1b75640?, 0xc0000a20e0?}) /remote-source/build/windows-machine-config-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:320 +0x33c sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc0005a3d60, {0x2086670, 0xc000c811c0}) /remote-source/build/windows-machine-config-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:273 +0x1d9 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2() /remote-source/build/windows-machine-config-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:234 +0x85 created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2 /remote-source/build/windows-machine-config-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:230 +0x333
Version-Release number of selected component (if applicable):
$ oc get cm -n openshift-windows-machine-config-operator NAME DATA AGE kube-root-ca.crt 1 52m openshift-service-ca.crt 1 52m windows-machine-config-operator-lock 0 52m windows-services-7.1.0-39ce397 2 52m [cloud-user@preserve-jfrancoa openshift-tests-private]$ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.12.0-0.nightly-2023-05-17-133811 True False 5h6m Cluster version is 4.12.0-0.nightly-2023-05-17-133811
How reproducible:
Not sure
Steps to Reproduce:
1. 2. 3.
Actual results:
WMCO crashes
Expected results:
WMCO runs correctly.
Additional info:
- links to
-
RHSA-2023:119726 Red Hat OpenShift support for Windows Containers 7.2.0 security update
- mentioned on