-
Bug
-
Resolution: Unresolved
-
Minor
-
None
-
4.19
-
None
-
Quality / Stability / Reliability
-
False
-
-
3
-
None
-
None
-
None
-
None
-
None
-
WINC - Sprint 279
-
1
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Description of problem:
Version Annotation Editing Results in Unusable Node
Version-Release number of selected component (if applicable):
10.19.0-c6693cf (4.19.0-0.nightly-2025-07-15-022514)
How reproducible:
100%
Steps to Reproduce:
1.Overwrite the node annotation: oc annotate node windows-fc72n windowsmachineconfig.openshift.io/version=invalidVersion --overwrite 2.Check node recovery: oc get nodes -l kubernetes.io/os=windows 3.
Actual results:
Node windows-fc72n is marked as NotReady and SchedulingDisabled. The version annotation appears to be lost or invalid, preventing the node from recovering properly. Example output: NAME STATUS ROLES AGE VERSION windows-fc72n NotReady,SchedulingDisabled worker 10h v1.32.5 windows-hkhft Ready worker 10h v1.32.5
Expected results:
The version annotation is successfully re-added to the node. The node recovers to the Ready state and maintains its functionality.
Additional info:
This behavior may be related to the regression issue identified in OCPBUGS-35285.
wmco:
{"level":"info","ts":"2025-07-16T12:51:00Z","logger":"controller.windowsmachine","msg":"instance requires upgrade","node":"windows-fc72n","version":"invalidVersion","expected version":"10.19.0-c6693cf"} {"level":"info","ts":"2025-07-16T12:51:00Z","logger":"nc 10.0.128.7","msg":"deconfiguring"} {"level":"info","ts":"2025-07-16T12:51:00Z","logger":"nc 10.0.128.7","msg":"evicting pod winc-test/win-webserver-69d485bdf8-s68z9\n"} {"level":"info","ts":"2025-07-16T12:51:00Z","logger":"nc 10.0.128.7","msg":"evicting pod winc-test/win-webserver-69d485bdf8-h995w\n"} {"level":"info","ts":"2025-07-16T12:51:00Z","logger":"nc 10.0.128.7","msg":"evicting pod winc-test/win-webserver-69d485bdf8-clmhf\n"} {"level":"info","ts":"2025-07-16T12:51:01Z","logger":"wc 10.0.128.7","msg":"deconfigured","service":"windows-instance-config-daemon"} {"level":"error","ts":"2025-07-16T12:57:32Z","msg":"Reconciler error","controller":"controllerconfig","controllerGroup":"machineconfiguration.openshift.io","controllerKind":"ControllerConfig","ControllerConfig":{"name":"machine-config-controller"},"namespace":"","name":"machine-config-controller","reconcileID":"812d7184-ad8c-4b92-b42e-2327b7b50456","error":"error updating kubelet CA certificate in node byoh-ll2hj: error creating nodeConfig for instance 10.0.128.9: error instantiating Windows instance from VM: unable to setup VM 10.0.128.9 sshConnectivity: error instantiating SSH client: unable to connect to Windows VM 10.0.128.9: timed out waiting for the condition","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/build/windows-machine-config-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:347\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/build/windows-machine-config-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:294\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/build/windows-machine-config-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:255"} {"level":"error","ts":"2025-07-16T12:57:32Z","msg":"Reconciler error","controller":"node","controllerGroup":"","controllerKind":"Node","Node":{"name":"byoh-ll2hj"},"namespace":"","name":"byoh-ll2hj","reconcileID":"717201a4-8df3-4b95-8669-b38afaa8fcf7","error":"failed to create new nodeconfig: error instantiating Windows instance from VM: unable to setup VM 10.0.128.9 sshConnectivity: error instantiating SSH client: unable to connect to Windows VM 10.0.128.9: timed out waiting for the condition","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/build/windows-machine-config-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:347\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/build/windows-machine-config-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:294\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/build/windows-machine-config-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:255"} {"level":"info","ts":"2025-07-16T12:57:47Z","logger":"wc 10.0.128.7","msg":"rebooting instance"} {"level":"info","ts":"2025-07-16T12:57:48Z","logger":"controllers.controllerconfig","msg":"updating kubelet CA client certificates in","node":"windows-fc72n"} {"level":"error","ts":"2025-07-16T12:57:49Z","msg":"Reconciler error","controller":"controllerconfig","controllerGroup":"machineconfiguration.openshift.io","controllerKind":"ControllerConfig","ControllerConfig":{"name":"machine-config-controller"},"namespace":"","name":"machine-config-controller","reconcileID":"bad875ca-83f9-4345-9549-03f2e3de6ebf","error":"error updating kubelet CA certificate in node windows-fc72n: error checking if file 'C:\\k\\kubelet-ca.crt' exists on the Windows VM: error checking if file C:\\k\\kubelet-ca.crt exists: error running powershell.exe -NonInteractive -ExecutionPolicy Bypass \"Test-Path C:\\k\\kubelet-ca.crt\": wait: remote command exited without exit status or exit signal","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/build/windows-machine-config-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:347\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/build/windows-machine-config-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:294\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/build/windows-machine-config-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:255"} {"level":"error","ts":"2025-07-16T13:03:14Z","msg":"Reconciler error","controller":"machine","controllerGroup":"machine.openshift.io","controllerKind":"Machine","Machine":{"name":"windows-fc72n","namespace":"openshift-machine-api"},"namespace":"openshift-machine-api","name":"windows-fc72n","reconcileID":"f630dc55-2155-4b1e-be63-e2b5d719ef05","error":"unable to configure instance windows-fc72n: timeout waiting for windowsmachineconfig.openshift.io/reboot-required to be cleared: timed out waiting for the condition","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/build/windows-machine-config-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:347\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/build/windows-machine-config-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:294\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/build/windows-machine-config-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:255"} {"level":"info","ts":"2025-07-16T13:03:14Z","logger":"controller.windowsmachine","msg":"processing","windowsmachine":{"name":"windows-fc72n","namespace":"openshift-machine-api"},"address":"10.0.128.7"}