-
Bug
-
Resolution: Unresolved
-
Normal
-
4.19.0, 4.20.0
-
Quality / Stability / Reliability
-
False
-
-
5
-
Moderate
-
None
-
None
-
None
-
MCO Sprint 272, MCO Sprint 273
-
2
-
Done
-
Release Note Not Required
-
None
-
None
-
None
-
None
-
None
Description of problem:
The machine-os-builder pod is not releasing the leader lease when it is removed
Version-Release number of selected component (if applicable):
4.20.0-0.nightly-2025-06-02-135911
How reproducible:
Configure OCL and apply a MC, the machine-os-builder pod will be restarted and we will that it reports this error I0603 09:18:47.013288 1 leaderelection.go:257] attempting to acquire leader lease openshift-machine-config-operator/machine-os-builder... I0603 09:18:47.016444 1 leaderelection.go:456] lock is held by machine-os-builder-84bd7fcc4-85hkq_7e7b8645-7df0-4d47-a7ce-01ef0dda602d and has not yet expired I0603 09:18:47.016458 1 leaderelection.go:266] failed to acquire lease openshift-machine-config-operator/machine-os-builder I0603 09:19:16.576468 1 leaderelection.go:456] lock is held by machine-os-builder-84bd7fcc4-85hkq_7e7b8645-7df0-4d47-a7ce-01ef0dda602d and has not yet expired I0603 09:19:16.576485 1 leaderelection.go:266] failed to acquire lease openshift-machine-config-operator/machine-os-builder I0603 09:20:02.284367 1 leaderelection.go:456] lock is held by machine-os-builder-84bd7fcc4-85hkq_7e7b8645-7df0-4d47-a7ce-01ef0dda602d and has not yet expired I0603 09:20:02.284383 1 leaderelection.go:266] failed to acquire lease openshift-machine-config-operator/machine-os-builder I0603 09:20:32.500332 1 leaderelection.go:456] lock is held by machine-os-builder-84bd7fcc4-85hkq_7e7b8645-7df0-4d47-a7ce-01ef0dda602d and has not yet expired I0603 09:20:32.500345 1 leaderelection.go:266] failed to acquire lease openshift-machine-config-operator/machine-os-builder I0603 09:21:01.783799 1 leaderelection.go:456] lock is held by machine-os-builder-84bd7fcc4-85hkq_7e7b8645-7df0-4d47-a7ce-01ef0dda602d and has not yet expired I0603 09:21:01.783813 1 leaderelection.go:266] failed to acquire lease openshift-machine-config-operator/machine-os-builder I0603 09:21:45.008349 1 leaderelection.go:271] successfully acquired lease openshift-machine-config-operator/machine-os-builder It means that when the pod was removed it didnt release the lease. When the pod is removed, we can see these logs I0603 09:07:15.660983 1 cleaner.go:60] Cleaning up ephemeral objects from build "test-78001-8c09eda06d65b1ea60801f0cb1a0737b" using selector "machineconfiguration.openshift.io/ephemeral-build-object,machineconfiguration.openshift.io/machine-os-build=test-78001-8c09eda06d65b1ea60801f0cb1a0737b,machineconfiguration.openshift.io/on-cluster-layering,machineconfiguration.openshift.io/rendered-machine-config=rendered-worker-16d68ff4ee42b6aaffb0c9720260858a,machineconfiguration.openshift.io/target-machine-config-pool=worker" I0603 09:07:15.681872 1 helpers.go:111] Received shutdown signal: terminated. Delaying shutdown... I0603 09:07:15.725497 1 reconciler.go:976] Deleting image image-registry.openshift-image-registry.svc:5000/openshift-machine-config-operator/ocb-worker-image:test-78001-8c09eda06d65b1ea60801f0cb1a0737b from internal registry for MachineOSBuild test-78001-8c09eda06d65b1ea60801f0cb1a0737b I0603 09:07:15.744335 1 reconciler.go:1108] Finished deleting MachineOSBuild "test-78001-8c09eda06d65b1ea60801f0cb1a0737b" after 91.61394ms I0603 09:07:20.726855 1 reconciler.go:1105] Updating MachineConfigPool "worker" I0603 09:07:20.726872 1 reconciler.go:1125] Syncing MachineOSConfigs and MachineOSBuilds I0603 09:07:20.726876 1 reconciler.go:1125] Syncing MachineOSConfigs I0603 09:07:20.726881 1 reconciler.go:1122] Finished syncing MachineOSConfigs after 5.252µs I0603 09:07:20.726885 1 reconciler.go:1125] Syncing MachineOSBuilds I0603 09:07:20.726888 1 reconciler.go:1122] Finished syncing MachineOSBuilds after 3.152µs I0603 09:07:20.726891 1 reconciler.go:1122] Finished syncing MachineOSConfigs and MachineOSBuilds after 19.702µs I0603 09:07:20.726894 1 reconciler.go:1108] Finished updating MachineConfigPool "worker" after 306.344µs I0603 09:07:25.754874 1 reconciler.go:1105] Updating MachineConfigPool "worker" I0603 09:07:25.754891 1 reconciler.go:1125] Syncing MachineOSConfigs and MachineOSBuilds I0603 09:07:25.754895 1 reconciler.go:1125] Syncing MachineOSConfigs I0603 09:07:25.754903 1 reconciler.go:1122] Finished syncing MachineOSConfigs after 7.083µs I0603 09:07:25.754911 1 reconciler.go:1125] Syncing MachineOSBuilds I0603 09:07:25.754914 1 reconciler.go:1122] Finished syncing MachineOSBuilds after 6.233µs I0603 09:07:25.754917 1 reconciler.go:1122] Finished syncing MachineOSConfigs and MachineOSBuilds after 28.045µs I0603 09:07:25.754922 1 reconciler.go:1108] Finished updating MachineConfigPool "worker" after 327.808µs We see in the log that the lease is not released
Steps to Reproduce:
Always
Actual results:
The machine-os-builder pod does not release the lease
Expected results:
The machine-os-builder pod should release the lease when it is shut down
Additional info:
Slack conversation: https://redhat-internal.slack.com/archives/GH7G2MANS/p1748594906387099
- blocks
-
OCPBUGS-60080 In OCL. Machine-os-builder pod does not release the lease when it is removed
-
- Closed
-
- is cloned by
-
OCPBUGS-60080 In OCL. Machine-os-builder pod does not release the lease when it is removed
-
- Closed
-
- links to