-
Bug
-
Resolution: Unresolved
-
Normal
-
None
-
4.17.z
-
Quality / Stability / Reliability
-
False
-
-
None
-
Moderate
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Description of problem:
We are provisioning 4.17.3 OCP cluster on bare metal using Agent Based Installer. After installation, MCO is getting degraded with content mismatch in master nodes.
Here is the nodes information:
> $ oc get nodes -owide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
c2-esx02.racka05.local Ready control-plane,master,worker 5d5h v1.30.5 20.13.172.102 <none> Red Hat Enterprise Linux CoreOS 417.94.202410211619-0 5.14.0-427.42.1.el9_4.x86_64 cri-o://1.30.6-6.rhaos4.17.git6ac6e96.el9
c2-esx03.racka05.local Ready control-plane,master,worker 5d5h v1.30.5 20.13.172.103 <none> Red Hat Enterprise Linux CoreOS 417.94.202410211619-0 5.14.0-427.42.1.el9_4.x86_64 cri-o://1.30.6-6.rhaos4.17.git6ac6e96.el9
c2-esx04.racka05.local Ready,SchedulingDisabled control-plane,master,worker 5d5h v1.30.5 20.13.172.104 <none> Red Hat Enterprise Linux CoreOS 417.94.202410211619-0 5.14.0-427.42.1.el9_4.x86_64 cri-o://1.30.6-6.rhaos4.17.git6ac6e96.el9
For the node without errors(c2-esx02.racka05.local and c2-esx03.racka05.local):
currentConfig: rendered-master-2a545301fdfe36a460760fc8734f7aaf
desiredConfig: rendered-master-2a545301fdfe36a460760fc8734f7aaf
For the node with errors(c2-esx04.racka05.local):
currentConfig: rendered-master-d8f7478567e27665f3128a439d9c5b50
desiredConfig: rendered-master-2a545301fdfe36a460760fc8734f7aaf
> $ OLD_RENDERED=rendered-master-d8f7478567e27665f3128a439d9c5b50
> $ NEW_RENDERED=rendered-master-2a545301fdfe36a460760fc8734f7aaf
> $ diff -NrU 5 <(oc get mc ${OLD_RENDERED} -o json) <(oc get mc ${NEW_RENDERED} -o json)
--- /proc/self/fd/11 2024-11-27 01:57:06.609339330 +0000
+++ /proc/self/fd/18 2024-11-27 01:57:06.609339330 +0000
@@ -4,25 +4,25 @@
"metadata": {
"annotations": {
"machineconfiguration.openshift.io/generated-by-controller-version": "6308e62a4edabb857def36e2ffdaad1f92cc4734",
"machineconfiguration.openshift.io/release-image-version": "4.17.3"
},
- "creationTimestamp": "2024-11-21T19:58:07Z",
+ "creationTimestamp": "2024-11-21T19:58:13Z",
"generation": 1,
- "name": "rendered-master-d8f7478567e27665f3128a439d9c5b50",
+ "name": "rendered-master-2a545301fdfe36a460760fc8734f7aaf",
"ownerReferences": [
{
"apiVersion": "machineconfiguration.openshift.io/v1",
"blockOwnerDeletion": true,
"controller": true,
"kind": "MachineConfigPool",
"name": "master",
"uid": "27cd9183-3ee9-4014-8b35-967ebb7da815"
}
],
- "resourceVersion": "9330",
- "uid": "c7f2a699-644c-4237-80c1-e6da5a089b6e"
+ "resourceVersion": "9466",
+ "uid": "5b9a97f1-cc98-4f01-b769-06fcce327f75"
},
"spec": {
"baseOSExtensionsContainerImage": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:669a24aafb6b665fc36549e3beab648b663ab7db7dce308ad0e0ef7affe94130",
"config": {
"ignition": {
@@ -281,11 +281,12 @@
"overwrite": true,
"path": "/etc/kubernetes/manifests/keepalived.yaml"
},
{
"contents": {
- "source": "data:,NODE_SIZING_ENABLED%3Dfalse%0ASYSTEM_RESERVED_MEMORY%3D1Gi%0ASYSTEM_RESERVED_CPU%3D500m%0ASYSTEM_RESERVED_ES%3D1Gi"
+ "compression": "",
+ "source": "data:text/plain;charset=utf-8;base64,Tk9ERV9TSVpJTkdfRU5BQkxFRD10cnVlClNZU1RFTV9SRVNFUlZFRF9NRU1PUlk9MUdpClNZU1RFTV9SRVNFUlZFRF9DUFU9NTAwbQpTWVNURU1fUkVTRVJWRURfRVM9MUdpCg=="
},
"mode": 420,
"overwrite": true,
"path": "/etc/node-sizing-enabled.env"
},
The oc client binary and openshift-install binary. oc: v4.17.2 and openshift-install: v4.17.3.
Additional info: