-
Bug
-
Resolution: Not a Bug
-
Normal
-
None
-
4.13.0
-
No
-
Rejected
-
False
-
Description of problem:
With the machine config pools paused, specific changes on the controller config are not rendered. When the pools are unpause after that, the machine config pools are not updated.
How reproducible: always
Steps to Reproduce:
1. Pause machine config pools
> oc patch MachineConfigPool worker --type=merge --patch '{ "spec":{ "paused": true } }';oc patch MachineConfigPool master --type=merge --patch '{ "spec":{ "paused": true } }'
2. Apply a network configuration change to set MTU migration:
> oc patch Network.operator.openshift.io cluster --type=merge --patch '{"spec": { "migration": { "mtu": { "network": { "from": 1360, "to": 1260 } , "machine": { "to" : 1360} } } } }'
3. Observe the corresponding controller config change
> oc get controllerconfigs machine-config-controller -o yaml | grep -B1 -A5 mtuMigration
...
network:
mtuMigration:
machine:
to: 1360
network:
from: 1360
to: 1260
...
4. Observe the latest rendered config
> oc get mc rendered-master-749f5e1422740330b898f6c04aa973a4 -o yaml | grep Environment=TARGET_MTU Environment=TARGET_MTU=1360
5. Observe MCO status
machine-config 4.13.0-0.nightly-2023-02-12-024338 True False True 49m Failed to resync 4.13.0-0.nightly-2023-02-12-024338 because: Required MachineConfigPool 'master' is paused and can not sync until it is unpaused
6. Reverse the network configuration change, so mtu migration is unset
> oc patch Network.operator.openshift.io cluster --type=merge --patch '{"spec": { "migration": { "mtu": { "network": { "from": 1260, "to": 1360 } , "machine": { "to" : 1460} } } } }'
> oc patch Network.operator.openshift.io cluster --type=merge --patch '{"spec": { "migration": null } }'
7. Observe the corresponding controller config change
> oc get controllerconfigs machine-config-controller -o yaml | grep -B1 -A5 mtuMigration // no output, expected
8. Observe the latest rendered config
❯ oc get mc rendered-master-84a18f8d4a2ac800e0adfb27d62cc296 -o yaml | grep Environment=TARGET_MTU
Environment=TARGET_MTU=1460
This is unexpected, output should be empty.
The observed output corresponds as if only the first command of step 6 had been executed.
The second command of step 6 unsets mtu migration, and that is correctly reflected in controller config spec where the mtu migration is unset as well.
But the machine config is rendered as it had not.
9. Unpause the pools
> oc patch MachineConfigPool worker --type=merge --patch '{ "spec":{ "paused": false } }';oc patch MachineConfigPool master --type=merge --patch '{ "spec":{ "paused": false } }'
10. The pools are unexpectedly not updated
❯ oc get mc --sort-by=.metadata.creationTimestamp NAME GENERATEDBYCONTROLLER IGNITIONVERSION AGE 99-master-ssh 3.2.0 120m 99-worker-ssh 3.2.0 120m 00-worker 55684a0a23d6d110331e8279af5d96f2f8e3a443 3.2.0 113m 01-master-container-runtime 55684a0a23d6d110331e8279af5d96f2f8e3a443 3.2.0 113m 01-master-kubelet 55684a0a23d6d110331e8279af5d96f2f8e3a443 3.2.0 113m 01-worker-container-runtime 55684a0a23d6d110331e8279af5d96f2f8e3a443 3.2.0 113m 01-worker-kubelet 55684a0a23d6d110331e8279af5d96f2f8e3a443 3.2.0 113m 00-master 55684a0a23d6d110331e8279af5d96f2f8e3a443 3.2.0 113m 97-worker-generated-kubelet 55684a0a23d6d110331e8279af5d96f2f8e3a443 3.2.0 113m 99-worker-generated-registries 55684a0a23d6d110331e8279af5d96f2f8e3a443 3.2.0 113m 97-master-generated-kubelet 55684a0a23d6d110331e8279af5d96f2f8e3a443 3.2.0 113m 99-master-generated-registries 55684a0a23d6d110331e8279af5d96f2f8e3a443 3.2.0 113m rendered-master-6a3430dbb83b1465f57926de01478309 55684a0a23d6d110331e8279af5d96f2f8e3a443 3.2.0 113m rendered-worker-ef360c5158169c8b7f8b56d39fffa5c0 55684a0a23d6d110331e8279af5d96f2f8e3a443 3.2.0 113m rendered-master-fa8e3ed364bf74668458ef9a7505fb84 55684a0a23d6d110331e8279af5d96f2f8e3a443 3.2.0 102m rendered-worker-cb192b72b065e2a8585bfcc1d3e97acc 55684a0a23d6d110331e8279af5d96f2f8e3a443 3.2.0 102m rendered-master-749f5e1422740330b898f6c04aa973a4 55684a0a23d6d110331e8279af5d96f2f8e3a443 3.2.0 72m rendered-worker-46b2e9c26c8ee82034c1ca3f11978444 55684a0a23d6d110331e8279af5d96f2f8e3a443 3.2.0 72m rendered-master-4fc095a2826536f96c9b177e8b9c2016 55684a0a23d6d110331e8279af5d96f2f8e3a443 3.2.0 61m rendered-worker-81dda56ee4f26d937b22eae6ac30e1a5 55684a0a23d6d110331e8279af5d96f2f8e3a443 3.2.0 61m rendered-worker-d920fca6ead8dee94f26367e87d846cc 55684a0a23d6d110331e8279af5d96f2f8e3a443 3.2.0 57m rendered-master-84a18f8d4a2ac800e0adfb27d62cc296 55684a0a23d6d110331e8279af5d96f2f8e3a443 3.2.0 57m ❯ oc get mcp NAME CONFIG UPDATED UPDATING DEGRADED MACHINECOUNT READYMACHINECOUNT UPDATEDMACHINECOUNT DEGRADEDMACHINECOUNT AGE master rendered-master-fa8e3ed364bf74668458ef9a7505fb84 True False False 3 3 3 0 113m worker rendered-worker-cb192b72b065e2a8585bfcc1d3e97acc True False False 3 3 3 0 113m
- relates to
-
MCO-452 [tech-preview] Proper state reporting when the MCO changes state
-
- Closed
-