-
Bug
-
Resolution: Unresolved
-
Major
-
None
-
4.19
-
Quality / Stability / Reliability
-
False
-
-
None
-
Important
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Description of problem:
Nodepool upgrade to 4.19.5 fails
Version-Release number of selected component (if applicable):
upgrading to image : quay.io/openshift-release-dev/ocp-release:4.19.5-x86_64 [kni@ocp-edge64 ~]$ oc version Client Version: 4.18.0-ec.4 Kustomize Version: v5.4.2 Server Version: 4.19.5 Kubernetes Version: v1.32.6 [kni@ocp-edge64 ~]$ ~/hypershift_working/hypershift/bin/hcp version Client Version: openshift/hypershift: ab02c94ed951d38fbc454e580e838705b52fc393. Latest supported OCP: 4.18.0 Server Version: da32b47345c6a1ce4fa2981390292bdbe79334e8 Server Supports OCP Versions: 4.19, 4.18, 4.17, 4.16, 4.15, 4.14 [kni@ocp-edge64 ~]$ oc get hc -A NAMESPACE NAME VERSION KUBECONFIG PROGRESS AVAILABLE PROGRESSING MESSAGE clusters hosted-0 4.19.5 hosted-0-admin-kubeconfig Completed True False The hosted control plane is available [kni@ocp-edge64 ~]$ oc get nodepool -A NAMESPACE NAME CLUSTER DESIRED NODES CURRENT NODES AUTOSCALING AUTOREPAIR VERSION UPDATINGVERSION UPDATINGCONFIG MESSAGE clusters hosted-0 hosted-0 2 1 False False 4.18.0 True True Scaling up MachineSet to 2 replicas (actual 1)
How reproducible:
100%
Steps to Reproduce:
1.deploy a hub and hosted clusters with 4.18
2.upgrade hub to 4.19.5
3.upgrade hosted to 4.19.5
4.upgrade nodepool to 4.19.5,
5.scale nodepool 2->1
6.scale nodepool 1->2 so the new nodes will come p with the new version
Actual results:
While hub and hosted cluster were upgraded succesfully to 4.19.5 - nodepool version remains 4.18 - nodepool fails upgrading to 2 nodes - all bmh but 1 are stuck in provisioning state.
[kni@ocp-edge64 ~]$ oc get bmh -A
NAMESPACE NAME STATE CONSUMER ONLINE ERROR AGE
hosted-0 hosted-worker-0-0-bmh provisioning true 47h
hosted-0 hosted-worker-0-1-bmh provisioning true 47h
hosted-0 hosted-worker-0-2-bmh provisioning true 47h
hosted-0 hosted-worker-0-3-bmh provisioning true 47h
hosted-0 hosted-worker-0-4-bmh provisioning true 47h
hosted-0 hosted-worker-0-5-bmh provisioned true 47h
openshift-machine-api openshift-master-0-0 provisioned ocp-edge-cluster-0-fqftf-master-0 true 2d1h
openshift-machine-api openshift-master-0-1 provisioned ocp-edge-cluster-0-fqftf-master-1 true 2d1h
openshift-machine-api openshift-master-0-2 provisioned ocp-edge-cluster-0-fqftf-master-2 true 2d1h
Expected results:
nodepool scaling to 2 nodes, the new one with the new version .
Additional info:
Must gather : https://drive.google.com/file/d/1h4v-gOV7gGfIIywm4uvTsN7ZpkPwI6Xc/view?usp=sharing