Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-16395

UPI Installation Failure: cluster operator control-plane-machine-set is not available

XMLWordPrintable

    • +
    • Critical
    • No
    • Rejected
    • False
    • Hide

      None

      Show
      None
    • N/A
    • Release Note Not Required
    • Done

      Description of problem:

      OCP 4.14 installation fails.
      
      Waiting for the UPI installation to complete using the wait-for, ends with a CO error:
      ```
      $ openshift-install wait-for install-complete --log-level=debug
      
      level=error msg=failed to initialize the cluster: Cluster operator control-plane-machine-set is not available
      ```
      
      ```
      $ oc get clusterversion
      NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
      version             False       True          122m    Unable to apply 4.14.0-0.nightly-2023-07-18-085740: the cluster operator control-plane-machine-set is not available
      ```
      
      ```
      $ oc get co | grep control-plane-machine-set
      control-plane-machine-set                  4.14.0-0.nightly-2023-07-18-085740   False       False         True       6h47m   Missing 3 available replica(s)
      ```

      Version-Release number of selected component (if applicable):

      Openshift on Openstack
      OCP 4.14.0-0.nightly-2023-07-18-085740
      RHOS-16.2-RHEL-8-20230413.n.1
      UPI installation

      How reproducible:

      Always

      Steps to Reproduce:

      Run the UPI openshift installation  

      Actual results:

      UPI installation fail

      Expected results:

      UPI installation pass

      Additional info:

      • Last UPI successful installation in D/S CI used: 4.14.0-0.nightly-2023-07-05-191022 
      • control-plane-machine-set-operator log:
      $ oc logs -n openshift-machine-api control-plane-machine-set-operator-5cbb7f68cc-h5f4p | tail
      E0719 14:20:52.645504       1 controller.go:649]  "msg"="Observed unmanaged control plane nodes" "error"="found unmanaged control plane nodes, the following node(s) do not have associated machines: ostest-c2drn-master-0, ostest-c2drn-master-1, ostest-c2drn-master-2" "controller"="controlplanemachineset" "name"="cluster" "namespace"="openshift-machine-api" "reconcileID"="1984ddf9-506f-4d10-88e5-0787b305484e" "unmanagedNodes"="ostest-c2drn-master-0,ostest-c2drn-master-1,ostest-c2drn-master-2"
      I0719 14:20:52.645530       1 controller.go:268]  "msg"="Cluster state is degraded. The control plane machine set will not take any action until issues have been resolved." "controller"="controlplanemachineset" "name"="cluster" "namespace"="openshift-machine-api" "reconcileID"="1984ddf9-506f-4d10-88e5-0787b305484e"
      I0719 14:20:52.667462       1 controller.go:212]  "msg"="Finished reconciling control plane machine set" "controller"="controlplanemachineset" "name"="cluster" "namespace"="openshift-machine-api" "reconcileID"="1984ddf9-506f-4d10-88e5-0787b305484e"
      I0719 14:20:52.668013       1 controller.go:156]  "msg"="Reconciling control plane machine set" "controller"="controlplanemachineset" "name"="cluster" "namespace"="openshift-machine-api" "reconcileID"="3f095b75-21af-4475-b0fd-25052e8c3bce"
      I0719 14:20:52.668718       1 controller.go:121]  "msg"="Reconciling control plane machine set" "controller"="controlplanemachinesetgenerator" "name"="cluster" "namespace"="openshift-machine-api" "reconcileID"="e80d898c-9a8d-4774-8f22-fb464be45758"
      I0719 14:20:52.668780       1 controller.go:142]  "msg"="Finished reconciling control plane machine set" "controller"="controlplanemachinesetgenerator" "name"="cluster" "namespace"="openshift-machine-api" "reconcileID"="e80d898c-9a8d-4774-8f22-fb464be45758"
      I0719 14:20:52.669005       1 status.go:119]  "msg"="Observed Machine Configuration" "controller"="controlplanemachineset" "name"="cluster" "namespace"="openshift-machine-api" "observedGeneration"=1 "readyReplicas"=0 "reconcileID"="3f095b75-21af-4475-b0fd-25052e8c3bce" "replicas"=0 "unavailableReplicas"=3 "updatedReplicas"=0
      E0719 14:20:52.669237       1 controller.go:649]  "msg"="Observed unmanaged control plane nodes" "error"="found unmanaged control plane nodes, the following node(s) do not have associated machines: ostest-c2drn-master-0, ostest-c2drn-master-1, ostest-c2drn-master-2" "controller"="controlplanemachineset" "name"="cluster" "namespace"="openshift-machine-api" "reconcileID"="3f095b75-21af-4475-b0fd-25052e8c3bce" "unmanagedNodes"="ostest-c2drn-master-0,ostest-c2drn-master-1,ostest-c2drn-master-2"
      I0719 14:20:52.669267       1 controller.go:268]  "msg"="Cluster state is degraded. The control plane machine set will not take any action until issues have been resolved." "controller"="controlplanemachineset" "name"="cluster" "namespace"="openshift-machine-api" "reconcileID"="3f095b75-21af-4475-b0fd-25052e8c3bce"
      I0719 14:20:52.669842       1 controller.go:212]  "msg"="Finished reconciling control plane machine set" "controller"="controlplanemachineset" "name"="cluster" "namespace"="openshift-machine-api" "reconcileID"="3f095b75-21af-4475-b0fd-25052e8c3bce"
      • The nodes are up:
      [cloud-user@installer-host ~]$ oc get nodes
      NAME                    STATUS   ROLES                  AGE     VERSION
      ostest-c2drn-master-0   Ready    control-plane,master   6h55m   v1.27.3+4aaeaec
      ostest-c2drn-master-1   Ready    control-plane,master   6h55m   v1.27.3+4aaeaec
      ostest-c2drn-master-2   Ready    control-plane,master   6h55m   v1.27.3+4aaeaec
      ostest-c2drn-worker-0   Ready    worker                 6h36m   v1.27.3+4aaeaec
      ostest-c2drn-worker-1   Ready    worker                 6h35m   v1.27.3+4aaeaec
      ostest-c2drn-worker-2   Ready    worker                 6h36m   v1.27.3+4aaeaec 

       

            emacchi@redhat.com Emilien Macchi
            rhn-support-imatza Itay Matza
            Itay Matza Itay Matza
            Votes:
            0 Vote for this issue
            Watchers:
            12 Start watching this issue

              Created:
              Updated:
              Resolved: