Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-41846

Add warning about etcd-to-ephemeral procedure rollback

XMLWordPrintable

    • Moderate
    • None
    • 1
    • OSDOCS Sprint 259
    • 1
    • False
    • Hide

      None

      Show
      None
    • Release Note Not Required
    • Done

      Description of problem:

      As part of the work on https://issues.redhat.com/browse/OCPBUGS-39190, we found that there is not a viable software solution to avoid customer to hit the bug. Therefore, we decided to add a warning message in the documentation.

      The warning message would say that the once the 98-var-lib-etcd.yaml is applied in the system, the user shouldn't remove it, as that would lead to broken etcd members.

      Therefore, the rollback procedure is to change the flavor in the CPMS object to one that does not include the ephemeral variable. 

      As an example:

      • With masters using below flavor:
       $ openstack server list -c Name -c Flavor  | grep master
      | ostest-8j6nm-master-6zp2t-2 | m4.xlarge   |
      | ostest-8j6nm-master-6fx27-1 | m4.xlarge   |
      | ostest-8j6nm-master-5wsq8-0 | m4.xlarge   |
      
      • Where below two flavors exists, with the only difference of the Ephemeral attribute:
      (shiftstack) [stack@undercloud-0 ~]$ openstack flavor list 
      +--------------------------------------+-------------+-------+------+-----------+-------+-----------+
      | ID                                   | Name        |   RAM | Disk | Ephemeral | VCPUs | Is Public |
      +--------------------------------------+-------------+-------+------+-----------+-------+-----------+
      | 12e1ef8d-c6d9-4499-bc61-a06dfa24a17f | m4.xlarge.2 | 16384 |   40 |         0 |     4 | True      |
      | ed39d66c-c240-4dec-b2ab-384e336f7a0d | m4.xlarge   | 16384 |   40 |        10 |     4 | True      |
      +--------------------------------------+-------------+-------+------+-----------+-------+-----------+   
      • The rollback consist on changing the flavor like below:
      $ oc patch ControlPlaneMachineSet/cluster -n openshift-machine-api --type json -p ' 
      [
          {
            "op": "add",
            "path": "/spec/template/machines_v1beta1_machine_openshift_io/spec/providerSpec/value/flavor", 
            "value": "m4.xlarge.2"
          }
        ]
      '

      As a consequence, the masters will be recreated without using any ephemeral disk for the etcd partition and the system-units loaded by 98-var-lib-etcd.yaml will not run in the systems because the condition of the ephemeral disk existance is not met.

       

              mbridges@redhat.com Max Bridges
              rlobillo Ramón Lobillo
              Ramón Lobillo Ramón Lobillo
              Matthew Booth
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: