Uploaded image for project: 'RHEL Documentation'
  1. RHEL Documentation
  2. RHELDOCS-20468

[DOC task] RSCT config wiped after update from RHEL 8.6 to 8.8

XMLWordPrintable

    • None
    • rhel-sst-ccs
    • ssg_virtualization
    • 3
    • False
    • False
    • Hide

      None

      Show
      None
    • None
    • RHELOPC Sprint 43, RHELOPC Sprint 44, RHELOPC Sprint 45, RHELOPC Sprint 46
    • None
    • Hide
      .Upgrading from RHEL 8.6 or prior to RHEL 8.7 or later might cause issues with instance functionality

      In RHEL 8.7 and later versions, `cloud-init` no longer uses a customized downstream-only configuration and instead uses a standard upstream configuration, located in the `/etc/cloud/cloud.cfg` file. Due to this, certain modules, such as `reset_rmc`, have been added to the configuration and now trigger in their respective `cloud-init` stages.

      As a consequence, after upgrading a RHEL instance to RHEL 8.7 or later, the `reset_rmc` module might run on nodes that were configured to run Reliable Scalable Cluster Technology (RSCT). This might in turn cause various functionality issues, such as the instance not being able to reconnect to its cluster.

      To work around the issue, manually create the lock file before upgrade, so that `reset_rmc` does not run after the update:

      ----
      $ touch /var/lib/cloud/instances/<INSTANCE_ID>/sem/config_reset_rmc
      ----

      This prevents the issue from occurring after the upgrade.

      Alternatively, remove the following modules from the `cloud_final_modules` section of the `/etc/cloud/cloud.cfg` file after every `cloud-init` upgrade, and then reboot the node:

      * reset_rmc
      * refresh_rmc_and_interface
      Show
      .Upgrading from RHEL 8.6 or prior to RHEL 8.7 or later might cause issues with instance functionality In RHEL 8.7 and later versions, `cloud-init` no longer uses a customized downstream-only configuration and instead uses a standard upstream configuration, located in the `/etc/cloud/cloud.cfg` file. Due to this, certain modules, such as `reset_rmc`, have been added to the configuration and now trigger in their respective `cloud-init` stages. As a consequence, after upgrading a RHEL instance to RHEL 8.7 or later, the `reset_rmc` module might run on nodes that were configured to run Reliable Scalable Cluster Technology (RSCT). This might in turn cause various functionality issues, such as the instance not being able to reconnect to its cluster. To work around the issue, manually create the lock file before upgrade, so that `reset_rmc` does not run after the update: ---- $ touch /var/lib/cloud/instances/<INSTANCE_ID>/sem/config_reset_rmc ---- This prevents the issue from occurring after the upgrade. Alternatively, remove the following modules from the `cloud_final_modules` section of the `/etc/cloud/cloud.cfg` file after every `cloud-init` upgrade, and then reboot the node: * reset_rmc * refresh_rmc_and_interface
    • Unspecified
    • Unspecified
    • Unspecified

      On an AWS instance, after updating from RHEL 8.6 to 8.8, the config in /var/ct/cfg is wiped since recfgct is run after the host is rebooted. This prevents the node from rejoining the peer domain and puts it in an unhealthy state.

      The file "cloudinit/config/:.py" seems to responsible for this behaviour, from the "cloud-init-22.1-8.el8_8.1.noarch" package.

      {{{{2025-03-28 14:50:25,858 - cc_reset_rmc.py[DEBUG]: Attempting to reset RMC.
      2025-03-28 14:50:25,858 - util.py[DEBUG]: Reading from /etc/ct_node_id (quiet=False)
      2025-03-28 14:50:25,858 - util.py[DEBUG]: Read 129 bytes from /etc/ct_node_id
      2025-03-28 14:50:25,858 - cc_reset_rmc.py[DEBUG]: Node ID at beginning of module: 6f26e400d297e97f
      2025-03-28 14:50:25,858 - subp.py[DEBUG]: Running command ['rmcctrl', '-z'] with allowed return codes [0] (shell=False, capture=True)
      2025-03-28 14:50:32,139 - subp.py[DEBUG]: Running command ['recfgct'] with allowed return codes [0] (shell=False, capture=True)
      2025-03-28 14:50:36,268 - cc_reset_rmc.py[DEBUG]: 0513-071 The ctcas Subsystem has been added.
      0513-071 The ctrmc Subsystem has been added.
      Enabling ctrmc.service for systemctl ...
      0513-059 The ctrmc Subsystem has been started. Subsystem PID is 8453.
      2025-03-28 14:50:36,268 - util.py[DEBUG]: Reading from /etc/ct_node_id (quiet=False)
      2025-03-28 14:50:36,268 - util.py[DEBUG]: Read 129 bytes from /etc/ct_node_id
      2025-03-28 14:50:36,268 - cc_reset_rmc.py[DEBUG]: Node ID at end of module: 9432a23ebe67a103}}}}

              jherrman@redhat.com Jiří Herrmann
              rh-ee-anisinha Ani Sinha
              Ryan Woodard (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

                Created:
                Updated:
                Resolved: