Uploaded image for project: 'Red Hat OpenStack Services on OpenShift'
  1. Red Hat OpenStack Services on OpenShift
  2. OSPRH-13697

FFU upgrade from 16.2.6 to 17.1.4 and missing warning on wrong mysql-pacemaker-puppet.yaml

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Done
    • Icon: Undefined Undefined
    • None
    • rhos-17.1.4
    • None
    • False
    • Hide

      None

      Show
      None
    • False
    • Important

      After completing the FFU from 16.2 to 17.1.4, a minor update of the OpenStack environment was invoked to test its behavior.

      As an unexpected event, the following FATAL error was detected during the Controllers' minor update

      ~~~
      2025-02-03 12:15:38.493341 | 566fdea1-0004-8528-a4ec-000000000122 | FATAL | assert | control0001-cdm | error=

      { "assertion": "mysql_upgrade_needed.stdout != \"mysql already upgraded\"", "changed": false, "evaluated_to": false, "msg": "MYSQL upgrade run was skipped during FFWD. RHBZ#2269509 you need to run openstack overcloud upgrade run --limit mysql --playbook <stack>/upgrade_steps_playbook.yaml --stack <stack>. This will cause t emporary API outage!" }

      ~~~

      To bypass this issue and made the cluster available to the Customer, the following change s were applied to the ./deployment/database/mysql-pacemaker-puppet.yaml file before restarting the minor update:

      ~~~
      [(LAB) root@director0001 openstack-tripleo-heat-templates]# diff ./deployment/database/mysql-pacemaker-puppet.yaml ./deployment/database/mysql-pacemaker-puppet.yaml.bck
      434d433
      < - - /var/lib/config-data/puppet-generated/mysql/root:/root:ro
      444c443
      < -u root --net=host UPGRADE_ENV UPGRADE_VOLUMES "UPGRADE_IMAGE" /bin/bash -cx "$UPGRADE_SCRIPT"'

      > -u mysql --net=host UPGRADE_ENV UPGRADE_VOLUMES "UPGRADE_IMAGE" /bin/bash -cx "$UPGRADE_SCRIPT"'
      453c452
      < mysql_upgrade --check-if-upgrade-is-needed

      > mysql_upgrade --check-if-upgrade-is-needed -umysql
      462d460
      < failed_when: false
      ~~~

      The workaround adopted and described above was derived from: https://issues.redhat.com/browse/OSPRH-12674

      Could you evaluate to include a warning in the FFU procedure to prevent a similar situation (minor update not working) post upgrade of also other Customers ?

              Unassigned Unassigned
              rhn-support-rbruzzon Riccardo Bruzzone
              rhos-dfg-upgrades
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: