Uploaded image for project: 'Red Hat OpenStack Services on OpenShift'
  1. Red Hat OpenStack Services on OpenShift
  2. OSPRH-13670

[DB surgery] Orphaned entries in nova.services table block customer from using Nova service

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • None
    • openstack-nova
    • None
    • False
    • Hide

      None

      Show
      None
    • False
    • ?
    • None
    • Important

      To Reproduce Steps to reproduce the behavior:

      • back in 2022 domain names were changed in this deployment for unknown reason;
      • new services records were introduced with new domain names;
      • services records with old domain names became orphanes;
      • environment was updated to RHOSP 17.1 recently and nova control plane fails to start because of the following error (it is reported slightly differently, but exception is the same):
      nova.exception.TooOldComputeService: Current Nova version does not support computes older than Victoria but the minimum compute service level in your system is 40 and the oldest supported service level is 52.
      

      We tried to remove orphaned services using nova command, but API moved to irresponsive state and started throwing HTTP 503 error with same TooOldComputeService exception reported via /var/log/containers/httpd/nova-api/nova_api_wsgi_error.log

      As a result, it looks like only reasonable path forward is DB surgery and removing orphaned services.

      Expected behavior
      I believe that our observations is what we expect to see in these circumstances. But we still need a solution.

      Bug impact
      This problem prevents customer from moving environment back to production after FFU is completed.

      Known workaround
      None

      Additional context
      We need to quickly address situation itself using DB surgery, then we can wait for permanent solution if needed.

              rh-ee-bgibizer Balazs Gibizer
              rhn-support-astupnik Aleksei Stupnikov
              rhos-dfg-compute
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated: