Uploaded image for project: 'Red Hat OpenStack Services on OpenShift'
  1. Red Hat OpenStack Services on OpenShift
  2. OSPRH-5315

When stopping infrastructure management and compute services virtlogd container is still up that causes further error in the EDPM adoption process

XMLWordPrintable

    • False
    • Hide

      None

      Show
      None
    • False
    • ?
    • ?
    • ?
    • ?
    • Important

      When stopping infrastructure management and compute services virtlogd container is still up that causes further error in the EDPM adoption process:

      After running https://github.com/openstack-k8s-operators/data-plane-adoption/blob/main/docs_user/modules/openstack-stop_remaining_services.adoc

      [root@overcloud-novacompute-0 ~]# podman ps
      CONTAINER ID  IMAGE                                                                                     COMMAND               CREATED       STATUS                 PORTS       NAMES
      c773b2ccaa7e  director.ctlplane.localdomain:8787/rhosp-rhel9/openstack-ovn-controller:17.1              kolla_start           46 hours ago  Up 16 hours (healthy)              ovn_controller
      52cb84409303  director.ctlplane.localdomain:8787/rhosp-rhel9/openstack-iscsid:17.1                      kolla_start           46 hours ago  Up 16 hours (healthy)              iscsid
      18f13ddfa08b  director.ctlplane.localdomain:8787/rhosp-rhel9/openstack-cron:17.1                        kolla_start           46 hours ago  Up 16 hours (healthy)              logrotate_crond
      2c2fe930d200  director.ctlplane.localdomain:8787/rhosp-rhel9/openstack-neutron-metadata-agent-ovn:17.1  kolla_start           46 hours ago  Up 16 hours (healthy)              ovn_metadata_agent
      c0e14f723e7f  director.ctlplane.localdomain:8787/rhosp-rhel9/openstack-nova-libvirt:17.1                /usr/sbin/virtlog...  16 hours ago  Up 16 hours                        nova_virtlogd 

      nova_virtlogd is remnant and causes errors later in the EDPM adoption process:

      [cloud-user@adoptionhelper pntae_lab_dp3]$  oc exec openstack-cell1-galera-0 -c galera -- mysql -rs -uroot -p$PODIFIED_DB_ROOT_PASSWORD \
            -e "select a.version from nova_cell1.services a join nova_cell1.services b where a.version!=b.version and a.binary='nova-compute';"
      56

       

      [root@overcloud-novacompute-0 ~]# systemctl status tripleo_nova_virtlogd_wrapper.service
      × tripleo_nova_virtlogd_wrapper.service - nova_virtlogd_wrapper container
           Loaded: loaded (/etc/systemd/system/tripleo_nova_virtlogd_wrapper.service; disabled; preset: disabled)
           Active: failed (Result: timeout) since Wed 2024-02-28 08:04:40 UTC; 10min ago
         Duration: 16h 14min 5.695s
         Main PID: 2483 (code=killed, signal=KILL)
              CPU: 230ms
      Feb 28 08:03:16 overcloud-novacompute-0 podman[229666]: 2024-02-28 08:03:16.552137953 +0000 UTC m=+0.101839997 container cleanup 89a97d572b1319554de223e32a785704c7cc6f055d74189efcae320e3d01a4a6 (image=director.ctlplane.localdomain:8787/rhosp-rhel9/openstack-nova-libvirt:17.1, name=nova_virtlogd_wrapper, a>
      Feb 28 08:03:16 overcloud-novacompute-0 podman[229666]: nova_virtlogd_wrapper
      Feb 28 08:04:40 overcloud-novacompute-0 systemd[1]: tripleo_nova_virtlogd_wrapper.service: State 'stop-sigterm' timed out. Killing.
      Feb 28 08:04:40 overcloud-novacompute-0 systemd[1]: tripleo_nova_virtlogd_wrapper.service: Killing process 2483 (conmon) with signal SIGKILL.
      Feb 28 08:04:40 overcloud-novacompute-0 systemd[1]: tripleo_nova_virtlogd_wrapper.service: Main process exited, code=killed, status=9/KILL
      Feb 28 08:04:40 overcloud-novacompute-0 podman[229921]: error opening file `/run/crun/89a97d572b1319554de223e32a785704c7cc6f055d74189efcae320e3d01a4a6/status`: No such file or directory
      Feb 28 08:04:40 overcloud-novacompute-0 podman[229913]: 2024-02-28 08:04:40.839132649 +0000 UTC m=+0.036947421 container cleanup 89a97d572b1319554de223e32a785704c7cc6f055d74189efcae320e3d01a4a6 (image=director.ctlplane.localdomain:8787/rhosp-rhel9/openstack-nova-libvirt:17.1, name=nova_virtlogd_wrapper, v>
      Feb 28 08:04:40 overcloud-novacompute-0 podman[229913]: nova_virtlogd_wrapper
      Feb 28 08:04:40 overcloud-novacompute-0 systemd[1]: tripleo_nova_virtlogd_wrapper.service: Failed with result 'timeout'.
      Feb 28 08:04:40 overcloud-novacompute-0 systemd[1]: Stopped nova_virtlogd_wrapper container.
      

      Docs says I should get empty instead of 56.
      I moved to the next section. When doing "Remove pre-FFU workarounds for Nova control plane services", I'm getting failures in the nova control plane:

      │ 2024-02-27 11:32:44.131 1 ERROR nova nova.exception.TooOldComputeService: Current Nova version does not support computes older than Yoga but the minimum compute service level in your system is 56 and the oldest supported service level is 61.   
      More logs here: https://privatebin.corp.redhat.com/?1a9afa93ffe62af2#Gs8TYN27KbLPftmB8c57SB7c6AxnccJ9Cn4VzfQhDfki

       

            ksambor@redhat.com Kamil Sambor
            pnavarro@redhat.com Pedro Navarro Perez
            rhos-dfg-compute
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: