Uploaded image for project: 'Red Hat OpenStack Services on OpenShift'
  1. Red Hat OpenStack Services on OpenShift
  2. OSPRH-3157

Migration gets stuck at pre-migrating status if source compute node is down but maintenance enabled

XMLWordPrintable

    • False
    • False
    • Committed
    • openstack-nova-27.2.1-18.0.20240105185939.7583267.el9ost
    • Committed
    • Committed
    • None
    • Undefined
    • Moderate

      Description of problem:

      Currently nova rejects migration(resize) if the source compute node is down.

      ~~~
      (undercloud) [stack@undercloud-0 ~]$ openstack baremetal node list
      ------------------------------------------------------------------------------------------------------------------------------+

      UUID Name Instance UUID Power State Provisioning State Maintenance

      ------------------------------------------------------------------------------------------------------------------------------+

      76fb384e-761d-4811-b03d-2ab981b8daa6 compute-0 1c039eaf-b2bf-4e52-843e-0a0221006dbf power off active False
      281613b5-c2ca-4741-bc6f-e7501b2fd6d8 compute-1 58730849-de47-4263-88f3-7e61b705c55b power on active False
      fc5681e1-6ae6-46fe-9bd6-c0b8cb43021e controller-0 b698b399-6ecd-4536-a7ed-e66554eedc9d power on active False
      e8efc933-3ea2-472b-8321-c8de52308e8d controller-1 d834fd79-3865-44a0-8a48-d5a7a5e1c720 power on active False
      7bdcaf64-0993-4878-95d5-1addb21925b2 controller-2 ea830026-efe6-4a4c-a702-4dcd4d2d7892 power on active False

      ------------------------------------------------------------------------------------------------------------------------------+
      (overcloud) [stack@undercloud-0 ~]$ openstack server list --long
      ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

      ID Name Status Task State Power State Networks Image Name Image ID Flavor Name Flavor ID Availability Zone Host Properties

      ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
      ...

      d11a09f4-cb44-48ad-9375-8e1be7d77bb2 testinstance SHUTOFF None Shutdown private=192.168.10.131 cirros-0.4.0-x86_64-disk.img_alt 3fb188fd-9902-4c41-a12e-5306edd65922     nova compute-0.redhat.local  

      ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
      (overcloud) [stack@undercloud-0 ~]$ openstack compute service list
      ---------------------------------------------------------------------------------------------------------------------------

      ID Binary Host Zone Status State Updated At

      ---------------------------------------------------------------------------------------------------------------------------
      ...

      956ece2d-be24-4618-8e8f-a82f616fc31b nova-compute compute-0.redhat.local nova enabled down 2021-07-25T07:16:11.000000

      ---------------------------------------------------------------------------------------------------------------------------
      (overcloud) [stack@undercloud-0 ~]$ openstack server migrate testinstance
      Service is unavailable at this time. (HTTP 409) (Request-ID: req-d76e2ba5-57eb-4ae9-8ce3-58dd76b64896)
      ~~~

      However this validation is bypassed if nova-compute on the source compute node has maintenance enabled,
      and this results in migration stuck in pre-migrating status.

      ~~~
      (overcloud) [stack@undercloud-0 ~]$ openstack compute service set --disable compute-0.redhat.local nova-compute
      (overcloud) [stack@undercloud-0 ~]$ openstack compute service list
      ----------------------------------------------------------------------------------------------------------------------------

      ID Binary Host Zone Status State Updated At

      ----------------------------------------------------------------------------------------------------------------------------
      ...

      956ece2d-be24-4618-8e8f-a82f616fc31b nova-compute compute-0.redhat.local nova disabled down 2021-07-25T07:17:05.000000

      ----------------------------------------------------------------------------------------------------------------------------
      (overcloud) [stack@undercloud-0 ~]$ openstack server migrate testinstance
      (overcloud) [stack@undercloud-0 ~]$
      (overcloud) [stack@undercloud-0 ~]$ nova migration-list --instance-uuid d11a09f4-cb44-48ad-9375-8e1be7d77bb2
      -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

      Id UUID Source Node Dest Node Source Compute Dest Compute Dest Host Status Instance UUID Old Flavor New Flavor Created At Updated At Type

      -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

      15 a7678b2e-4a4f-4fcc-a438-e057cc8796d6 compute-0.redhat.local compute-1.redhat.local compute-0.redhat.local compute-1.redhat.local 172.17.1.45 pre-migrating d11a09f4-cb44-48ad-9375-8e1be7d77bb2 5 5 2021-07-25T07:17:23.000000 2021-07-25T07:17:27.000000 migration

      -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
      (overcloud) [stack@undercloud-0 ~]$ openstack server show testinstance
      --------------------------------------------------------------------------------------------------------------------------------------------------------+

      Field Value

      --------------------------------------------------------------------------------------------------------------------------------------------------------+

      OS-DCF:diskConfig MANUAL
      OS-EXT-AZ:availability_zone nova
      OS-EXT-SRV-ATTR:host compute-0.redhat.local
      OS-EXT-SRV-ATTR:hostname testinstance
      OS-EXT-SRV-ATTR:hypervisor_hostname compute-0.redhat.local
      OS-EXT-SRV-ATTR:instance_name instance-00000024
      OS-EXT-SRV-ATTR:kernel_id  
      OS-EXT-SRV-ATTR:launch_index 0
      OS-EXT-SRV-ATTR:ramdisk_id  
      OS-EXT-SRV-ATTR:reservation_id r-jth003rm
      OS-EXT-SRV-ATTR:root_device_name /dev/vda
      OS-EXT-SRV-ATTR:user_data None
      OS-EXT-STS:power_state Shutdown
      OS-EXT-STS:task_state resize_prep
      OS-EXT-STS:vm_state stopped
      OS-SRV-USG:launched_at 2021-07-14T07:31:30.000000
      OS-SRV-USG:terminated_at None
      accessIPv4  
      accessIPv6  
      addresses private=192.168.10.131
      config_drive  
      created 2021-07-14T07:31:22Z
      description None
      flavor disk='1', ephemeral='0', extra_specs.hw_rng:allowed='True', original_name='m1.nano', ram='128', swap='0', vcpus='1'
      hostId ede2468f5fe92029a8a42760bfe4a90f20f5c064e12ab911a8c80b22
      host_status MAINTENANCE
      id d11a09f4-cb44-48ad-9375-8e1be7d77bb2
      image cirros-0.4.0-x86_64-disk.img_alt (3fb188fd-9902-4c41-a12e-5306edd65922)
      key_name None
      locked False
      locked_reason None
      name testinstance
      progress 0
      project_id 942783ae248c4e9eb353a6e6b327bda5
      properties  
      security_groups name='default'
      server_groups []
      status RESIZE
      tags []
      trusted_image_certificates None
      updated 2021-07-25T07:17:27Z
      user_id 06b547a0af8f49fd8239c85ce5d9571b
      volumes_attached  

      --------------------------------------------------------------------------------------------------------------------------------------------------------+
      ~~~

      Version-Release number of selected component (if applicable):

      How reproducible:
      Always

      Steps to Reproduce:
      1. Create an instance
      2. Shutdown the compute node where the instance is started
      3. Enable maintenance of the nova-compute service on the source compute node
      4. Migrate the instance

      Actual results:
      Migration is accepted but gets stuck in pre-migrating status

      Expected results:
      Migration is rejected

      Additional info:

              Unassigned Unassigned
              jira-bugzilla-migration RH Bugzilla Integration
              rhos-dfg-compute
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: