Uploaded image for project: 'Red Hat OpenStack Services on OpenShift'
  1. Red Hat OpenStack Services on OpenShift
  2. OSPRH-9557

resizing from flavor with swap to one without swap puts instance into Error status

XMLWordPrintable

    • Compute Epoxy Phase 2 Cont'd
    • 1
    • Moderate

      What problem/issue/behavior are you having trouble with? What do you expect to see?

      • Create flavor with swap - FLAVOR_SWAP
      • Create flavor without swap - FLAVOR_NO_SWAP
      • Create instance using the "FLAVOR_SWAP" flavor.
      • Start a resize operation with the new flavor "FLAVOR_NO_SWAP"
      • Instance will be migrated to the new compute.
      • Resize operation will fail after the migration to the new compute (before the actual resize).
      • VM will be placed in an ERROR state and shutdown.

      Please see the following Nova Launchpad bug which seems to be same : https://bugs.launchpad.net/nova/+bug/1552777

      Here are the relevant logs :

      Output of "openstack server show":

      fault {u'message': u"Cannot access storage file '/var/lib/nova/instances/a3c1a8a3-7b54-4ebf-9660-935c87881c07/disk.swap' (as uid:107, gid:107): No such file or directory", u'code': 500, u'details': u' File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 202, in decorated_function\n return function(self, context, *args, **kwargs)\n File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4486, in finish_resize\n self._revert_allocation(context, instance, migration)\n File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4483, in finish_resize\n migration)\n File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4502, in _finish_resize_helper\n disk_info, image_meta, bdms)\n File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4453, in _finish_resize\n old_instance_type)\n File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4448, in _finish_resize\n block_device_info, power_on)\n File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 8299, in finish_migration\n post_xml_callback=gen_confdrive)\n File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5604, in _create_domain_and_network\n destroy_disks_on_failure)\n File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5573, in _create_domain_and_network\n post_xml_callback=post_xml_callback)\n File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5508, in _create_domain\n guest.launch(pause=pause)\n File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 144, in launch\n self._encoded_xml, errors=\'ignore\')\n File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 139, in launch\n return self._domain.createWithFlags(flags)\n File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit\n result = proxy_call(self._autowrap, f, *args, **kwargs)\n File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call\n rv = execute(f, *args, **kwargs)\n File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute\n six.reraise(c, e, tb)\n File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker\n rv = meth(*args, **kwargs)\n File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1110, in createWithFlags\n if ret == -1: raise libvirtError (\'virDomainCreateWithFlags() failed\', dom=self)\n', u'created': u'2019-11-19T19:20:49Z'}

      Looking on the new compute – I am seeing that the root disk was migrated but the swap disk wasn’t :

      [root@tenlab1-compute013 a3c1a8a3-7b54-4ebf-9660-935c87881c07]# ls -alh
      total 848M
      drwxr-xr-x. 2 42436 root 73 Nov 19 19:20 .
      drwxr-xr-x. 5 42436 42436 97 Nov 19 19:20 ..
      rw-rr-. 1 42436 42436 0 Nov 19 19:20 console.log
      rw-rr-. 1 42436 root 847M Nov 19 19:19 disk
      rw-rr-. 1 42436 root 512K Nov 19 19:19 disk.config
      rw-rr-. 1 42436 42436 245 Nov 19 19:20 disk.info

      The disk.info file also seems to be still listing the swap disk :

      [root@tenlab1-compute013 a3c1a8a3-7b54-4ebf-9660-935c87881c07]# cat disk.info

      {"/var/lib/nova/instances/a3c1a8a3-7b54-4ebf-9660-935c87881c07/disk.swap": "qcow2", "/var/lib/nova/instances/a3c1a8a3-7b54-4ebf-9660-935c87881c07/disk.config": "raw", "/var/lib/nova/instances/a3c1a8a3-7b54-4ebf-9660-935c87881c07/disk": "qcow2"}

      The instance itself is still showing the old flavor with the swap.
      (overcloud) [stack@tenlab1-director ~]$ openstack server show a3c1a8a3-7b54-4ebf-9660-935c87881c07 | grep flavor

      flavor SBC_PIM (68a1ea4b-e5e8-4b5f-93c8-db37290c0f46)

      ORIGINAL FLAVOR
      (overcloud) [stack@tenlab1-director ~]$ openstack flavor show 68a1ea4b-e5e8-4b5f-93c8-db37290c0f46
      ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

      Field Value

      ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

      OS-FLV-DISABLED:disabled False
      OS-FLV-EXT-DATA:ephemeral 0
      access_project_ids 1e152c0f15be40dcbd421fe4a5d8d0fc
      disk 21
      id 68a1ea4b-e5e8-4b5f-93c8-db37290c0f46
      name SBC_PIM
      os-flavor-access:is_public False
      properties aggregate_instance_extra_specs:vIMS_SRIOV='true', hw:cpu_policy='dedicated', hw:cpu_thread_policy='prefer', hw:mem_page_size='1GB', hw:numa_cpus.0='1-7', hw:numa_cpus.1='0', hw:numa_mem.0='8192', hw:numa_mem.1='8192', hw:numa_mempolicy='strict', hw:numa_nodes='2'
      ram 16384
      rxtx_factor 1.0
      swap 1024
      vcpus 8

      ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

      NEW FLAVOR :
      (overcloud) [stack@tenlab1-director ~]$ openstack flavor show SBC_PIM_TEST
      ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

      Field Value

      ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

      OS-FLV-DISABLED:disabled False
      OS-FLV-EXT-DATA:ephemeral 0
      access_project_ids 1e152c0f15be40dcbd421fe4a5d8d0fc
      disk 21
      id 49d67215-c86a-4ae1-bc12-ff7f245b112a
      name SBC_PIM_TEST
      os-flavor-access:is_public False
      properties aggregate_instance_extra_specs:vIMS_SRIOV='true', hw:cpu_policy='dedicated', hw:cpu_thread_policy='prefer', hw:mem_page_size='1GB', hw:numa_cpus.0='1-7', hw:numa_cpus.1='0', hw:numa_mem.0='8192', hw:numa_mem.1='8192', hw:numa_mempolicy='strict', hw:numa_nodes='2'
      ram 16384
      rxtx_factor 1.0
      swap  
      vcpus 8

      ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

              rh-ee-auniyal Amit Uniyal
              jira-bugzilla-migration RH Bugzilla Integration
              RH Bugzilla Integration RH Bugzilla Integration
              rhos-workloads-compute
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: