+++ This bug was initially created as a clone of Bug #1444369 +++

Need ability to attach, hotplug, second SRIOV VF to instance.

Version-Release number of selected component (if applicable):
RHEL OSP 10

How reproducible:
Everytime.

Steps to Reproduce:
1. Spawn an instance using SRIOV Port
2. Tried to attach second SRIOV port to an instance.
3. It's getting failed with following message:

~~~
$ nova interface-attach --port-id 4c87802f-0f31-4087-9376-d639801be588 225b9842-aa1b-4a28-b997-537e645953b8
ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'PortBindingFailed_Remote'> (HTTP 500) (Request-ID: req-51dd3020-bbd6-4db0-bfea-3fa4e6b8ddff)
~~~

Actual results:
Not able to attach second SRIOV port to an instance.

Expected results:
Need ability to hotplug SRIOV port to instance.

There is an abandonen blueprint[1] for this functionality.

[1] https://review.openstack.org/#/c/139910/

— Additional comment from Assaf Muller on 2017-04-24 12:29:56 UTC —

— Additional comment from Stephen Gordon on 2017-04-24 15:53:25 UTC —

This will be re-assessed in planning for the Queens (13) release. Please note that given differences in the code bases between Newton and Queens as well as the areas a solution to this problem will touch it is currently considered unlikely that this will be backportable.

— Additional comment from Red Hat Bugzilla Rules Engine on 2018-04-13 14:48:52 UTC —

This bugzilla has been removed from the release since it is missing the Triaged keyword or does not have an acked release flag. For details, see https://mojo.redhat.com/docs/DOC-1144661#jive_content_id_OSP_Release_Planning.

— Additional comment from Stephen Finucane on 2018-08-22 14:48:23 UTC —

— Additional comment from Meiyan Zheng on 2019-05-28 05:22:22 UTC —

Hi,

Customer who's using RHOSP13 said they can get SR-IOV interface allocated to instance using it normally with:

1. $ nova interface-attach --port-id [SRIOV-PORT-ID] [INSTANCE-ID]
(this command will failed with error, but instance will get port allocated)
2. $ openstack port set --binding-profile pci_slot='0000:5e:02.7' --binding-profile pci_vendor_info='8086:154c' --binding-profile physical_network='sriov2' [PORT-UUID]

After updated the port information, customer said instance can use this SR-IOV VF device.

But we want to confirm:
1. can we use above as workaround for now?
2. any concerns about the workaround?

Best Regards,
Meiyan

— Additional comment from on 2019-05-28 07:31:04 UTC —

no workaound above is not supported.
there is no upstream support for attacing sriov interface to existing instance after they have booted.
while the above may work in some situations we do not support it.

using admin credentials to set the pci_slot info in the neutron port bypasses the pci resouce tracker.
The openstack port set command is setting a field that should only ever be set by nova so the user is
volating that contract.

The new interface will only become available after a hard reboot of the instance which will force the
xml to be regenerated but there is not a guarantee that the pci device will be correctly claimed in the pci
resource manager. As a result a new vm could be scheduled to the host which will attempt to use that pci device
and as a result the the instance could crash.

in osp14+ we have an explicit api check to prevent attaching sriov interfaces because we do not correctly
support that usecase as such without a explicit support exception being granted that declares that redhat will not provide support
for any issue arising from this workaroud, the customer would be explcitly using a procedure that we do not support.
A support exception would allow redhat to continue to support issue that are unrelated to sriov for instance that have been modified this
way. without a support exception the customer risk invaliding there support for future openstack issues on instance or host where this
has been applied.

to summurise i have concerns that the workaound could lead to multiple vms trying to use the same pci device as the pci resocue tracker may not be updated
correctly. the instance requst spec object may not be updated with a new pci_request_spec object for the sriov interface meaning any move operations
such as migrate,resize etctra may fail or result in the pci device not being claimed/freed on teh destination/source hosts.
finally the --binding-profile pci_vendor_info='8086:154c' --binding-profile physical_network='sriov2' options will not validate/enforce that the pci device
at 0000:5e:02.7 is attache to physnet sriov2 nor will it validate that the vendor id and product id of the devicve is an intel fortvile nic.

with all that in mind it highly likely that even if this works for there usecase today that it will break in the futrue when then upgrade so we cannot
advise that they build a production workflow around this technique.

— Additional comment from Meiyan Zheng on 2019-05-29 00:39:28 UTC —

Hi,

Million thanks for your detailed explanation! Will talk with customer about those concerns.

Best Regards,
Meiyan

— Additional comment from Vincent S. Cojot on 2019-08-01 21:10:03 UTC —

Might it be possible to add the API check to OSP13 as to give a better message than just an API error 500?

— Additional comment from on 2019-08-01 23:20:44 UTC —

i believe we added API check in a later release.

it questionable if backport that change would violate our policy of never back porting API changes.
i can bring it up in our next team call but my instinct would be no in general however since its a 500 error currently
it is less concerning as we have upstream exceptions for 500 error that allow us to change the error code to a more specific one.

i believe we may have change it to a 409 conflict
https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/409

we also need to reevaluate if we still intent to allow this in the future or if we should change this to wont fix.

The new API error upstream was introduced as we decided that we wont fix this as a bug and would treat it as an RFE
if we choose to support this in the future.

— Additional comment from on 2020-03-09 19:14:40 UTC —

removing the triaged keyword so we can discuss this in or next team call.
i dont think we have capasity to do this in 17 so i would like to disucss how to track this against 18 and disucss its priortiy.
should we remove somethign to priorties this? if so what? and what level of priortiy and severity shoudl we set.

we previously triaged it as low low with the intent of evenually adressing this.
if we still plan to do that i think we should target it to a release if not we likely should close as wont fix.

— Additional comment from jingjing niu on 2020-03-17 12:42:59 UTC —

Since we stopped collaboration with Huawei, it is just RH decision to track or close this RFE.

— Additional comment from on 2020-03-18 11:40:01 UTC —

this feature request has nothing to do with Huawei.
this has been a longstandign feature reques tsince well before this BZ was filed.
This was actully discussed as part of the initall support for sriov via neuton and was
declared out of scope at the time.

i havent spokend to the team about it for a few days but it will likely morph into a more generic
feature for osp 18 or we will close it. we currently dont have a good way to track rfes for 18.

— Additional comment from on 2020-03-20 13:48:11 UTC —

erwan/stephen
this is still waiting on the outcome of your discussion on weather we close this or figure out a way
to target this to osp 18

— Additional comment from Stephen Finucane on 2020-09-08 15:55:05 UTC —

— Additional comment from Stephen Finucane on 2020-09-16 14:48:26 UTC —

This has landed upstream.