Description of problem:

At a customer's shift-on-stack production environment, an attempt to scale-out OCP worker nodes failed, leaving cinder root volumes in "reserved" state. Because policy prevents users with "member" role from deleting or resetting the volume state, the volume is left unusable until an administrator can fix them.

See reproduction steps without OCP being a factor.

Version-Release number of selected component (if applicable):
RHOSP 16.2, 17.1

How reproducible:
Always reproducible

Steps to Reproduce:

All of the following steps are executed as a user with "member" role on the project.

1. Create a server group:
~~~
(projectmember) (overcloud) [stack@director.example.lab ~]$ openstack server group create --policy soft-anti-affinity servers-anti-1 --os-compute-api-version 2.15
------------------------------------------------+

------------------------------------------------+

------------------------------------------------+
~~~

2. Server groups have a default 10 members:
~~~
(projectmember) (overcloud) [stack@director.example.lab ~]$ openstack quota show -c server-group-members
---------------------------+

---------------------------+

---------------------------+
~~~

3. Create enough root volumes:
~~~
(projectmember) (overcloud) [stack@director.example.lab ~]$ for x in

;do
> openstack volume create -image cirros-0.5.2 --size 1 --bootable vol${x}
> done
...
(projectmember) (overcloud) [stack@director.example.lab ~]$ openstack volume list
--------------------------------------------------------------------

--------------------------------------------------------------------

--------------------------------------------------------------------
~~~

4. Request creation of 11 servers using existing root volume:
~~~
(projectmember) (overcloud) [stack@director.example.lab ~]$ for x in 11 ;do openstack server create -volume vol${x} -flavor 1c1g --hint group=4af0bb95-6eb0-47d7-a6fa-94992730848d testvm${x} -c id; done
Quota exceeded, too many servers in group (HTTP 403) (Request-ID: req-a7bd8ba7-0fe2-42e6-8b61-c04770baa10b)
(projectmember) (overcloud) [stack@director.example.lab ~]$
~~~

Actual results:

1. The 11th server leaves vol-11 in reserved state:
~~~
(projectmember) (overcloud) [stack@director.example.lab ~]$ openstack volume list
------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------
(projectmember) (overcloud) [stack@director.example.lab ~]$
~~~

2. The user that created the volume can now neither reset the state nor delete it:
~~~
(projectmember) (overcloud) [stack@director.example.lab ~]$ openstack volume delete vol-11
Failed to delete volume with name or ID 'vol-11': Invalid volume: Volume status must be available or error or error_restoring or error_extending or error_managing and must not be migrating, attached, belong to a group, have snapshots or be disassociated from snapshots after volume transfer. (HTTP 400) (Request-ID: req-260992f9-5f76-46ad-ab30-ec6090691644)
1 of 1 volumes failed to delete.
(projectmember) (overcloud) [stack@director.example.lab ~]$
(projectmember) (overcloud) [stack@director.example.lab ~]$ cinder delete vol-11
Delete for volume vol-11 failed: Invalid volume: Volume status must be available or error or error_restoring or error_extending or error_managing and must not be migrating, attached, belong to a group, have snapshots or be disassociated from snapshots after volume transfer. (HTTP 400) (Request-ID: req-7f8ec1e0-a6e8-47c7-a47f-5c44e3681edc)
ERROR: Unable to delete any of the specified volumes.
(projectmember) (overcloud) [stack@director.example.lab ~]$
(projectmember) (overcloud) [stack@director.example.lab ~]$ cinder reset-state --state Available vol-11
Policy doesn't allow volume_extension:volume_admin_actions:reset_status to be performed. (HTTP 403) (Request-ID: req-08e650ea-a8ca-4f41-944b-7e5906965a5d)
ERROR: Unable to reset the state for the specified entity(s).
(projectmember) (overcloud) [stack@director.example.lab ~]$
~~~

Expected results:

1. Ideally, the failed server create request should cleanup the cinder volume reservation, or
2. A user with member role should be able to delete, or reset-state and then delete the volume (although granted, there could be situations in which the root cause needs to be investigated before resetting the volume state and therefore and admin is required).

Additional info:

Note that this seems to be specifically related to using server groups. A subsequent build without a hint to a server group suceeds:
~~~
(projectmember) (overcloud) [stack@director.example.lab ~]$ openstack volume create --image cirros-0.5.2 --size 1 vol-12 -c id
-------------------------------------------+

-------------------------------------------+

-------------------------------------------+
(projectmember) (overcloud) [stack@director.example.lab ~]$ openstack server create --volume vol-12 --flavor 1c1g testvm-12 -c id --wait

-------------------------------------------+

-------------------------------------------+

-------------------------------------------+
(projectmember) (overcloud) [stack@director.example.lab ~]$
(projectmember) (overcloud) [stack@director.example.lab ~]$ openstack volume list
------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------
(projectmember) (overcloud) [stack@director.example.lab ~]$
~~

Moreover, a server create request that exceeds a different quota (instances) does not leave the volume in reserved state:
~~~
(projectmember) (overcloud) [stack@director.example.lab ~]$ openstack volume create --image cirros-0.5.2 --size 1 vol-13
---------------------------------------------------------+

---------------------------------------------------------+

---------------------------------------------------------+
(projectmember) (overcloud) [stack@director.example.lab ~]$
(projectmember) (overcloud) [stack@director.example.lab ~]$ openstack server create --volume vol-13 --flavor 1c1g testvm-13 -c id --wait
Quota exceeded for instances: Requested 1, but already used 11 of 11 instances (HTTP 403) (Request-ID: req-1de133c1-9f4c-4a05-a4d3-e87b6125e7be)
(projectmember) (overcloud) [stack@director.example.lab ~]$
(projectmember) (overcloud) [stack@director.example.lab ~]$ openstack volume list
-------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------
(projectmember) (overcloud) [stack@director.example.lab ~]$
~~~