-
Bug
-
Resolution: Duplicate
-
Normal
-
rhos-18.0.0
-
5
-
False
-
-
False
-
?
-
?
-
?
-
?
-
None
-
-
-
DFG Security: UC Sprint 98, DFG Security: UC Sprint 99, DFG Security: UC Sprint 100, DFG Security: UC Sprint 101, DFG Security: UC Sprint 102, DFG Security: UC Sprint 103
-
Important
When a service password is changed in osp-secret, i.e. to rotate the password. The service operators start using the new keystone user pass in their config but it seems that the new password is not accepted by keystone.
Example scenario with cinder but I think all the service operators are affected in the same way.
$ oc set data secret/osp-secret CinderPassword=foobar
$ oc rsh -c cinder-api cinder-api-0 grep password -R /etc/cinder/cinder.conf.d/ /etc/cinder/cinder.conf.d/00-global-defaults.conf:auth_type = password /etc/cinder/cinder.conf.d/00-global-defaults.conf:password = foobar /etc/cinder/cinder.conf.d/00-global-defaults.conf:auth_type = password /etc/cinder/cinder.conf.d/00-global-defaults.conf:password = foobar /etc/cinder/cinder.conf.d/00-global-defaults.conf:auth_type = password /etc/cinder/cinder.conf.d/00-global-defaults.conf:password = foobar /etc/cinder/cinder.conf.d/..data/00-global-defaults.conf:auth_type = password /etc/cinder/cinder.conf.d/..data/00-global-defaults.conf:password = foobar /etc/cinder/cinder.conf.d/..data/00-global-defaults.conf:auth_type = password /etc/cinder/cinder.conf.d/..data/00-global-defaults.conf:password = foobar /etc/cinder/cinder.conf.d/..data/00-global-defaults.conf:auth_type = password /etc/cinder/cinder.conf.d/..data/00-global-defaults.conf:password = foobar /etc/cinder/cinder.conf.d/..2024_06_26_08_08_11.2847890368/00-global-defaults.conf:auth_type = password /etc/cinder/cinder.conf.d/..2024_06_26_08_08_11.2847890368/00-global-defaults.conf:password = foobar /etc/cinder/cinder.conf.d/..2024_06_26_08_08_11.2847890368/00-global-defaults.conf:auth_type = password /etc/cinder/cinder.conf.d/..2024_06_26_08_08_11.2847890368/00-global-defaults.conf:password = foobar /etc/cinder/cinder.conf.d/..2024_06_26_08_08_11.2847890368/00-global-defaults.conf:auth_type = password /etc/cinder/cinder.conf.d/..2024_06_26_08_08_11.2847890368/00-global-defaults.conf:password = foobar
$ openstack volume service list
The server is currently unavailable. Please try again at a later time.<br /><br />
The Keystone service is temporarily unavailable.
(HTTP 503)
command terminated with exit code 1
: keystoneauth1.exceptions.http.Unauthorized: The request you have made requires authentication. (HTTP 401) (Request-ID: req-723ae722-4ce4-475d-866c-afaac2949720)
2024-06-26 08:15:18.865 17 WARNING keystonemiddleware.auth_token [None req-1fbb4ad8-8658-41a7-99df-af048bdb8548 - - - - - -] Identity response: {"error":{"code":401,"message":"The request you have made requires authentication.","title":"Unauthorized"}}
: keystoneauth1.exceptions.http.Unauthorized: The request you have made requires authentication. (HTTP 401) (Request-ID: req-b80c8a86-a726-4845-b126-0fcda513b798)
2024-06-26 08:15:18.866 17 CRITICAL keystonemiddleware.auth_token [None req-1fbb4ad8-8658-41a7-99df-af048bdb8548 - - - - - -] Unable to validate token: Identity server rejected authorization necessary to fetch token data: keystonemiddleware.auth_token._exceptions.ServiceError: Identity server rejected authorization necessary to fetch token data
Possible workaround (but it is really ugly):
- delete the KeystoneService for the given service e.g.
oc delete KeystoneService/cinderv3
- remove the finalizers from the KeystoneService to unblock the CR deletion.
oc edit KeystoneService/cinderv3
- delete the KeystoneEndpoint for the given service e.g.
oc delete KeystoneEndpoint/cinderv3
- remove the finalizers from the KeystoneEndpoint.
- the service operator will re-create the CRs and now the new password will be accepted by keystone
- is related to
-
OSPRH-9554 Zero downtime password rotation
-
- Backlog
-
