Uploaded image for project: 'Red Hat OpenStack Services on OpenShift'
  1. Red Hat OpenStack Services on OpenShift
  2. OSPRH-7402

Glance accessing oslo_limits using keystone limits fails with 500 error

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Cannot Reproduce
    • Icon: Normal Normal
    • rhos-18.0.1
    • None
    • keystone-operator
    • None
    • False
    • Hide

      None

      Show
      None
    • False
    • ?
    • ?
    • ?
    • ?
    • None
    • DFG Security: UC Sprint 97, DFG Security: UC Sprint 98, DFG Security: UC Sprint 99, DFG Security: UC Sprint 100
    • Important

      When glance enables its quota feature which is using keystone limits, trying to create image, delete image, import image or accessing quota usage information fails with 500 error.

      How to reproduce:
      1. Ensure you have glance deployment with quotas enabled [1]
      2. Connect to glance pod and run below command:
      $ glance usage

      [1] https://github.com/openstack-k8s-operators/glance-operator/tree/main/config/samples/quotas#glance-per-tenant-quotas

      expected result;
      It should show available keystone limits for glance.

      Actual result:
      HTTP 500 Internal Server Error: The server has either erred or is incapable of performing the requested operation.

      Glance logs:
      2024-05-30 15:06:17.344 19 ERROR oslo.limit.limit [None req-ef29ce86-4929-47bd-b9ee-a33665caa685 22c079b3a48f4fa6b99b9ca5fd4cc395 dcd4ab5cd09c412d8b24cce6adde4037 - - default default] Unable to initialize OpenStackSDK session: The request you have made requires authentication. (HTTP 401) (Request-ID: req-a01be27c-1333-4ad6-b6d0-18ae4fba4501): keystoneauth1.exceptions.http.Unauthorized: The request you have made requires authentication. (HTTP 401) (Request-ID: req-a01be27c-1333-4ad6-b6d0-18ae4fba4501)
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi [None req-ef29ce86-4929-47bd-b9ee-a33665caa685 22c079b3a48f4fa6b99b9ca5fd4cc395 dcd4ab5cd09c412d8b24cce6adde4037 - - default default] Caught error: Can't initialise OpenStackSDK session: The request you have made requires authentication. (HTTP 401) (Request-ID: req-a01be27c-1333-4ad6-b6d0-18ae4fba4501).: oslo_limit.exception.SessionInitError: Can't initialise OpenStackSDK session: The request you have made requires authentication. (HTTP 401) (Request-ID: req-a01be27c-1333-4ad6-b6d0-18ae4fba4501).
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi Traceback (most recent call last):
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi   File "/usr/lib/python3.9/site-packages/oslo_limit/limit.py", line 44, in _get_keystone_connection
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi     _SDK_CONNECTION = connection.Connection(session=session).identity
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi   File "/usr/lib/python3.9/site-packages/openstack/service_description.py", line 87, in _get_
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi     proxy = self._make_proxy(instance)
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi   File "/usr/lib/python3.9/site-packages/openstack/service_description.py", line 262, in _make_proxy
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi     found_version = temp_adapter.get_api_major_version()
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi   File "/usr/lib/python3.9/site-packages/keystoneauth1/adapter.py", line 354, in get_api_major_version
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi     return self.session.get_api_major_version(auth or self.auth, **kwargs)
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi   File "/usr/lib/python3.9/site-packages/keystoneauth1/session.py", line 1276, in get_api_major_version
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi     return auth.get_api_major_version(self, **kwargs)
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi   File "/usr/lib/python3.9/site-packages/keystoneauth1/identity/base.py", line 500, in get_api_major_version
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi     data = get_endpoint_data(discover_versions=discover_versions)
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi   File "/usr/lib/python3.9/site-packages/keystoneauth1/identity/base.py", line 271, in get_endpoint_data
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi     service_catalog = self.get_access(session).service_catalog
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi   File "/usr/lib/python3.9/site-packages/keystoneauth1/identity/base.py", line 134, in get_access
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi     self.auth_ref = self.get_auth_ref(session)
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi   File "/usr/lib/python3.9/site-packages/keystoneauth1/identity/generic/base.py", line 208, in get_auth_ref
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi     return self._plugin.get_auth_ref(session, **kwargs)
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi   File "/usr/lib/python3.9/site-packages/keystoneauth1/identity/v3/base.py", line 188, in get_auth_ref
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi     resp = session.post(token_url, json=body, headers=headers,
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi   File "/usr/lib/python3.9/site-packages/keystoneauth1/session.py", line 1149, in post
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi     return self.request(url, 'POST', **kwargs)
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi   File "/usr/lib/python3.9/site-packages/keystoneauth1/session.py", line 986, in request
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi     raise exceptions.from_response(resp, method, url)
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi keystoneauth1.exceptions.http.Unauthorized: The request you have made requires authentication. (HTTP 401) (Request-ID: req-a01be27c-1333-4ad6-b6d0-18ae4fba4501)
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi During handling of the above exception, another exception occurred:
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi Traceback (most recent call last):
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi   File "/usr/lib/python3.9/site-packages/glance/common/wsgi.py", line 1286, in _call_
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi     action_result = self.dispatch(self.controller, action,
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi   File "/usr/lib/python3.9/site-packages/glance/common/wsgi.py", line 1329, in dispatch
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi     return method(*args, **kwargs)
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi   File "/usr/lib/python3.9/site-packages/glance/api/v2/discovery.py", line 160, in get_usage
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi     project_usage = ks_quota.get_usage(req.context)
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi   File "/usr/lib/python3.9/site-packages/glance/quota/keystone.py", line 169, in get_usage
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi     enforcer = limit.Enforcer(callback)
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi   File "/usr/lib/python3.9/site-packages/oslo_limit/limit.py", line 74, in _init_
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi     self.connection = _get_keystone_connection()
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi   File "/usr/lib/python3.9/site-packages/oslo_limit/limit.py", line 52, in _get_keystone_connection
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi     raise exception.SessionInitError(e)
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi oslo_limit.exception.SessionInitError: Can't initialise OpenStackSDK session: The request you have made requires authentication. (HTTP 401) (Request-ID: req-a01be27c-1333-4ad6-b6d0-18ae4fba4501).
      2024-05-30 15:06:17.345 19 ERROR glance.common.wsgi
      2024-05-30 15:06:17.350 19 INFO eventlet.wsgi.server [None req-ef29ce86-4929-47bd-b9ee-a33665caa685 22c079b3a48f4fa6b99b9ca5fd4cc395 dcd4ab5cd09c412d8b24cce6adde4037 - - default default] ::1 - - [30/May/2024 15:06:17] "GET /v2/info/usage HTTP/1.1" 500 454 0.626022

      Keystone logs:

      [Thu May 30 15:06:17.341526 2024] [wsgi:error] [pid 25:tid 59] [remote 10.217.1.172:57096] 2024-05-30 15:06:17.341 25 DEBUG keystone.models.token_model [None req-a01be27c-1333-4ad6-b6d0-18ae4fba4501 - - - - - -] User b0cf8e0284ca40ab824b84e2b5001d2b has no access to the system _validate_system_scope /usr/lib/python3.9/site-packages/keystone/models/token_model.py:523\x1b[00m                                                                                                                        │exit
      [Thu May 30 15:06:17.342622 2024] [wsgi:error] [pid 25:tid 59] [remote 10.217.1.172:57096] 2024-05-30 15:06:17.342 25 WARNING keystone.server.flask.application [None req-a01be27c-1333-4ad6-b6d0-18ae4fba4501 - - - - - -] Authorization failed. The request you have made requires authentication. from 10.217.1.172: keystone.exception.Unauthorized: The request you have made requires authentication.\x1b[00m 
      [30/May/2024:15:06:17 +0000] "POST /v3/auth/tokens HTTP/1.1" 401 109 "-" "glance-api keystoneauth1/5.1.2 python-requests/2.25.1 CPython/3.9.18"

       

              dwilde@redhat.com Dave Wilde
              akekane@redhat.com Abhishek Kekane
              rhos-dfg-security
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: