Uploaded image for project: 'Red Hat OpenShift Data Science'
  1. Red Hat OpenShift Data Science
  2. RHODS-4818

JH unable to show and spawn images when GPU Addon is

XMLWordPrintable

    • False
    • None
    • False
    • Hide

      JH spawner page lets users spawn images

      Show
      JH spawner page lets users spawn images
    • Release Notes
    • No
    • No
    • Hide
      JupyterHub was unable to display images when the NVIDIA GPU add-on was installed:: The *Start a notebook server* page did not display notebook images after installing the NVIDIA GPU add-on. Images are now correctly displayed, and can be started from the *Start a notebook server* page.
      Show
      JupyterHub was unable to display images when the NVIDIA GPU add-on was installed:: The *Start a notebook server* page did not display notebook images after installing the NVIDIA GPU add-on. Images are now correctly displayed, and can be started from the *Start a notebook server* page.
    • Documented as Resolved Issue
    • Yes
    • Pending
    • None
    • Urgent

      Description of problem:

      After triggering the GPU addon installation, JH starts showing this error:

      Prerequisites (if any, like setup, operators/versions):

      Steps to Reproduce

      1. install RHODS
      2. Launch JupyterHub
      3. check you can see the images in JH spawner page
      4. trigger GPU Addon installation
      5. go back to JH spawner page (maybe wait some minutes and/or refresh the page)

      Actual results:

      JH spawner page does not show notebook images but an error message saying "Unable to load notebook server configuration options"

      Expected results:

      JH spawner page lets users spawn images

      Reproducibility (Always/Intermittent/Only Once):

      Always

      Build Details:

      RHODS v1.14
      OSD 4.11

      Workaround:

      None

      Additional info:

      [2022-08-03 16:27:41,262] ERROR in app: Exception on /services/jsp-api/api/ui/config [GET]
      Traceback (most recent call last):
        File "/opt/app-root/lib64/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
          response = self.full_dispatch_request()
        File "/opt/app-root/lib64/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
          rv = self.handle_user_exception(e)
        File "/opt/app-root/lib64/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
          reraise(exc_type, exc_value, tb)
        File "/opt/app-root/lib64/python3.8/site-packages/flask/_compat.py", line 39, in reraise
          raise value
        File "/opt/app-root/lib64/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
          rv = self.dispatch_request()
        File "/opt/app-root/lib64/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
          return self.view_functions[rule.endpoint](**req.view_args)
        File "/opt/app-root/lib64/python3.8/site-packages/connexion/decorators/decorator.py", line 68, in wrapper
          response = function(request)
        File "/opt/app-root/lib64/python3.8/site-packages/connexion/decorators/uri_parsing.py", line 149, in wrapper
          response = function(request)
        File "/opt/app-root/lib64/python3.8/site-packages/connexion/decorators/parameter.py", line 115, in wrapper
          return function(**kwargs)
        File "/opt/app-root/lib64/python3.8/site-packages/jupyterhub_singleuser_profiles/api/api.py", line 54, in decorated
          return f(user=user, *args, **kwargs)
        File "/opt/app-root/lib64/python3.8/site-packages/jupyterhub_singleuser_profiles/api/api.py", line 91, in get_ui_config
          cm = _PROFILES.get_ui_configuration()
        File "/opt/app-root/lib64/python3.8/site-packages/jupyterhub_singleuser_profiles/profiles.py", line 144, in get_ui_configuration
          return ui.validate_ui_cm()
        File "/opt/app-root/lib64/python3.8/site-packages/jupyterhub_singleuser_profiles/ui_config.py", line 128, in validate_ui_cm
          self.ui_cfg['gpuConfig']['clusterGpuCount'] = self.openshift.get_gpu_number()
        File "/opt/app-root/lib64/python3.8/site-packages/jupyterhub_singleuser_profiles/openshift.py", line 206, in get_gpu_number
          headers={"Authorization": "Bearer " + self.get_openshift_prometheus_token()},
        File "/opt/app-root/lib64/python3.8/site-packages/jupyterhub_singleuser_profiles/openshift.py", line 170, in get_openshift_prometheus_token
          token_secret_name = [s for s in service_account.secrets if 'token' in s.name][0].name
      IndexError: list index out of range

              Unassigned Unassigned
              rhn-support-bdattoma Berto D'Attoma
              Milind Waykole Milind Waykole
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: