Uploaded image for project: 'Red Hat OpenShift Data Science'
  1. Red Hat OpenShift Data Science
  2. RHODS-5822

[DSG] RHODS admins are not warned when usage exceeds 90% or 100% for PVCs created by Data Science Projects

XMLWordPrintable

    • False
    • None
    • False
    • Hide

      RHODS Prometheus has alerts about PVC usage above 90% and 100%

      Show
      RHODS Prometheus has alerts about PVC usage above 90% and 100%
    • Documentation (Ref Guide, User Guide, etc.), Release Notes
    • Testable
    • No
    • 1.21.0-z
    • No
    • Hide
      == Admin users were not warned when usage exceeded 90% and 100% for PVCs created by data science projects.
      Warnings indicating when a PVC exceeded 90% and 100% of its capacity failed to display to admin users for PVCs created by data science projects. Admin users can now view warnings about when a PVC exceeds 90% and 100% of its capacity from the dashboard.
      Show
      == Admin users were not warned when usage exceeded 90% and 100% for PVCs created by data science projects. Warnings indicating when a PVC exceeded 90% and 100% of its capacity failed to display to admin users for PVCs created by data science projects. Admin users can now view warnings about when a PVC exceeds 90% and 100% of its capacity from the dashboard.
    • Bug Fix
    • Done
    • No
    • Pending
    • None
    • RHODS 1.21
    • Medium

      Description of problem:

      Prometheus usually has an alerts checking for PVC usage and triggering when oen PVC is filled above 90% or at 100%

      But it works only for PVC created by the classic Spawner and not for DSG.

      Prerequisites (if any, like setup, operators/versions):

      Create a DS Project from Dashboard UI

      Create a workbench with a PVC (1GB size for convenience of testing) from Dashboard UI 

      Steps to Reproduce

      1. Launch the workbench
      2. fill in the underlying PVC above 90% or at 100% (example code)
      3. check prometheus alerts are not firing (redhat-ods-monitoring namespace)

      Actual results:

      no alerts defined in prometheus to check PVC usage above 90% and 100%

      Expected results:

      alerts defined in prometheus to check PVC usage above 90% and 100%

      Reproducibility (Always/Intermittent/Only Once):

      Always

      Build Details:

      RHODS v1.19.9-10

      Workaround:

      Additional info:

       

       

        1. image-2022-11-10-12-33-56-389.png
          image-2022-11-10-12-33-56-389.png
          113 kB
        2. image-2022-11-10-12-35-05-605.png
          image-2022-11-10-12-35-05-605.png
          73 kB
        3. rhods-1.21.0-21-unable-to-get-storage-data.png
          rhods-1.21.0-21-unable-to-get-storage-data.png
          326 kB
        4. Screenshot from 2022-11-14 13-28-30.png
          Screenshot from 2022-11-14 13-28-30.png
          70 kB
        5. Screenshot from 2022-11-14 13-29-06.png
          Screenshot from 2022-11-14 13-29-06.png
          33 kB

            juntwang@redhat.com Juntao Wang
            rhn-support-bdattoma Berto D'Attoma
            Jorge Garcia Oncins Jorge Garcia Oncins
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: