XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Won't Do
    • Icon: Blocker Blocker
    • None
    • None
    • Monitoring
    • False
    • Hide

      None

      Show
      None
    • False
    • None
    • Yes
    • No
    • No
    • Yes
    • None
    • Low

      Multiple incidents over the weekend due to cluster upgrades. They all come with the same alert:

      JupyterHub image builds are failing

      Checking the clusters, there are no faling builds:

      ~$ oc get builds -n redhat-ods-applications
      NAME                                        TYPE     FROM          STATUS     STARTED       DURATION
      11.4.2-cuda-s2i-core-ubi8-1                 Docker   Git@f390807   Complete   6 weeks ago   6m11s
      11.4.2-cuda-s2i-base-ubi8-1                 Docker   Git@f390807   Complete   6 weeks ago   5m37s
      11.4.2-cuda-s2i-py38-ubi8-1                 Docker   Git@4d85c35   Complete   6 weeks ago   5m16s
      11.4.2-cuda-s2i-thoth-ubi8-py38-1           Docker   Git@f485d7e   Complete   6 weeks ago   11m9s
      s2i-minimal-gpu-cuda-11.4.2-notebook-1      Docker   Git@fed57a3   Complete   6 weeks ago   12m34s
      s2i-pytorch-gpu-cuda-11.4.2-notebook-1      Source   Git@a71e007   Complete   6 weeks ago   14m30s
      s2i-tensorflow-gpu-cuda-11.4.2-notebook-1   Source   Git@f78e140   Complete   6 weeks ago   14m54s
      openvino-notebooks-experimental-1           Docker   Git@1a41588   Complete   3 weeks ago   13m0s
      

      but in comon, all clusters are progressing upgrade:

      ~$ oc get clusterversion
      NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
      version   4.9.28    True        True          59m     Working towards 4.9.29: 71 of 738 done (9% complete)
      

      No other simptoms or issues are detected, so this ticket is about having alerts that are resilient to upgrades.

      References:

              cchase@redhat.com Chris Chase
              asegundo+sd-mt-sre Amador Pahim
              Jorge Garcia Oncins Jorge Garcia Oncins
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated:
                Resolved: