Uploaded image for project: 'Red Hat Advanced Cluster Management'
  1. Red Hat Advanced Cluster Management
  2. ACM-20542

Upgrade from ACM 2.11.3 to 2.13.2 causes some Observability Pods to use wrong Image

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Normal Normal
    • None
    • ACM 2.13.2
    • Observability
    • Quality / Stability / Reliability
    • 1
    • False
    • Hide

      None

      Show
      None
    • False
    • MCO Core Sprint 44
    • None

      One of our partner is trying to upgrade their disconnected hub cluster from 4.16 to 4.18. They have mirrored all required images to their local registry. When they try to upgrade ACM operator from 2.11.3 to 2.13.0 , some of the pods in open-cluster-management-observability starts failing due to "image pull back off". It tries to use quay.io/stolostron/origin-oauth-proxy:4.5 image.

      open-cluster-management-observability observability-alertmanager-2 3/4 ImagePullBackOff 0 71m
      open-cluster-management-observability observability-grafana-7dd876989c-4gms6 2/3 ImagePullBackOff 0 71m
      open-cluster-management-observability observability-rbac-query-proxy-5859b69fb-npvpx 1/2 ImagePullBackOff 0 71m
       
      After some discussion in #forum-acm-observability (https://redhat-internal.slack.com/archives/CUU609ZQC/p1745327709396519 ), we agreed on that upgrading to 2.13.2 might fix the issue based on this link: https://docs.redhat.com/en/documentation/red_hat_advanced_cluster_management_for_kubernetes/2.13/html/release_notes/acm-release-notes#upgrade-stuck 
       
      Our partner has tried directly upgrading from 2.11.3 to 2.13.2 as well but again some these pods stuck for "ImagePullBackOff" and 2 of these pods are self fixed automatically to correct image and 1 of pod is manually deleted and then fixed to correct image. But at least this time, upgrade was completed and all pods are running. Upgrade to 2.13.0 was not getting completed. Upgrade to 2.13.2 seems better but still some bug is happening during upgrade.

              rh-ee-jachanse Jacob Baungard Hansen
              skoksal@redhat.com Sarp Koksal
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated: