Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-6055

thanos-ruler-user-workload-1 pod is getting repeatedly re-created after upgrade do 4.10.41

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • None
    • 4.13, 4.12, 4.11, 4.10
    • Monitoring
    • None
    • None
    • MON Sprint 231, MON Sprint 232
    • 2
    • Rejected
    • False
    • Hide

      None

      Show
      None
    • Hide
      * Previously, if more than one extra label was defined for Thanos Ruler, then the statefulset could enter a recreation loop because prometheus-operator would not add the labels in a specified order each time it reconciled the custom resource. After this fix, prometheus-operator now sorts extra labels before adding them to the statefulset. (link:https://issues.redhat.com/browse/OCPBUGS-6055[*OCPBUGS-6055*])
      Show
      * Previously, if more than one extra label was defined for Thanos Ruler, then the statefulset could enter a recreation loop because prometheus-operator would not add the labels in a specified order each time it reconciled the custom resource. After this fix, prometheus-operator now sorts extra labels before adding them to the statefulset. (link: https://issues.redhat.com/browse/OCPBUGS-6055 [* OCPBUGS-6055 *])
    • Bug Fix
    • Done

      Description of problem:

      - After upgrading to OCP 4.10.41, thanos-ruler-user-workload-1 in the openshift-user-workload-monitoring namespace is consistently being created and deleted.
- We had to scale down the Prometheus operator multiple times so that the upgrade is considered as successful.
- This fix is temporary. After some time it appears again and Prometheus operator needs to be scaled down and up again.
- The issue is present on all clusters in this customer environment which are upgraded to 4.10.41.

      Version-Release number of selected component (if applicable):

       

      How reproducible:

      N/A, I wasn't able to reproduce the issue.

      Steps to Reproduce:

       

      Actual results:

       

      Expected results:

       

      Additional info:

       

              rhn-support-bburt Brian Burt
              rhn-support-rbobek Roman Bobek
              Junqi Zhao Junqi Zhao
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated:
                Resolved: