Uploaded image for project: 'Red Hat Advanced Cluster Management'
  1. Red Hat Advanced Cluster Management
  2. ACM-26173

receive hashring is never updated on scaling due to missing permissions

XMLWordPrintable

    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • False
    • Moderate
    • None

      Description of problem:

      If increasing he number of thanos-receive replicas, thanos-receive-controller should update the hashring to include the new node. This never happens because it's missing permission to get the pod status.

      Version-Release number of selected component (if applicable):

      Probably all current ACM releases.

      How reproducible:

      Always

      Steps to Reproduce:

      1. Install MCO, wait for everything time come up
      2. Increase the number of thanos receive replicas in the MCO CR.

      Actual results:

      1. The hashring in observability-thanos-receive-controller-tenants-generated is never updated
      2. thanos-receive-controller errors:
      level=warn caller=main.go:585 ts=2025-11-07T16:07:51.550767141Z msg="failed polling until pod is ready" pod=observability-thanos-receive-default-3 duration=3.072254ms err="pods \"observability-thanos-receive-default-3\" is forbidden: User \"system:serviceaccount:open-cluster-management-observability:observability-thanos-receive-controller\" cannot get resource \"pods\" in API group \"\" in the namespace \"open-cluster-management-observability\""
      

      Expected results:

      1. The hashring in observability-thanos-receive-controller-tenants-generated is actually updated

      Additional info:

              smeduri1@redhat.com Subbarao Meduri
              rh-ee-jachanse Jacob Baungard Hansen
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated: