Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-3952

Console shows default labels for relabeled alerts in RulerConfig CR

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • Logging 5.7.0
    • Log Console
    • False
    • None
    • False
    • NEW
    • NEW

      We can relabel alerts with RulerConfig definition. The relabeled alerting rules for labels are present when querying the Alert Manager but not on Console. Console still shows the rules before relabeling.

      Even external labels added to the RulerConfig are not visible on console

      RulerConfig CR: http://pastebin.test.redhat.com/1097239

      loki config and runtime-config manifest attached

      When querying AlertManager:

      $ oc -n openshift-monitoring exec -c prometheus prometheus-k8s-0 – curl -k -H "Authorization: Bearer $(oc whoami -t)" 'https://alertmanager-user-workload.openshift-user-workload-monitoring.svc:9095/api/v2/alerts' | jq
      .....
        {
          "annotations":
      {       "description": "My application 1 has high amount of logs.",       "summary": "project \"my-app-1\" log volume is high."     }
      ,
          "endsAt": "2023-04-13T19:25:20.791Z",
          "fingerprint": "2eebfe209afffa26",
          "receivers": [
           
      {         "name": "Default"       }
          ],
          "startsAt": "2023-04-13T18:55:20.791Z",
          "status":
      {       "inhibitedBy": [],       "silencedBy": [],       "state": "active"     }
      ,
          "updatedAt": "2023-04-13T19:21:20.800Z",
          "generatorURL": "/graph?g0.expr=%28count_over_time%28%7Bkubernetes_namespace_name%3D%22my-app-1%22%7D%5B2m%5D%29+%3E+10%29&g0.tab=1",
          "labels":
      {       "alertname": "MyApplication1LogVolumeIsHigh",       "environment": "prod",       "kubernetes_container_name": "centos-logtest",       "kubernetes_host": "kbharti-0413-gcp3-gdc8g-worker-a-5ndxk.c.openshift-qe.internal",       "kubernetes_namespace_name": "my-app-1",       "kubernetes_pod_name": "centos-logtest-pwndm",       "log_type": "application",       "project": "my-app-1",       "region": "us-east-1",       "severity": "warning",       "tenantId": "application"     }
        }
      ]
      

      Console Screen Attached.

      Steps to reproduce:
      1) Deploy CLO and EO
      2) Forward logs to Loki
      3) Create Alerting rules for a tenant
      4) Create RulerConfig CR with alert relabeling configuration.
      5) Query AM for Alerts and validate labels.
      6) Check labels on UI. 

      How reproducible: always

      Expected Result: UI should show alerts firing with relabeled labels

      Actual Result: Default labels for alerting rules are shown on the UI

            gbernal@redhat.com Gabriel Bernal
            rhn-support-kbharti Kabir Bharti
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: