Uploaded image for project: 'Observability Documentation'
  1. Observability Documentation
  2. OBSDOCS-256

honorLabels not respected in Prometheus config

XMLWordPrintable

    • 5
    • False
    • Hide

      None

      Show
      None
    • False
    • OBSDOCS (Oct 23 - Nov 13) #244, OBSDOCS (Nov 13 - Dev 4) #245, OBSDOCS (Dec 4 - Dev 25) #246, OBSDOCS (Jan 1 - Jan 22) #247

      Description of problem:

      Customer noticed when they set honorLabels: true, "exported_" was still getting prepended to the labels. prometheus-user-workload-0:/etc/prometheus/config_out/prometheus.env.yaml shows:   honor_labels: false
      

      Version-Release number of selected component (if applicable):

      4.11

      How reproducible:

      I tested this in my lab and got the same results.

      Steps to Reproduce:

      Their "user-workload-monitoring-config" config map is as follows:
      alertmanagerMain:
        logLevel: info
      prometheusOperator:
        logLevel: info
      prometheus: 
        overrideHonorLabels: false
      ...
      
      
      The ServiceMonitor config is:
      spec:
        endpoints:
          - honorLabels: true
            interval: 1m
            path: /metrics
            port: example
            scheme: http
        selector:
          matchLabels:
            app: example
      
      

      Actual results:

      The target scraping works, but the labels are still renamed to "exported_namespace", "exported_...". Looking in the prometheus config of the user-workload prometheus pod, we can also see that honor_labels is false for the ServiceMonitor: prometheus-user-workload-0:/etc/prometheus/config_out/prometheus.env.yaml:
      
      ...
      - job_name: serviceMonitor/#Redacted#-core-monitoring/#Redacted#-core-pushgateway/0
        honor_labels: false
        honor_timestamps: false
        kubernetes_sd_configs:
      ...

      Expected results:

      Labels should be honored

      Additional info:

       

              rhn-support-bburt Brian Burt
              rhn-support-stwalter Steven Walter
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: