Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-58102

ExternalDNS metrics are not available

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • 4.16.z
    • ExternalDNS Operator
    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • Important
    • None
    • None
    • None
    • Rejected
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem

      Customer is using the External DNS Operator on AWS and on-premise. The External DNS Operator itself provides metrics via `:8080/metrics`. However when an ExternalDNS object is created, the metrics exposed by the ExternalDNS Pod are not exposed and not scraped by Prometheus.

      Retrieving metrics via `localhost` works as expected:

      sh-5.1$ curl localhost:7979/metrics
      # HELP external_dns_controller_last_reconcile_timestamp_seconds Timestamp of last attempted sync with the DNS provider
      # TYPE external_dns_controller_last_reconcile_timestamp_seconds gauge
      external_dns_controller_last_reconcile_timestamp_seconds 1.7504298579781175e+09
      # HELP external_dns_controller_last_sync_timestamp_seconds Timestamp of last successful sync with the DNS provider
      # TYPE external_dns_controller_last_sync_timestamp_seconds gauge
      external_dns_controller_last_sync_timestamp_seconds 0
      # HELP external_dns_controller_no_op_runs_total Number of reconcile loops ending up with no changes on the DNS provider side.
      # TYPE external_dns_controller_no_op_runs_total counter
      external_dns_controller_no_op_runs_total 0

      However the metrics are only exposed for localhost as the ExternalDNS Pod is started with the following arguments:

      [..]
            args:
              - '--domain-filter=testextdnsoperator.apacshift.support'
              - '--metrics-address=127.0.0.1:7979'
      [..]

      Trying to scrape them from outside the Pod fails as the port is only bound locally:

      sh-5.1$ curl 10.131.0.26:7979/metrics
      curl: (7) Failed to connect to 10.131.0.26 port 7979: Connection refused

      The customer expects that the metrics from the ExternalDNS Pod are automatically exposed and automatically scraped (for example via a `PodMonitor` or a `ServiceMonitor`).

      Version-Release number of selected component (if applicable):

      external-dns-operator.v1.3.0

      How reproducible:

      Always

      Steps to Reproduce:

      • Install the External DNS Operator
      • Create an ExternalDNS object with any configuration, confirm that the Pod for that object is running
      • By default, the "external-dns-operator" namespace is not labeled with "openshift.io/cluster-monitoring: 'true'". Label the namespace with this label so the Platform Prometheus consumes the ServiceMonitors / PodMonitors.
      • Check metrics for the namespace where the Operator is installed

      Actual results:

      No metrics are being scraped, metrics are only exposed on `127.0.0.1`

      Expected results:

      Metrics are exposed and are being scraped automatically by Prometheus. The customer expects to see "external_dns_*" metrics in Prometheus.

      Additional info:

      -

              nid-team-bot NID Team Bot
              rhn-support-skrenger Simon Krenger
              None
              None
              Melvin Joseph Melvin Joseph
              None
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated: