Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-8905

Loki Operator configures service URLs without FQDN trailing dot, causing excessive upstream DNS queries (ndots:5)

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Unresolved
    • Icon: Normal Normal
    • None
    • Logging 6.4.z
    • Loki
    • None
    • Future Sustainability
    • False
    • Hide

      None

      Show
      None
    • False
    • Not Selected
    • NEW
    • NEW
    • If Release Note Needed, Set a Value

      Description of problem:

      • The OpenShift Logging Loki Operator configures internal service endpoints in the logging-loki-config ConfigMap using a format that triggers unintended and inefficient DNS lookups. Specifically, the operator configures the endpoint as logging-loki-query-frontend-grpc.openshift-logging.svc.cluster.local:9095.

         # oc get cm logging-loki-config -oyaml | grep server_name
          tls_server_name: logging-loki-querier-http.openshift-logging.svc.cluster.local
          tls_server_name: logging-loki-query-frontend-grpc.openshift-logging.svc.cluster.local
          tls_server_name: logging-loki-ingester-grpc.openshift-logging.svc.cluster.local
          tls_server_name: logging-loki-compactor-grpc.openshift-logging.svc.cluster.local
          tls_server_name: logging-loki-index-gateway-grpc.openshift-logging.svc.cluster.local

      • Because this hostname contains exactly four dots, it falls below the default ndots:5 threshold. As a result, the DNS resolver treats it as a relative name and appends the cluster's search domains before attempting to resolve it as an absolute domain. Since these configurations are strictly managed and reconciled by the Loki operator, cluster administrators do not have a supported method to manually modify the ConfigMap to correct this behavior.

      Actual results:
      CoreDNS generates an excessive number of unnecessary DNS requests by appending search domains to the query (e.g., logging-loki-query-frontend-grpc.openshift-logging.svc.cluster.local.ocptest.integration). While CoreDNS rejects some of these locally, a high volume of these internal OpenShift requests are inadvertently forwarded to the customer's external/upstream DNS servers, causing unnecessary load and polluting upstream environments.

      Proposed Solutions:

      • The operator's codebase should be updated to optimize how Loki resolves its internal endpoints either using explicit FQDN by append a trailing dot or short names.

      Additional Information:

      Related KCS Articles:

          https://access.redhat.com/solutions/2518321

          https://access.redhat.com/solutions/4681131

              Unassigned Unassigned
              rhn-support-dgupte Dhananjay Gupte
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: