Uploaded image for project: 'OpenShift Service Mesh'
  1. OpenShift Service Mesh
  2. OSSM-6700

Bad URL in rest_client_request_latency_seconds metrics

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done-Errata
    • Icon: Normal Normal
    • OSSM 2.6.0
    • OSSM 2.5.2
    • Customer Impact
    • None

      The operator uses the wrong URL in the rest_client_request_latency_seconds metrics. The URL is always http://apiserver/%7Bprefix%7D, causing virtually all requests to be counted in a single metric, instead having one metric per URL. Here's how the metrics look like right now:

      # HELP rest_client_request_latency_seconds Request latency in seconds. Broken down by verb and URL.
      # TYPE rest_client_request_latency_seconds histogram
      rest_client_request_latency_seconds_bucket{url="https://10.217.4.1:443/%7Bprefix%7D",verb="GET",le="0.001"} 4
      rest_client_request_latency_seconds_bucket{url="https://10.217.4.1:443/%7Bprefix%7D",verb="GET",le="0.002"} 24
      rest_client_request_latency_seconds_bucket{url="https://10.217.4.1:443/%7Bprefix%7D",verb="GET",le="0.004"} 55
      rest_client_request_latency_seconds_bucket{url="https://10.217.4.1:443/%7Bprefix%7D",verb="GET",le="0.008"} 139
      rest_client_request_latency_seconds_bucket{url="https://10.217.4.1:443/%7Bprefix%7D",verb="GET",le="0.016"} 177
      rest_client_request_latency_seconds_bucket{url="https://10.217.4.1:443/%7Bprefix%7D",verb="GET",le="0.032"} 179
      rest_client_request_latency_seconds_bucket{url="https://10.217.4.1:443/%7Bprefix%7D",verb="GET",le="0.064"} 198
      rest_client_request_latency_seconds_bucket{url="https://10.217.4.1:443/%7Bprefix%7D",verb="GET",le="0.128"} 257
      rest_client_request_latency_seconds_bucket{url="https://10.217.4.1:443/%7Bprefix%7D",verb="GET",le="0.256"} 271
      rest_client_request_latency_seconds_bucket{url="https://10.217.4.1:443/%7Bprefix%7D",verb="GET",le="0.512"} 295
      rest_client_request_latency_seconds_bucket{url="https://10.217.4.1:443/%7Bprefix%7D",verb="GET",le="+Inf"} 364
      

      This is caused by a bug in the ancient version of the client-go library we are using. The finalURLTemplate function calculates groupIndex incorrectly, causing it to not find /api/ or /apis/ in the URL and thus defaults to this %7Bprefix%7D URL instead of producing the correct one.

      Note: we've had this issue since the start, so it might not make sense to fix it in 2.6. In 3.0, this issue is not present.

            mluksa@redhat.com Marko Luksa
            mluksa@redhat.com Marko Luksa
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: