Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-48790

Insights-runtime-extractor pod return 500 after patching trustedCA using proxy

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Normal Normal
    • None
    • 4.18
    • None
    • None
    • False
    • Hide

      None

      Show
      None
    • Hide
      When the OpenShift cluster CA Bundle is updated with a custom CA bundle, it will take some time before the change is propagated to the insights-runtime-extractor container.

      If the Insights Operator gathers data right after the CA Bundle is updated, the container might not have been restarted and the workload runtime data will not be gathered.
      Show
      When the OpenShift cluster CA Bundle is updated with a custom CA bundle, it will take some time before the change is propagated to the insights-runtime-extractor container. If the Insights Operator gathers data right after the CA Bundle is updated, the container might not have been restarted and the workload runtime data will not be gathered.
    • Technology Preview
    • In Progress

      Description of problem:

          After creating a pair of self-signed tls cert and private key, then add it into trustde-ca-bundle by using the following cmd:
      
      oc patch proxy/cluster \
           --type=merge \
           --patch='{"spec":{"trustedCA":{"name":"custom-ca"}}}'
      
      The insights-runtime-extractor pod will return response with 500 status code, this is the https flow details:
      
      *   Trying 10.129.2.15:8000...
      * Connected to exporter.openshift-insights.svc.cluster.local (10.129.2.15) port 8000 (#0)
      * ALPN, offering h2
      * ALPN, offering http/1.1
      *  CAfile: /var/run/configmaps/service-ca-bundle/service-ca.crt
      * TLSv1.0 (OUT), TLS header, Certificate Status (22):
      * TLSv1.3 (OUT), TLS handshake, Client hello (1):
      * TLSv1.2 (IN), TLS header, Certificate Status (22):
      * TLSv1.3 (IN), TLS handshake, Server hello (2):
      * TLSv1.2 (IN), TLS header, Finished (20):
      * TLSv1.2 (IN), TLS header, Unknown (23):
      * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
      * TLSv1.2 (IN), TLS header, Unknown (23):
      * TLSv1.3 (IN), TLS handshake, Request CERT (13):
      * TLSv1.2 (IN), TLS header, Unknown (23):
      * TLSv1.3 (IN), TLS handshake, Certificate (11):
      * TLSv1.2 (IN), TLS header, Unknown (23):
      * TLSv1.3 (IN), TLS handshake, CERT verify (15):
      * TLSv1.2 (IN), TLS header, Unknown (23):
      * TLSv1.3 (IN), TLS handshake, Finished (20):
      * TLSv1.2 (OUT), TLS header, Finished (20):
      * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
      * TLSv1.2 (OUT), TLS header, Unknown (23):
      * TLSv1.3 (OUT), TLS handshake, Certificate (11):
      * TLSv1.2 (OUT), TLS header, Unknown (23):
      * TLSv1.3 (OUT), TLS handshake, Finished (20):
      * SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
      * ALPN, server accepted to use h2
      * Server certificate:
      *  subject: CN=*.exporter.openshift-insights.svc
      *  start date: Jan  2 02:19:07 2025 GMT
      *  expire date: Jan  2 02:19:08 2027 GMT
      *  subjectAltName: host "exporter.openshift-insights.svc.cluster.local" matched cert's "exporter.openshift-insights.svc.cluster.local"
      *  issuer: CN=openshift-service-serving-signer@1735784302
      *  SSL certificate verify ok.
      * Using HTTP2, server supports multi-use
      * Connection state changed (HTTP/2 confirmed)
      * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
      * TLSv1.2 (OUT), TLS header, Unknown (23):
      * TLSv1.2 (OUT), TLS header, Unknown (23):
      * TLSv1.2 (OUT), TLS header, Unknown (23):
      * Using Stream ID: 1 (easy handle 0x5577a19094a0)
      * TLSv1.2 (OUT), TLS header, Unknown (23):
      > GET /gather_runtime_info HTTP/2
      > Host: exporter.openshift-insights.svc.cluster.local:8000
      > accept: */*
      > user-agent: insights-operator/one10time200gather184a34f6a168926d93c330 cluster/_f19625f5-ee5f-40c0-bc49-23a8ba1abe61_
      > authorization: Bearer sha256~x9jj_SnjJf6LVlhhWFdUG8UqnPDHzZW0xMYa0WU05Gw
      >
      * TLSv1.2 (IN), TLS header, Unknown (23):
      * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
      * TLSv1.2 (IN), TLS header, Unknown (23):
      * TLSv1.2 (OUT), TLS header, Unknown (23):
      * TLSv1.2 (IN), TLS header, Unknown (23):
      * TLSv1.2 (IN), TLS header, Unknown (23):
      < HTTP/2 500
      < content-type: text/plain; charset=utf-8
      < date: Thu, 02 Jan 2025 08:18:59 GMT
      < x-content-type-options: nosniff
      < content-length: 33
      <
      * TLSv1.2 (IN), TLS header, Unknown (23):
      stat : no such file or directory

      Version-Release number of selected component (if applicable):

          4.19

      How reproducible:

          True

      Steps to Reproduce:

          1. Create a pair of self-signed tls cert and key
          2. Update trusted-ca-bundle by using following cmd:
            oc patch proxy/cluster \ --type=merge \ --patch='{"spec":{"trustedCA":{"name":"custom-ca"}}}'     
          3. Pull a request to insights-runtime-extractor pod via the following cmd:
          curl  -v --cacert  /var/run/configmaps/trusted-ca-bundle/ca-bundle.crt  -H "User-Agent: insights-operator/one10time200gather184a34f6a168926d93c330 cluster/_<cluster_id>_" -H "Authorization: <token>" -H 'Cache-Control: no-cache' https://api.openshift.com/api/accounts_mgmt/v1/certificates     

      Actual results:

          3. The status code of response to this request is 500

      Expected results:

      3. The status code of response to this request should be 200 and return the runtime info as expected.

      Additional info:

          

              jmesnil1@redhat.com Jeff Mesnil
              rh-ee-bazhou baiyang zhou
              Jeff Mesnil Jeff Mesnil
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: