Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-18280

osus does not respect the trustedca in proxy/cluster resource when http/https is not set

XMLWordPrintable

    • Important
    • No
    • 3
    • OTA 248, OTA 255, OTA 258, OTA 259
    • 4
    • Rejected
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      Fail to create osus instance on a cluster with cluster-wide ca cert due to cluster-wide cert is not picked up.
      
      # ./oc -n osus get po
      NAME                                      READY   STATUS    RESTARTS       AGE
      graph-data-tag-digest                     1/1     Running   0              56s
      sample-74dffbdffd-9nrx9                   0/2     Running   1 (2m2s ago)   6m6s
      sample-7c995c9565-4sb29                   0/2     Running   1 (115s ago)   5m59s
      updateservice-operator-5fb779d67b-dn5bk   1/1     Running   0              6m22s
      
      # ./oc -n osus logs sample-74dffbdffd-9nrx9
      ...
      [2023-08-29T08:33:48Z INFO  graph_builder::graph] graph update triggered
      [2023-08-29T08:33:48Z TRACE cincinnati::plugins] Running next plugin 'release-scrape-dockerv2'
      [2023-08-29T08:33:48Z ERROR graph_builder::graph] failed to fetch all release metadata from ec2-13-59-235-86.us-east-2.compute.amazonaws.com:5000/osus/ocp-release
      [2023-08-29T08:33:48Z ERROR graph_builder::graph] http transport error: error sending request for url (https://ec2-13-59-235-86.us-east-2.compute.amazonaws.com:5000/v2/): error trying to connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1915: (unable to get local issuer certificate)
      [2023-08-29T08:33:48Z ERROR graph_builder::graph] error sending request for url (https://ec2-13-59-235-86.us-east-2.compute.amazonaws.com:5000/v2/): error trying to connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1915: (unable to get local issuer certificate)
      [2023-08-29T08:33:48Z ERROR graph_builder::graph] error trying to connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1915: (unable to get local issuer certificate)
      [2023-08-29T08:33:48Z ERROR graph_builder::graph] error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1915: (unable to get local issuer certificate)
      [2023-08-29T08:33:48Z ERROR graph_builder::graph] error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1915:
      
      # ./oc -n osus get cm
      NAME                       DATA   AGE
      kube-root-ca.crt           1      7m11s
      openshift-service-ca.crt   1      7m11s
      sample-config              1      6m30s
      sample-env                 7      6m30s
      
      //as above, no cluster-trusted-ca cm created in osus ns, and so no cert mounted.
      # ./oc -n osus get po sample-74dffbdffd-9nrx9 -oyaml|grep mountPath:
          - mountPath: /etc/configs
          - mountPath: /var/lib/cincinnati/graph-data
          - mountPath: /var/lib/cincinnati/registry-credentials
          - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
          - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
          - mountPath: /var/lib/cincinnati/graph-data
          - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      
      //But there is indeed cluster wide trusted-ca in proxy/cluster.
      # ./oc get proxy cluster -ojson|jq .spec
      {
        "trustedCA": {
          "name": "user-ca-bundle"
        }
      }
      
      # ./oc -n openshift-config get cm user-ca-bundle -ojson|jq .data
      {
        "ca-bundle.crt": ...
      }
      
      And in another image-registry pod, the cert is mounted correctly.
      # ./oc -n openshift-image-registry get po image-registry-967cb4d66-thqkx -oyaml|grep mountPath:
          - mountPath: /var/run/secrets/cloud
          - mountPath: /etc/secrets
          - mountPath: /etc/pki/ca-trust/extracted
          - mountPath: /etc/pki/ca-trust/source/anchors
          - mountPath: /usr/share/pki/ca-trust-source
          - mountPath: /var/lib/kubelet/
          - mountPath: /var/run/secrets/openshift/serviceaccount
          - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      

      Version-Release number of selected component (if applicable):

      OCP 4.13.9
      osus 5.0.2

      How reproducible:

      always

      Steps to Reproduce:

      1. Trigger installation with "additionalTrustBundlePolicy: Always" and "additionalTrustBundle" set in install-config.yaml, but no proxy configured. (the additionalTrustBundle is the CA cert signing private registry cert too)
      #cat install-config.yaml
      ...
      additionalTrustBundle: |
      -----BEGIN CERTIFICATE-----
      MIIFqTCCA5GgAwIBAgICEAAwDQYJKoZIhvcNAQELBQAwaTELMAkGA1UEBhMCQ04x
      -----END CERTIFICATE-----
      -----BEGIN CERTIFICATE-----
      -----END CERTIFICATE-----
      additionalTrustBundlePolicy: Always
      ...
      
      2. Create osus operator
      3. Create osus instance
      

      Actual results:

      Fail to install osus instance

      Expected results:

      OSUS instance can be installed successfully

      Additional info:

       

              pratikam Pratik Mahajan
              rhn-support-jiajliu Jia Liu
              Jia Liu Jia Liu
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Created:
                Updated: