-
Bug
-
Resolution: Unresolved
-
Major
-
4.13.0
-
Quality / Stability / Reliability
-
False
-
-
3
-
Important
-
No
-
None
-
Rejected
-
OTA 248, OTA 255, OTA 258, OTA 259, Cincinnati Backlog
-
5
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Description of problem:
Fail to create osus instance on a cluster with cluster-wide ca cert due to cluster-wide cert is not picked up.
# ./oc -n osus get po
NAME READY STATUS RESTARTS AGE
graph-data-tag-digest 1/1 Running 0 56s
sample-74dffbdffd-9nrx9 0/2 Running 1 (2m2s ago) 6m6s
sample-7c995c9565-4sb29 0/2 Running 1 (115s ago) 5m59s
updateservice-operator-5fb779d67b-dn5bk 1/1 Running 0 6m22s
# ./oc -n osus logs sample-74dffbdffd-9nrx9
...
[2023-08-29T08:33:48Z INFO graph_builder::graph] graph update triggered
[2023-08-29T08:33:48Z TRACE cincinnati::plugins] Running next plugin 'release-scrape-dockerv2'
[2023-08-29T08:33:48Z ERROR graph_builder::graph] failed to fetch all release metadata from ec2-13-59-235-86.us-east-2.compute.amazonaws.com:5000/osus/ocp-release
[2023-08-29T08:33:48Z ERROR graph_builder::graph] http transport error: error sending request for url (https://ec2-13-59-235-86.us-east-2.compute.amazonaws.com:5000/v2/): error trying to connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1915: (unable to get local issuer certificate)
[2023-08-29T08:33:48Z ERROR graph_builder::graph] error sending request for url (https://ec2-13-59-235-86.us-east-2.compute.amazonaws.com:5000/v2/): error trying to connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1915: (unable to get local issuer certificate)
[2023-08-29T08:33:48Z ERROR graph_builder::graph] error trying to connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1915: (unable to get local issuer certificate)
[2023-08-29T08:33:48Z ERROR graph_builder::graph] error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1915: (unable to get local issuer certificate)
[2023-08-29T08:33:48Z ERROR graph_builder::graph] error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1915:
# ./oc -n osus get cm
NAME DATA AGE
kube-root-ca.crt 1 7m11s
openshift-service-ca.crt 1 7m11s
sample-config 1 6m30s
sample-env 7 6m30s
//as above, no cluster-trusted-ca cm created in osus ns, and so no cert mounted.
# ./oc -n osus get po sample-74dffbdffd-9nrx9 -oyaml|grep mountPath:
- mountPath: /etc/configs
- mountPath: /var/lib/cincinnati/graph-data
- mountPath: /var/lib/cincinnati/registry-credentials
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
- mountPath: /var/lib/cincinnati/graph-data
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
//But there is indeed cluster wide trusted-ca in proxy/cluster.
# ./oc get proxy cluster -ojson|jq .spec
{
"trustedCA": {
"name": "user-ca-bundle"
}
}
# ./oc -n openshift-config get cm user-ca-bundle -ojson|jq .data
{
"ca-bundle.crt": ...
}
And in another image-registry pod, the cert is mounted correctly.
# ./oc -n openshift-image-registry get po image-registry-967cb4d66-thqkx -oyaml|grep mountPath:
- mountPath: /var/run/secrets/cloud
- mountPath: /etc/secrets
- mountPath: /etc/pki/ca-trust/extracted
- mountPath: /etc/pki/ca-trust/source/anchors
- mountPath: /usr/share/pki/ca-trust-source
- mountPath: /var/lib/kubelet/
- mountPath: /var/run/secrets/openshift/serviceaccount
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
Version-Release number of selected component (if applicable):
OCP 4.13.9 osus 5.0.2
How reproducible:
always
Steps to Reproduce:
1. Trigger installation with "additionalTrustBundlePolicy: Always" and "additionalTrustBundle" set in install-config.yaml, but no proxy configured. (the additionalTrustBundle is the CA cert signing private registry cert too) #cat install-config.yaml ... additionalTrustBundle: | -----BEGIN CERTIFICATE----- MIIFqTCCA5GgAwIBAgICEAAwDQYJKoZIhvcNAQELBQAwaTELMAkGA1UEBhMCQ04x -----END CERTIFICATE----- -----BEGIN CERTIFICATE----- -----END CERTIFICATE----- additionalTrustBundlePolicy: Always ... 2. Create osus operator 3. Create osus instance
Actual results:
Fail to install osus instance
Expected results:
OSUS instance can be installed successfully
Additional info:
- relates to
-
OCPBUGS-55463 OSUS Operator - graph-builder container does start due to unable to get issuer certificate
-
- New
-
- links to