-
Bug
-
Resolution: Done
-
Critical
-
OADP 1.1.0
-
False
-
-
False
-
QE - Ack
-
oadp-velero-plugin-for-vsm-container-1.2.0-13, oadp-volume-snapshot-mover-container-1.2.0-27, oadp-operator-container-1.2.0-32
-
ToDo
-
0
-
0
-
Very Likely
-
0
-
None
-
Unset
-
Unknown
-
Proposed
-
No
Description of problem: TLS-verification related config doesn't apply on volsync:
- setting insecureSkipTLSVerify: 'true' doesn't apply using Volsync CSI backup and a secured URL.
- setting caCert and insecureSkipTLSVerify: 'false' also doesn't apply
In both cases, volsync-src-vsb pods error with "x509: certificate signed by unknown authority".
Version-Release number of selected component (if applicable):
OADP 1.1
Build: 1.1.0-37
How reproducible: 100%
Steps to Reproduce:
1. Create secret, restic-secret and DPA:
apiVersion: v1
items:
- apiVersion: oadp.openshift.io/v1alpha1
kind: DataProtectionApplication
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"oadp.openshift.io/v1alpha1","kind":"DataProtectionApplication","metadata":{"annotations":{},"name":"example-velero","namespace":"openshift-adp"},"spec":{"backupLocations":[{"name":"default","velero":{"config":{"insecureSkipTLSVerify":"true","profile":"noobaa","region":"noobaa","s3ForcePathStyle":"true","s3Url":"https://s3-openshift-storage.apps.oadp-13370.0722-xz8.qe.rhcloud.com"},"credential":{"key":"cloud","name":"cloud-credentials"},"default":true,"objectStorage":{"bucket":"oadpbucket123227","prefix":"velero"},"provider":"aws"}}],"configuration":{"restic":{"enable":true},"velero":{"defaultPlugins":["openshift","csi","aws"]}},"features":{"enableDataMover":true}}}
creationTimestamp: "2022-07-22T15:51:18Z"
generation: 1
name: example-velero
namespace: openshift-adp
resourceVersion: "1510829"
uid: b81798e9-9ce8-4b28-a0c3-0e9efb3b9033
spec:
backupLocations:
- velero:
config:
insecureSkipTLSVerify: "true"
profile: noobaa
region: noobaa
s3ForcePathStyle: "true"
s3Url: https://s3-openshift-storage.apps.oadp-13370.0722-xz8.qe.rhcloud.com
credential:
key: cloud
name: cloud-credentials
default: true
objectStorage:
bucket: oadpbucket123227
prefix: velero
provider: aws
configuration:
restic:
enable: true
velero:
defaultPlugins:
- openshift
- csi
- aws
features:
enableDataMover: true
status:
conditions:
- lastTransitionTime: "2022-07-22T15:51:18Z"
message: Reconcile complete
reason: Complete
status: "True"
type: Reconciled
kind: List
metadata:
resourceVersion: ""
selfLink: ""
2. Create VSC.
apiVersion: snapshot.storage.k8s.io/v1 deletionPolicy: Retain driver: openshift-storage.rbd.csi.ceph.com kind: VolumeSnapshotClass metadata: creationTimestamp: "2022-07-22T12:33:49Z" generation: 1 labels: velero.io/csi-volumesnapshot-class: "true" name: ocs-storagecluster-rbdplugin-snapclass-velero resourceVersion: "1167956" uid: 2bce503a-192b-4b5b-8a85-d3094bcd2aa0 parameters: clusterID: openshift-storage csi.storage.k8s.io/snapshotter-secret-name: rook-csi-rbd-provisioner csi.storage.k8s.io/snapshotter-secret-namespace: openshift-storage
3. deploy an app with PV and create a backup of the app
4. Create CSI backup of the app
Actual results:
volsync-src-vsb pods fail with "certificate signed by unknown authority":
[mperetz@mperetz oadp-qe-automation]$ oc get pods -n openshift-adp NAME READY STATUS RESTARTS AGE openshift-adp-controller-manager-75646957f6-ks5sh 1/1 Running 0 3h28m restic-lsz94 1/1 Running 0 3m57s restic-sl97s 1/1 Running 0 3m57s restic-v4hl6 1/1 Running 0 3m57s velero-8d686754d-zrnjj 1/1 Running 0 3m57s volsync-src-vsb-velero-mysql-df9vx-rep-src-dg6nv 0/1 Error 0 41s volsync-src-vsb-velero-mysql-df9vx-rep-src-r44cf 0/1 Error 0 21s volsync-src-vsb-velero-mysql-df9vx-rep-src-wkmsz 0/1 ContainerCreating 0 1s volume-snapshot-mover-64cdcf4b97-mzspr 1/1 Running 0 3m57s vsb-velero-mysql-df9vx-pod 1/1 Running 0 4m56s [mperetz@mperetz oadp-qe-automation]$ oc logs volsync-src-vsb-velero-mysql-df9vx-rep-src-vmcrt -nopenshift-adp volsync-src-vsb-velero-mysql-df9vx-rep-src-r44cf Error from server (NotFound): pods "volsync-src-vsb-velero-mysql-df9vx-rep-src-vmcrt" not found [mperetz@mperetz oadp-qe-automation]$ oc logs -nopenshift-adp volsync-src-vsb-velero-mysql-df9vx-rep-src-r44cf Starting container VolSync restic container version: ACM-0.4.1-e6dde1b backup Testing mandatory env variables == Checking directory for content === == Initialize Dir ======= Fatal: create repository at s3:https://s3-openshift-storage.apps.oadp-13370.0722-xz8.qe.rhcloud.com/oadpbucket123227/openshift-adp/snapcontent-2e8bb060-640c-4529-9b01-20c847f3a136-pvc failed: client.BucketExists: Get "https://s3-openshift-storage.apps.oadp-13370.0722-xz8.qe.rhcloud.com/oadpbucket123227/?location=": x509: certificate signed by unknown authority[mperetz@mperetz oadp-qe-automation]$
Same issue happens when using caCert and and insecureSkipTLSVerify: 'false'
Expected results: TLS-verification related config should be respected
Additional info:
- links to
- mentioned on
(1 links to, 2 mentioned on)
