-
Sub-task
-
Resolution: Done
-
Undefined
-
None
-
None
-
None
-
None
-
4
-
False
-
-
False
-
Passed
-
0
-
0.000
-
Very Likely
-
0
-
None
-
Unset
-
Unknown
Description of problem: During a backup using DataMover, the volsync-src-vsb may get stuck on ContainerCreating for some unknown reason at the moment.
Happened mostly with MSSQL app, but not sure exactly if related
Version-Release number of selected component (if applicable): downstream build 1.1.0-41
How reproducible: can happen a lot, not sure exactly what is the trigger.
Steps to Reproduce:
1.Install Volsync operator
2. Create Volumesnapshotclass
3. Deploy DPA with DataMover enabled
4. Deploy MSSQL app
4. Create a backup
Actual results:
Volsync pod is stuck on ContainerCreating, and thus the VSB and ReplicationSource are also stuck on InProgress
[mperetz@mperetz oadp-e2e-qe]$ oc get pods -n openshift-adp NAME READY STATUS RESTARTS AGE openshift-adp-controller-manager-7b59c9fd84-nspb4 1/1 Running 0 58m restic-64t6l 1/1 Running 0 14m restic-krtnv 1/1 Running 0 14m restic-prnz6 1/1 Running 0 14m velero-b6964fc5-s64g4 1/1 Running 0 14m volsync-src-vsb-velero-mssql-pvc-ntx6s-rep-src-fdxnq 0/1 ContainerCreating 0 10m volume-snapshot-mover-64cdcf4b97-txlgk 1/1 Running 0 14m vsb-velero-mssql-pvc-ntx6s-pod 1/1 Running 0 11m [mperetz@mperetz oadp-e2e-qe]$ oc adm must-gather
VSB:
[mperetz@mperetz oadp-e2e-qe]$ oc get vsb -A -o yaml apiVersion: v1 items: - apiVersion: datamover.oadp.openshift.io/v1alpha1 kind: VolumeSnapshotBackup metadata: creationTimestamp: "2022-07-27T12:29:33Z" generation: 1 labels: velero.io/backup-name: mssql-6fe27969-0da7-11ed-9387-902e163f806c name: vsb-velero-mssql-pvc-ntx6s namespace: mssql-persistent resourceVersion: "1199123" uid: 018bce9a-e5b6-4aba-92f5-506e42c14435 spec: protectedNamespace: openshift-adp volumeSnapshotContent: name: snapcontent-d9b2fa24-87af-45ce-b0a4-d357352771e0 status: phase: InProgress resticrepository: s3:http://s3.openshift-storage.svc.cluster.local:80/oadpbucket124736/openshift-adp/snapcontent-d9b2fa24-87af-45ce-b0a4-d357352771e0-pvc sourcePVCData: name: mssql-pvc size: 10Gi storageClassName: ocs-storagecluster-ceph-rbd volumeSnapshotClassName: example-snapclass kind: List metadata: resourceVersion: "" selfLink: ""
ReplicationSource:
[mperetz@mperetz oadp-e2e-qe]$ oc get replicationsource -A -o yaml
apiVersion: v1
items:
- apiVersion: volsync.backube/v1alpha1
kind: ReplicationSource
metadata:
creationTimestamp: "2022-07-27T12:31:13Z"
generation: 1
labels:
datamover.oadp.openshift.io/vsb: vsb-velero-mssql-pvc-ntx6s
name: vsb-velero-mssql-pvc-ntx6s-rep-src
namespace: openshift-adp
resourceVersion: "1197306"
uid: 83ab4711-5ce4-46bd-baa8-f7f68486ef6d
spec:
restic:
copyMethod: None
repository: vsb-velero-mssql-pvc-ntx6s-secret
sourcePVC: snapcontent-d9b2fa24-87af-45ce-b0a4-d357352771e0-pvc
trigger:
manual: vsb-velero-mssql-pvc-ntx6s-trigger
status:
conditions:
- lastTransitionTime: "2022-07-27T12:31:13Z"
message: Synchronization in-progress
reason: SyncInProgress
status: "True"
type: Synchronizing
- lastTransitionTime: "2022-07-27T12:31:14Z"
message: Reconcile complete
reason: ReconcileComplete
status: "True"
type: Reconciled
lastSyncStartTime: "2022-07-27T12:31:13Z"
restic: {}
kind: List
metadata:
resourceVersion: ""
selfLink: ""
VolumeSnapshot:
[mperetz@mperetz oadp-e2e-qe]$ oc get volumesnapshot -A -o yaml apiVersion: v1 items: - apiVersion: snapshot.storage.k8s.io/v1 kind: VolumeSnapshot metadata: creationTimestamp: "2022-07-27T12:30:13Z" finalizers: - snapshot.storage.kubernetes.io/volumesnapshot-as-source-protection generation: 1 labels: datamover.oadp.openshift.io/vsb: vsb-velero-mssql-pvc-ntx6s name: snapcontent-d9b2fa24-87af-45ce-b0a4-d357352771e0-clone-volumesnapshot namespace: openshift-adp resourceVersion: "1193574" uid: b9d4a34c-e7d8-4937-8ddf-42357464d054 spec: source: volumeSnapshotContentName: snapcontent-d9b2fa24-87af-45ce-b0a4-d357352771e0-clone status: boundVolumeSnapshotContentName: snapcontent-d9b2fa24-87af-45ce-b0a4-d357352771e0-clone creationTime: "2022-07-27T12:29:43Z" readyToUse: true restoreSize: "0" kind: List metadata: resourceVersion: "" selfLink: ""
Expected results:
Additional info:
Volumesnapshotclass/volumesnapshotcontent can be found under the attached general-must-gather.tar.xz file
OADP must-gather logs can be found under the must-gather.tar.gz