Uploaded image for project: 'OpenShift API for Data Protection'
  1. OpenShift API for Data Protection
  2. OADP-1161

Incorporate restic unlock feature from volsync

XMLWordPrintable

    • False
    • Hide

      None

      Show
      None
    • False
    • QE - Ack
    • ToDo
    • No
    • 0
    • 0
    • Very Likely
    • 0
    • None
    • Unset
    • Unknown

      Delete backup using  velero   :   

      velero backup delete single-ns-5000-pods-restic-rbd-backup-adp-1152new
      or 
      velero backup delete --all  

      this operation create a new 'Deletebackuprequests' CR which failed to delete the backup 
      the 'Deletebackuprequests'  STATUS is 'Processed' and stuck for ever  the backup status is 'Deleting'
      OADP version v1.2.0 iib:387699
      VELERO version v1.9.2

      the backup : 

      Name:         single-ns-5000-pods-restic-rbd-backup-adp-1152new
      Namespace:    openshift-adp
      Labels:       velero.io/storage-location=example-velero-1
      Annotations:  velero.io/source-cluster-k8s-gitversion: v1.24.0+3882f8f
                    velero.io/source-cluster-k8s-major-version: 1
                    velero.io/source-cluster-k8s-minor-version: 24
      API Version:  velero.io/v1
      Kind:         Backup
      Metadata:
        Creation Timestamp:  2023-01-09T12:33:43Z
        Generation:          2
        Managed Fields:
          API Version:  velero.io/v1
          Fields Type:  FieldsV1
          fieldsV1:
            f:metadata:
              f:annotations:
                .:
                f:velero.io/source-cluster-k8s-gitversion:
                f:velero.io/source-cluster-k8s-major-version:
                f:velero.io/source-cluster-k8s-minor-version:
              f:labels:
                .:
                f:velero.io/storage-location:
            f:spec:
              .:
              f:csiSnapshotTimeout:
              f:defaultVolumesToFsBackup:
              f:hooks:
              f:includedNamespaces:
              f:metadata:
              f:snapshotVolumes:
              f:storageLocation:
              f:ttl:
            f:status:
              .:
              f:completionTimestamp:
              f:errors:
              f:expiration:
              f:formatVersion:
              f:phase:     Manager:         velero-server
          Operation:       Update
          Time:            2023-01-09T12:33:43Z
        Resource Version:  80769692
        UID:               81aa583f-3fc2-4459-89a1-c3d17e7ba456
      
      
       Spec:
        Csi Snapshot Timeout:          10m0s
        Default Volumes To Fs Backup:  true
        Hooks:
        Included Namespaces:
          busybox-perf-single-ns-5000-pods
        Metadata:
        Snapshot Volumes:  false
        Storage Location:  example-velero-1
        Ttl:               720h0m0s
      Status:
        Completion Timestamp:  2023-01-09T06:14:55Z
        Errors:                3365
        Expiration:            2023-02-07T19:39:52Z
        Format Version:        1.1.0
        Phase:                 Deleting
        Progress:
          Items Backed Up:  40022
          Total Items:      40022
        Start Timestamp:    2023-01-08T19:39:52Z
        Version:            1
      Events:               <none>
      

       

      the Deletebackuprequests : 

       Name:         single-ns-5000-pods-restic-rbd-backup-adp-1152new-84w9z
      Namespace:    openshift-adp
      Labels:       velero.io/backup-name=single-ns-5000-pods-restic-rbd-backup-adp-1152new
                    velero.io/backup-uid=81aa583f-3fc2-4459-89a1-c3d17e7ba456
      Annotations:  <none>
      API Version:  velero.io/v1
      Kind:         DeleteBackupRequest
      Metadata:
        Creation Timestamp:  2023-01-09T12:39:29Z
        Generate Name:       single-ns-5000-pods-restic-rbd-backup-adp-1152new-
        Generation:          4
        Managed Fields:
          API Version:  velero.io/v1
          Fields Type:  FieldsV1
          fieldsV1:
            f:metadata:
              f:generateName:
              f:labels:
                .:
                f:velero.io/backup-name:
                f:velero.io/backup-uid:
            f:spec:
              .:
              f:backupName:
            f:status:
          Manager:      velero
          Operation:    Update
          Time:         2023-01-09T12:39:29Z
          API Version:  velero.io/v1
          Fields Type:  FieldsV1
          fieldsV1:
            f:status:
              f:errors:
              f:phase:
          Manager:         velero-server
          Operation:       Update
          Time:            2023-01-09T12:55:41Z
           API Version:  velero.io/v1
           Fields Type:  FieldsV1
           fieldsV1:
             f:status:
               f:errors:
               f:phase:
           Manager:         velero-server
           Operation:       Update
           Time:            2023-01-09T12:55:41Z
         Resource Version:  80825461
         UID:               5ec047cb-2ab8-4304-b2fa-affef9a56d16
       Spec:
         Backup Name:  single-ns-5000-pods-restic-rbd-backup-adp-1152new
       Status:
         Errors:
           error running command=restic forget --repo=s3:https://s3-openshift-storage.apps.vlan608.rdu2.scalelab.redhat.com/oadp-bucket/velero/restic/busybox-perf-single-ns-5
       000-pods --password-file=/tmp/credentials/openshift-adp/velero-repo-credentials-repository-password --cache-dir=/scratch/.cache/restic d4d6a4c2 --insecure-tls=true, st
       dout=, stderr=unable to create lock in backend: repository is already locked exclusively by PID 695052 on velero-5bbdd484c9-7tsfw by 1000760000 (UID 1000760000, GID 0)
       lock was created at 2023-01-09 12:41:28 (14m13.399648571s ago)
       storage ID 88fe4233
       the `unlock` command can be used to remove stale locks
       : exit status 1
           error running command=restic forget --repo=s3:https://s3-openshift-storage.apps.vlan608.rdu2.scalelab.redhat.com/oadp-bucket/velero/restic/busybox-perf-single-ns-5
       000-pods --password-file=/tmp/credentials/openshift-adp/velero-repo-credentials-repository-password --cache-dir=/scratch/.cache/restic 591837c5 --insecure-tls=true, st
       dout=, stderr=unable to create lock in backend: repository is already locked exclusively by PID 695052 on velero-5bbdd484c9-7tsfw by 1000760000 (UID 1000760000, GID 0)
       lock was created at 2023-01-09 12:41:28 (14m13.997269869s ago)
       storage ID 88fe4233
       the `unlock` command can be used to remove stale locks
       : exit status 1
           error running command=restic forget --repo=s3:https://s3-openshift-storage.apps.vlan608.rdu2.scalelab.redhat.com/oadp-bucket/velero/restic/busybox-perf-single-ns-5
       000-pods --password-file=/tmp/credentials/openshift-adp/velero-repo-credentials-repository-password --cache-dir=/scratch/.cache/restic e985f2df --insecure-tls=true, st
       dout=, stderr=unable to create lock in backend: repository is already locked exclusively by PID 695052 on velero-5bbdd484c9-7tsfw by 1000760000 (UID 1000760000, GID 0)
       lock was created at 2023-01-09 12:41:28 (14m14.595544974s ago)
       storage ID 88fe4233
          error running command=restic forget --repo=s3:https://s3-openshift-storage.apps.vlan608.rdu2.scalelab.redhat.com/oadp-bucket/velero/restic/busybox-perf-single-ns-5000-pods --password-file=/tmp/credentials/openshift-adp/velero-repo-credentials-repository-password --cache-dir=/scratch/.cache/restic b9c614b7 --insecure-tls=true, stdout=, stderr=unable to create lock in backend: repository is already locked exclusively by PID 695052 on velero-5bbdd484c9-7tsfw by 1000760000 (UID 1000760000, GID 0)
      lock was created at 2023-01-09 12:41:28 (28m11.702539451s ago)
      storage ID 88fe4233
      the `unlock` command can be used to remove stale locks
      : exit status 1
          error running command=restic forget --repo=s3:https://s3-openshift-storage.apps.vlan608.rdu2.scalelab.redhat.com/oadp-bucket/velero/restic/busybox-perf-single-ns-5000-pods --password-file=/tmp/credentials/openshift-adp/velero-repo-credentials-repository-password --cache-dir=/scratch/.cache/restic 8ddba7c8 --insecure-tls=true, stdout=, stderr=unable to create lock in backend: repository is already locked exclusively by PID 695052 on velero-5bbdd484c9-7tsfw by 1000760000 (UID 1000760000, GID 0)
      lock was created at 2023-01-09 12:41:28 (28m12.171337881s ago)
      storage ID 88fe4233
      the `unlock` command can be used to remove stale locks
      : exit status 1
          error running command=restic forget --repo=s3:https://s3-openshift-storage.apps.vlan608.rdu2.scalelab.redhat.com/oadp-bucket/velero/restic/busybox-perf-single-ns-5000-pods --password-file=/tmp/credentials/openshift-adp/velero-repo-credentials-repository-password --cache-dir=/scratch/.cache/restic 451758e2 --insecure-tls=true, stdout=, stderr=unable to create lock in backend: repository is already locked exclusively by PID 695052 on velero-5bbdd484c9-7tsfw by 1000760000 (UID 1000760000, GID 0)
      lock was created at 2023-01-09 12:41:28 (28m12.657032469s ago)
      storage ID 88fe4233
      the `unlock` command can be used to remove stale locks
      : exit status 1
          error running command=restic forget --repo=s3:https://s3-openshift-storage.apps.vlan608.rdu2.scalelab.redhat.com/oadp-bucket/velero/restic/busybox-perf-single-ns-5000-pods --password-file=/tmp/credentials/openshift-adp/velero-repo-credentials-repository-password --cache-dir=/scratch/.cache/restic 513c0377 --insecure-tls=true, stdout=, stderr=unable to create lock in backend: repository is already locked exclusively by PID 695052 on velero-5bbdd484c9-7tsfw by 1000760000 (UID 1000760000, GID 0)
      lock was created at 2023-01-09 12:41:28 (28m13.220450893s ago)
      storage ID 88fe4233
      the `unlock` command can be used to remove stale locks
      : exit status 1
          error running command=restic forget --repo=s3:https://s3-openshift-storage.apps.vlan608.rdu2.scalelab.redhat.com/oadp-bucket/velero/restic/busybox-perf-single-ns-5000-pods --password-file=/tmp/credentials/openshift-adp/velero-repo-credentials-repository-password --cache-dir=/scratch/.cache/restic bc468326 --insecure-tls=true, stdout=, stderr=unable to create lock in backend: repository is already locked exclusively by PID 695052 on velero-5bbdd484c9-7tsfw by 1000760000 (UID 1000760000, GID 0)
      lock was created at 2023-01-09 12:41:28 (28m13.7823843s ago)
      storage ID 88fe4233
      the `unlock` command can be used to remove stale locks
      : exit status 1
        Phase:  Processed
      Events:   <none>
       

       

       

            spampatt@redhat.com Shubham Pampattiwar
            tzahia Tzahi Ashkenazi
            Tzahi Ashkenazi Tzahi Ashkenazi
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: