Uploaded image for project: 'OpenShift API for Data Protection'
  1. OpenShift API for Data Protection
  2. OADP-1398

[OADP] Velero has wrong timezone and we are unable to change it

XMLWordPrintable

    • False
    • Hide

      None

      Show
      None
    • False
    • QE - Ack
    • oadp-velero-container-1.1.4-1
    • ToDo
    • 0
    • 0
    • Very Likely
    • 0
    • Customer Escalated, Customer Facing
    • None
    • Unset
    • Unknown
    • No

      Validation Steps

      Set env TZ like so to desired timezone.

      apiVersion: oadp.openshift.io/v1alpha1
kind: DataProtectionApplication
metadata:
  name: dpa-sample
  namespace: openshift-adp
spec:
  configuration:
    velero:
      podConfig:
        env:
          - name: TZ
            value: America/New_York

      Notice in velero pod logs that timestamps now include offsets from UTC seen here `-04:00`

      time="2023-03-14T10:57:24-04:00" level=info msg="Starting Velero server v1.9.5-OADP (-)" logSource="/remote-source/velero/app/pkg/cmd/server/server.go:183"

      Create a schedule
      Notice in velero pod logs that backup generated name has the exact same hour/minute numbers
      Seen below 2023-03-14T11:00:04 matches up with 20230314110004

      time="2023-03-14T11:00:04-04:00" level=info msg="No Schedule.template.metadata.labels set - using Schedule.labels for backup object" backup=openshift-adp/schedule-20230314110004 labels="map[]"

      Description of problem:

      In this case: TimeZone

      Customer Issue is here: Please read! https://access.redhat.com/support/cases/#/case/03405780
      https://github.com/vmware-tanzu/velero/pull/2944

       

      We could ensure the oadp-controller also has the TZ config or we could better handle any env config available to velero container running in docker.

       

      Notes from Emily:
      Emily  12:34 PM

      I wonder if we could append it to the velero container here: https://github.com/openshift/oadp-operator/blob/master/controllers/velero.go#L486 (edited) 
       

      maybe could do something like add the value to the OADP api in the velero config (what Dylan pointed out). Write a func to check whether or not that value is set. If so, append that env var with the value in DPA to the velero container
       
      Notes from Dylan:
      https://github.com/openshift/oadp-operator/blob/master/api/v1alpha1/oadp_types.go#L66
       
       
      basically just add it here
       

      Which version we'll fix this in is TBD.

          1.
          		 [RedHat QE] Verify Bug OADP-1150 - Handle docker env config changes in the oadp-operator Sub-task Closed Undefined Shahaf Bahar
          2.
          		 [IBM QE-P] Verify Bug OADP-1150 - Handle docker env config changes in the oadp-operator Sub-task Release Pending Undefined Sonia Garudi
          3.
          		 [IBM QE-Z] Verify Bug OADP-1150 - Handle docker env config changes in the oadp-operator Sub-task Release Pending Undefined SHIVA SAI K (Inactive)

            [OADP-1398] [OADP] Velero has wrong timezone and we are unable to change it

            Errata Tool added a comment -

            Since the problem described in this issue should be resolved in a recent advisory, it has been closed.

            For information on the advisory (OpenShift API for Data Protection (OADP) 1.1.4 security and bug fix update), and where to find the updated files, follow the link below.

            If the solution does not work for you, open a new bug report.
            https://access.redhat.com/errata/RHBA-2023:3227

            Errata Tool added a comment - Since the problem described in this issue should be resolved in a recent advisory, it has been closed. For information on the advisory (OpenShift API for Data Protection (OADP) 1.1.4 security and bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2023:3227

            Tiger Kaovilai added a comment -

            Validation Steps

            Set env TZ like so to desired timezone.

            apiVersion: oadp.openshift.io/v1alpha1
kind: DataProtectionApplication
metadata:
  name: dpa-sample
  namespace: openshift-adp
spec:
  configuration:
    velero:
      podConfig:
        env:
          - name: TZ
            value: America/New_York

            Notice in velero pod logs that timestamps now include offsets from UTC seen here `-04:00`

            time="2023-03-14T10:57:24-04:00" level=info msg="Starting Velero server v1.9.5-OADP (-)" logSource="/remote-source/velero/app/pkg/cmd/server/server.go:183"

            Create a schedule
            Notice in velero pod logs that backup generated name has the exact same hour/minute numbers
            Seen below 2023-03-14T11:00:04 matches up with 20230314110004

            time="2023-03-14T11:00:04-04:00" level=info msg="No Schedule.template.metadata.labels set - using Schedule.labels for backup object" backup=openshift-adp/schedule-20230314110004 labels="map[]"

            Tiger Kaovilai added a comment - Validation Steps Set env TZ like so to desired timezone. apiVersion: oadp.openshift.io/v1alpha1 kind: DataProtectionApplication metadata: name: dpa-sample namespace: openshift-adp spec: configuration: velero: podConfig: env: - name: TZ value: America/New_York Notice in velero pod logs that timestamps now include offsets from UTC seen here `-04:00` time= "2023-03-14T10:57:24-04:00" level=info msg= "Starting Velero server v1.9.5-OADP (-)" logSource= "/remote-source/velero/app/pkg/cmd/server/server.go:183" Create a schedule Notice in velero pod logs that backup generated name has the exact same hour/minute numbers Seen below 2023-03-14T11:00:04 matches up with 20230314110004 time= "2023-03-14T11:00:04-04:00" level=info msg= "No Schedule.template.metadata.labels set - using Schedule.labels for backup object" backup=openshift-adp/schedule-20230314110004 labels= "map[]"

            Wes Hayutin added a comment -

            moving back to assigned until dev posts validation steps and proof this is working in 1.1.3

            Wes Hayutin added a comment - moving back to assigned until dev posts validation steps and proof this is working in 1.1.3

            Tiger Kaovilai added a comment -

            Moved to modified. Further testing reveals that latest velero downstream container now has the fixes required. Upstream PRs in velero is not necessary.

            Tiger Kaovilai added a comment - Moved to modified. Further testing reveals that latest velero downstream container now has the fixes required. Upstream PRs in velero is not necessary.

            Tiger Kaovilai added a comment -

            PR for release-1.9 https://github.com/vmware-tanzu/velero/pull/5979

            Until this is rebased or cherry-picked into oadp-1.1 this issues cannot be moved to modified.

            Tiger Kaovilai added a comment - PR for release-1.9 https://github.com/vmware-tanzu/velero/pull/5979 Until this is rebased or cherry-picked into oadp-1.1 this issues cannot be moved to modified.

            Tiger Kaovilai added a comment -

            Filed upstream issue
            https://github.com/vmware-tanzu/velero/issues/5976

            Tiger Kaovilai added a comment - Filed upstream issue https://github.com/vmware-tanzu/velero/issues/5976

            Guillermo Ojea Quintana added a comment -

            Please note that this also happens with the restic pods.

            Thanks!

            Guillermo Ojea Quintana added a comment - Please note that this also happens with the restic pods. Thanks!

            Wes Hayutin added a comment -

            rjohnson@redhat.com I ran the same test.. however now I'm attempting to see the velero log stamp change tz. Which is the crux of the issue.

            Wes Hayutin added a comment - rjohnson@redhat.com I ran the same test.. however now I'm attempting to see the velero log stamp change tz. Which is the crux of the issue.

            Wes Hayutin added a comment -

            These two help articles may speak the issue.
            https://access.redhat.com/solutions/2567961
            https://access.redhat.com/solutions/5616681

            Still investigating

            Wes Hayutin added a comment - These two help articles may speak the issue. https://access.redhat.com/solutions/2567961 https://access.redhat.com/solutions/5616681 Still investigating

            Wes Hayutin added a comment -

            bash-4.4$ TZ='UTC'; export TZ
            bash-4.4$ date
            Thu Mar 9 20:54:43 UTC 2023
            bash-4.4$ TZ='America/Argentina/Buenos_Aires'; export TZ
            bash-4.4$ date
            Thu Mar 9 20:54:51 America 2023

            it doesn't change the time, but the timezone is changed.

            Wes Hayutin added a comment - bash-4.4$ TZ='UTC'; export TZ bash-4.4$ date Thu Mar 9 20:54:43 UTC 2023 bash-4.4$ TZ='America/Argentina/Buenos_Aires'; export TZ bash-4.4$ date Thu Mar 9 20:54:51 America 2023 it doesn't change the time, but the timezone is changed.

              tkaovila@redhat.com Tiger Kaovilai
              wnstb Wes Hayutin
              Shahaf Bahar Shahaf Bahar
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Created:
                Updated:
                Resolved: