Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-17393

On error, ReaR deletes the logs useful to debug NetBackup issues

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done-Errata
    • Icon: Undefined Undefined
    • rhel-9.4
    • rhel-9.3.0
    • rear
    • rear-2.6-22.el9
    • None
    • Moderate
    • rhel-sst-cs-system-management
    • ssg_core_services
    • 24
    • 26
    • 3
    • False
    • Hide

      None

      Show
      None
    • Yes
    • Red Hat Enterprise Linux
    • None
    • Bug Fix
    • Hide
      .ReaR now preserves logs from the `bprestore` command of NetBackup in the rescue system and the recovered system

      Previously, when using the NetBackup integration (`BACKUP=NBU`), ReaR added the log from the `bprestore` command during recovery to a directory that was deleted on exit. Additionally, ReaR did not save further logs produced by the command under the `/usr/openv/netbackup/logs/bprestore/` directory on the recovered system.

      As a consequence, if the `bprestore` command failed during recovery, the logs were deleted unless the `rear recover` command was run with the `-d` or `-D` option. Moreover, even if the recovery finished successfully, the logs under `/usr/openv/netbackup/logs/bprestore/` directory were lost after a reboot and could not be examined.

      With this update, ReaR keeps the log from the `bprestore` command in the `/var/lib/rear/restore` directory in the rescue system where it persists after the `rear recover` command has finished until the rescue system is rebooted. If the system is recovered, all logs from `/usr/openv/netbackup/logs/bprestore/` are copied to the `/var/log/rear/recover/restore` directory together with the log from `/var/lib/rear/restore` in case further examination is required.
      Show
      .ReaR now preserves logs from the `bprestore` command of NetBackup in the rescue system and the recovered system Previously, when using the NetBackup integration (`BACKUP=NBU`), ReaR added the log from the `bprestore` command during recovery to a directory that was deleted on exit. Additionally, ReaR did not save further logs produced by the command under the `/usr/openv/netbackup/logs/bprestore/` directory on the recovered system. As a consequence, if the `bprestore` command failed during recovery, the logs were deleted unless the `rear recover` command was run with the `-d` or `-D` option. Moreover, even if the recovery finished successfully, the logs under `/usr/openv/netbackup/logs/bprestore/` directory were lost after a reboot and could not be examined. With this update, ReaR keeps the log from the `bprestore` command in the `/var/lib/rear/restore` directory in the rescue system where it persists after the `rear recover` command has finished until the rescue system is rebooted. If the system is recovered, all logs from `/usr/openv/netbackup/logs/bprestore/` are copied to the `/var/log/rear/recover/restore` directory together with the log from `/var/lib/rear/restore` in case further examination is required.
    • Done
    • None

      What were you trying to do that didn't work?

      This is a continuation of https://issues.redhat.com/browse/RHEL-17390.
      Due to the issue with NetBackup in above Jira, ReaR tells to check the log file for details:

      ERROR: bprestore failed (return code = 5)
Some latest log messages since the last called script 400_restore_with_nbu.sh:
  2023-11-24 16:19:26.272726847 Including restore/NBU/default/400_restore_with_nbu.sh
  2023-11-24 16:19:26.274085332 NetBackup: restoring / into /mnt/local
  2023-11-24 16:19:26.275589429 RUN: /usr/openv/netbackup/bin/bprestore -B -H -L /var/tmp/rear.oWPDfOxxrtXC3m5/tmp/bplog.restore -8 -R /var/tmp/rear.oWPDfOxxrtXC3m5/tmp/nbu_change_file -t 0 -w 0 -C is
  2023-11-24 16:19:26.276993690 Restore progress: see /var/tmp/rear.oWPDfOxxrtXC3m5/tmp/bplog.restore
  EXIT STATUS 2800: Standard policy restore error
Aborting due to an error, check /var/log/rear/rear-HOSTNAME.log for details

      Unfortunately, unless rear -d is used, the log file is deleted because it's stored on the temporary storage.
      Additionally there is nothing useful in that log at that failure step, the useful log is /usr/openv/netbackup/logs/bprestore/root.112423_00001.log instead.

      Please provide the package NVR for which bug is seen:

      rear-2.6-19.el9.x86_64

      How reproducible:

      Always on customer system, no way to test internally

            [RHEL-17393] On error, ReaR deletes the logs useful to debug NetBackup issues

            Errata Tool added a comment -

            Since the problem described in this issue should be resolved in a recent advisory, it has been closed.

            For information on the advisory (rear bug fix update), and where to find the updated files, follow the link below.

            If the solution does not work for you, open a new bug report.
            https://access.redhat.com/errata/RHBA-2024:2283

            Errata Tool added a comment - Since the problem described in this issue should be resolved in a recent advisory, it has been closed. For information on the advisory (rear bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2024:2283

            Mugdha Soni added a comment -

            Hi rhn-support-pcahyna  

            Hope you are doing well !

            Please provide information in the Release Note Text field if any documentation efforts are required here .

            See How to report a release note for reference.

             

            Thanks

            Mugdha

            Mugdha Soni added a comment - Hi rhn-support-pcahyna    Hope you are doing well ! Please provide information in the Release Note Text field if any documentation efforts are required here . See How to report a release note  for reference.   Thanks Mugdha

            Joerg Kastning added a comment -

            Hi rhn-support-pcahyna as promised here is the feedback from DHL:

            "I've finished all my tests here for both versions (2.6.19 and 2.6.22). Everything works as expected from our side - I was able to boot the servers, restore all backups from my Netbackup - and everything was validaded using our internal processes. So this way you can give some positive feedback to your engineer. Thank you so much!"

            Cheers,

            Joerg

            Joerg Kastning added a comment - Hi rhn-support-pcahyna as promised here is the feedback from DHL: "I've finished all my tests here for both versions (2.6.19 and 2.6.22). Everything works as expected from our side - I was able to boot the servers, restore all backups from my Netbackup - and everything was validaded using our internal processes. So this way you can give some positive feedback to your engineer. Thank you so much!" Cheers, Joerg

            Renaud Métrich added a comment - - edited

            This /var/tmp/rear.oWPDfOxxrtXC3m5/tmp/bplog.restore is not necessary for this very specific case because bprestore fails too early, but in case bprestore fails while restoring, it may be useful.

            I think it needs to be copied to /var/log/rear and name adjusted

            Renaud Métrich added a comment - - edited This /var/tmp/rear.oWPDfOxxrtXC3m5/tmp/bplog.restore is not necessary for this very specific case because bprestore fails too early, but in case bprestore fails while restoring, it may be useful. I think it needs to be copied to /var/log/rear and name adjusted

            Pavel Cahyna added a comment - - edited

            Due to the issue with NetBackup in above Jira, ReaR tells to check the log file for details:

            ReaR tells to check this log file:

            Aborting due to an error, check /var/log/rear/rear-HOSTNAME.log for details

            and this file gets copied into the recovered system. Which file do you want to save, is it /var/tmp/rear.oWPDfOxxrtXC3m5/tmp/bplog.restore ? If so, is it really worth having it, since there is nothing useful in your case? Do you think that in some other case there could be anything useful?

            Regarding /usr/openv/netbackup/logs/bprestore/root.112423_00001.log. What would you like ReaR to do with the file? Copy it into the recovered system? Mention it in the error message? Or both?

            Pavel Cahyna added a comment - - edited Due to the issue with NetBackup in above Jira, ReaR tells to check the log file for details: ReaR tells to check this log file: Aborting due to an error, check /var/log/rear/rear-HOSTNAME.log for details and this file gets copied into the recovered system. Which file do you want to save, is it /var/tmp/rear.oWPDfOxxrtXC3m5/tmp/bplog.restore ? If so, is it really worth having it, since there is nothing useful in your case? Do you think that in some other case there could be anything useful? Regarding /usr/openv/netbackup/logs/bprestore/root.112423_00001.log . What would you like ReaR to do with the file? Copy it into the recovered system? Mention it in the error message? Or both?

              rhn-support-pcahyna Pavel Cahyna
              rhn-support-rmetrich Renaud Métrich
              Pavel Cahyna Pavel Cahyna
              Jakub Haruda Jakub Haruda
              Mugdha Soni Mugdha Soni
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated:
                Resolved: