TALM Backup fails consistently with error:

2024-03-28T22:25:37Z	INFO	controllers.ClusterGroupUpgrade	[triggerBackup]	{"cluster": "worker-0", "err": "can't find recovery image pull spec in environment"}    

    TALM 4.16.0-16

    Always

    1. Install latest TALM 4.16 from Brew
    2. Create policy and CGU with backup: True
    3. Enable CGU
    4. CGU Fails with Unrecoverable Error
    5. TALM pod logs show the above error.

    

2024-03-28T22:17:04Z    INFO    controllers.ClusterGroupUpgrade    Start reconciling CGU    {"name": {"name":"generated-cgu-backupsequence","namespace":"talm-test"}}
2024-03-28T22:17:04Z    INFO    controllers.ClusterGroupUpgrade    Loaded CGU    {"name": {"name":"generated-cgu-backupsequence","namespace":"talm-test"}, "version": "4333025"}
2024-03-28T22:17:04Z    INFO    controllers.ClusterGroupUpgrade    Remediation plan    {"remediatePlan": [["worker-0"]]}
2024-03-28T22:17:04Z    INFO    controllers.ClusterGroupUpgrade    [blockingCRsNotCompleted]    {"blockingCRs": []}
2024-03-28T22:17:04Z    INFO    controllers.ClusterGroupUpgrade    [reconcileBackup]    {"FindStatusCondition": "&Condition{Type:BackupSuceeded,Status:False,ObservedGeneration:0,LastTransitionTime:2024-03-28 22:15:34 +0000 UTC,Reason:InProgress,Message:Backup in progress for 1 clusters,}"}
2024-03-28T22:17:04Z    INFO    controllers.ClusterGroupUpgrade    [triggerBackup]    {"currentState": "PreparingToStart", "cluster": "worker-0"}
2024-03-28T22:17:04Z    INFO    controllers.ClusterGroupUpgrade    [triggerBackup]    {"currentState": "PreparingToStart", "condition": "entry", "cluster": "worker-0", "nextState": "Starting"}
2024-03-28T22:17:04Z    INFO    controllers.ClusterGroupUpgrade    [getBackupJobTemplateData]    {"workload image": ""}
2024-03-28T22:17:04Z    INFO    controllers.ClusterGroupUpgrade    [triggerBackup]    {"cluster": "worker-0", "err": "can't find recovery image pull spec in environment"}
2024-03-28T22:17:04Z    INFO    controllers.ClusterGroupUpgrade    Finish reconciling CGU    {"name": {"name":"generated-cgu-backupsequence","namespace":"talm-test"}, "requeueAfter": 30}

    Backup is successful