Description of problem - Provide a detailed description of the issue encountered, including logs/command-output snippets and screenshots if the issue is observed in the UI:

The bug is in the MCO code deletion path of mirrorpeer.

This only happens when you are using hosted and base clusters both as the managed clusters with multiple drpolices.
So basically, there are multiple mirrorpeers.
When only one mirrorpeer is deleted, the agent is required to delete the s3 bucket of that cluster. In agent_mirrorpeer_controller.go, in the deletion of mirrorpeer if block, you have the code: Code link
 
and what we see is that it keeps looping here. It never proceeds to the s3 bucket deletion and mirrorpeer deletion.
 

The OCP platform infrastructure and deployment type (AWS, Bare Metal, VMware, etc. Please clarify if it is platform agnostic deployment), (IPI/UPI):

HCP

The ODF deployment type (Internal, External, Internal-Attached (LSO), Multicluster, DR, Provider, etc):

DR

The version of all relevant components (OCP, ODF, RHCS, ACM whichever is applicable):

Base cluster OCP version - 4.19.5

Hosted cluster OCP version - 4.18.20

Hub cluster version - 4.19.3

Does this issue impact your ability to continue to work with the product?

Yes

Is there any workaround available to the best of your knowledge?

Yes. User can proceed with manual deletion of finalizers from mirrorpeer, to continue with DR Uninstall.

Can this issue be reproduced? If so, please provide the hit rate

Yes. If both the base clusters are in DR relationship, and a pair of hosted clusters also is in DR relationship.
Now if you try to remove DR from any of the above pair, mirrorpeer deletion will get stuck.

Can this issue be reproduced from the UI?

Yes. If we follow same steps from UI

If this is a regression, please provide more details to justify this:

Steps to Reproduce:

1.

2.

3.

The exact date and time when the issue was observed, including timezone details:

Actual results:

Mirrorpeer deletion got stuck

Expected results:

DR removal from the managed cluster pair should be done successfully.

Logs collected and log location:

Additional info: