Description of problem:

The Assert object in the test helpers library (https://github.com/openshift/machine-config-operator/blob/c6faace50b71d89f694e9f3a8ee7a6635ac36f7c/test/helpers/assertions.go) does not adequately handle the case where an object is deleted and then a new object is created in its place.

This is primarily observed in the following CI runs where TestRebuildAnnotationRestartsBuild fails:

• https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_machine-config-operator/5268/pull-ci-openshift-machine-config-operator-main-e2e-gcp-op-ocl/1977764027626950656
• https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_machine-config-operator/5268/pull-ci-openshift-machine-config-operator-main-e2e-gcp-op-ocl/1973133837785370624
• https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_machine-config-operator/5268/pull-ci-openshift-machine-config-operator-main-e2e-gcp-op-ocl/1973046397330526208
• https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_machine-config-operator/5268/pull-ci-openshift-machine-config-operator-main-e2e-gcp-op-ocl/1966568518447009792 

How reproducible:

In the wild, the bug is a bit hard to reproduce. However, using a specially-created test setup, it is trivial.

How to reproduce:

See included code snippet for reproduction: reproduce_test.go

Although this test case is written to require a cluster, it can be trivially reproduced using a FakeClient implementation as well.

Actual results:

In the wild, the assertion will eventually fail due to timeout or context cancellation when polling. When using the non-polling option as shown in the attached code snippet, it will fail immediately.

Expected results:

The assertion should succeed because the original object no longer exists.

Additional info:

The proximate cause of this is because the *DoesNotExist() methods only consider the name of the object and not its UID. When a new object with the same name is created, the object will have a different UID. The timeout occurs because the assertion is never reached.

It is perhaps worth carving out a separate and specific assertion that ensures that the object is actually replaced. It may also be worth revisiting how this helper is used in order to determine whether it should continue to be used in this way.