Description of problem:

The install-dynamic-plugins pod is not correctly dealing with the lockfile in some scenarios. We think it might be when a new deployment is triggered before the prior deployment's install-dynamic-plugins container has finished. It appears that when an existing install-dynamic-plugins pod is terminated it doesn't always cleanup the lockfile correctly - or maybe it happens if the plugin install encounters an error.

We believe it can be triggered by modifying configmaps/secrets in quick succession, since this causes the operator to trigger new rollouts. It's not totally reliable, but has happened to us a few times now.

Prerequisites (if any, like setup, operators/versions):

RHDH Operator for v1.7.1

Steps to Reproduce

Modify configmaps/secrets in quick sucession.

Actual results:

New containers cannot install plugins and never deploy.

Expected results:

The new containers eventually deploy.

Reproducibility (Always/Intermittent/Only Once):

Intermittent

Build Details:

1.7.1 on OCP 4.18.11

Additional info (Such as Logs, Screenshots, etc):