Two OCP clusters (4.5 and 4.6) recently have been left unusable by seemingly runaway secret creation due to RH PAM operator.
Common factor in both cases was RH PAM operator installed, and some time afterwards, the cluster was degraded due to large volume of data writes to ETCD DB.
PAM operator was version 7.9.1-1
One cluster found these "console-cr" secrets in post analysis of ETCD DB
~~~
65901 /kubernetes.io/secrets/bpm-dva/console-cr-form-dockercfg
190686 /kubernetes.io/secrets/bpm-dva/console-cr-form-token
~~~
Another customer:
"It had over 8700 secrets created for the console-cr service account ( 110 MB of secrets )"
No further info available from either cluster sorry as they were nuked after issue could not be resolved.