Description of problem:

Loading time for the console can increase in cases up to 5 minutes or more when the pod has been running continuously for several months and the cluster is under heavy load, accumulating resource deletions over time.

Version-Release number of selected component (if applicable):

All currently supported versions back to ACM 2.7.z / MCE 2.2.z

How reproducible:

Always

Steps to Reproduce:

While it takes a lot of data to reproduce the initial loading time problem, by accumulating resource deletions, it is easy to see the root cause.

When resource are deleted, the deletion event is sent to all clients with an open connection, but an event is also cached in the server. This causes the pod memory usage to slowly increase over time, and it also causes initial loading time to gradually increase.

To see this, begin a cluster import, giving the cluster a unique name, then delete it and wait for it to disappear from the UI. Open the event stream at /api/proxy/plugin/mce/console/multicloud/events. Search for "<cluster-name>" where <cluster-name> is the name of the cluster import you created and deleted.

Actual results:

The event stream will contain many references to this cluster name, even though new clients have no need to know that these resources ever existed and will do nothing useful with the DELETED events. These events consume network bandwith and client CPU with no impact.

Expected results:

Existing clients should see the DELETED events; clients connecting after deletion should not.