Description of problem:
Loading time for the console can increase in cases up to 5 minutes or more when the pod has been running continuously for several months and the cluster is under heavy load, accumulating resource deletions over time.
Version-Release number of selected component (if applicable):
All currently supported versions back to ACM 2.7.z / MCE 2.2.z
How reproducible:
Always
Steps to Reproduce:
While it takes a lot of data to reproduce the initial loading time problem, by accumulating resource deletions, it is easy to see the root cause.
When resource are deleted, the deletion event is sent to all clients with an open connection, but an event is also cached in the server. This causes the pod memory usage to slowly increase over time, and it also causes initial loading time to gradually increase.
To see this, begin a cluster import, giving the cluster a unique name, then delete it and wait for it to disappear from the UI. Open the event stream at /api/proxy/plugin/mce/console/multicloud/events. Search for "<cluster-name>" where <cluster-name> is the name of the cluster import you created and deleted.
Actual results:
The event stream will contain many references to this cluster name, even though new clients have no need to know that these resources ever existed and will do nothing useful with the DELETED events. These events consume network bandwith and client CPU with no impact.
Expected results:
Existing clients should see the DELETED events; clients connecting after deletion should not.
- clones
-
ACM-9883 Console initial loading time increases over time
-
- Closed
-
- is cloned by
-
-
- Closed
-
- links to
-
RHSA-2024:126795
Red Hat Advanced Cluster Management 2.9.3 security and bug fix container updates
[ACM-9885] Console initial loading time increases over time [ACM 2.9.z]
Since the problem described in this issue should be resolved in a recent advisory, it has been closed.
For information on the advisory (Moderate: Red Hat Advanced Cluster Management 2.9.3 security and bug fix container updates), and where to find the updated files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHSA-2024:1328
Hi rhn-support-ashafi - would you be able to verify this fix for 2.9.3 or let rh-ee-kcormier know if there is a QE env he can check? We need the verification to move forward with the scheduled release. Please let me know if you need anything or have questions. Thanks!
rhn-support-ashafi I know you verified for 2.10... can you verify for 2.9.3 as well? Or is there a QE 2.9 env I can check? Thanks.
Hi rh-ee-kcormier - can you verify the fix for this issue? We are working to have these items verified asap so we can determine if the release will remain on schedule. Please let me know if you have concerns or questions. Thank you!
Since the problem described in this issue should be resolved in a recent advisory, it has been closed.
For information on the advisory (Moderate: Red Hat Advanced Cluster Management 2.9.3 security and bug fix container updates), and where to find the updated files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHSA-2024:1328