Description of problem:

Red Hat Advanced Cluster Management Observability stack is experiencing intermittent issues where managed clusters oscillate between Available and Degraded states. The root cause appears to be TLS handshake errors in the observatorium-api pods, preventing proper metric collection and transmission between hub and managed clusters. Despite attempts to fix the underlying configuration, the observability components continue to alternate between operational and degraded states.

Version-Release number of selected component (if applicable):

ACM 2.10.7

Hub Cluster - 4.16.27

Managed Cluster (gtdvvocthecl) - 4.16.27

How reproducible:

Steps to Reproduce:

1. Deploy ACM 2.10.7 on OpenShift 4.16 cluster
2. Create MultiClusterObservability resource with appropriate storage configuration.
3. Apply observability addon to a managed cluster.
4. Monitor the ManagedClusterAddOn status over several hours.

Actual results:

• Observability addon status alternates between Available and Degraded states
• Metrics-collector pods report "502 Bad Gateway" and connection errors
• TLS handshake errors appear in observatorium-api pod logs

Expected results:

• Stable "Available" status for the observability addon
• Consistent metrics collection without TLS handshake errors

Additional info:

• The issue may be related to known bug ACM-11757 "Managed clusters flip between Available and Degraded"
• Case attached has must-gathers from both hub and managed clusters for inspection.