Why is this important?:
Infra owners managing a very large number of bare metal clusters from RHACM need to have a notification system to monitor their clusters. When managing hundreds of clusters, infra owners need complete observability during the lifecycle of their clusters. Today, there is such proactive notification when the cluster gets installed or destroyed using the acm_managed_cluster_info metric. However, additional observability is needed for errors/error conditions.

Scenario[1]:
1. During installation the Kubernetes API client is watching relevant object(s).
2. Error condition occurs in Cluster
3. Kubernetes API client detects error
4. Message sent to AMQ queue (notification)

[1] The other “scenarios” would include clusters created or destroyed.

Example of what could be productized from above scenario:
Using a Kubernetes API client to watch relevant object(s) and react to the events received. For example, in Python, someone could install the `kubernetes` package and do the following:

{{{{{{ #!/usr/bin/env python

1. {} {}coding: utf-8 --}}

import kubernetes.client
import kubernetes.config
import kubernetes.dynamic

1. Create the API client:
   client = kubernetes.dynamic.DynamicClient(
   kubernetes.client.ApiClient(configuration=kubernetes.config.load_kube_config())
   )

1. Watch cluster deployments in all namespaces:
   resource = client.resources.get(
   api_version="hive.openshift.io/v1",
   kind="ClusterDeployment"
   )
   while True:
   stream = resource.watch(timeout=10)
   for event in stream:
   if event["type"] == "MODIFIED":
   cluster = event["object"]
   if cluster.spec.installed:
   print(
   f"Cluster ' {cluster.metadata.name}
   ' in namespace ' {cluster.metadata.namespace}
   ' is "
   f"now installed"
   )}}}}

Then replace the `print ...` with code to integrate with an eventing system, for example - sending a message to an AMQ queue..

Acceptance Criteria:

• CI - MUST be running successfully with tests automated
• Release Technical Enablement - Provide necessary release enablement details and documents.

Done Checklist:

• CI - CI is running, tests are automated and merged.
• Release Enablement <link to Feature Enablement Presentation>
• DEV - Upstream code and tests merged: <link to meaningful PR or GitHub Issue>
• DEV - Upstream documentation merged: <link to meaningful PR or GitHub Issue>
• DEV - Downstream build attached to advisory: <link to errata>
• QE - Test plans in Polarion: <link or reference to Polarion>
• QE - Automated tests merged: <link or reference to automated tests>
• DOC - Downstream documentation merged: <link to meaningful PR>