Feature Overview (aka. Goal Summary)  

Provide a declarative configurations for the monitoring stack and enable GitOps workflow with Argo CD to manage the monitoring stack configuration across clusters.  

Goals (aka. expected user outcomes)

• Enable Argo CD and GitOps workflow to manage monitoring stack configuration across clusters
• Enable use of secret managers for providing sensitive data (e.g. credentials, certs, etc) to the monitoring stack configuration

Requirements (aka. Acceptance Criteria):

Background

The monitoring stack is configured through a configmap that contains a yaml in the data field (YAML-in-YAML)(link to docs). As a result, the specific changes to the monitoring stack configurations (the nested YAML) are not detectable with Git diffs and are not visible to the GitOps workflow.

The immediate consequence in a GitOps workflow is that when the monitoring stack configmap is changed in the Git repository, Argo CD (or other GitOps tools) would detect the change but are not able to surface what has changed which makes is non-trivial for the platform teams to determine the impact and make an approve/decline decision. 

The YAML-in-YAML approach creates additional challenges when customers need to embed sensitive data in the monitoring stack configuration such as auth token for external altermanagers. In the GitOps approach, the sensitive data would be stored in a secret manager and get pulled to the cluster where it is needed. However the current approach prevents customers to use secret managers for storing the credentials related to the monitoring stack in a secret manager and forces them to commit the credentials in a Git repo, which violates security practises and is automatically flagged by GitHub and GitLab secret detectors ( and also Red Hat). 

In the following ArgoCon recording, Ford elaborates the above issues:
https://www.youtube.com/watch?v=zB-EWRVSQpY&t=796s

Customer Considerations

Provide any additional customer-specific considerations that must be made when designing and delivering the Feature.  Initial completion during Refinement status.

<your text here>

Documentation Considerations

Provide information that needs to be considered and planned so that documentation will meet customer needs.  If the feature extends existing functionality, provide a link to its current documentation. Initial completion during Refinement status.

<your text here>

Interoperability Considerations

Which other projects, including ROSA/OSD/ARO, and versions in our portfolio does this feature impact?  What interoperability test scenarios should be factored by the layered products?  Initial completion during Refinement status.

<your text here>