Epic Goal

• Follow up on https://issues.redhat.com/browse/MON-2209
• Develop a notion of optional scrape profiles for service monitors and handle them in CMO
• Give users the option to influence the number of metrics the in-cluster stack collects
• Improve CMOs scaling behavior in very small and very large environments

Why is this important?

• In some environments CMO exhibits bad behavior. In single node environments its one of the main consumers of available resource budgets, in very large clusters memory usage requires very large nodes to keep up.
• Not all metrics collected are strictly necessary for functionality of other components. These can optionally be dropped if the admin is not interested in them.

Scenarios

1. A single node deployment always wants to minimize resource usage and the admin might want to choose to collect as few metrics as possible.
2. A user with many clusters often has an existing monitoring setup and wants to spend as few resource on OpenShift internal monitoring as possible. We want to give them an option to scrape as few metrics as possible.

Docs

• Document how to make use of scrape profiles to limit resource consumption.
• Document how to drop metrics from existing configs.