Summary

Investigate creation of alerts when collector status is not valid.    Alerts for status conditions would provide better visibility and reduce the time to resolve any issue with the configuration.

Acceptance Criteria

• Document the recommended alerts and their criteria
  • If the collector is not working as expected and this is reflected in the "clusterLogForwarder.status.conditions" where the "ValidationSucess" is not "True" and/or it's not permitted to collect some log types for the "serviceAccount" set, then an alert should be triggered
• Identify the content needed to create the associated runbook(s) needed for each new alert
• Create an epic and supporting JIRA tasks for recommendations we intend to implement
• Gain consensus from the Logging team regarding the outcome of the analysis

Notes

• Based on feature request https://issues.redhat.com/browse/OBSDA-1167
• When validation fails during initial setup, the collector does not start.   But if a running collector is modified and validation fails, the collectors will continue working with the "old" valid configuration