This is a goal to help Service Delivery with unmanageable alerts during upgrades. Reach out to SD and get an initial list of alerts that are causing problems, then see how they are appearing in our data.

If they do appear in CI, work out a testing framework for specific alerts that we can make flake/fail. This might be a threshold, but I suspect more likely will be things that shouldn't fire at all.

Add test, make it a flake, file the bug for relevant team, and graduate the test to a failure once the issue is resolved.

cblecker.openshift could you get us a list of 3-5 of your top problematic alerts?