Context: Add-ons tooling now supports deploying alerts that are defined in the managed-tenants repos rather than being baked into the product. Adopting this means that we can tweak alerts without having to do a build (and therefore could do so whenever we want).

rh-ee-magautie  is driving this effort, and we could be his guinea pig.