Goal: 

• as part of https://issues.redhat.com/browse/HMS-2422  we created a metrics around pending tasks (average time, total tasks, oldest task)
• Lets let these run for a month or so in production and see what the normal behavior is like
• once that has passed, lets create alerts to warn us if things are backing up too much

Acceptance Criteria:

• an alert when pending tasks are waiting too long, or the number of tasks is too high
• Verify Y
• Verify ...