Currently each stabilization-changes container pushes Slack waiting notifications on its first attempt, and then waits at least 24h before sending a new waiting-notification (actual changes like PRs being opened are always notified immediately, the 24h cool-down is just the Recommend waiting to promote... messages). Usually this isn't a problem, because we don't get new stabilization-changes containers all that often. But when we do happen to get a bunch of new containers, e.g. because our Python base image is triggering rebuilds or because the host PSI cluster is rolling compute, the #forum-release notifications can get annoying.

We should teach the stabilization-changes script an option like -cache PATH or some such, to cache its next notification to disk before sleeping. When loading any previous cache on start-up, the incoming container would pick up the previous container's next-notification, only defaulting to next_notification = datetime.datetime.now() when there was no cached value, or the cache was non-existent. Then we should update the DeploymentConfig to set -cache /cache/cache.json or whatever around here, and update the existing emptyDir volume to use a persistent volume claim.