Uploaded image for project: 'Project Quay'
  1. Project Quay
  2. PROJQUAY-7306

The ability for High Quota warning be able to be sent via email, Slack webhooks, etc. 

XMLWordPrintable

    • Icon: Feature Feature
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • None
    • quay
    • Product / Portfolio Work
    • False
    • Hide

      None

      Show
      None
    • False
    • Not Selected

      Goal:

      High Quota warnings are implemented as notifications that can be sent via email, Slack webhooks, and other external channels, similar to the existing repository notification system.

      Background:

      Currently, quota warnings only appear in the Quay UI (in-app bell notifications), but most customers rely on asynchronous notifications because they don't check the UI regularly.  Without a proactive notification via a supported channel, users may only become aware of their quota utilization when it is too late (quota exhausted, pushes denied).

      This is especially critical for large organizations managing many teams and pipelines.

      Why is this important?

      Current Problem

      • Quota warnings are in-app only: The existing quota_warning and quota_error notifications create entries in the Notification database table (the bell icon in the UI).  They are never dispatched through the NotificationMethod system that drives Email, Slack, Webhook, and other external delivery channels.
      • No organization-level notification configuration exists: The entire external notification system (RepositoryNotification model) is scoped to individual repositories.  There is no equivalent namespace/org-level construct.
      • Quota is namespace-scoped, not repository-scoped: This means existing per-repository email confirmation (RepositoryAuthorizedEmail) is architecturally incompatible with quota notifications, which have no single repository anchor.
      • No periodic threshold checks: Warnings are currently only fired at push time.  Storage that grows through other means (e.g., proxy cache, replication) will not trigger warnings until the next push.

      Customer Impact (from RFE-5257, RFE-4378)

      • Customers cannot receive proactive quota alerts via email or Slack.  They only discover quota issues when a push is blocked.
      • Organizations with 2000+ repositories cannot feasibly set up per-repository email confirmations to route any email-based notification.
      • Org admins want to set a single team distribution list (e.g., ops-team@example.com) to receive all quota warnings without per-repository setup required.

      Required Deliverables 

      • Quota threshold notifications via external channels: High Quota warnings at user-defined threshold levels (e.g., 75%, 80%, 95%, 100%) shall trigger notifications via email, Slack webhooks, and other external channels.  The send methods shall be similar to the existing repository notification system.
      • New quota_warning and quota_error notification events: Add QuotaWarningEvent and QuotaErrorEvent with payloads including: "namespace", "quota_bytes_used", "quota_bytes_limit", "percent_used", and "threshold_percent".
      • Organization-level notification configuration: A new namespace-scoped notification model (NamespaceNotification, similar to RepositoryNotification) allows org admins to configure one or more notification channels (email, Slack, webhook, etc.) at the organization level.  These apply to all quota events for that namespace.
      • Email routing via contact_email: The email address used for quota email notifications is separately configurable in line with the existing email repository notifications.  This leverages the organization contact_email field (introduced by PROJQUAY-6975) as the primary pre-authorized email target, eliminating the need for per-repository email confirmation.
        • Fallback: org owner email addresses (if contact_email is not set).
      • Deduplication and throttling: A quota_warning notification for a given namespace and threshold level is not re-fired more than once per threshold-crossing event.  Re-notification is appropriate once:
        • i) usage drops below the threshold and rises again, or
        • ii) past a configurable cool-down period (e.g., 24 hours).

      Open Questions

      • 1) Deduplication TTL: What is the expected cool-down period for re-notification?  Should we come up with a sensible default and make it configurable via a flag?
      • 2) Retroactive notification: If a quota is set while the namespace is already above a threshold, should a notification fire immediately?  Perhaps yes, so it's fired once whenever the quota configuration is saved.
      • 3) Interaction with RFE-4378 (email without confirmation for repo events): The per-repository email confirmation bypass (for mirror sync, push events, etc.) is a separate concern from org-level quota emails.  Should it be addressed as a sub-task here or as a standalone item?  Perhaps: a sub-task, since the contact_email routing partially addresses it for org-affiliated repos.

              Unassigned Unassigned
              DanielMesser Daniel Messer
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated: