Uploaded image for project: 'Subscription Watch'
  1. Subscription Watch
  2. SWATCH-2099

Produce failed messages to dead-letter-queue for swatch-azure-producer

    XMLWordPrintable

Details

    • Story
    • Resolution: Done
    • Critical
    • 2024-02-05 - API
    • None
    • None

    Description

      swatch-azure-producer CONSUMES messages from billable_usage topic

      • Processing FAILS due to azureUsageContext not available yet.  using 72 hours from snapshot_date to in order to determine hard failure vs. retryable failure
      • Make sure 72 isn’t a magic number.  A constant with java comment is ok.  Putting it as an app properties value that COULD be overridden is ok.  Don’t expose via env var/clowdapp.  The number is unlikely to change unless Azure does things that make the missing subscription id causing this mess available earlier.
      • swatch-azure-producer PRODUCES messages to billable-usage.dlt regardless of hard vs retryable failures
      • Preserve the original message, so we can reuse this DLQ for any billable-usage failures in the future
      • Include message headers
      • retry_after = now() + 1hr
      • On hard failures that we know (such as we are after the 72 hour window since snapshot_date for update from Azure), errors should be logged at Warning level, with a descriptive log message & structured log data to enable debugging/alerting. On unknown hard failures ERROR with stack trace

      AC:

      • When swatch-azure-producer receives a billableUsage and the subscription can not be determined (subscriptionId unavailable), a message is produced to billable-usage.dlt with retry_after header set to now() + 1hr
      • When the subscriptionId is unavailable and the usage is past 72 hours (configurable) old, log a warning
      • Log non retriable other errors at either warning or error

      Attachments

        Issue Links

          Activity

            People

              jcarvaja@redhat.com Jose Carvajal Hilario
              kflahert@redhat.com Kevin Flaherty
              Sumedh Sidhaye Sumedh Sidhaye
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: