Uploaded image for project: 'Subscription Watch'
  1. Subscription Watch
  2. SWATCH-4511

Fix unreliable test for billing account separation in usage aggregation

XMLWordPrintable

    • 3
    • False
    • Hide

      None

      Show
      None
    • False
    • subs-swatch-lightning
    • Swatch Lightning Sprint 9

      We have a flaky test that's causing unnecessary build failures and blocking our CI pipeline.

      The test test_multiple_tally_summaries_not_aggregated_for_different_billing_account_ids is designed to verify that usage data from different billing accounts gets properly separated when we process tally snapshots. It's an important test because it ensures we're correctly handling scenarios where the same organization has multiple billing accounts (like different AWS accounts).

      The test works by sending tally snapshots for the same organization but with two different billing account IDs, then checking that we get separate hourly aggregate messages for each account. The problem is that it's failing intermittently when waiting for those Kafka messages to arrive, which suggests there might be timing issues or race conditions in how we're verifying the results.

      This flakiness is currently marked with an iqe_blocker annotation, which means every time it fails (even though it's just a test stability issue, not a real product bug), it can block our entire test suite.

      Example link: https://ci.ext.devshift.net/job/RedHatInsights-rhsm-subscriptions-pr-check/14219/testReport/junit/tests.component.swatch_billable_usage/test_swatch_billable_usage/test_multiple_tally_summaries_not_aggregated_for_different_billing_account_ids/

      Acceptance Criteria:

      • Fix the test
      • Remove the iqe_blocker annotation

              jcarvaja@redhat.com Jose Carvajal Hilario
              jcarvaja@redhat.com Jose Carvajal Hilario
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: