Uploaded image for project: 'Subscription Watch'
  1. Subscription Watch
  2. SWATCH-4516

Migrate all the flaky billable usage component tests written in IQE to java

XMLWordPrintable

    • Icon: Task Task
    • Resolution: Unresolved
    • Icon: Critical Critical
    • None
    • None
    • None
    • None
    • 5
    • False
    • Hide

      None

      Show
      None
    • False
    • subs-swatch-lightning

      The following billable-usage component tests written in IQE are flaky:

      • test_verify_remittance_status_for_inactive_subscription
      • test_remittance_matches_tally_snapshot_data_for_non_contract_products_with_billing_factor_below_one
      • test_remittance_matches_tally_snapshot_data_for_non_contract_products_with_billing_factor_equal_one
      • test_multiple_tally_summaries_not_aggregated_for_different_org_ids
      • test_multiple_tally_summaries_not_aggregated_for_different_metric_ids
      • test_last_month_usage_not_tallied_with_current_month
      • test_verify_remittance_diff_when_tally_summary_total_greater_than_remittance_usage

      The coverage for these flaky tests should be captured in the java component tests and removed from IQE.

      See some examples next:

      https://ci.ext.devshift.net/job/RedHatInsights-rhsm-subscriptions-pr-check/13865/

      tests.component.swatch_billable_usage.test_swatch_billable_usage.test_multiple_tally_summaries_not_aggregated_for_different_metric_ids                   
>       assert False, error_message
E       AssertionError: The action 'wait for 2 kafka messages' was not completed within the timeout period. Last result was 'None'.
E       assert False

swatch-billable-usage-service-6cc44f4846-r58tl_swatch-billable-usage-service.log
2026-01-08 18:03:36,041 WARN  [org.apache.kafka.streams.kstream.internals.KStreamWindowAggregate] (platform.rhsm-subscriptions.swatch-billable-usage-aggregator-670817a3-6dc4-4b59-b4ca-136d35225319-StreamThread-1) Skipping record for expired window. topic=[platform.rhsm-subscriptions.swatch-billable-usage-aggregator-billable-usage-store-repartition] partition=[0] offset=[55] timestamp=[1767895415932] window=[1767895413000,1767895416000) expiration=[1767895416008] streamTime=[1767895416008]

       
      https://ci.ext.devshift.net/job/RedHatInsights-rhsm-subscriptions-pr-check/13864/

      tests.component.swatch_billable_usage.test_swatch_billable_usage.test_remittance_matches_tally_snapshot_data_for_non_contract_products_with_billing_factor_below_one
# verify value is 20*0.25=5.0
> verify_kafka_hourly_aggregate_data(application, billing_provider, billing_account_id, 5.0)

      Resources:

      • Logs are attached for each of these runs.
      • zipped artifacts are attached
      • A screenshot with the start times.

      Acceptance Criteria:

      • All the mentioned component tests written in IQE for billable usage are migrated from IQE to java component tests.

        1. subscriptions-pr-check-13865.txt
          16.06 MB
        2. subscriptions-pr-check-13864.txt
          2.74 MB
        3. Screenshot From 2026-01-08 15-14-34.png
          Screenshot From 2026-01-08 15-14-34.png
          398 kB
        4. artifacts-13865.zip
          2.66 MB
        5. artifacts-13864.zip
          2.55 MB

              Unassigned Unassigned
              buschv Vanessa Busch
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: