Uploaded image for project: 'Subscription Watch'
  1. Subscription Watch
  2. SWATCH-4064

rhsm-subscriptions-egress fails to upload some tables to s3 in production

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Critical Critical
    • None
    • None
    • CI/Infrastructure
    • None
    • False
    • Hide

      None

      Show
      None
    • False

      Example: https://console-openshift-console.apps.crcp01ue1.o9m8.p1.openshiftapps.com/k8s/ns/rhsm-prod/pods/rhsm-subscriptions-egress-29331360-x2pdt

      Output:

      Table 'subscription_product_ids': Data collection started.
      + echo 'Table '\''subscription_product_ids'\'': Data collection started.'
      + psql -h rhsm-prod.ckxxru2ayexw.us-east-1.rds.amazonaws.com -U postgres rhsm-subscriptions -c 'COPY subscription_product_ids TO STDOUT CSV HEADER'
      ++ date -I
      + pipenv run aws s3 cp - s3://rhsm-subscriptions-egress-prod/production_data/subscription_product_ids/historic/2025-10-08-subscription_product_ids.csv
      ERROR:  relation "subscription_product_ids" does not exist
      
      
      + pipenv run aws s3 cp - s3://rhsm-subscriptions-egress-prod/production_data/tally_measurements/historic/2025-10-08-tally_measurements.csv
      upload failed: - to s3://rhsm-subscriptions-egress-prod/production_data/tally_measurements/historic/2025-10-08-tally_measurements.csv An error occurred (InvalidArgument) when calling the UploadPart operation: Part number must be an integer between 1 and 10000, inclusive
      

      All the other tables worked fine:

      Table 'subscription_measurements': Data collection started.
      + psql -h rhsm-prod.ckxxru2ayexw.us-east-1.rds.amazonaws.com -U postgres rhsm-subscriptions -c 'COPY subscription_measurements TO STDOUT CSV HEADER'
      ++ date -I
      + pipenv run aws s3 cp - s3://rhsm-subscriptions-egress-prod/production_data/subscription_measurements/historic/2025-10-08-subscription_measurements.csv
      ++ date -I
      + pipenv run aws s3 cp s3://rhsm-subscriptions-egress-prod/production_data/subscription_measurements/historic/2025-10-08-subscription_measurements.csv s3://rhsm-subscriptions-egress-prod/production_data/subscription_measurements/latest/full_data.csv
      Completed 8.0 MiB/37.2 MiB (27.7 MiB/s) with 1 file(s) remaining
      Completed 13.2 MiB/37.2 MiB (31.9 MiB/s) with 1 file(s) remaining
      Completed 21.2 MiB/37.2 MiB (49.7 MiB/s) with 1 file(s) remaining
      Completed 29.2 MiB/37.2 MiB (64.1 MiB/s) with 1 file(s) remaining
      Completed 37.2 MiB/37.2 MiB (78.0 MiB/s) with 1 file(s) remaining
      copy: s3://rhsm-subscriptions-egress-prod/production_data/subscription_measurements/historic/2025-10-08-subscription_measurements.csv to s3://rhsm-subscriptions-egress-prod/production_data/subscription_measurements/latest/full_data.csv
      + echo 'Table '\''subscription_measurements'\'': Dump uploaded to intermediate storage.'
      

      Acceptance Criteria

      • Investigate why tally_measurements is failing to be uploaded
      • Remove the table subscription_product_ids to be uploaded which does not exist anylonger

              Unassigned Unassigned
              jcarvaja@redhat.com Jose Carvajal Hilario
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: