Uploaded image for project: 'Automation Hub'
  1. Automation Hub
  2. AAH-2287

nightly collection sync on beta community site fail due to OOM

      Description

      The collection sync tasks triggered via the nightly cron job fail due to an OOM error on the pulp worker. Even when the requirements file only contain a single collection version. 

      This began occurring on beta-dev 2023-03-02,

      beta-stage 2023-03-03, 

      and beta(prod)  2023-03-04

      Key Dependency Versions

      {"available_versions":\{"v3":"v3/","pulp-v3":"pulp/api/v3/","v1":"v1/"}

      ,"server_version":"4.8.0dev","galaxy_ng_version":"4.8.0dev","galaxy_ng_commit":"master:d7225180","galaxy_importer_version":"0.4.7","pulp_core_version":"3.23.1","pulp_ansible_version":"0.17.0","pulp_container_version":"2.14.4"}

      Steps to Reproduce

      Send a POST request to https://beta-galaxy[-dev|-stage].ansible.com/api/content/published/v3/sync/

      Always reproducible

      Actual Behavior

      the sync task fails

      the pulp-worker pod metrics show a rise in memory utilization up to 4GiB (the current applied memory limit), then a sudden drop when the OOM occurs. The logs indicate the task process existed with non zero exitcode -9.

      Expected Behavior

      The sync task should complete successfully.

              drodowic@redhat.com Daniel Rodowicz
              drodowic@redhat.com Daniel Rodowicz
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: