Uploaded image for project: 'Automation Hub'
  1. Automation Hub
  2. AAH-2332

Concurrency bugs in repo management with many parallel collection publishes

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • None
    • Unspecified
    • None

      Description

      In the ansible-core CI, we have a process that creates and publishes many collections in parallel for later testing. This is largely to reduce the time of testing.

      While updating the version of galaxy_ng that we test against, we have run into several concurrency issues with repo management in galaxy_ng.

      Ultimately I believe they fall into 2 categories:

      1. Race conditions removing versions that exceed retain_repo_versions
      2. Race conditions creating new repo versions

      Key Dependency Versions

      galaxy-importer==0.4.10
      galaxy-ng==4.7.0
      pulp-ansible==0.17.0
      pulp-container==2.14.5
      pulp-glue==0.19.0
      pulpcore==3.23.3
      

      Steps to Reproduce

      In short, create something like 50 collections, and publish them all in parallel to a single repository.

      Actual Behavior

      Various tracebacks. See attached files.

      Expected Behavior

      No tracebacks, all collections uploaded.

        1. cannot-delete-some.txt
          2 kB
        2. duplicate-key.txt
          4 kB
        3. dynaconf-list.txt
          12 kB
        4. index-out-of-range.txt
          2 kB
        5. last-remaining.txt
          2 kB
        6. still-referenced.txt
          3 kB

            jtanner@redhat.com James Tanner
            mmartz@redhat.com sivel -
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: