Uploaded image for project: 'Satellite'
  1. Satellite
  2. SAT-28575

Full Capsule Sync always triggers pulp optimised sync

XMLWordPrintable

    • Sprint 140, Sprint 141, Sprint 142
    • Moderate
    • Automated
    • Yes

      Description of problem:
      Trying to fix corrupted metadata of a repository on Capsule, an attempt to run Full Capsule Sync does not help. The reason is Pulp sync task returns "Skipping Sync (no change from previous sync)" - and this is caused by the fact katello always call `RepositoriesRpmApi.sync` endpoint with `"optimize":true`. That is caused by:

      https://github.com/Katello/katello/blob/master/app/lib/actions/pulp3/capsule_content/sync.rb#L16-L17

      That needs to be:

      {noblock}
      sync_options[:optimize] = false if input[:options][:skip_metadata_check]
      output[:pulp_tasks] = repo.backend_service(smart_proxy).with_mirror_adapter.sync(sync_options){noblock}

       

      How reproducible:
      100%
       

      Is this issue a regression from an earlier version:
      Yes, a regression from 6.9
       

      Steps to Reproduce:
      1. Have one CV promoted to LE that is associated to a Capsule and synced there. Ideally have just one pulp repository on the Capsule, to simplify the next step.
      2. Manually corrupt some metadata on filesystem on the Capsule - e.g. via `file $(find /var/lib/pulp/media/artifact/ -type f)`, identify `primary.xml` file and remove it.
      3. Have a Content Host registered via the Capsule and enabled the newly-broken repo. Run there e.g. `dnf update` to confirm metadata are missing. Check `/var/log/messages` on Capsule for the details.
      4. Run Full Capsule Sync
      5. Once it completes, try `dnf update` again.

      Actual behavior:
      3. `dnf update` fails with something like:

      zoo_repo                                                                                                                                                      304  B/s |  55  B     00:00    
      Errors during downloading metadata for repository 'RedHat_zoo_product_zoo_repo':
        - Status code: 500 for https://pmoravec-caps615.example.com/pulp/content/RedHat/RECYCLE/cv_zoo/custom/zoo_product/zoo_repo/repodata/repomd.xml (IP: 1.2.3.4)
      Error: Failed to download metadata for repo 'RedHat_zoo_product_zoo_repo': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
      

      `/var/log/messages` on Capsule shows:

      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]: [2024-10-04 08:58:55 +0000] [2043] [ERROR] Error handling request
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]: Traceback (most recent call last):
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]:  File "/usr/lib64/python3.11/site-packages/aiohttp/web_protocol.py", line 452, in _handle_request
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]:    resp = await request_handler(request)
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]:           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]:  File "/usr/lib64/python3.11/site-packages/aiohttp/web_app.py", line 543, in _handle
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]:    resp = await handler(request)
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]:           ^^^^^^^^^^^^^^^^^^^^^^
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]:  File "/usr/lib64/python3.11/site-packages/aiohttp/web_middlewares.py", line 114, in impl
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]:    return await handler(request)
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]:           ^^^^^^^^^^^^^^^^^^^^^^
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]:  File "/usr/lib/python3.11/site-packages/pulpcore/content/authentication.py", line 48, in authenticate
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]:    return await handler(request)
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]:           ^^^^^^^^^^^^^^^^^^^^^^
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]:  File "/usr/lib/python3.11/site-packages/pulpcore/cache/cache.py", line 339, in cached_function
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]:    response = await self.make_entry(
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]:               ^^^^^^^^^^^^^^^^^^^^^^
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]:  File "/usr/lib/python3.11/site-packages/pulpcore/cache/cache.py", line 378, in make_entry
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]:    response = await handler(*args, **kwargs)
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]:               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]:  File "/usr/lib/python3.11/site-packages/pulpcore/content/handler.py", line 249, in stream_content
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]:    return await self._match_and_stream(path, request)
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]:           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]:  File "/usr/lib/python3.11/site-packages/pulpcore/content/handler.py", line 610, in _match_and_stream
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]:    return await self._serve_content_artifact(ca, headers, request)
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]:           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]:  File "/usr/lib/python3.11/site-packages/pulpcore/content/handler.py", line 918, in _serve_content_artifact
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]:    raise Exception(_("Expected path '{}' is not found").format(path))
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]: Exception: Expected path '/var/lib/pulp/media/artifact/3b/bd4af9a5cac6195a2b138e9754df0c3d8acf7cdf34ed736bc313342409aa45' is not found
      Oct  4 10:58:55 pmoravec-caps615 pulpcore-content[2043]: [04/Oct/2024:08:58:55 +0000] "GET /pulp/content/RedHat/RECYCLE/cv_zoo/custom/zoo_product/zoo_repo/repodata/repomd.xml HTTP/1.1" 500 245 "-" "libdnf (Red Hat Enterprise Linux 8.0; generic; Linux.x86_64)"
      

      So far, so good.

      BUT the same persists also after Full Capsule Sync - that will have "Skipping Sync (no change from previous sync)" message in the pulp task details (see e.g. in Actions::Pulp3::CapsuleContent::Sync dynflow step of the task). Step 5 will fail the same way..

      Expected behavior:
      4. Full Caps Sync should not skip syncing the repo in Actions::Pulp3::CapsuleContent::Sync step.
      5. `dnf update` should not fail after the Full Caps Sync

      Business Impact / Additional info:
      This prolonged resolution of a support case where we had to manually remove or fix individual repos instead of simply clicking to Full Caps Sync (which was tried, of course)

          There are no Sub-Tasks for this issue.

              rhn-support-pmoravec Pavel Moravec
              rhn-support-pmoravec Pavel Moravec
              Samir Jha Samir Jha
              Vladimír Sedmík Vladimír Sedmík
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated: