Uploaded image for project: 'CPE Infrastructure'
  1. CPE Infrastructure
  2. CPE-2645

f41 - createRepo/mergeRepo failures related to zstd in mirrors

XMLWordPrintable

    • False
    • None
    • False
    • Testable

      https://pagure.io/fedora-infrastructure/issue/12282

      1. Describe what you would like us to do:

      I run a local "fedora-like" koji build environment at $dayjob. I just recently (Nov-8) completed mass branching steps for our internal f41 release and noticed that all `createrepo` tasks for f41 fail due to "Cannot detect compression type" errors in the `updates` repo.

      I have tested against two different external mirrors, leaseweb.net, and facebook.net. The failure exists in both.

      See log output below:

      ```
      ==> 50465:DEFAULT:createrepo.log <==
      $ /usr/bin/createrepo_c --error-exit-val -vd -o /tmp/koji/tasks/465/50465/repo -g /mnt/koji/repos/local41-build/15703/groups/comps.xml /tmp/koji/tasks/465/50465/repo
      13:59:13: Version: 0.20.1 (Features: LegacyWeakdeps )
      13:59:13: Signal handler setup
      13:59:13: Thread pool ready
      13:59:13: Dir to scan: /tmp/koji/tasks/465/50465/repo/.repodata
      13:59:13: Package count: 0
      13:59:13: Copy metadatum /mnt/koji/repos/local41-build/15703/groups/comps.xml -> /tmp/koji/tasks/465/50465/repo/.repodata/comps.xml
      13:59:13: Creating .xml.gz files
      13:59:13: Setting number of packages
      13:59:13: Creating databases
      13:59:13: Thread pool user data ready
      13:59:13: Generating repomd.xml
      13:59:13: Old repodata doesn't exists: Cannot rename /tmp/koji/tasks/465/50465/repo/repodata/ -> /tmp/koji/tasks/465/50465/repo/repodata.old.1302231.20241112135913.309512: cp: cannot stat '/tmp/koji/tasks/465/50465/repo/repodata/': No such file or directory : Child process exited with code 1
      13:59:13: Renamed /tmp/koji/tasks/465/50465/repo/.repodata/ -> /tmp/koji/tasks/465/50465/repo/repodata/
      13:59:13: Memory cleanup
      13:59:13: All done
      Directory walk started
      Directory walk done - 0 packages
      Using comps.xml from target repo
      Temporary output repo path: /tmp/koji/tasks/465/50465/repo/.repodata/
      Preparing sqlite DBs
      Pool started (with 5 workers)
      Pool finished

      ==> 50465:DEFAULT:mergerepos.log <==
      $ /usr/bin/mergerepo_c --koji -b /mnt/koji/repos/local41-build/15703/x86_64/blocklist -a x86_64 -o /tmp/koji/tasks/465/50465/repo --arch-expand -g /mnt/koji/repos/local41-build/15703/groups/comps.xml -r file:///tmp/koji/tasks/465/50465/repo_15703_premerge/ -r https://mirror.facebook.net/fedora/linux/updates/41/Everything/x86_64/ -r https://mirror.facebook.net/fedora/linux/releases/41/Everything/x86_64/os/ -r http://repos.redacted.com/cblr/repo_mirror/local41-stable/
      C_CREATEREPOLIB: Critical: cr_metadata_load_xml: Error encountered while parsing
      C_CREATEREPOLIB: Critical: Cannot load repo: "https://mirror.facebook.net/fedora/linux/updates/41/Everything/x86_64/"
      C_CREATEREPOLIB: Critical: cr_metadata_load_xml: Error encountered while parsing
      Critical: Cannot load repo: "https://mirror.facebook.net/fedora/linux/updates/41/Everything/x86_64/" : Error encountered while parsing:primary.xml parsing: Cannot open /tmp/koji/tasks/465/50465/createrepo_c_tmp_repo_BAZ7wH/repodata/7fc131418a87157ce23f4d4bd6083ab68cdccd81911864b9f5a2d01657ad75cf-primary.xml.zst: Cannot detect compression type
      type group added to list from path: https://mirror.facebook.net/fedora/linux/releases/41/Everything/x86_64/os/repodata/e0139f84a99188422fc0b0ce898b3eea06ee4b9702158c5a24adc70047c5a760-comps-Everything.x86_64.xml.gz
      type group_zck added to list from path: https://mirror.facebook.net/fedora/linux/releases/41/Everything/x86_64/os/repodata/7911c7e6fda34417e3f36955ec4c93ac6d86714b11a2ebcb78832a2e97f3c558-comps-Everything.x86_64.xml.zck
      type group added to list from path: /tmp/koji/tasks/465/50465/createrepo_c_tmp_repo_Tznesr/repodata/e0139f84a99188422fc0b0ce898b3eea06ee4b9702158c5a24adc70047c5a760-comps-Everything.x86_64.xml.gz
      type group_zck added to list from path: /tmp/koji/tasks/465/50465/createrepo_c_tmp_repo_Tznesr/repodata/7911c7e6fda34417e3f36955ec4c93ac6d86714b11a2ebcb78832a2e97f3c558-comps-Everything.x86_64.xml.zck
      type group added to list from path: https://mirror.facebook.net/fedora/linux/updates/41/Everything/x86_64/repodata/d345a0c1291d32f3e166b85152226b37eac5442ab466088f3444f27a24263fee-comps-Everything.x86_64.xml.zst
      type group_zck added to list from path: https://mirror.facebook.net/fedora/linux/updates/41/Everything/x86_64/repodata/aaddd0ad8d29afa50e97dda8e059e10863c62bcbb4ff26c1d34948c6d8fa575e-comps-Everything.x86_64.xml.zck
      type updateinfo added to list from path: https://mirror.facebook.net/fedora/linux/updates/41/Everything/x86_64/repodata/0e99af50d2e8ddb2f6f6725d98fc7a2d7fe3c3acadfc95d3fa228ab54201ccd8-updateinfo.xml.zst
      type updateinfo_zck added to list from path: https://mirror.facebook.net/fedora/linux/updates/41/Everything/x86_64/repodata/9b2013712bedb1a78837448d8de6d6212d8ae103c8cd0c0f08792623f809aa27-updateinfo.xml.zck
      type group added to list from path: /tmp/koji/tasks/465/50465/createrepo_c_tmp_repo_BAZ7wH/repodata/d345a0c1291d32f3e166b85152226b37eac5442ab466088f3444f27a24263fee-comps-Everything.x86_64.xml.zst
      type group_zck added to list from path: /tmp/koji/tasks/465/50465/createrepo_c_tmp_repo_BAZ7wH/repodata/aaddd0ad8d29afa50e97dda8e059e10863c62bcbb4ff26c1d34948c6d8fa575e-comps-Everything.x86_64.xml.zck
      type updateinfo added to list from path: /tmp/koji/tasks/465/50465/createrepo_c_tmp_repo_BAZ7wH/repodata/0e99af50d2e8ddb2f6f6725d98fc7a2d7fe3c3acadfc95d3fa228ab54201ccd8-updateinfo.xml.zst
      type updateinfo_zck added to list from path: /tmp/koji/tasks/465/50465/createrepo_c_tmp_repo_BAZ7wH/repodata/9b2013712bedb1a78837448d8de6d6212d8ae103c8cd0c0f08792623f809aa27-updateinfo.xml.zck
      type group added to list from path: /tmp/koji/tasks/465/50465/repo_15703_premerge/repodata/e72b6b237a363e7da04bdb2a175b2010e0253db6015e865668a630bca68618c5-comps.xml
      type group_gz added to list from path: /tmp/koji/tasks/465/50465/repo_15703_premerge/repodata/be56b12220aed934709e1d9d9ac48d3981896ed74f1c26c7688745e5630eaf88-comps.xml.gz

      ==> Task has completed <==
      ```

      1. When do you need this to be done by? (YYYY/MM/DD)

        As soon as is convenient.

              rhn-engineering-jantill James Antill
              rh-ee-mkonecny Michal Konecny
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: