+++ This bug was initially created as a clone of Bug #2161993 +++

Description of problem:

If i will upload rpm for example xxx-1.2.3.rpm into repository, but inside that rpm the name is for example yyy, repository metadata will point into yyy even though physically the file is named xxx and pulpcore db data pointing correctly to xxx-1.2.3.rpm.

Version-Release number of selected component (if applicable):
Satellite 6.12.0-4

How reproducible:
Always

Steps to Reproduce:
1. Create custom repository
2. Upload rpm
3. Install rpm on client

Actual results:
[Errno 14] HTTPS Error 404 - Not Found

Expected results:
Successful installation

Additional info:
As workaround repository can be removed and manually removed records in pulpcore database, after that physically rename rpm to correspond to its metadata and after that file will be pointed correctly to actual location.

More details internally.

— Additional comment from on 2023-01-18T14:30:22Z

Comment private because it is about 3rd party rpm and results are from my lab Satellite, so yes if reproducer will be needed i can share it.

RPM: agent-2.18.7_x64.rpm

1. rpm -qi ~/Downloads/hurukai_agent-2.18.7-1.x86_64.rpm
   Name : hurukai_agent

Related pulpcore details:

1. select content_ptr_id, name, version, release, location_href from rpm_package where name ilike '%hurukai%';
   content_ptr_id | name | version | release | location_href
   --------------------------------------------------------------------------------------------------
   7e937578-255a-4e7d-94ec-f0ddd65f4d72 | hurukai_agent | 2.18.7 | 1 | hurukai_agent-2.18.7-1.x86_64.rpm

1. select * from core_content where pulp_id = '7e937578-255a-4e7d-94ec-f0ddd65f4d72';
   pulp_id | pulp_created | pulp_last_updated | pulp_type | upstream_id | timestamp_of_interest
   ---------------------------------------------------------------------------------------------------------------------+------------------------------
   7e937578-255a-4e7d-94ec-f0ddd65f4d72 | 2023-01-18 15:18:33.132096+01 | 2023-01-18 15:18:33.132124+01 | rpm.package | | 2023-01-18 15:18:33.797957+01
   (1 row)

1. select * from core_contentartifact where content_id = '7e937578-255a-4e7d-94ec-f0ddd65f4d72';
   pulp_id | pulp_created | pulp_last_updated | relative_path | artifact_id | content_id
   -------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------
   c7984961-d2f3-42f3-95b3-1edf5115a55c | 2023-01-18 15:18:33.152946+01 | 2023-01-18 15:18:33.152972+01 | agent-2.18.7_x64.rpm | e61ae1e9-11a8-4159-a1e1-7b50586e69ac | 7e937578-255a-4e7d-94ec-f0ddd65f4d72
   (1 row)

1. select * from core_artifact where pulp_id = 'e61ae1e9-11a8-4159-a1e1-7b50586e69ac';
   pulp_id | pulp_created | pulp_last_updated | file | size | md5 |
   sha1 | sha224 | sha256 | sha384

   sha512

   timestamp_of_interest ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ----------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------- ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- e61ae1e9-11a8-4159-a1e1-7b50586e69ac

   2023-01-18 15:18:31.99271+01

   2023-01-18 15:18:31.992743+01

   artifact/a6/dd20f0974bbfc60a9d61370ac6a662a15966fac0eab4ab204f1fd1aa78eecd

   35921280

   5a0cc8b6a1ba035108559702e2dd66d6

   f3497ccd45ea a200774ab0577ef9550a11c89328

   bafdb64a4450d1b34ab0c52e79483bc47b591cef562f7966d7bad640

   a6dd20f0974bbfc60a9d61370ac6a662a15966fac0eab4ab204f1fd1aa78eecd

   911f6ba6b1c9c84c93c479e1a2713849853bc81f4952fe86678b0b9618c1f1cdc14f42307f827c9ff 361babf6498609e

   35f585923af0de422f8921c17a35357c71593f27d1d445dbcbcb295c12ba0c81f401359b466819e7b9c60ff4e5375444832c8a5becca4d17cb1908aa26e3a7c7

   2023-01-18 15:18:31.994784+01 (1 row)

metadata in repository (0953940c937c9eccaa3b15f858cef07c9ffd3f660b0aae74042866782f2bed1c-primary.xml):

<metadata xmlns="http://linux.duke.edu/metadata/common" xmlns:rpm="http://linux.duke.edu/metadata/rpm" packages="1">
<package type="rpm">
<name>hurukai_agent</name>
<arch>x86_64</arch>
<version epoch="0" ver="2.18.7" rel="1"/>
<checksum type="sha256" pkgid="YES">a6dd20f0974bbfc60a9d61370ac6a662a15966fac0eab4ab204f1fd1aa78eecd</checksum>
<summary>Harfanglab's Hurukai agent</summary>
<description>EDR agent from Harfanglab</description>
<packager/>
<url/>
<time file="1674051512" build="1666262437"/>
<size package="35921280" installed="58472679" archive="58746112"/>
<location href="Packages/h/hurukai_agent-2.18.7-1.x86_64.rpm"/> <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
<format>
<rpm:license>unknown</rpm:license>
<rpm:vendor>HarfangLab</rpm:vendor>
<rpm:group>unknown</rpm:group>
<rpm:buildhost>runner-ifsmsd8v-project-12-concurrent-0zxlp8</rpm:buildhost>
<rpm:sourcerpm>hurukai_agent-2.18.7-1.src.rpm</rpm:sourcerpm>
<rpm:header-range start="280" end="219580"/>
<rpm:provides>
<rpm:entry name="hurukai_agent(x86-64)" flags="EQ" epoch="0" ver="2.18.7" rel="1"/>
<rpm:entry name="hurukai_agent" flags="EQ" epoch="0" ver="2.18.7" rel="1"/>
</rpm:provides>
<rpm:requires>
<rpm:entry name="/bin/sh"/>
<rpm:entry name="/bin/sh" pre="1"/>
<rpm:entry name="/bin/sh"/>
<rpm:entry name="/bin/sh" pre="1"/>
<rpm:entry name="logrotate"/>
</rpm:requires>
<file>/opt/hurukai/bin/hurukai-run</file>
<file>/opt/hurukai/bin/hurukai-doctor</file>
<file>/opt/hurukai/bin/hurukai-configure</file>
</format>
</package>
</metadata>

If reproducer is needed please let me know, i can share the details.

— Additional comment from on 2023-01-18T14:34:22Z

1. yum install hurakai_agent on clien

[MIRROR] hurukai_agent-2.18.7-1.x86_64.rpm: Status code: 404 for https://jjansat612.usersys.redhat.com/pulp/content/JJAN_ORG/Library/custom/Custom/test/Packages/h/hurukai_agent-2.18.7-1.x86_64.rpm (IP: 192.168.38.83)

but physical location in repo is

https://jjansat612.usersys.redhat.com/pulp/content/JJAN_ORG/Library/custom/Custom/test/Packages/a/agent-2.18.7_x64.rpm

— Additional comment from on 2023-01-20T11:54:10Z

• • • Bug 2161992 has been marked as a duplicate of this bug. ***

— Additional comment from on 2023-01-23T14:41:14Z

Ian,

Would this be on Katello or Pulp?

Thanks!

— Additional comment from on 2023-01-23T15:19:26Z

I'm going to put this on Pulp for now. It sounds like Pulp is exposing the RPM with the name that is in the RPM's metadata rather than the given filename. It would be ideal if Pulp could use the given name instead.

@dalley@redhat.com does that sounds plausible? I feel like I remember a conversation around a technical issue with uploading an RPM that has a different name from its metadata, but I can't fully remember.

— Additional comment from on 2023-02-23T05:42:06Z

• • • Bug 2166093 has been marked as a duplicate of this bug. ***

— Additional comment from on 2023-02-23T05:43:49Z

Copying over info from duplicate: https://bugzilla.redhat.com/show_bug.cgi?id=2166093

"Uploading RPMS seems to have name mismatches as reported in the community. This seems to be some behavior where the names are not entirely driven by pulp which seems to break dnf installs due to wrong package name.
https://community.theforeman.org/t/rpm-packages-renamed-when-manually-uploaded/31776

Created from redmine issue https://projects.theforeman.org/issues/35952"

— Additional comment from on 2023-02-23T06:21:25Z

• • • Bug 2162591 has been marked as a duplicate of this bug. ***

— Additional comment from on 2023-02-23T06:23:26Z

Lots of discussion on https://bugzilla.redhat.com/show_bug.cgi?id=2162591, marked as duplicate of this one.

— Additional comment from on 2023-02-23T16:32:11Z

Target Milestone is set to Unspecified, since this bug has Target Milestone set to 6.13.0 and approved release flag is not sat-6.13.0+

Once the bug has been granted the sat-6.13.0+ flag, the Target Milestone can be set to the desired value.

— Additional comment from on 2023-03-08T15:44:28Z

Just from curiosity when it will be fixed, will be there some mechanism how to fix the issue if it already happened.

In meaning that we will for example say customer to upgrade to Satellite 6.13 and run simple rake script in opposite of saying to remove multiple CV version, delete repository, manually clean pulpcore db related records, rename the file and re-create the repo/CVs ?

— Additional comment from on 2023-03-21T18:38:25Z

My customer is not happy about the issue and wanted a workaround / fix ASAP

— Additional comment from on 2023-03-22T06:41:32Z

This seems to be the mismatch of package relative_path and location_href. The mismatch seems to happen when relative_path parameter is provided to the API when uploading the rpm.

As we can see below, relative_path will only set to match the location href if it is not provided.
~~~
In "app/serializers/package.py"

def deferred_validate(self, data):
<snip>
new_pkg["location_href"] = (
format_nevra_short(
new_pkg["name"],
new_pkg["epoch"],
new_pkg["version"],
new_pkg["release"],
new_pkg["arch"],
)
+ ".rpm"
)
if not data.get("relative_path"):
data["relative_path"] = new_pkg["location_href"] <======================
~~~

Previously in Satelite 6.10 and 6.11, location_href is matching the relative_path
~~~
In app/serializers/package.py:

new_pkg = _prepare_package(data["artifact"], data["relative_path"]) <============

In app/shared_utils.py:

def _prepare_package(artifact, filename):
<snip>
package = Package.createrepo_to_dict(cr_pkginfo)

package["location_href"] = filename <=============================
artifact_file.close()
return package
~~~

— Additional comment from on 2023-03-22T06:45:11Z

I think this is the change that cause the regression.

commit 274054d8946d3a0bfc30fd91896e2eac2a10f9b9
Author: Pavel Picka <ppicka@redhat.com>
Date: Wed May 18 11:14:57 2022 +0200

Optional relative_path when upload

User doesn't have to specify ``relative_path``.
It is generated by NeVRA info from package if not specified.

Also remove relative_path from test.

closes: #2440
https://github.com/pulp/pulp_rpm/issues/2440

Require PR: https://github.com/pulp/pulp-cli/pull/507

[nocoverage

— Additional comment from on 2023-03-22T07:45:40Z

I applied the following patch and it prevented the issue.
~~~
— a/app/serializers/package.py 2023-03-22 03:14:08.474670651 -0400
+++ b/app/serializers/package.py 2023-03-22 03:17:30.603622517 -0400
@@ -274,7 +274,7 @@
_("There is already a package with:

{values}

.").format(values=error_data)
)

• new_pkg["location_href"] = (
  + filename = (
  format_nevra_short(
  new_pkg["name"],
  new_pkg["epoch"],
  @@ -285,7 +285,10 @@
  + ".rpm"
  )
  if not data.get("relative_path"):
• data["relative_path"] = new_pkg["location_href"]
  + data["relative_path"] = filename
  + new_pkg["location_href"] = filename
  + else:
  + new_pkg["location_href"] = data["relative_path"]

data.update(new_pkg)
return data
~~~

Unfortunately, to fix the already broken packages in the custom repo we need to:
1. Remove all the packages in the repo and then remove the repo. This is because Satellite doesn't let you to delete repo versions.
2. Set orphan protection time setting to 1. After that call remove orphans. Wait for the task to finish successfully.
3. Recreate the repo and upload the packages.

Alternatively, we can go to the DB to manually correct the "rpm_package.location_href" to match "core_contentartifact.relative_path". After that force publication.

@dalley any idea?

— Additional comment from on 2023-03-27T04:38:24Z

Pulp upstream issue created

https://github.com/pulp/pulp_rpm/issues/3039

— Additional comment from on 2023-03-29T02:25:12Z

@hyu@redhat.com re: comment #15

Probably the former procedure is best in most situations. The latter is an option. We could also publish the metadata based off of the relative_path rather than the location_href. location_href should never have been in the package metadata to begin with, it's a historical mistake we should eventually move away from.

Probably #2 is easiest and fastest in a support context if a user can't do #1 for some reason.

— Additional comment from on 2023-03-29T03:19:27Z

Hi Daniel

(In reply to Daniel Alley from comment #17)
> @hyu@redhat.com re: comment #15
>
> We could also publish the metadata based off of the relative_path
> rather than the location_href. location_href should never have been in the
> package metadata to begin with, it's a historical mistake we should
> eventually move away from.
>

Ah, good to know tthis.

> Probably #2 is easiest and fastest in a support context if a user can't do
> #1 for some reason.

Yeah, in the case that the custom repo is also published to the content views, it will be very troublesome for the customer delete all of them.

— Additional comment from on 2023-03-30T17:47:40Z

Yum repositories are only supposed to contain RPMs named $name-$version-$release.$arch.rpm
(Note: No epoch! I'm not entirely sure why the epoch isn't included, maybe for historical reasons, but this seems to be a convention that is hard-coded in lots of things.)

Unfortunately, some vendors that provide RPMs do not correctly name their RPM files. For example, IBM provides RPMs for Tivoli, but does not include the version/release in the filename: TIVsm-API64.x86_64.rpm

In Satellite versions that used Pulp 2, the filename of an uploaded RPM file was ignored, and RPMs served by Satellite/Pulp used an automatically-generated filename based on the name/version/release/arch extracted from the metadata within the RPM.

However, in Satellite 6.10/6.11, the filename on uploaded RPMs is used as-is. Among other problems, this can break things if multiple RPMs are uploaded with the same (improper) filename. See RedHat case 03309188 and https://bugzilla.redhat.com/show_bug.cgi?id=2151657 for example.

In Satellite 6.12, it appears that uploaded RPMs might now be renamed automatically.
However, the generated name in the <location> tag in the relevant repodata file seems to includes the epoch. For example: <location href="Packages/o/openssl-1:1.0.2k-26.el7_9.x86_64.rpm"/>
But the name that is served by Pulp does not include the epoch, which breaks things for any package name that includes an epoch. (I haven't tested whether Pulp is correctly renaming the file without the epoch, or whether it is still using the uploaded filename as-is which happened to be correct in this test.) For example:

1. yum update
   ...
   openssl-1:1.0.2k-26.el7_9 FAILED 65% [=============================================- ] 30 MB/s | 56 MB 00:00:00 ETA
   https://satellite/pulp/repos/ORG/Env/RHEL7_COMP_VIEW/custom/Test/RHEL7_Test/Packages/o/openssl-1%3A1.0.2k-26.el7_9.x86_64.rpm: [Errno 14] HTTPS Error 404 - Not Found
2. curl --silent --cacert /etc/pki/ca-trust/source/anchors/katello-server-ca.pem --cert "/etc/pki/entitlement/$EntSerial.pem" --key "/etc/pki/entitlement/$EntSerial-key.pem" "https://$SatServer/pulp/repos/ORG/Env/RHEL7_COMP_VIEW/custom/Test/RHEL7_Test/Packages/o/" | grep openssl-1
   <li><a href="openssl-1.0.2k-26.el7_9.x86_64.rpm">openssl-1.0.2k-26.el7_9.x86_64.rpm</a></li>

At first glance, it looks like the above location_href code in "app/serializers/package.py" that is using the epoch is probably the issue here.

— Additional comment from on 2023-03-30T18:06:11Z

With a little more digging, it looks like Satellite 6.12 only generates the name in the <location> tag, and does not rename the files that are imported into / served by Pulp.

So, basically, this breaks an even wider range of use cases than the Satellite 6.10/6.11 behavior. (6.10/6.11 only broke uploads of RPMs if both the filename was incorrect and two different RPMs with the same filename were uploaded; 6.12 now breaks any upload with an incorrect filename or with an epoch used in the RPM.)

— Additional comment from on 2023-03-31T16:08:55Z

Fixes and more discussion:
https://bugzilla.redhat.com/show_bug.cgi?id=2151657
https://github.com/pulp/pulp_rpm/issues/3039
https://github.com/Katello/katello/pull/10329