Description of problem:

Sometimes when end-users obtain their CA and Signed certificates for satellites and capsules, often those certs can have some comments mentioned before each certificate block.

If any such comment or any of the content of the CA bundle contains a non-ascii character, Then despite the Satellite server functioning just fine, The capsule server will never be able to sync content from the satellite.

Version-Release number of selected component (if applicable):

Any Version of Satellite 6 on Pulp3 ( Tested on Satellite 6.14 \ Reported on 6.14 + 6.11 )

How reproducible:

Always and easily

Steps to Reproduce:
1. Install a Satellite and Capsule server with default certs
2. Create/Obtain SSL and CA bundle certificates for satellite and capsule
3. Add a comment at the very top of the CA bundle cert with a non-ascii charecter e.g. something like

1. saydas Intermédiaire CA

4. Now install those certs in both satellite and capsule.

5. Add Library lifecycle to capsule server for content syncing.

6. Import manifest in satellite and then Enable and Sync some repos in Satellite server.

7. Observe the auto-sync triggered for the capsule server or else manually trigger one.

Actual results:

Sync fails for every repo on the capsule server

~~~
Jan 14 13:22:03 saydas-capsule pulpcore-worker-1[38266]: pulp [27024eb5-c48c-4d8a-be6f-c47f1d633911]: pulpcore.tasking.pulpcore_worker:INFO: Starting task 381b7d4b-57e8-46b1-b585-ced536b3a6ea
Jan 14 13:22:03 saydas-capsule pulpcore-worker-1[38266]: pulp [27024eb5-c48c-4d8a-be6f-c47f1d633911]: pulp_rpm.app.tasks.synchronizing:INFO: Synchronizing: repository=54d852d4-2c81-4776-bb4c-8a5bdbe53166 remote=54d852d4-2c81-4776-bb4c-8a5bdbe53166
Jan 14 13:22:03 saydas-capsule pulpcore-worker-1[38266]: pulp [27024eb5-c48c-4d8a-be6f-c47f1d633911]: pulpcore.tasking.pulpcore_worker:INFO: Task 381b7d4b-57e8-46b1-b585-ced536b3a6ea failed (cadata should be an ASCII string or a bytes-like object)
Jan 14 13:22:03 saydas-capsule pulpcore-worker-1[38266]: pulp [27024eb5-c48c-4d8a-be6f-c47f1d633911]: pulpcore.tasking.pulpcore_worker:INFO: File "/usr/lib/python3.9/site-packages/pulpcore/tasking/pulpcore_worker.py", line 460, in execute_task
Jan 14 13:22:03 saydas-capsule pulpcore-worker-1[38266]: result = func(*args, **kwargs)
Jan 14 13:22:03 saydas-capsule pulpcore-worker-1[38266]: File "/usr/lib/python3.9/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 482, in synchronize
Jan 14 13:22:03 saydas-capsule pulpcore-worker-1[38266]: remote_url = fetch_remote_url(remote, url)
Jan 14 13:22:03 saydas-capsule pulpcore-worker-1[38266]: File "/usr/lib/python3.9/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 285, in fetch_remote_url
Jan 14 13:22:03 saydas-capsule pulpcore-worker-1[38266]: get_repomd_file(remote, normalized_remote_url)
Jan 14 13:22:03 saydas-capsule pulpcore-worker-1[38266]: File "/usr/lib/python3.9/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 240, in get_repomd_file
Jan 14 13:22:03 saydas-capsule pulpcore-worker-1[38266]: downloader = remote.get_downloader(url=urlpath_sanitize(url, "repodata/repomd.xml"))
Jan 14 13:22:03 saydas-capsule pulpcore-worker-1[38266]: File "/usr/lib/python3.9/site-packages/pulp_rpm/app/models/repository.py", line 106, in get_downloader
Jan 14 13:22:03 saydas-capsule pulpcore-worker-1[38266]: return super().get_downloader(remote_artifact=remote_artifact, url=url, **kwargs)
Jan 14 13:22:03 saydas-capsule pulpcore-worker-1[38266]: File "/usr/lib/python3.9/site-packages/pulpcore/app/models/repository.py", line 476, in get_downloader
Jan 14 13:22:03 saydas-capsule pulpcore-worker-1[38266]: download_factory = self.download_factory
Jan 14 13:22:03 saydas-capsule pulpcore-worker-1[38266]: File "/usr/lib/python3.9/site-packages/pulp_rpm/app/models/repository.py", line 74, in download_factory
Jan 14 13:22:03 saydas-capsule pulpcore-worker-1[38266]: self._download_factory = DownloaderFactory(
Jan 14 13:22:03 saydas-capsule pulpcore-worker-1[38266]: File "/usr/lib/python3.9/site-packages/pulpcore/download/factory.py", line 78, in _init_
Jan 14 13:22:03 saydas-capsule pulpcore-worker-1[38266]: self._session = self._make_aiohttp_session_from_remote()
Jan 14 13:22:03 saydas-capsule pulpcore-worker-1[38266]: File "/usr/lib/python3.9/site-packages/pulpcore/download/factory.py", line 109, in _make_aiohttp_session_from_remote
Jan 14 13:22:03 saydas-capsule pulpcore-worker-1[38266]: sslcontext = ssl.create_default_context(cadata=self._remote.ca_cert)
Jan 14 13:22:03 saydas-capsule pulpcore-worker-1[38266]: File "/usr/lib64/python3.9/ssl.py", line 746, in create_default_context
Jan 14 13:22:03 saydas-capsule pulpcore-worker-1[38266]: context.load_verify_locations(cafile, capath, cadata)
~~~

Reason is well explained in the traceback i.e. the ca_cert contains a non-ascii charecter.

From pulpcore DB:

pulpcore=# select distinct ca_cert from core_remote;
ca_cert
------------------------------------------------------------------

1. saydas Intermédiaire CA +
   ----BEGIN CERTIFICATE---- +
   MIIF3DCCA8SgAwIBAgIBATANBgkqhkiG9w0BAQsFADCBgTELMAkGA1UEBhMCSU4x+
   CzAJBgNVBAgTAldCMQwwCgYDVQQHEwNLT0wxDzANBgNVBAoTBlJlZEhhdDELMAkG+
   A1UECxMCWEUxFzAVBgNVBAMTDnNheWRhcy5wbnEuY3NiMSAwHgYJKoZIhvcNAQkB+
   FhFzYXlkYXNAcmVkaGF0LmNvbTAeFw0yMzA0MjUxMTUwMDNaFw0zMDA3MjcxMTUw+
   MDNaMHkxCzAJBgNVBAYTAklOMQswCQYDVQQIEwJXQjEPMA0GA1UEChMGUmVkSGF0+
   ...
   ..
   .. output snipped ..

here "é" is the non-ascii character creating the problem.

Expected results:

Either katello-certs-check itself will be able to detect the non-ascii character and then inform end-user
Or, Satellite\Capsule\Pulp\Katello would only use the content of valid certificate blocks in a file ( ignoring any comments inbetween ).

Additional info:

Fix is rather simple here i.e.

• On capsule, clear all the remote objects:

1. PULP_SETTINGS='/etc/pulp/settings.py' DJANGO_SETTINGS_MODULE='pulpcore.app.settings' pulpcore-manager shell << EOF
   from pulpcore.app.models import Remote
   Remote.objects.all().delete()
   EOF

• On Satellite, Remove the ascii charecter from the CA bundle and re-apply the certs with "--certs-update-server --certs-update-server-ca" flag. Re-deploy the same CA on capsule as well via capsule-certs-generate and satellite-installer.

• Perform a "Complete Sync" of the capsule server to ReCreate the Pulp Remotes with correct certs.