-
Bug
-
Resolution: Unresolved
-
Critical
-
None
-
rhos-17.1.8, rhos-17.1.9
-
None
-
False
-
-
False
-
?
-
None
-
-
-
-
Critical
Customer with RHOSP 17.1.8 is unable to upload signed images to Glance. Problem is reproducible in RHOSP 17.1.9 lab.
To Reproduce Steps to reproduce the behavior: basically, to follow steps from https://docs.redhat.com/en/documentation/red_hat_openstack_platform/17.1/html/managing_secrets_with_the_key_manager_service/assembly-encrypting-validating-openstack-services_rhosp#proc-validating-image-service-images_key-manager-services . Here are example commands from my lab:
openssl genrsa -out private_key.pem 1024 openssl rsa -pubout -in private_key.pem -out public_key.pem openssl req -new -key private_key.pem -out cert_request.csr You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [XX]:CZ State or Province Name (full name) []:Jihomoravsky kraj Locality Name (eg, city) [Default City]:Brno Organization Name (eg, company) [Default Company Ltd]:Example corp Organizational Unit Name (eg, section) []:Support Common Name (eg, your name or your server's hostname) []: Email Address []: Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: An optional company name []: openssl x509 -req -days 14 -in cert_request.csr -signkey private_key.pem -out x509_signing_cert.crt Certificate request self-signature ok subject=C = CZ, ST = Jihomoravsky kraj, L = Brno, O = Example corp, OU = Support openstack secret store --name signing-cert --algorithm RSA --secret-type certificate --payload-content-type "application/octet-stream" --payload-content-encoding base64 --payload "$(base64 x509_signing_cert.crt)" -c 'Secret href' -f value http://10.0.0.148:9311/v1/secrets/54740174-e675-475e-a2ae-f89047493b53 openstack image save cirros --file cirros.img openssl dgst -sha256 -sign private_key.pem -sigopt rsa_padding_mode:pss -out cirros.signature cirros.img base64 -w 0 cirros.signature > cirros_signature_base64 cirros_disk_signature_base64=$(cat cirros_signature_base64) openstack image create -vv --container-format bare --disk-format qcow2 --property img_signature="$cirros_disk_signature_base64" --property img_signature_certificate_uuid="54740174-e675-475e-a2ae-f89047493b53" --property img_signature_hash_method="SHA-256" --property img_signature_key_type="RSA-PSS" cirros_signed --file ./cirros.img --public --fit-width OMITTED REQ: curl -g -i -X POST http://10.0.0.148:9292/v2/images -H "Content-Type: application/json" -H "User-Agent: openstacksdk/0.55.1 keystoneauth1/4.4.0 python-requests/2.25.1 CPython/3.9.16" -H "X-Auth-Token: {SHA256}3d1dd3e7387fb0b876ccaef830ade51247590b98b16783190e20e359e58c5d1c" -d '{"disk_format": "qcow2", "visibility": "public", "name": "cirros_signed", "container_format": "bare", "img_signature": "gQIuC0CLU8dMLUglKx7Ac127TGwCeQJukqt5WY7kQHSjVM3j/wy88s7tSIXdkhS+cwv68onRGKIqjNVXu9LmGPJD3Qz05chASaGoXiKTer32oWjSVEcoBqySLxNgqQGYmi7QOhcqMP8idDiVh7qdLSEbz9usTxFQLYq1ZwEMx9Y=", "img_signature_certificate_uuid": "54740174-e675-475e-a2ae-f89047493b53", "img_signature_hash_method": "SHA-256", "img_signature_key_type": "RSA-PSS", "owner_specified.openstack.md5": "", "owner_specified.openstack.sha256": "", "owner_specified.openstack.object": "images/cirros_signed"}' http://10.0.0.148:9292 "POST /v2/images HTTP/1.1" 201 1103 RESP: [201] content-length: 1103 content-type: application/json date: Mon, 08 Sep 2025 18:13:48 GMT location: http://10.0.0.148:9292/v2/images/92d32974-bb62-4d41-9f99-ae4b83a6c510 openstack-image-import-methods: web-download openstack-image-store-ids: default_backend x-openstack-request-id: req-467363ac-e7d4-4bd9-9c83-3a6289e7608d RESP BODY: {"img_signature": "gQIuC0CLU8dMLUglKx7Ac127TGwCeQJukqt5WY7kQHSjVM3j/wy88s7tSIXdkhS+cwv68onRGKIqjNVXu9LmGPJD3Qz05chASaGoXiKTer32oWjSVEcoBqySLxNgqQGYmi7QOhcqMP8idDiVh7qdLSEbz9usTxFQLYq1ZwEMx9Y=", "img_signature_certificate_uuid": "54740174-e675-475e-a2ae-f89047493b53", "img_signature_hash_method": "SHA-256", "img_signature_key_type": "RSA-PSS", "owner_specified.openstack.md5": "", "owner_specified.openstack.sha256": "", "owner_specified.openstack.object": "images/cirros_signed", "name": "cirros_signed", "disk_format": "qcow2", "container_format": "bare", "visibility": "public", "size": null, "virtual_size": null, "status": "queued", "checksum": null, "protected": false, "min_ram": 0, "min_disk": 0, "owner": "9a3cfa6bd2b1463daf4655254affbe56", "os_hidden": false, "os_hash_algo": null, "os_hash_value": null, "id": "92d32974-bb62-4d41-9f99-ae4b83a6c510", "created_at": "2025-09-08T18:13:48Z", "updated_at": "2025-09-08T18:13:48Z", "tags": [], "self": "/v2/images/92d32974-bb62-4d41-9f99-ae4b83a6c510", "file": "/v2/images/92d32974-bb62-4d41-9f99-ae4b83a6c510/file", "schema": "/v2/schemas/image"} POST call to image for http://10.0.0.148:9292/v2/images used request id req-467363ac-e7d4-4bd9-9c83-3a6289e7608d REQ: curl -g -i -X PUT http://10.0.0.148:9292/v2/images/92d32974-bb62-4d41-9f99-ae4b83a6c510/file -H "Accept: " -H "Content-Type: application/octet-stream" -H "User-Agent: openstacksdk/0.55.1 keystoneauth1/4.4.0 python-requests/2.25.1 CPython/3.9.16" -H "X-Auth-Token: {SHA256}3d1dd3e7387fb0b876ccaef830ade51247590b98b16783190e20e359e58c5d1c" -d '<_io.BufferedReader name='./cirros.img'>' http://10.0.0.148:9292 "PUT /v2/images/92d32974-bb62-4d41-9f99-ae4b83a6c510/file HTTP/1.1" 400 129 RESP: [400] content-length: 129 content-type: text/plain; charset=UTF-8 date: Mon, 08 Sep 2025 18:13:52 GMT x-openstack-request-id: req-9c72a8cb-6ed7-430a-b5fe-00d926f5c64b RESP BODY: Omitted, Content-Type is set to text/plain; charset=UTF-8. Only application/json responses have their bodies logged. PUT call to image for http://10.0.0.148:9292/v2/images/92d32974-bb62-4d41-9f99-ae4b83a6c510/file used request id req-9c72a8cb-6ed7-430a-b5fe-00d926f5c64b OMITTED
Extracts from glance logs:
2025-09-08 18:13:49.710 46 DEBUG glance_store._drivers.rbd [req-9c72a8cb-6ed7-430a-b5fe-00d926f5c64b 4c622554343c4f57b012f8755efe3a3b 9a3cfa6bd2b1463daf4655254affbe56 - default default] creating image 92d32974-bb62-4d41-9f99-ae4b83a6c510 with order 23 and size 12716032 add /usr/lib/python3.9/site-packages/glance_store/_drivers/rbd.py:570 2025-09-08 18:13:51.060 46 DEBUG glance_store.multi_backend [req-9c72a8cb-6ed7-430a-b5fe-00d926f5c64b 4c622554343c4f57b012f8755efe3a3b 9a3cfa6bd2b1463daf4655254affbe56 - default default] Attempting to import store rbd _load_multi_store /usr/lib/python3.9/site-packages/glance_store/multi_backend.py:171 2025-09-08 18:13:51.080 46 DEBUG glance_store.capabilities [req-9c72a8cb-6ed7-430a-b5fe-00d926f5c64b 4c622554343c4f57b012f8755efe3a3b 9a3cfa6bd2b1463daf4655254affbe56 - default default] Store glance_store._drivers.rbd.Store doesn't support updating dynamic storage capabilities. Please overwrite 'update_capabilities' method of the store to implement updating logics if needed. update_capabilities /usr/lib/python3.9/site-packages/glance_store/capabilities.py:91 2025-09-08 18:13:51.081 46 DEBUG glance_store.driver [req-9c72a8cb-6ed7-430a-b5fe-00d926f5c64b 4c622554343c4f57b012f8755efe3a3b 9a3cfa6bd2b1463daf4655254affbe56 - default default] Late loading location class glance_store._drivers.rbd.StoreLocation get_store_location_class /usr/lib/python3.9/site-packages/glance_store/driver.py:117 2025-09-08 18:13:51.081 46 DEBUG glance_store.location [req-9c72a8cb-6ed7-430a-b5fe-00d926f5c64b 4c622554343c4f57b012f8755efe3a3b 9a3cfa6bd2b1463daf4655254affbe56 - default default] Registering scheme rbd with {'default_backend': {'store': <glance_store._drivers.rbd.Store object at 0x7f9b669fd3d0>, 'location_class': <class 'glance_store._drivers.rbd.StoreLocation'>, 'store_entry': 'rbd'}} register_scheme_backend_map /usr/lib/python3.9/site-packages/glance_store/location.py:132 2025-09-08 18:13:51.607 47 DEBUG eventlet.wsgi.server [-] (47) accepted ('172.17.1.39', 34648) server /usr/lib/python3.9/site-packages/eventlet/wsgi.py:992 2025-09-08 18:13:51.611 47 INFO eventlet.wsgi.server [-] 172.17.1.39 - - [08/Sep/2025 18:13:51] "GET /healthcheck HTTP/1.0" 200 137 0.002678 2025-09-08 18:13:52.202 46 ERROR glance.api.v2.image_data [req-9c72a8cb-6ed7-430a-b5fe-00d926f5c64b 4c622554343c4f57b012f8755efe3a3b 9a3cfa6bd2b1463daf4655254affbe56 - default default] Signature verification failed for image 92d32974-bb62-4d41-9f99-ae4b83a6c510: Signature verification failed: cursive.exception.SignatureVerificationError: Signature verification failed 2025-09-08 18:13:52.245 46 INFO eventlet.wsgi.server [req-9c72a8cb-6ed7-430a-b5fe-00d926f5c64b 4c622554343c4f57b012f8755efe3a3b 9a3cfa6bd2b1463daf4655254affbe56 - default default] 10.0.0.60,172.17.1.39 - - [08/Sep/2025 18:13:52] "PUT /v2/images/92d32974-bb62-4d41-9f99-ae4b83a6c510/file HTTP/1.1" 400 322 3.617276
Expected behavior
Signed Glance image is successfully upload when documentation is followed as is.
Bug impact
Usage of signed glance images is blocked
Known workaround
None
Additional context
Sosreports attached on 9/6/2025 contain relevant logs, "Barbican Implementation steps and Testing.docx" contains customer's steps to reproduce. But problem is also reproducible in lab