Uploaded image for project: 'Red Hat OpenStack Services on OpenShift'
  1. Red Hat OpenStack Services on OpenShift
  2. OSPRH-19756

cursive.exception.SignatureVerificationError is returned when trying to upload signed image

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Critical Critical
    • None
    • rhos-17.1.8, rhos-17.1.9
    • None
    • False
    • Hide

      None

      Show
      None
    • False
    • ?
    • None
    • Critical

      Customer with RHOSP 17.1.8 is unable to upload signed images to Glance. Problem is reproducible in RHOSP 17.1.9 lab.

      To Reproduce Steps to reproduce the behavior: basically, to follow steps from https://docs.redhat.com/en/documentation/red_hat_openstack_platform/17.1/html/managing_secrets_with_the_key_manager_service/assembly-encrypting-validating-openstack-services_rhosp#proc-validating-image-service-images_key-manager-services . Here are example commands from my lab:

      openssl genrsa -out private_key.pem 1024
      
      openssl rsa -pubout -in private_key.pem -out public_key.pem
      
      openssl req -new -key private_key.pem -out cert_request.csr
      You are about to be asked to enter information that will be incorporated
      into your certificate request.
      What you are about to enter is what is called a Distinguished Name or a DN.
      There are quite a few fields but you can leave some blank
      For some fields there will be a default value,
      If you enter '.', the field will be left blank.
      -----
      Country Name (2 letter code) [XX]:CZ
      State or Province Name (full name) []:Jihomoravsky kraj
      Locality Name (eg, city) [Default City]:Brno
      Organization Name (eg, company) [Default Company Ltd]:Example corp
      Organizational Unit Name (eg, section) []:Support
      Common Name (eg, your name or your server's hostname) []:
      Email Address []:
      
      Please enter the following 'extra' attributes
      to be sent with your certificate request
      A challenge password []:
      An optional company name []:
      
      openssl x509 -req -days 14 -in cert_request.csr -signkey private_key.pem -out x509_signing_cert.crt
      Certificate request self-signature ok
      subject=C = CZ, ST = Jihomoravsky kraj, L = Brno, O = Example corp, OU = Support
      
      openstack secret store --name signing-cert --algorithm RSA --secret-type certificate --payload-content-type "application/octet-stream" --payload-content-encoding base64  --payload "$(base64 x509_signing_cert.crt)" -c 'Secret href' -f value
      http://10.0.0.148:9311/v1/secrets/54740174-e675-475e-a2ae-f89047493b53
      
      openstack image save cirros --file cirros.img
      
      openssl dgst -sha256 -sign private_key.pem -sigopt rsa_padding_mode:pss -out cirros.signature cirros.img
      
      base64 -w 0 cirros.signature  > cirros_signature_base64
      
      cirros_disk_signature_base64=$(cat cirros_signature_base64)
      
      
      openstack image create -vv --container-format bare --disk-format qcow2 --property img_signature="$cirros_disk_signature_base64" --property img_signature_certificate_uuid="54740174-e675-475e-a2ae-f89047493b53"  --property img_signature_hash_method="SHA-256" --property img_signature_key_type="RSA-PSS" cirros_signed --file ./cirros.img --public --fit-width
      OMITTED
      REQ: curl -g -i -X POST http://10.0.0.148:9292/v2/images -H "Content-Type: application/json" -H "User-Agent: openstacksdk/0.55.1 keystoneauth1/4.4.0 python-requests/2.25.1 CPython/3.9.16" -H "X-Auth-Token: {SHA256}3d1dd3e7387fb0b876ccaef830ade51247590b98b16783190e20e359e58c5d1c" -d '{"disk_format": "qcow2", "visibility": "public", "name": "cirros_signed", "container_format": "bare", "img_signature": "gQIuC0CLU8dMLUglKx7Ac127TGwCeQJukqt5WY7kQHSjVM3j/wy88s7tSIXdkhS+cwv68onRGKIqjNVXu9LmGPJD3Qz05chASaGoXiKTer32oWjSVEcoBqySLxNgqQGYmi7QOhcqMP8idDiVh7qdLSEbz9usTxFQLYq1ZwEMx9Y=", "img_signature_certificate_uuid": "54740174-e675-475e-a2ae-f89047493b53", "img_signature_hash_method": "SHA-256", "img_signature_key_type": "RSA-PSS", "owner_specified.openstack.md5": "", "owner_specified.openstack.sha256": "", "owner_specified.openstack.object": "images/cirros_signed"}'
      http://10.0.0.148:9292 "POST /v2/images HTTP/1.1" 201 1103
      RESP: [201] content-length: 1103 content-type: application/json date: Mon, 08 Sep 2025 18:13:48 GMT location: http://10.0.0.148:9292/v2/images/92d32974-bb62-4d41-9f99-ae4b83a6c510 openstack-image-import-methods: web-download openstack-image-store-ids: default_backend x-openstack-request-id: req-467363ac-e7d4-4bd9-9c83-3a6289e7608d
      RESP BODY: {"img_signature": "gQIuC0CLU8dMLUglKx7Ac127TGwCeQJukqt5WY7kQHSjVM3j/wy88s7tSIXdkhS+cwv68onRGKIqjNVXu9LmGPJD3Qz05chASaGoXiKTer32oWjSVEcoBqySLxNgqQGYmi7QOhcqMP8idDiVh7qdLSEbz9usTxFQLYq1ZwEMx9Y=", "img_signature_certificate_uuid": "54740174-e675-475e-a2ae-f89047493b53", "img_signature_hash_method": "SHA-256", "img_signature_key_type": "RSA-PSS", "owner_specified.openstack.md5": "", "owner_specified.openstack.sha256": "", "owner_specified.openstack.object": "images/cirros_signed", "name": "cirros_signed", "disk_format": "qcow2", "container_format": "bare", "visibility": "public", "size": null, "virtual_size": null, "status": "queued", "checksum": null, "protected": false, "min_ram": 0, "min_disk": 0, "owner": "9a3cfa6bd2b1463daf4655254affbe56", "os_hidden": false, "os_hash_algo": null, "os_hash_value": null, "id": "92d32974-bb62-4d41-9f99-ae4b83a6c510", "created_at": "2025-09-08T18:13:48Z", "updated_at": "2025-09-08T18:13:48Z", "tags": [], "self": "/v2/images/92d32974-bb62-4d41-9f99-ae4b83a6c510", "file": "/v2/images/92d32974-bb62-4d41-9f99-ae4b83a6c510/file", "schema": "/v2/schemas/image"}
      POST call to image for http://10.0.0.148:9292/v2/images used request id req-467363ac-e7d4-4bd9-9c83-3a6289e7608d
      REQ: curl -g -i -X PUT http://10.0.0.148:9292/v2/images/92d32974-bb62-4d41-9f99-ae4b83a6c510/file -H "Accept: " -H "Content-Type: application/octet-stream" -H "User-Agent: openstacksdk/0.55.1 keystoneauth1/4.4.0 python-requests/2.25.1 CPython/3.9.16" -H "X-Auth-Token: {SHA256}3d1dd3e7387fb0b876ccaef830ade51247590b98b16783190e20e359e58c5d1c" -d '<_io.BufferedReader name='./cirros.img'>'
      http://10.0.0.148:9292 "PUT /v2/images/92d32974-bb62-4d41-9f99-ae4b83a6c510/file HTTP/1.1" 400 129
      RESP: [400] content-length: 129 content-type: text/plain; charset=UTF-8 date: Mon, 08 Sep 2025 18:13:52 GMT x-openstack-request-id: req-9c72a8cb-6ed7-430a-b5fe-00d926f5c64b
      RESP BODY: Omitted, Content-Type is set to text/plain; charset=UTF-8. Only application/json responses have their bodies logged.
      PUT call to image for http://10.0.0.148:9292/v2/images/92d32974-bb62-4d41-9f99-ae4b83a6c510/file used request id req-9c72a8cb-6ed7-430a-b5fe-00d926f5c64b
      OMITTED
      

      Extracts from glance logs:

      2025-09-08 18:13:49.710 46 DEBUG glance_store._drivers.rbd [req-9c72a8cb-6ed7-430a-b5fe-00d926f5c64b 4c622554343c4f57b012f8755efe3a3b 9a3cfa6bd2b1463daf4655254affbe56 - default default] creating image 92d32974-bb62-4d41-9f99-ae4b83a6c510 with order 23 and size 12716032 add /usr/lib/python3.9/site-packages/glance_store/_drivers/rbd.py:570
      2025-09-08 18:13:51.060 46 DEBUG glance_store.multi_backend [req-9c72a8cb-6ed7-430a-b5fe-00d926f5c64b 4c622554343c4f57b012f8755efe3a3b 9a3cfa6bd2b1463daf4655254affbe56 - default default] Attempting to import store rbd _load_multi_store /usr/lib/python3.9/site-packages/glance_store/multi_backend.py:171
      2025-09-08 18:13:51.080 46 DEBUG glance_store.capabilities [req-9c72a8cb-6ed7-430a-b5fe-00d926f5c64b 4c622554343c4f57b012f8755efe3a3b 9a3cfa6bd2b1463daf4655254affbe56 - default default] Store glance_store._drivers.rbd.Store doesn't support updating dynamic storage capabilities. Please overwrite 'update_capabilities' method of the store to implement updating logics if needed. update_capabilities /usr/lib/python3.9/site-packages/glance_store/capabilities.py:91
      2025-09-08 18:13:51.081 46 DEBUG glance_store.driver [req-9c72a8cb-6ed7-430a-b5fe-00d926f5c64b 4c622554343c4f57b012f8755efe3a3b 9a3cfa6bd2b1463daf4655254affbe56 - default default] Late loading location class glance_store._drivers.rbd.StoreLocation get_store_location_class /usr/lib/python3.9/site-packages/glance_store/driver.py:117
      2025-09-08 18:13:51.081 46 DEBUG glance_store.location [req-9c72a8cb-6ed7-430a-b5fe-00d926f5c64b 4c622554343c4f57b012f8755efe3a3b 9a3cfa6bd2b1463daf4655254affbe56 - default default] Registering scheme rbd with {'default_backend': {'store': <glance_store._drivers.rbd.Store object at 0x7f9b669fd3d0>, 'location_class': <class 'glance_store._drivers.rbd.StoreLocation'>, 'store_entry': 'rbd'}} register_scheme_backend_map /usr/lib/python3.9/site-packages/glance_store/location.py:132
      2025-09-08 18:13:51.607 47 DEBUG eventlet.wsgi.server [-] (47) accepted ('172.17.1.39', 34648) server /usr/lib/python3.9/site-packages/eventlet/wsgi.py:992
      2025-09-08 18:13:51.611 47 INFO eventlet.wsgi.server [-] 172.17.1.39 - - [08/Sep/2025 18:13:51] "GET /healthcheck HTTP/1.0" 200 137 0.002678
      2025-09-08 18:13:52.202 46 ERROR glance.api.v2.image_data [req-9c72a8cb-6ed7-430a-b5fe-00d926f5c64b 4c622554343c4f57b012f8755efe3a3b 9a3cfa6bd2b1463daf4655254affbe56 - default default] Signature verification failed for image 92d32974-bb62-4d41-9f99-ae4b83a6c510: Signature verification failed: cursive.exception.SignatureVerificationError: Signature verification failed
      2025-09-08 18:13:52.245 46 INFO eventlet.wsgi.server [req-9c72a8cb-6ed7-430a-b5fe-00d926f5c64b 4c622554343c4f57b012f8755efe3a3b 9a3cfa6bd2b1463daf4655254affbe56 - default default] 10.0.0.60,172.17.1.39 - - [08/Sep/2025 18:13:52] "PUT /v2/images/92d32974-bb62-4d41-9f99-ae4b83a6c510/file HTTP/1.1" 400 322 3.617276

      Expected behavior
      Signed Glance image is successfully upload when documentation is followed as is.

      Bug impact
      Usage of signed glance images is blocked

      Known workaround
      None

      Additional context
      Sosreports attached on 9/6/2025 contain relevant logs, "Barbican Implementation steps and Testing.docx" contains customer's steps to reproduce. But problem is also reproducible in lab

              croeland@redhat.com Cyril Roelandt
              rhn-support-astupnik Alex Stupnikov
              rhos-storage-glance
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated: