Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-107224

openssl cms -encrypt fails when using ECDSA certificate

Linking RHIVOS CVEs to...Migration: Automation ...Sync from "Extern...XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Minor Minor
    • None
    • rhel-10.1
    • fips-provider-next
    • None
    • No
    • Low
    • rhel-security-crypto-clubs
    • 0
    • False
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • None
    • None
    • Unspecified
    • Unspecified
    • Unspecified
    • All
    • None

      What were you trying to do that didn't work?

      # openssl cms -encrypt -in message.txt -out message.enc -recip client/cert.pem
      
      A0442C98FF7F0000:error:1C80007A:Provider routines:ossl_fips_ind_digest_exch_check:invalid digest:providers/common/securitycheck_fips.c:91:
      A0442C98FF7F0000:error:17000074:CMS routines:cms_EnvelopedData_Encryption_init_bio:error setting recipientinfo:crypto/cms/cms_env.c:1199:
      A0442C98FF7F0000:error:17000068:CMS routines:CMS_final:cms lib:crypto/cms/cms_smime.c:907:

      Please provide the package NVR for which the bug is seen:

      openssl-3.5.1-3.el10
      fips-provider-next-1.2.0-1.el10

      Steps to reproduce

      1. Enable FIPS mode.
      2. Generate ECDSA certificate (signed by selfsigned ECDSA CA).
      3. See the description.

      Expected results

      ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
      ::   encryption with ECDSA keys
      ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
      :: [ 07:59:05 ] :: [  BEGIN   ] :: Running 'openssl cms -encrypt -in message.txt -out message.enc -recip client/cert.pem '
      :: [ 07:59:05 ] :: [   PASS   ] :: Command 'openssl cms -encrypt -in message.txt -out message.enc -recip client/cert.pem ' (Expected 0, got 0)
      :: [ 07:59:05 ] :: [  BEGIN   ] :: Running 'sed -e '/^MIME/d' -e '/^Content/d' -e '/^$/d' message.enc > no_header.enc.b64'
      :: [ 07:59:05 ] :: [   PASS   ] :: Command 'sed -e '/^MIME/d' -e '/^Content/d' -e '/^$/d' message.enc > no_header.enc.b64' (Expected 0, got 0)
      :: [ 07:59:05 ] :: [  BEGIN   ] :: Running 'openssl base64 -d -in no_header.enc.b64 -out no_header.enc'
      :: [ 07:59:05 ] :: [   PASS   ] :: Command 'openssl base64 -d -in no_header.enc.b64 -out no_header.enc' (Expected 0, got 0)
      :: [ 07:59:05 ] :: [  BEGIN   ] :: Running 'openssl asn1parse -inform DER -in no_header.enc'
          0:d=0  hl=4 l= 292 cons: SEQUENCE
          4:d=1  hl=2 l=   9 prim: OBJECT            :pkcs7-envelopedData
         15:d=1  hl=4 l= 277 cons: cont [ 0 ]
         19:d=2  hl=4 l= 273 cons: SEQUENCE
         23:d=3  hl=2 l=   1 prim: INTEGER           :02
         26:d=3  hl=3 l= 189 cons: SET
         29:d=4  hl=3 l= 186 cons: cont [ 1 ]
         32:d=5  hl=2 l=   1 prim: INTEGER           :03
         35:d=5  hl=2 l=  81 cons: cont [ 0 ]
         37:d=6  hl=2 l=  79 cons: cont [ 1 ]
         39:d=7  hl=2 l=   9 cons: SEQUENCE
         41:d=8  hl=2 l=   7 prim: OBJECT            :id-ecPublicKey
         50:d=7  hl=2 l=  66 prim: BIT STRING
        118:d=5  hl=2 l=  24 cons: SEQUENCE
        120:d=6  hl=2 l=   9 prim: OBJECT            :dhSinglePass-stdDH-sha1kdf-scheme
        131:d=6  hl=2 l=  11 cons: SEQUENCE
        133:d=7  hl=2 l=   9 prim: OBJECT            :id-aes256-wrap
        144:d=5  hl=2 l=  72 cons: SEQUENCE
        146:d=6  hl=2 l=  70 cons: SEQUENCE
        148:d=7  hl=2 l=  26 cons: SEQUENCE
        150:d=8  hl=2 l=  21 cons: SEQUENCE
        152:d=9  hl=2 l=  19 cons: SET
        154:d=10 hl=2 l=  17 cons: SEQUENCE
        156:d=11 hl=2 l=   3 prim: OBJECT            :organizationName
        161:d=11 hl=2 l=  10 prim: UTF8STRING        :Example CA
        173:d=8  hl=2 l=   1 prim: INTEGER           :02
        176:d=7  hl=2 l=  40 prim: OCTET STRING      [HEX DUMP]:0F5560847D4AC5E4D7E25B73EE44F5994C62739B3A9830369033FC6DB9FC01FEB72DCAC83F9F9FA0
        218:d=3  hl=2 l=  76 cons: SEQUENCE
        220:d=4  hl=2 l=   9 prim: OBJECT            :pkcs7-data
        231:d=4  hl=2 l=  29 cons: SEQUENCE
        233:d=5  hl=2 l=   9 prim: OBJECT            :aes-256-cbc
        244:d=5  hl=2 l=  16 prim: OCTET STRING      [HEX DUMP]:A164C56B875381FF8260ED89C4C437D3
        262:d=4  hl=2 l=  32 prim: cont [ 0 ]
      :: [ 07:59:05 ] :: [   PASS   ] :: Command 'openssl asn1parse -inform DER -in no_header.enc' (Expected 0, got 0)
      :: [ 07:59:05 ] :: [   PASS   ] :: File '/var/tmp/rlRun_LOG.yM2Z8i3o' should contain ':aes-256-cbc'
      :: [ 07:59:05 ] :: [   PASS   ] :: File '/var/tmp/rlRun_LOG.yM2Z8i3o' should contain ':dhSinglePass-stdDH-sha1kdf-scheme'
      :: [ 07:59:05 ] :: [  BEGIN   ] :: Running 'openssl cms -decrypt -in message.enc -out message.dec -recip client/cert.pem -inkey client/key.pem'
      :: [ 07:59:05 ] :: [   PASS   ] :: Command 'openssl cms -decrypt -in message.enc -out message.dec -recip client/cert.pem -inkey client/key.pem' (Expected 0, got 0)
      :: [ 07:59:05 ] :: [   PASS   ] :: File 'message.dec' should contain 'something to encrypt'
      :: [ 07:59:05 ] :: [  BEGIN   ] :: Running 'rm message.enc no_header.enc message.dec'
      :: [ 07:59:05 ] :: [   PASS   ] :: Command 'rm message.enc no_header.enc message.dec' (Expected 0, got 0)
      ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
      ::   RESULT: PASS (encryption with ECDSA keys)
      

      Actual results

      ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
      ::   encryption with ECDSA keys
      :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: [ 08:00:15 ] :: [  BEGIN   ] :: Running 'openssl cms -encrypt -in message.txt -out message.enc -recip client/cert.pem '
      8072BD29E17F0000:error:1C80007A:Provider routines:ossl_fips_ind_digest_exch_check:invalid digest:providers/common/securitycheck_fips.c:91:
      8072BD29E17F0000:error:17000074:CMS routines:cms_EnvelopedData_Encryption_init_bio:error setting recipientinfo:crypto/cms/cms_env.c:1199:
      8072BD29E17F0000:error:17000068:CMS routines:CMS_final:cms lib:crypto/cms/cms_smime.c:907:
      :: [ 08:00:15 ] :: [   FAIL   ] :: Command 'openssl cms -encrypt -in message.txt -out message.enc -recip client/cert.pem ' (Expected 0, got 3)
      ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
      ::   RESULT: FAIL (encryption with ECDSA keys)
      

      Additional information

      Tested by /CoreOS/openssl/Regression/bz2160797-openssl-smime-and-cms-commands-default-to-3DES-and.

              rhn-engineering-ssorce Simo Sorce
              omoris Ondrej Moris
              Simo Sorce Simo Sorce
              Ondrej Moris Ondrej Moris
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated: