What were you trying to do that didn't work?

Establish server/client connection with ML-DSA-65

What is the impact of this issue to you?

I can't sue ML-DSA signature algorithms with GnuTLS.
Applications such as Libvirt can't support ML-DSA / Post Quantum Cryptography

Please provide the package NVR for which the bug is seen:

gnutls-3.8.9-17.el10

How reproducible is this bug?:

100%

Steps to reproduce

1. Create ca and server keys and certs with certtool but make sure not to include encryption_key in the template and use mldsa65 for the private key generations

   certtool --generate-privkey --key-type=mldsa65

   https://libvirt.org/kbase/tlscerts.html

2. server$ GNUTLS_SYSTEM_PRIORITY_FILE=/dev/null gnutls-serv --priority=NORMAL:+SIGN-ML-DSA-65 --x509certfile ml-dsa/server/servercert.pem --x509keyfile ml-dsa/server/serverkey.pem --debug 9

3. client$ GNUTLS_SYSTEM_PRIORITY_FILE=/dev/null gnutls-cli --priority=NORMAL:+SIGN-ML-DSA-65 --x509cafile ml-dsa/ca/cacert.pem -p 5556 localhost
   

Expected results

ML-DSA-65 is used and certificate is trusted and connection succeeds.

Actual results

Client doesn't trust certificate:

- Status: The certificate is NOT trusted. The certificate chain uses insecure algorithm. 
*** PKI verification of server certificate failed...
*** Fatal error: Error in the certificate.

Additional info

1. ML-DSA-65 can also be enabled through

   cat /etc/crypto-policies/back-ends/gnutls.config |grep ML-DSA
   secure-sig = ML-DSA-65
   secure-sig-for-cert = ML-DSA-65
   

    GNUTLS_SYSTEM_PRIORITY_FILE=/etc/crypto-policies/back-ends/gnutls.config gnutls-(serv|cli) ...

2. We've observed that private key generation 0 bit are reported during generation with certtool.

   Generating a 0 bit ML-DSA-65 private key...