Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-149735

[RFE] Automated rotation of outdated IPA service keys

Linking RHIVOS CVEs to...Migration: Automation ...Sync from "Extern...XMLWordPrintable

    • Icon: Story Story
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • rhel-10.3
    • rhel-8.10.z
    • ipa
    • None
    • rhel-idm-ipa
    • None
    • False
    • False
    • Hide

      None

      Show
      None
    • Yes
    • None
    • None
    • None
    • Unspecified Release Note Type - Unknown
    • Unspecified
    • Unspecified
    • Unspecified
    • None

      The specification in this RFE relies on the changes that were already implement in freeipa#7861.

      On some IPA domains, the IPA services (HTTP API, LDAP, DogTag, named, ...) were initialized with a list of key type that has since been extended with stronger types. However, there is no mechanism to add keys with these new types to IPA services.

      This prevents old domains running on RHEL8 servers in FIPS mode to upgrade to RHEL9 in FIPS mode, because such an upgrade requires AES HMAC-SHA2 support. If IPA services on RHEL8 do not support and use AES HMAC-SHA2 by default, then they are unable to interoperated with their counterparts on FIPS RHEL9 servers.

      This is why the IPA server should monitor the key types of its own services and rotate them to create keys with new types when there are added to the list of supported key types.

      Here is the list of keytabs that should be checked and rotated if necessary:

      /etc/dirsrv/ds.keytab
      /var/lib/ipa/gssproxy/http.keytab
      /var/lib/ipa/api/anon.keytab
      /etc/pki/pki-tomcat/dogtag.keytab
      /etc/named.keytab
      /etc/ipa/dnssec/ipa-dnskeysyncd.keytab
      /etc/samba/samba.keytab
      /var/lib/sss/keytabs/trusted.domain.com.keytab
      

      Rotating the service keys can be done without any impact on domain's operations if timing is handled properly. When new keys are saved in the KDC, the old ones can be overwritten, because no new tickets should be created for the old keys. However the tickets delivered to clients for these services should still be accepted until they are no longer valid. Passed this delay, the old keys have not reason to remain in the keytab, because the KDC is not able to generate tickets with them anymore. Hence they can be removed after the maximum renewable time it passed.

      Generate new IPA service keys

      The best time to initialize the check of IPA service keys is on IPA startup. This can be done by calling:

      /usr/bin/klist -ekt /var/lib/ipa/gssproxy/http.keytab
      

      Which will print this kind of output:

      Keytab name: FILE:/var/lib/ipa/gssproxy/http.keytab
      KVNO Timestamp           Principal
      ---- ------------------- ------------------------------------------------------
         1 02/13/2026 08:59:28 HTTP/server.ipa.domain@IPA.DOMAIN (aes256-cts-hmac-sha1-96) 
         1 02/13/2026 08:59:28 HTTP/server.ipa.domain@IPA.DOMAIN (aes128-cts-hmac-sha1-96) 
         1 02/13/2026 08:59:28 HTTP/server.ipa.domain@IPA.DOMAIN (camellia256-cts-cmac) 
         1 02/13/2026 08:59:28 HTTP/server.ipa.domain@IPA.DOMAIN (camellia128-cts-cmac) 
         2 02/13/2026 10:11:46 HTTP/server.ipa.domain@IPA.DOMAIN (aes256-cts-hmac-sha1-96) 
         2 02/13/2026 10:11:46 HTTP/server.ipa.domain@IPA.DOMAIN (aes128-cts-hmac-sha1-96) 
         2 02/13/2026 10:11:46 HTTP/server.ipa.domain@IPA.DOMAIN (camellia256-cts-cmac) 
         2 02/13/2026 10:11:46 HTTP/server.ipa.domain@IPA.DOMAIN (camellia128-cts-cmac) 
      

      Host keytab aside (/etc/krb5.keytab), all lines in the keytab should have the same. In this case, the lines to be taken into account are the ones with the highest KVNO (here it's 2). Out these lines, collect the key types between parentheses (here they are aes256-cts-hmac-sha1-96, aes128-cts-hmac-sha1-96, camellia256-cts-cmac, and camellia128-cts-cmac).

      This list must then be compared to the one generated by ipa-getkeytab:

      /usr/bin/ipa-getkeytab --permitted-enctypes
      

      Which should generate this kind of list (based on the configuration from /etc/krb5.conf.d/crypto-policies):

      aes256-cts-hmac-sha384-192
      aes128-cts-hmac-sha256-128
      aes256-cts-hmac-sha1-96
      aes128-cts-hmac-sha1-96
      camellia256-cts-cmac
      camellia128-cts-cmac
      

      The ipa-getkeytab-generated list is ordered by preference. If the types from the keytab are not the first types in this list, it means the keytab is outdated, and the service keys have to be rotated.

      This can be done using ipa-getkeytab (but note that ipa-getkeytab requires administration permissions on the service to renew its key, the host keytab can be use for this):

      /usr/bin/kinit -kt /etc/krb5.keytab host/server.ipa.domain@IPA.DOMAIN
      /usr/bin/ipa-getkeytab -p HTTP/server.ipa.domain@IPA.DOMAIN -k /var/lib/ipa/gssproxy/http.keytab
      /usr/bin/kdestroy
      

      After this operation, the keytab should look like this:

      Keytab name: FILE:/var/lib/ipa/gssproxy/http.keytab
      KVNO Timestamp           Principal
      ---- ------------------- ------------------------------------------------------
         1 02/13/2026 08:59:28 HTTP/server.ipa.domain@IPA.DOMAIN (aes256-cts-hmac-sha1-96) 
         1 02/13/2026 08:59:28 HTTP/server.ipa.domain@IPA.DOMAIN (aes128-cts-hmac-sha1-96) 
         1 02/13/2026 08:59:28 HTTP/server.ipa.domain@IPA.DOMAIN (camellia256-cts-cmac) 
         1 02/13/2026 08:59:28 HTTP/server.ipa.domain@IPA.DOMAIN (camellia128-cts-cmac) 
         2 02/13/2026 10:11:46 HTTP/server.ipa.domain@IPA.DOMAIN (aes256-cts-hmac-sha1-96) 
         2 02/13/2026 10:11:46 HTTP/server.ipa.domain@IPA.DOMAIN (aes128-cts-hmac-sha1-96) 
         2 02/13/2026 10:11:46 HTTP/server.ipa.domain@IPA.DOMAIN (camellia256-cts-cmac) 
         2 02/13/2026 10:11:46 HTTP/server.ipa.domain@IPA.DOMAIN (camellia128-cts-cmac) 
         3 02/13/2026 11:18:48 HTTP/server.ipa.domain@IPA.DOMAIN (aes256-cts-hmac-sha384-192) 
         3 02/13/2026 11:18:48 HTTP/server.ipa.domain@IPA.DOMAIN (aes128-cts-hmac-sha256-128) 
         3 02/13/2026 11:18:48 HTTP/server.ipa.domain@IPA.DOMAIN (aes256-cts-hmac-sha1-96) 
         3 02/13/2026 11:18:48 HTTP/server.ipa.domain@IPA.DOMAIN (aes128-cts-hmac-sha1-96) 
         3 02/13/2026 11:18:48 HTTP/server.ipa.domain@IPA.DOMAIN (camellia256-cts-cmac) 
         3 02/13/2026 11:18:48 HTTP/server.ipa.domain@IPA.DOMAIN (camellia128-cts-cmac) 
      

      Keys with a new KVNO (3) were added, and now types AES HMAC-SHA2 types are there too (aes256-cts-hmac-sha384-192 and aes128-cts-hmac-sha256-128).

      (Note that kadmin.local ktadd would be a more convenient way to add the new keys to the service keytabs. However, the recent work on the upstream pull request has shown that the list of key types for service principals when generated using kadmin, is based on the supported_enctypes parameter from kdc.conf, which is never update. Hence we have no guarantee this list is up-to-date, while ipa-getkeytab relies on the permitted_enctypes list controlled by the crypto-policies system)

      Remove outdated keys

      The old keys should be kept in the IPA service keytabs until the maximum renewable age is passed (by default, this value is 7 days):

      ldapsearch -o ldif-wrap=no -LLL -Q -s base -b 'cn=IPA.DOMAIN,cn=kerberos,dc=ipa,dc=domain' krbMaxRenewableAge
      dn: cn=IPA.DOMAIN,cn=kerberos,dc=ipa,dc=domain
      krbMaxRenewableAge: 604800
      

      This could either be implemented by, on each IPA startup, checking the IPA service keytabs where multiple KVNOs are present, and look at the last change of the local IPA services:

      ldapsearch -o ldif-wrap=no -LLL -Q -s base -b 'krbPrincipalName=HTTP/server.ipa.domain@IPA.DOMAIN,cn=services,cn=accounts,dc=ipa,dc=domain' krbLastPwdChange
      
      dn: krbprincipalname=HTTP/server.ipa.domain@IPA.DOMAIN,cn=services,cn=accounts,dc=ipa,dc=domain
      krbLastPwdChange: 20260218155005Z
      

      But a even better way, would be to have a task planned in advance using CRON or systemd.

      The outdated keys can be removed using kadmin.local:

      /usr/bin/kadmin.local ktremove -k /var/lib/ipa/gssproxy/http.keytab HTTP/server.ipa.domain@IPA.DOMAIN 1
      
      Entry for principal HTTP/server.ipa.domain@IPA.DOMAIN with kvno 1 removed from keytab WRFILE:/var/lib/ipa/gssproxy/http.keytab.
      Entry for principal HTTP/server.ipa.domain@IPA.DOMAIN with kvno 1 removed from keytab WRFILE:/var/lib/ipa/gssproxy/http.keytab.
      Entry for principal HTTP/server.ipa.domain@IPA.DOMAIN with kvno 1 removed from keytab WRFILE:/var/lib/ipa/gssproxy/http.keytab.
      Entry for principal HTTP/server.ipa.domain@IPA.DOMAIN with kvno 1 removed from keytab WRFILE:/var/lib/ipa/gssproxy/http.keytab.
      
      /usr/bin/kadmin.local ktremove -k /var/lib/ipa/gssproxy/http.keytab HTTP/server.ipa.domain@IPA.DOMAIN 2
      
      Entry for principal HTTP/server.ipa.domain@IPA.DOMAIN with kvno 2 removed from keytab WRFILE:/var/lib/ipa/gssproxy/http.keytab.
      Entry for principal HTTP/server.ipa.domain@IPA.DOMAIN with kvno 2 removed from keytab WRFILE:/var/lib/ipa/gssproxy/http.keytab.
      Entry for principal HTTP/server.ipa.domain@IPA.DOMAIN with kvno 2 removed from keytab WRFILE:/var/lib/ipa/gssproxy/http.keytab.
      Entry for principal HTTP/server.ipa.domain@IPA.DOMAIN with kvno 2 removed from keytab WRFILE:/var/lib/ipa/gssproxy/http.keytab.
      

              frenaud@redhat.com Florence Renaud
              jrische@redhat.com Julien Rische
              Florence Renaud Florence Renaud
              Sudhir Menon Sudhir Menon
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Created:
                Updated: