Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-139997

NVMe-TCP TLS: need to load nvme_tcp module before issuing nvme connect post reboot

Linking RHIVOS CVEs to...Migration: Automation ...SWIFT: POC ConversionSync from "Extern...XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • rhel-10.2
    • nvme-cli
    • None
    • None
    • None
    • rhel-storage-io-2
    • None
    • False
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • None
    • None
    • Unspecified
    • Unspecified
    • Unspecified
    • x86_64
    • None

      What were you trying to do that didn't work?

      On a test system, I performed the following steps to connect to an nvme namespace with NVMe-TCP TLS:

       

      root@rhel-storage-103 ~]# subsysnqn=nqn.1992-08.com.netapp:sn.48c34f56b71f11ef8a10d039ea98949f:subsystem.rhel_103_ss_1_tls 
      [root@rhel-storage-103 ~]#  keydata=NVMeTLSkey-1:01:ZGWG+gp2Wt22Qnc3Bjw2UwVndVgnwB/yTozbmsf0SBQKexut: 
      [root@rhel-storage-103 ~]# yum install -y ktls-utils # sed -i '/^\[authenticate\]/a keyrings=.nvme' /etc/tlshd.conf [
      root@rhel-storage-103 ~]# systemctl start tlshd [root@rhel-storage-103 ~]# systemctl enable tlshd Created symlink '/etc/systemd/system/remote-fs.target.wants/tlshd.service' → '/usr/lib/systemd/system/tlshd.service'. 
      [root@rhel-storage-103 ~]# nvme check-tls-key -vv --identity=1 --subsysnqn=$subsysnqn --keydata=$keydata --insert --keyfile /etc/nvme/tls-keys --compat Inserted TLS key 00dd7e77
      
      [root@rhel-storage-103 ~]# nvme connect -t tcp --tls -a 172.18.50.171 -n nqn.1992-08.com.netapp:sn.48c34f56b71f11ef8a10d039ea98949f:subsystem.rhel_103_ss_1_tls connecting to device: nvme6
      

       

      The connection has been established successfully. However, after rebooting the system, I encountered an issue when trying to connect again:

       

      # nvme connect -t tcp --tls -a 172.18.50.171 -n nqn.1992-08.com.netapp:sn.48c34f56b71f11ef8a10d039ea98949f:subsystem.rhel_103_ss_1_tls
      could not add new controller: pre-shared TLS key is missing
      
      Jan 08 14:28:04 rhel-storage-103.fast.eng.rdu2.dc.redhat.com kernel: nvme nvme6: no valid PSK found
       

      If I issue the connect command a 2nd time, it works:

       

      # nvme connect -t tcp --tls -a 172.18.50.171 -n nqn.1992-08.com.netapp:sn.48c34f56b71f11ef8a10d039ea98949f:subsystem.rhel_103_ss_1_tls
      connecting to device: nvme6

      If I manually load the nvme_tcp module after a reboot, the connection is also successful without having to issue the command twice. cleech@redhat.com has indicated that this is likely a race condition with loading the TLS pre-shared-keys from a key-file (/etc/nvme/tls-keys) into the %.nvme kernel keyring. 

      Please provide the package NVR for which the bug is seen:

      kernel-6.12.0-165.el10

      nvme-cli-2.16-2.el10

      How reproducible is this bug?: Often

      Steps to reproduce

      1. See above

              mlombard@redhat.com Maurizio Lombardi
              mpatalan Marco Patalano
              Maurizio Lombardi Maurizio Lombardi
              Marco Patalano Marco Patalano
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated: