Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-32402

ovn-ipsec-host pod fails to configure cert on nss db

XMLWordPrintable

    • No
    • Proposed
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      It is noticed that ovs-monitor-ipsec fails to import cert into nss db with following error.
      
      2024-04-17T19:57:21.140989157Z 2024-04-17T19:57:21Z |  6  | reconnect | INFO | unix:/var/run/openvswitch/db.sock: connecting...
      2024-04-17T19:57:21.142234972Z 2024-04-17T19:57:21Z |  9  | reconnect | INFO | unix:/var/run/openvswitch/db.sock: connected
      2024-04-17T19:57:21.170709468Z 2024-04-17T19:57:21Z |  14 | ovs-monitor-ipsec | INFO | Tunnel ovn-69b991-0 appeared in OVSDB
      2024-04-17T19:57:21.171379359Z 2024-04-17T19:57:21Z |  16 | ovs-monitor-ipsec | INFO | Tunnel ovn-52bc87-0 appeared in OVSDB
      2024-04-17T19:57:21.171826906Z 2024-04-17T19:57:21Z |  18 | ovs-monitor-ipsec | INFO | Tunnel ovn-3e78bb-0 appeared in OVSDB
      2024-04-17T19:57:21.172300675Z 2024-04-17T19:57:21Z |  20 | ovs-monitor-ipsec | INFO | Tunnel ovn-12fb32-0 appeared in OVSDB
      2024-04-17T19:57:21.172726970Z 2024-04-17T19:57:21Z |  22 | ovs-monitor-ipsec | INFO | Tunnel ovn-8a4d01-0 appeared in OVSDB
      2024-04-17T19:57:21.178644919Z 2024-04-17T19:57:21Z |  24 | ovs-monitor-ipsec | ERR | Import cert and key failed.
      2024-04-17T19:57:21.178644919Z b"No cert in -in file '/etc/openvswitch/keys/ipsec-cert.pem' matches private key\n80FBF36CDE7F0000:error:05800074:x509 certificate routines:X509_check_private_key:key values mismatch:crypto/x509/x509_cmp.c:405:\n"
      2024-04-17T19:57:21.179581526Z 2024-04-17T19:57:21Z |  25 | ovs-monitor-ipsec | ERR | traceback
      2024-04-17T19:57:21.179581526Z Traceback (most recent call last):
      2024-04-17T19:57:21.179581526Z   File "/usr/share/openvswitch/scripts/ovs-monitor-ipsec", line 1382, in <module>
      2024-04-17T19:57:21.179581526Z     main()
      2024-04-17T19:57:21.179581526Z   File "/usr/share/openvswitch/scripts/ovs-monitor-ipsec", line 1369, in main
      2024-04-17T19:57:21.179581526Z     monitor.run()
      2024-04-17T19:57:21.179581526Z   File "/usr/share/openvswitch/scripts/ovs-monitor-ipsec", line 1176, in run
      2024-04-17T19:57:21.179581526Z     if self.ike_helper.config_global(self):
      2024-04-17T19:57:21.179581526Z   File "/usr/share/openvswitch/scripts/ovs-monitor-ipsec", line 521, in config_global
      2024-04-17T19:57:21.179581526Z     self._nss_import_cert_and_key(cert, key, name)
      2024-04-17T19:57:21.179581526Z   File "/usr/share/openvswitch/scripts/ovs-monitor-ipsec", line 809, in _nss_import_cert_and_key
      2024-04-17T19:57:21.179581526Z     os.remove(path)
      2024-04-17T19:57:21.179581526Z FileNotFoundError: [Errno 2] No such file or directory: '/tmp/ovs_certkey_ef9cf1a5-bfb2-4876-8fb3-69c6b22561a2.p12'
      

      Version-Release number of selected component (if applicable):

       4.16.0   

      How reproducible:

      Hit on the CI: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_release/50690/rehearse-50690-pull-ci-openshift-cluster-network-operator-master-e2e-ovn-ipsec-step-registry/1780660589492703232

      Steps to Reproduce:

          1.
          2.
          3.
          

      Actual results:

      openshift-install failed with error:
      
      time="2024-04-17T19:34:47Z" level=error msg="Cluster initialization failed because one or more operators are not functioning properly.\nThe cluster should be accessible for troubleshooting as detailed in the documentation linked below,\nhttps://docs.openshift.com/container-platform/latest/support/troubleshooting/troubleshooting-installations.html\nThe 'wait-for install-complete' subcommand can then be used to continue the installation"
      time="2024-04-17T19:34:47Z" level=error msg="failed to initialize the cluster: Multiple errors are preventing progress:\n* Cluster operator authentication is degraded\n* Cluster operators monitoring, openshift-apiserver are not available"
      
      https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openshift_release/50690/rehearse-50690-pull-ci-openshift-cluster-network-operator-master-e2e-ovn-ipsec-step-registry/1780660589492703232/artifacts/e2e-ovn-ipsec-step-registry/ipi-install-install/artifacts/.openshift_install-1713382487.log
      
      

      Expected results:

      Cluster must come up COs running with IPsec enabled for EW traffic.    

      Additional info:

      It seems like ovn-ipsec-host pod's ovn-keys init container write empty content into /etc/openvswitch/keys/ipsec-cert.pem though corresponding csr request containing certificate in its status.    

              pepalani@redhat.com Periyasamy Palanisamy
              pepalani@redhat.com Periyasamy Palanisamy
              Huiran Wang Huiran Wang
              Jamo Luhrsen
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated:
                Resolved: