Uploaded image for project: 'OpenShift Virtualization'
  1. OpenShift Virtualization
  2. CNV-20110

[2110905] [4.9.6] NMState pods not compatible with Calico CNI provider preventing deploy of Openshift Virtualization

XMLWordPrintable

    • Moderate
    • No

      +++ This bug was initially created as a clone of Bug #2078247 +++

      Description of problem:
      The nmstate-handler pod in the Openshift Virtualization deployment does not have any configuration toggles to remove the hostPath mount of openvswitch/db.sock
      ```

      • hostPath:
        path: /run/openvswitch/db.sock
        type: Socket
        ```

      This means that Openshift virtualization cannot be used with any other CNI plugins that don't utlize OpenVSwitch like calico despite in the documentation that being mentioned as a possibility:
      https://docs.openshift.com/container-platform/4.10/virt/about-virt.html

      ```
      You can use OpenShift Virtualization with the OVN-Kubernetes, OpenShift SDN, or one of the other certified default Container Network Interface (CNI) network providers listed in Certified OpenShift CNI Plug-ins.

      ```

      Note when I manually scale down the operators and modify the daemonset to have the hostPath removed I am able to provision bridge interfaces and virtual machines using Openshift Virtualization. However: ultimately the operators will revert my changes and the pod will get stuck waiting for the socket which never exists

      ```
      nmstate-handler-5fjqb 1/1 Running 0 9d
      nmstate-handler-bjcx6 0/1 ContainerCreating 0 9d
      ```

      Version-Release number of selected component (if applicable):
      4.10
      4.9

      How reproducible:
      100%

      Steps to Reproduce:
      1. Provision an Openshift cluster with Calico as SDN (can use ROKS for example)
      2. Deploy Openshift Virtualization components and look for nmstate pods. Notice they are stuck in ContainerCreating due to missing mount

      Actual results:
      NMState pods are stuck in ContainerCreating

      Expected results:
      NMState pods to be running

      Additional info:

      — Additional comment from Ben Nemec on 2022-05-09 15:46:58 UTC —

      Moving to CNV team since they support kubernetes-nmstate in 4.9.

      — Additional comment from Sean Haselden on 2022-05-09 16:44:43 UTC —

      — Additional comment from Sean Haselden on 2022-05-09 16:47:02 UTC —

      Additional comment from customer:

      Note that I think the main point I was trying to point out and the main gap I see:
      NMSTATE pods seem to always mount
      ```

      • hostPath:
        path: /run/openvswitch/db.sock
        type: Socket
        ```

      However: some providers like calico do not use openvswitch: this means this file doesnt exist and the deployment never rolls out: When I manually remove it: I am able to proceed with properly using Openshift virtualization however it gets rereconciled to the bad value.

      I think that ultimately could be an "optional mount"

      — Additional comment from Tyler Lisowski on 2022-05-18 15:02:12 UTC —

      Can we get an update on this?

      — Additional comment from Petr Horáček on 2022-05-19 11:55:22 UTC —

      Hello Tyler. This bug should be fixed in 4.10 via https://github.com/nmstate/kubernetes-nmstate/pull/861. 4.9 is in maintenance phase now, so we are not planning to backport it there. Let me know if you have any questions, otherwise I will close this BZ.

      — Additional comment from Tyler Lisowski on 2022-05-19 14:13:40 UTC —

      No worries at all that's awesome:
      The only question I had was looking at this PR:
      https://github.com/nmstate/kubernetes-nmstate/pull/861

      It still looks like the `/run/openvswitch/db.sock` is mounted. To me it looks like this would still fail if that directory doesn't exist (like in calico environments). I would have expected it to be an "optional" mount. Could someone clarify that for me? I can also try and test in one of my environments if there is a release.

      — Additional comment from Tyler Lisowski on 2022-05-19 14:14:52 UTC —

      Actually it almost looks like before this PR there was a way to not have the mount but now there isn't:
      https://github.com/nmstate/kubernetes-nmstate/pull/861/files

      Is what is being stated that this PR will be reverted?

      — Additional comment from Tyler Lisowski on 2022-05-21 03:23:15 UTC —

      edit: this appears to work on 4.10! Thank you for pointing to this appreciate your help

      — Additional comment from Petr Horáček on 2022-05-26 09:41:37 UTC —

      Any time!

      For the record, by removing "type: Socket" in 4.10: https://github.com/nmstate/kubernetes-nmstate/pull/861/files#diff-77cffd7d279779f5da1bb47b90d78fc7c5e2c53cf2b2f082e9ebfefe0db2b367L285, we make sure that if the socket is not found on host, it does not prevent knmstate from starting.

              rhrazdil Radim Hrazdil (Inactive)
              phoracek@redhat.com Petr Horacek
              Anat Wax Anat Wax
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: