Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-54574

Installing Cilium CNI on management cluster prevents starting hosted cluster via Kubevirt

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • 4.19
    • HyperShift
    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • Important
    • None
    • None
    • None
    • Rejected
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

      When using Cilium CNI on a management cluster, a hosted cluster can't be started successfully as a specific NetworkPolicy for virt-launcher prevents downloading Ignition files from the Ignition server. This is caused by a known issue in Cilium https://github.com/cilium/cilium/issues/9209 . The network policy uses ipBlock.CIDR for egress traffic. This block is used to allow egress traffic to all IP addresses except cluster network and service network of the management cluster. However, Cilium fails to allow the traffic and the communication between the virt-launcher nad ignition-server fails.

      The NetworkPolicy for virt-launcher deployed by Hypershift looks like this: https://gist.github.com/mgencur/e4bb0058eae897a512a4e4dabfaf5ad2

      When this policy is manually removed (I verified this) the nodes successfully start and join the NodePool.

      Seen in this CI run. The complete set up can be seen on this branch

       

      Version-Release number of selected component (if applicable):

          4.19

      How reproducible:

          Always

      Steps to Reproduce:

          1. Setup OCP management cluster with Cilium network stack (the install-config.yaml should specify ingressVIPs for configuring the virtual IP address for ingress traffic)
          3. Install the Hypershift operator on the management cluster to allow creating hosted clusters. 
          4. Created a hosted cluster using kubevirt
          

      Actual results:

      The NodePool is waiting for Nodes to join.

        - lastTransitionTime: "2025-04-02T07:53:11Z"
          message: Minimum availability requires 3 replicas, current 0 available
          observedGeneration: 1
          reason: WaitingForAvailableMachines
          status: "False"
          type: Ready

      The console-logger pod for VMs created by Kubevirt shows errors like this one

      [ 3272.778076] ignition[868]: GET error: Get "https://ignition-server-clusters-7d2ddb0516585ff76ed8.apps.ostest.test.metalkube.org/ignition": dial tcp 192.168.111.4:443: i/o timeout

      Note: The IP address 192.168.111.4 is configured in install-config.yaml of the management cluster like this:

      ingressVIPs:
      - 192.168.111.4 

      Expected results:

          The nodes successfully join the NodePool and the hosted cluster starts successfully.

      Additional info:

      NetworkPolicies affect traffic from/to virtual machine instances through the virt-launcher Pod as indicated in Kubevirt docs.

              rh-ee-aabdelre Ahmed Abdalla Abdelrehim
              mgencur@redhat.com Martin Gencur
              None
              None
              Martin Gencur Martin Gencur
              None
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Created:
                Updated: