Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-77985

Windows pod-to-pod communication fails on port 80 in OVN-Kubernetes hybrid-overlay on Azure IPI

    • None
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description

      Windows pods cannot communicate with each other on port 80 when using OVN-Kubernetes hybrid-overlay networking. Port 8080 and other non-80 ports work fine. Linux-to-Windows communication on port 80 also works.

      Version Information

      • OCP Version: 4.20.0-0.nightly-2026-03-05-003951
      • Platform: Azure (IPI)
      • Network Type: OVN-Kubernetes (hybrid-overlay for Windows)
      • Windows Nodes: 2 nodes running Windows Server 2022 Datacenter

      Affected Connectivity Matrix

      Source Destination Port Result
      Windows Pod Windows Pod 8080 WORKS
      Windows Pod Windows Pod 80 FAILS
      Linux Pod Windows Pod 8080 WORKS
      Linux Pod Windows Pod 80 WORKS
      Windows Pod Localhost (127.0.0.1) 80 WORKS

      Reproduction Steps

      1. Deploy 2 Windows pods with web servers listening on port 80:
      apiVersion: apps/v1
      kind: Deployment
      metadata: 
        name: win-port80-test
        namespace: test
      spec: 
        replicas: 2
        selector: 
          matchLabels: 
            app: win-port80-test
        template: 
          metadata: 
            labels: 
              app: win-port80-test
          spec: 
            tolerations: 
            - key: "os"
              value: "Windows"
              effect: "NoSchedule"
            containers: 
            - name: webserver
              image: mcr.microsoft.com/powershell:lts-nanoserver-ltsc2022
              securityContext: 
                runAsNonRoot: false
                windowsOptions: 
                  runAsUserName: "ContainerAdministrator"
              command: 
              - pwsh.exe
              - -Command
              - |
                $listener = New-Object System.Net.HttpListener;
                $listener.Prefixes.Add('http://*:80/');
                $listener.Start();
                Write-Host('Listening at http://*:80/');
                while ($listener.IsListening) {
                  $context = $listener.GetContext();
                  $response = $context.Response;
                  $content='<html><body><H1>Port 80 Test</H1></body></html>';
                  $buffer = [System.Text.Encoding]::UTF8.GetBytes($content);
                  $response.ContentLength64 = $buffer.Length;
                  $response.OutputStream.Write($buffer, 0, $buffer.Length);
                  $response.Close();
                };
            nodeSelector: 
              kubernetes.io/os: windows
      
      1. Verify pods are on different nodes:
        oc get pods -o wide
        # Example output:
        # win-port80-test-b5cddb64-8k8jl   10.132.0.19   windows-zbwms
        # win-port80-test-b5cddb64-rlsjs   10.132.1.15   windows-njtnq
        
      2. Test Windows pod to Windows pod connectivity on port 80:
        oc exec win-port80-test-b5cddb64-rlsjs -- pwsh.exe -Command \
          "(Invoke-WebRequest -Uri http://10.132.0.19 -UseBasicParsing -ErrorAction SilentlyContinue).Content"
        

      Expected: Web page content returned
      Actual: Connection timeout after 21 seconds

      Invoke-WebRequest: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
      command terminated with exit code 1
      
      1. Test same pods on port 8080 - WORKS FINE:
        # Change listener to port 8080 and redeploy
        oc exec win-port8080-test-xxx -- pwsh.exe -Command \
          "(Invoke-WebRequest -Uri http://10.132.0.18:8080 -UseBasicParsing -ErrorAction SilentlyContinue).Content"
        # Returns: <html><body><H1>Windows Pod Test</H1></body></html>
        

      Additional Evidence

      Pods are listening on port 80:

      oc exec win-port80-test-xxx -- pwsh.exe -Command "netstat -an | Select-String ':80 '"
      # Output:
      #   TCP    0.0.0.0:80             0.0.0.0:0              LISTENING
      #   TCP    [::]:80                [::]:0                 LISTENING
      

      Localhost connectivity works:

      oc exec win-port80-test-xxx -- pwsh.exe -Command \
        "(Invoke-WebRequest -Uri http://127.0.0.1 -UseBasicParsing -ErrorAction SilentlyContinue).Content"
      # Returns: <html><body><H1>Port 80 Test</H1></body></html>
      

      Linux to Windows on port 80 works:

      oc run test-linux --image=curlimages/curl --restart=Never -- sleep 3600
      oc exec test-linux -- curl -m 10 http://10.132.0.19
      # Returns: <html><body><H1>Port 80 Test</H1></body></html>
      

      Hybrid-Overlay Configuration

      Windows nodes are using hybrid-overlay:

      oc get nodes -l kubernetes.io/os=windows -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.metadata.annotations.k8s\.ovn\.org/hybrid-overlay-node-subnet}{"\n"}{end}'
      # Output:
      # windows-njtnq    10.132.1.0/24
      # windows-zbwms    10.132.0.0/24
      

      Impact

      This blocks automated test OCP-66352 "Windows workloads support CSI persistent storage" which tests pod-to-pod connectivity as part of validating CSI volume functionality. The test creates web servers on port 80 and attempts to verify pod-to-pod communication.

      Root Cause Hypothesis

      Port 80 appears to be specifically blocked for Windows-to-Windows traffic in the hybrid-overlay implementation, possibly by:

      1. Windows Firewall rules specific to port 80
      2. OVN-Kubernetes hybrid-overlay filtering port 80 for Windows nodes
      3. Azure network security affecting port 80 inter-pod communication

      Workarounds

      • Use ports other than 80 (e.g., 8080) for Windows pod-to-pod communication
      • Use Kubernetes ClusterIP services instead of direct pod IP communication

      Environment Details

      • Network Plugin: OVN-Kubernetes with hybrid-overlay
      • Windows Nodes: 2 nodes, Windows Server 2022 Datacenter, kernel 10.0.20348.4773
      • Container Runtime: containerd 1.7.27
      • Cluster: Azure IPI deployment

              rh-ee-mapower Marty Power
              rrasouli Aharon Rasouli
              Anurag Saxena Anurag Saxena
              None
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: