-
Bug
-
Resolution: Unresolved
-
Undefined
-
None
-
4.19.0
-
Quality / Stability / Reliability
-
False
-
-
None
-
None
-
No
-
None
-
None
-
Rejected
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Description of problem:
Multinetworkpolicy applied to a specific pod to allow ingress traffic from pod in selected namespace does not work in the user defined network.
The pods are configured with static addresses in the scenario.
Version-Release number of selected component (if applicable):
build 4.19.0-0.nightly, openshift/cluster-network-operator#2678,openshift/ovn-kubernetes#2503
How reproducible:
Always
Steps to Reproduce:
- Create a bridge mapping to br-ex with nncp
oc get nncp bridge-mapping -oyaml
apiVersion: nmstate.io/v1 kind: NodeNetworkConfigurationPolicy metadata: annotations: nmstate.io/webhook-mutating-timestamp: "1745257440995920612" creationTimestamp: "2025-04-21T17:44:01Z" generation: 1 name: bridge-mapping resourceVersion: "969842" uid: b422c3b8-6fb5-4387-b9ce-fb404c1ed020 spec: desiredState: ovn: bridge-mappings: - bridge: br-ex localnet: mylocalnet state: present nodeSelector: node-role.kubernetes.io/worker: "" status: conditions: - lastHeartbeatTime: "2025-04-23T21:13:38Z" lastTransitionTime: "2025-04-23T21:13:38Z" message: 5/5 nodes successfully configured reason: SuccessfullyConfigured status: "True" type: Available - lastHeartbeatTime: "2025-04-23T21:13:38Z" lastTransitionTime: "2025-04-23T21:13:38Z" reason: SuccessfullyConfigured status: "False" type: Degraded - lastHeartbeatTime: "2025-04-23T21:13:38Z" lastTransitionTime: "2025-04-23T21:13:38Z" reason: ConfigurationProgressing status: "False" type: Progressing lastUnavailableNodeCountUpdate: "2025-04-21T17:44:08Z"
2. Create two UDN namespaces a1 and a2, add label team=operation to a2
oc get ns -l team=operation
NAME STATUS AGE
a2 Active 2d7h
3. Create two CUDN one in primary role and the other in secondary role
oc get clusteruserdefinednetwork
NAME AGE
pri-l3-network 2d7h sec-localnet-net 12h
primary CUDN
apiVersion: k8s.ovn.org/v1 kind: ClusterUserDefinedNetwork metadata: creationTimestamp: "2025-04-21T17:53:37Z" finalizers: - k8s.ovn.org/user-defined-network-protection generation: 1 name: pri-l3-network resourceVersion: "70468" uid: 9d29fd56-7a20-4c07-ac46-7db829eb316c spec: namespaceSelector: matchExpressions: - key: kubernetes.io/metadata.name operator: In values: - a1 - a2 network: layer3: mtu: 1300 role: Primary subnets: - cidr: 10.150.0.0/16 hostSubnet: 24 topology: Layer3 status: conditions: - lastTransitionTime: "2025-04-21T17:53:37Z" message: 'NetworkAttachmentDefinition has been created in following namespaces: [a1, a2]' reason: NetworkAttachmentDefinitionCreated status: "True" type: NetworkCreated
oc get clusteruserdefinednetwork sec-localnet-net -oyaml
apiVersion: k8s.ovn.org/v1 kind: ClusterUserDefinedNetwork metadata: creationTimestamp: "2025-04-23T12:56:55Z" finalizers: - k8s.ovn.org/user-defined-network-protection generation: 1 name: sec-localnet-net resourceVersion: "835873" uid: e5af6194-9d8a-4eb8-8dc3-b5c079b3eccd spec: namespaceSelector: matchExpressions: - key: kubernetes.io/metadata.name operator: In values: - a1 - a2 network: localnet: ipam: mode: Disabled physicalNetworkName: mylocalnet role: Secondary topology: Localnet status: conditions: - lastTransitionTime: "2025-04-23T12:56:55Z" message: 'NetworkAttachmentDefinition has been created in following namespaces: [a1, a2]' reason: NetworkAttachmentDefinitionCreated status: "True" type: NetworkCreated
4. Create pods in namespace a1 and a2 with static address assignment with YAML
MAC Address 02:03:04:05:06:10 - 02:03:04:05:06:60
IP address 192.100.2.10, 192.100.2.20, 192.100.2.30 in a1 and 192.100.2.40, 192.100.2.50, 192.100.2.60 in a2
kind: Pod
apiVersion: v1
metadata:
name: mylocalnet-pod1
labels:
name: mylocalnet-pod1
annotations:
k8s.v1.cni.cncf.io/networks: '[
{ "name": "sec-localnet-net", "mac": "02:03:04:05:06:10", "interface": "ovn-udn2", "ips": [ "192.100.2.10/24" ] }
]'
spec:
securityContext:
runAsNonRoot: true
seccompProfile:
type: RuntimeDefault
containers:
- name: mylocalnet-pod1
image: quay.io/openshifttest/hello-sdn@sha256:c89445416459e7adea9a5a416b3365ed3d74f2491beb904d61dc8d1eb89a72a4
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
nodeSelector:
kubernetes.io/hostname: worker-0
oc -n a1 get pods -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES mylocalnet-pod1 1/1 Running 0 11h 10.128.2.34 worker-0 <none> <none> mylocalnet-pod2 1/1 Running 0 11h 10.128.2.35 worker-0 <none> <none> mylocalnet-pod3 1/1 Running 0 11h 10.131.0.41 worker-1 <none> <none>
oc -n a2 get pods -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES mylocalnet-pod4 1/1 Running 0 11h 10.128.2.36 worker-0 <none> <none> mylocalnet-pod5 1/1 Running 0 11h 10.128.2.38 worker-0 <none> <none> mylocalnet-pod6 1/1 Running 0 11h 10.131.0.42 worker-1 <none> <none>
Add label to first pod in a1 and a2 team=team1
oc get pods -A -l team=team1
NAMESPACE NAME READY STATUS RESTARTS AGE a1 mylocalnet-pod1 1/1 Running 0 11h a2 mylocalnet-pod4 1/1 Running 0 11h
5. Create MNP in a1
oc -n a1 get multi-networkpolicy
NAME AGE
allow-ingress-pod-ns 167m
default-deny-ingress 167m
oc -n a1 get multi-networkpolicy allow-ingress-pod-ns -oyaml
apiVersion: k8s.cni.cncf.io/v1beta1 kind: MultiNetworkPolicy metadata: annotations: k8s.v1.cni.cncf.io/policy-for: sec-localnet-net creationTimestamp: "2025-04-23T22:42:41Z" generation: 1 name: allow-ingress-pod-ns namespace: a1 resourceVersion: "996558" uid: 2e0f43aa-6a7d-4ded-8006-8184343ff44b spec: ingress: - from: - namespaceSelector: matchLabels: team: operation podSelector: matchLabels: team: team1 podSelector: matchLabels: team: team1 policyTypes: - Ingress
oc -n a1 get multi-networkpolicy default-deny-ingress -oyaml
apiVersion: k8s.cni.cncf.io/v1beta1 kind: MultiNetworkPolicy metadata: annotations: k8s.v1.cni.cncf.io/policy-for: sec-localnet-net creationTimestamp: "2025-04-23T22:42:35Z" generation: 1 name: default-deny-ingress namespace: a1 resourceVersion: "996534" uid: 2b51f85e-8dd9-4b9b-8e29-2d2a5093eacc spec: podSelector: {} policyTypes: - Ingress
6. Test ingress traffic from labeled pod in namespace a2 to labeled pod in namespace a1
oc -n a2 exec -it mylocalnet-pod4 – curl 192.100.2.10:8080 --connect-timeout 5
curl: (28) Connection timeout after 5000 ms command terminated with exit code 28
Actual results:
curl fails
Expected results:
curl to succeed
Additional info:
Please fill in the following template while reporting a bug and provide as much relevant information as possible. Doing so will give us the best chance to find a prompt resolution.
Affected Platforms:
Is it an
- internal CI failure
- customer issue / SD
- internal RedHat testing failure
If it is an internal RedHat testing failure:
- Please share a kubeconfig or creds to a live cluster for the assignee to debug/troubleshoot along with reproducer steps (specially if it's a telco use case like ICNI, secondary bridges or BM+kubevirt).
If it is a CI failure:
- Did it happen in different CI lanes? If so please provide links to multiple failures with the same error instance
- Did it happen in both sdn and ovn jobs? If so please provide links to multiple failures with the same error instance
- Did it happen in other platforms (e.g. aws, azure, gcp, baremetal etc) ? If so please provide links to multiple failures with the same error instance
- When did the failure start happening? Please provide the UTC timestamp of the networking outage window from a sample failure run
- If it's a connectivity issue,
- What is the srcNode, srcIP and srcNamespace and srcPodName?
- What is the dstNode, dstIP and dstNamespace and dstPodName?
- What is the traffic path? (examples: pod2pod? pod2external?, pod2svc? pod2Node? etc)
If it is a customer / SD issue:
- Provide enough information in the bug description that Engineering doesn’t need to read the entire case history.
- Don’t presume that Engineering has access to Salesforce.
- Do presume that Engineering will access attachments through supportshell.
- Describe what each relevant attachment is intended to demonstrate (failed pods, log errors, OVS issues, etc).
- Referring to the attached must-gather, sosreport or other attachment, please provide the following details:
- If the issue is in a customer namespace then provide a namespace inspect.
- If it is a connectivity issue:
- What is the srcNode, srcNamespace, srcPodName and srcPodIP?
- What is the dstNode, dstNamespace, dstPodName and dstPodIP?
- What is the traffic path? (examples: pod2pod? pod2external?, pod2svc? pod2Node? etc)
- Please provide the UTC timestamp networking outage window from must-gather
- Please provide tcpdump pcaps taken during the outage filtered based on the above provided src/dst IPs
- If it is not a connectivity issue:
- Describe the steps taken so far to analyze the logs from networking components (cluster-network-operator, OVNK, SDN, openvswitch, ovs-configure etc) and the actual component where the issue was seen based on the attached must-gather. Please attach snippets of relevant logs around the window when problem has happened if any.
- When showing the results from commands, include the entire command in the output.
- For OCPBUGS in which the issue has been identified, label with “sbr-triaged”
- For OCPBUGS in which the issue has not been identified and needs Engineering help for root cause, label with “sbr-untriaged”
- Do not set the priority, that is owned by Engineering and will be set when the bug is evaluated
- Note: bugs that do not meet these minimum standards will be closed with label “SDN-Jira-template”
- For guidance on using this template please see
OCPBUGS Template Training for Networking components
