Details
-
Bug
-
Resolution: Done-Errata
-
Undefined
-
None
-
4.13.0
-
None
-
No
-
False
-
Description
Reproducer instructions:
You can set the whereabouts reconciler cronjob to run every minute with "*/1 * * * *"
oc patch clusterversion version --type json -p '[{"op":"add","path":"/spec/overrides","value":[{"kind":"Deployment","group":"apps","name":"network-operator","namespace":"openshift-network-operator","unmanaged":true}]}]' oc scale -n openshift-network-operator deployment.apps/network-operator --replicas=0;
Then:
oc edit ds -n openshift-multus multus-additional-cni-plugins (...) "reconciler_cron_expression": "*/1 * * * *" (...)
Now, restart a single reconciler pod:
oc delete pod -n openshift-multus whereabouts-reconciler-n4m8j
The pod should now run the reconciler every minute.
Now, take a deployment, and scale it to 0 replicas:
[akaris@linux whereabouts-clusterbot]$ oc scale deployment netshoot-deployment --replicas 0 deployment.apps/netshoot-deployment scaled
Now, start the following loops:
while true; do echo ===================== ; date ; oc get pods; oc get pods -o jsonpath='.items[].status'; oc get pods -o name | grep netshoot | while read p ; do oc exec $p -- ip a ls dev net1; done; done
while true; do echo ==================; date; oc get overlappingrangeipreservations -A -o yaml; oc get ippools -A -o yaml; sleep 5; done
oc logs -n openshift-multus whereabouts-reconciler-n4m8j --tail=0 -f
Scale to 3 replicas:
oc scale deployment netshoot-deployment --replicas 3
We can see now that the pods scale up ...:
akaris@linux whereabouts-clusterbot]$ while true; do echo ===================== ; date ; oc get pods; oc get pods -o jsonpath='.items[].status'; oc get pods -o name | grep netshoot | while read p ; do oc exec $p -- ip a ls dev net1; done; done ===================== Wed Mar 15 01:58:14 PM CET 2023 No resources found in default namespace. .items[].status===================== Wed Mar 15 01:58:16 PM CET 2023 No resources found in default namespace. .items[].status===================== Wed Mar 15 01:58:18 PM CET 2023 No resources found in default namespace. .items[].status===================== Wed Mar 15 01:58:19 PM CET 2023 NAME READY STATUS RESTARTS AGE netshoot-deployment-6db7547555-s89pm 0/1 ContainerCreating 0 1s netshoot-deployment-6db7547555-smhc8 0/1 ContainerCreating 0 1s netshoot-deployment-6db7547555-xhfvr 0/1 ContainerCreating 0 1s .items[].status3: net1@if40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 26:8c:9a:e5:79:e6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.21/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::248c:9aff:fee5:79e6/64 scope link valid_lft forever preferred_lft forever 3: net1@if40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 52:58:1f:9d:50:ca brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.22/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::5058:1fff:fe9d:50ca/64 scope link valid_lft forever preferred_lft forever 3: net1@if24: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 22:28:95:11:2e:af brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.20/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::2028:95ff:fe11:2eaf/64 scope link valid_lft forever preferred_lft forever ===================== Wed Mar 15 01:58:26 PM CET 2023 NAME READY STATUS RESTARTS AGE netshoot-deployment-6db7547555-s89pm 1/1 Running 0 8s netshoot-deployment-6db7547555-smhc8 1/1 Running 0 8s netshoot-deployment-6db7547555-xhfvr 1/1 Running 0 8s .items[].status3: net1@if40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 26:8c:9a:e5:79:e6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.21/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::248c:9aff:fee5:79e6/64 scope link valid_lft forever preferred_lft forever 3: net1@if40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 52:58:1f:9d:50:ca brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.22/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::5058:1fff:fe9d:50ca/64 scope link valid_lft forever preferred_lft forever 3: net1@if24: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 22:28:95:11:2e:af brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.20/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::2028:95ff:fe11:2eaf/64 scope link valid_lft forever preferred_lft forever ===================== Wed Mar 15 01:58:33 PM CET 2023 NAME READY STATUS RESTARTS AGE netshoot-deployment-6db7547555-s89pm 1/1 Running 0 15s netshoot-deployment-6db7547555-smhc8 1/1 Running 0 15s netshoot-deployment-6db7547555-xhfvr 1/1 Running 0 15s .items[].status3: net1@if40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 26:8c:9a:e5:79:e6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.21/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::248c:9aff:fee5:79e6/64 scope link valid_lft forever preferred_lft forever 3: net1@if40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 52:58:1f:9d:50:ca brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.22/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::5058:1fff:fe9d:50ca/64 scope link valid_lft forever preferred_lft forever 3: net1@if24: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 22:28:95:11:2e:af brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.20/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::2028:95ff:fe11:2eaf/64 scope link valid_lft forever preferred_lft forever ===================== Wed Mar 15 01:58:40 PM CET 2023 NAME READY STATUS RESTARTS AGE netshoot-deployment-6db7547555-s89pm 1/1 Running 0 22s netshoot-deployment-6db7547555-smhc8 1/1 Running 0 22s netshoot-deployment-6db7547555-xhfvr 1/1 Running 0 22s .items[].status3: net1@if40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 26:8c:9a:e5:79:e6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.21/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::248c:9aff:fee5:79e6/64 scope link valid_lft forever preferred_lft forever 3: net1@if40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 52:58:1f:9d:50:ca brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.22/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::5058:1fff:fe9d:50ca/64 scope link valid_lft forever preferred_lft forever 3: net1@if24: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 22:28:95:11:2e:af brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.20/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::2028:95ff:fe11:2eaf/64 scope link valid_lft forever preferred_lft forever ===================== Wed Mar 15 01:58:47 PM CET 2023 NAME READY STATUS RESTARTS AGE netshoot-deployment-6db7547555-s89pm 1/1 Running 0 29s netshoot-deployment-6db7547555-smhc8 1/1 Running 0 29s netshoot-deployment-6db7547555-xhfvr 1/1 Running 0 29s .items[].status3: net1@if40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 26:8c:9a:e5:79:e6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.21/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::248c:9aff:fee5:79e6/64 scope link valid_lft forever preferred_lft forever 3: net1@if40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 52:58:1f:9d:50:ca brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.22/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::5058:1fff:fe9d:50ca/64 scope link valid_lft forever preferred_lft forever 3: net1@if24: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 22:28:95:11:2e:af brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.20/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::2028:95ff:fe11:2eaf/64 scope link valid_lft forever preferred_lft forever ===================== Wed Mar 15 01:58:54 PM CET 2023 NAME READY STATUS RESTARTS AGE netshoot-deployment-6db7547555-s89pm 1/1 Running 0 35s netshoot-deployment-6db7547555-smhc8 1/1 Running 0 35s netshoot-deployment-6db7547555-xhfvr 1/1 Running 0 35s .items[].status3: net1@if40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 26:8c:9a:e5:79:e6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.21/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::248c:9aff:fee5:79e6/64 scope link valid_lft forever preferred_lft forever 3: net1@if40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 52:58:1f:9d:50:ca brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.22/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::5058:1fff:fe9d:50ca/64 scope link valid_lft forever preferred_lft forever 3: net1@if24: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 22:28:95:11:2e:af brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.20/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::2028:95ff:fe11:2eaf/64 scope link valid_lft forever preferred_lft forever ===================== Wed Mar 15 01:59:00 PM CET 2023 NAME READY STATUS RESTARTS AGE netshoot-deployment-6db7547555-s89pm 1/1 Running 0 42s netshoot-deployment-6db7547555-smhc8 1/1 Running 0 42s netshoot-deployment-6db7547555-xhfvr 1/1 Running 0 42s .items[].status3: net1@if40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 26:8c:9a:e5:79:e6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.21/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::248c:9aff:fee5:79e6/64 scope link valid_lft forever preferred_lft forever 3: net1@if40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 52:58:1f:9d:50:ca brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.22/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::5058:1fff:fe9d:50ca/64 scope link valid_lft forever preferred_lft forever 3: net1@if24: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 22:28:95:11:2e:af brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.20/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::2028:95ff:fe11:2eaf/64 scope link valid_lft forever preferred_lft forever ===================== Wed Mar 15 01:59:07 PM CET 2023 NAME READY STATUS RESTARTS AGE netshoot-deployment-6db7547555-s89pm 1/1 Running 0 48s netshoot-deployment-6db7547555-smhc8 1/1 Running 0 48s netshoot-deployment-6db7547555-xhfvr 1/1 Running 0 48s .items[].status3: net1@if40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 26:8c:9a:e5:79:e6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.21/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::248c:9aff:fee5:79e6/64 scope link valid_lft forever preferred_lft forever 3: net1@if40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 52:58:1f:9d:50:ca brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.22/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::5058:1fff:fe9d:50ca/64 scope link valid_lft forever preferred_lft forever 3: net1@if24: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 22:28:95:11:2e:af brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.20/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::2028:95ff:fe11:2eaf/64 scope link valid_lft forever preferred_lft forever ===================== Wed Mar 15 01:59:13 PM CET 2023 NAME READY STATUS RESTARTS AGE netshoot-deployment-6db7547555-s89pm 1/1 Running 0 55s netshoot-deployment-6db7547555-smhc8 1/1 Running 0 55s netshoot-deployment-6db7547555-xhfvr 1/1 Running 0 55s .items[].status 3: net1@if40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 26:8c:9a:e5:79:e6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.21/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::248c:9aff:fee5:79e6/64 scope link valid_lft forever preferred_lft forever
... but that their allocations are erroneously removed:
[akaris@linux whereabouts-clusterbot]$ while true; do echo ==================; date; oc get overlappingrangeipreservations -A -o yaml; oc get ippools -A -o yaml; sleep 5; done ================== Wed Mar 15 01:58:12 PM CET 2023 apiVersion: v1 items: [] kind: List metadata: resourceVersion: "" apiVersion: v1 items: - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: IPPool metadata: creationTimestamp: "2023-03-15T11:30:14Z" generation: 14 name: 192.168.123.0-24 namespace: openshift-multus resourceVersion: "60944" uid: c8a4f758-17fc-4177-8f0a-dd2894f9271f spec: allocations: {} range: 192.168.123.0/24 kind: List metadata: resourceVersion: "" ================== Wed Mar 15 01:58:18 PM CET 2023 apiVersion: v1 items: [] kind: List metadata: resourceVersion: "" apiVersion: v1 items: - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: IPPool metadata: creationTimestamp: "2023-03-15T11:30:14Z" generation: 14 name: 192.168.123.0-24 namespace: openshift-multus resourceVersion: "60944" uid: c8a4f758-17fc-4177-8f0a-dd2894f9271f spec: allocations: {} range: 192.168.123.0/24 kind: List metadata: resourceVersion: "" ================== Wed Mar 15 01:58:24 PM CET 2023 apiVersion: v1 items: - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: OverlappingRangeIPReservation metadata: creationTimestamp: "2023-03-15T12:58:20Z" generation: 1 name: 192.168.123.20 namespace: openshift-multus resourceVersion: "79302" uid: e052e83f-6e1c-4078-a6c0-a8febe80e49e spec: containerid: 87aead53324fc5d3346797a6c26ea438ccd0bbbf47bbb3e011d9b5522ab8926c podref: default/netshoot-deployment-6db7547555-xhfvr - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: OverlappingRangeIPReservation metadata: creationTimestamp: "2023-03-15T12:58:21Z" generation: 1 name: 192.168.123.21 namespace: openshift-multus resourceVersion: "79320" uid: 8c573e81-f81e-46b0-bdbd-f47a01f7b5d9 spec: containerid: 54e7773be81acb5a26a282960c96a055d74cd105e55d09528d0a3fd50c4d4e13 podref: default/netshoot-deployment-6db7547555-s89pm - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: OverlappingRangeIPReservation metadata: creationTimestamp: "2023-03-15T12:58:22Z" generation: 1 name: 192.168.123.22 namespace: openshift-multus resourceVersion: "79342" uid: c7c2dfb9-18e7-4e2f-b2cf-49299c859774 spec: containerid: d075e6291dc82519f93205ce49eceed935dc3caa72196a4317684bb1276443d4 podref: default/netshoot-deployment-6db7547555-smhc8 kind: List metadata: resourceVersion: "" apiVersion: v1 items: - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: IPPool metadata: creationTimestamp: "2023-03-15T11:30:14Z" generation: 17 name: 192.168.123.0-24 namespace: openshift-multus resourceVersion: "79341" uid: c8a4f758-17fc-4177-8f0a-dd2894f9271f spec: allocations: "20": id: 87aead53324fc5d3346797a6c26ea438ccd0bbbf47bbb3e011d9b5522ab8926c podref: default/netshoot-deployment-6db7547555-xhfvr "21": id: 54e7773be81acb5a26a282960c96a055d74cd105e55d09528d0a3fd50c4d4e13 podref: default/netshoot-deployment-6db7547555-s89pm "22": id: d075e6291dc82519f93205ce49eceed935dc3caa72196a4317684bb1276443d4 podref: default/netshoot-deployment-6db7547555-smhc8 range: 192.168.123.0/24 kind: List metadata: resourceVersion: "" ================== Wed Mar 15 01:58:30 PM CET 2023 apiVersion: v1 items: - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: OverlappingRangeIPReservation metadata: creationTimestamp: "2023-03-15T12:58:20Z" generation: 1 name: 192.168.123.20 namespace: openshift-multus resourceVersion: "79302" uid: e052e83f-6e1c-4078-a6c0-a8febe80e49e spec: containerid: 87aead53324fc5d3346797a6c26ea438ccd0bbbf47bbb3e011d9b5522ab8926c podref: default/netshoot-deployment-6db7547555-xhfvr - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: OverlappingRangeIPReservation metadata: creationTimestamp: "2023-03-15T12:58:21Z" generation: 1 name: 192.168.123.21 namespace: openshift-multus resourceVersion: "79320" uid: 8c573e81-f81e-46b0-bdbd-f47a01f7b5d9 spec: containerid: 54e7773be81acb5a26a282960c96a055d74cd105e55d09528d0a3fd50c4d4e13 podref: default/netshoot-deployment-6db7547555-s89pm - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: OverlappingRangeIPReservation metadata: creationTimestamp: "2023-03-15T12:58:22Z" generation: 1 name: 192.168.123.22 namespace: openshift-multus resourceVersion: "79342" uid: c7c2dfb9-18e7-4e2f-b2cf-49299c859774 spec: containerid: d075e6291dc82519f93205ce49eceed935dc3caa72196a4317684bb1276443d4 podref: default/netshoot-deployment-6db7547555-smhc8 kind: List metadata: resourceVersion: "" apiVersion: v1 items: - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: IPPool metadata: creationTimestamp: "2023-03-15T11:30:14Z" generation: 17 name: 192.168.123.0-24 namespace: openshift-multus resourceVersion: "79341" uid: c8a4f758-17fc-4177-8f0a-dd2894f9271f spec: allocations: "20": id: 87aead53324fc5d3346797a6c26ea438ccd0bbbf47bbb3e011d9b5522ab8926c podref: default/netshoot-deployment-6db7547555-xhfvr "21": id: 54e7773be81acb5a26a282960c96a055d74cd105e55d09528d0a3fd50c4d4e13 podref: default/netshoot-deployment-6db7547555-s89pm "22": id: d075e6291dc82519f93205ce49eceed935dc3caa72196a4317684bb1276443d4 podref: default/netshoot-deployment-6db7547555-smhc8 range: 192.168.123.0/24 kind: List metadata: resourceVersion: "" ================== Wed Mar 15 01:58:36 PM CET 2023 apiVersion: v1 items: - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: OverlappingRangeIPReservation metadata: creationTimestamp: "2023-03-15T12:58:20Z" generation: 1 name: 192.168.123.20 namespace: openshift-multus resourceVersion: "79302" uid: e052e83f-6e1c-4078-a6c0-a8febe80e49e spec: containerid: 87aead53324fc5d3346797a6c26ea438ccd0bbbf47bbb3e011d9b5522ab8926c podref: default/netshoot-deployment-6db7547555-xhfvr - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: OverlappingRangeIPReservation metadata: creationTimestamp: "2023-03-15T12:58:21Z" generation: 1 name: 192.168.123.21 namespace: openshift-multus resourceVersion: "79320" uid: 8c573e81-f81e-46b0-bdbd-f47a01f7b5d9 spec: containerid: 54e7773be81acb5a26a282960c96a055d74cd105e55d09528d0a3fd50c4d4e13 podref: default/netshoot-deployment-6db7547555-s89pm - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: OverlappingRangeIPReservation metadata: creationTimestamp: "2023-03-15T12:58:22Z" generation: 1 name: 192.168.123.22 namespace: openshift-multus resourceVersion: "79342" uid: c7c2dfb9-18e7-4e2f-b2cf-49299c859774 spec: containerid: d075e6291dc82519f93205ce49eceed935dc3caa72196a4317684bb1276443d4 podref: default/netshoot-deployment-6db7547555-smhc8 kind: List metadata: resourceVersion: "" apiVersion: v1 items: - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: IPPool metadata: creationTimestamp: "2023-03-15T11:30:14Z" generation: 17 name: 192.168.123.0-24 namespace: openshift-multus resourceVersion: "79341" uid: c8a4f758-17fc-4177-8f0a-dd2894f9271f spec: allocations: "20": id: 87aead53324fc5d3346797a6c26ea438ccd0bbbf47bbb3e011d9b5522ab8926c podref: default/netshoot-deployment-6db7547555-xhfvr "21": id: 54e7773be81acb5a26a282960c96a055d74cd105e55d09528d0a3fd50c4d4e13 podref: default/netshoot-deployment-6db7547555-s89pm "22": id: d075e6291dc82519f93205ce49eceed935dc3caa72196a4317684bb1276443d4 podref: default/netshoot-deployment-6db7547555-smhc8 range: 192.168.123.0/24 kind: List metadata: resourceVersion: "" ================== Wed Mar 15 01:58:42 PM CET 2023 apiVersion: v1 items: - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: OverlappingRangeIPReservation metadata: creationTimestamp: "2023-03-15T12:58:20Z" generation: 1 name: 192.168.123.20 namespace: openshift-multus resourceVersion: "79302" uid: e052e83f-6e1c-4078-a6c0-a8febe80e49e spec: containerid: 87aead53324fc5d3346797a6c26ea438ccd0bbbf47bbb3e011d9b5522ab8926c podref: default/netshoot-deployment-6db7547555-xhfvr - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: OverlappingRangeIPReservation metadata: creationTimestamp: "2023-03-15T12:58:21Z" generation: 1 name: 192.168.123.21 namespace: openshift-multus resourceVersion: "79320" uid: 8c573e81-f81e-46b0-bdbd-f47a01f7b5d9 spec: containerid: 54e7773be81acb5a26a282960c96a055d74cd105e55d09528d0a3fd50c4d4e13 podref: default/netshoot-deployment-6db7547555-s89pm - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: OverlappingRangeIPReservation metadata: creationTimestamp: "2023-03-15T12:58:22Z" generation: 1 name: 192.168.123.22 namespace: openshift-multus resourceVersion: "79342" uid: c7c2dfb9-18e7-4e2f-b2cf-49299c859774 spec: containerid: d075e6291dc82519f93205ce49eceed935dc3caa72196a4317684bb1276443d4 podref: default/netshoot-deployment-6db7547555-smhc8 kind: List metadata: resourceVersion: "" apiVersion: v1 items: - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: IPPool metadata: creationTimestamp: "2023-03-15T11:30:14Z" generation: 17 name: 192.168.123.0-24 namespace: openshift-multus resourceVersion: "79341" uid: c8a4f758-17fc-4177-8f0a-dd2894f9271f spec: allocations: "20": id: 87aead53324fc5d3346797a6c26ea438ccd0bbbf47bbb3e011d9b5522ab8926c podref: default/netshoot-deployment-6db7547555-xhfvr "21": id: 54e7773be81acb5a26a282960c96a055d74cd105e55d09528d0a3fd50c4d4e13 podref: default/netshoot-deployment-6db7547555-s89pm "22": id: d075e6291dc82519f93205ce49eceed935dc3caa72196a4317684bb1276443d4 podref: default/netshoot-deployment-6db7547555-smhc8 range: 192.168.123.0/24 kind: List metadata: resourceVersion: "" ================== Wed Mar 15 01:58:48 PM CET 2023 apiVersion: v1 items: - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: OverlappingRangeIPReservation metadata: creationTimestamp: "2023-03-15T12:58:20Z" generation: 1 name: 192.168.123.20 namespace: openshift-multus resourceVersion: "79302" uid: e052e83f-6e1c-4078-a6c0-a8febe80e49e spec: containerid: 87aead53324fc5d3346797a6c26ea438ccd0bbbf47bbb3e011d9b5522ab8926c podref: default/netshoot-deployment-6db7547555-xhfvr - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: OverlappingRangeIPReservation metadata: creationTimestamp: "2023-03-15T12:58:21Z" generation: 1 name: 192.168.123.21 namespace: openshift-multus resourceVersion: "79320" uid: 8c573e81-f81e-46b0-bdbd-f47a01f7b5d9 spec: containerid: 54e7773be81acb5a26a282960c96a055d74cd105e55d09528d0a3fd50c4d4e13 podref: default/netshoot-deployment-6db7547555-s89pm - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: OverlappingRangeIPReservation metadata: creationTimestamp: "2023-03-15T12:58:22Z" generation: 1 name: 192.168.123.22 namespace: openshift-multus resourceVersion: "79342" uid: c7c2dfb9-18e7-4e2f-b2cf-49299c859774 spec: containerid: d075e6291dc82519f93205ce49eceed935dc3caa72196a4317684bb1276443d4 podref: default/netshoot-deployment-6db7547555-smhc8 kind: List metadata: resourceVersion: "" apiVersion: v1 items: - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: IPPool metadata: creationTimestamp: "2023-03-15T11:30:14Z" generation: 17 name: 192.168.123.0-24 namespace: openshift-multus resourceVersion: "79341" uid: c8a4f758-17fc-4177-8f0a-dd2894f9271f spec: allocations: "20": id: 87aead53324fc5d3346797a6c26ea438ccd0bbbf47bbb3e011d9b5522ab8926c podref: default/netshoot-deployment-6db7547555-xhfvr "21": id: 54e7773be81acb5a26a282960c96a055d74cd105e55d09528d0a3fd50c4d4e13 podref: default/netshoot-deployment-6db7547555-s89pm "22": id: d075e6291dc82519f93205ce49eceed935dc3caa72196a4317684bb1276443d4 podref: default/netshoot-deployment-6db7547555-smhc8 range: 192.168.123.0/24 kind: List metadata: resourceVersion: "" ================== Wed Mar 15 01:58:54 PM CET 2023 apiVersion: v1 items: - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: OverlappingRangeIPReservation metadata: creationTimestamp: "2023-03-15T12:58:20Z" generation: 1 name: 192.168.123.20 namespace: openshift-multus resourceVersion: "79302" uid: e052e83f-6e1c-4078-a6c0-a8febe80e49e spec: containerid: 87aead53324fc5d3346797a6c26ea438ccd0bbbf47bbb3e011d9b5522ab8926c podref: default/netshoot-deployment-6db7547555-xhfvr - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: OverlappingRangeIPReservation metadata: creationTimestamp: "2023-03-15T12:58:21Z" generation: 1 name: 192.168.123.21 namespace: openshift-multus resourceVersion: "79320" uid: 8c573e81-f81e-46b0-bdbd-f47a01f7b5d9 spec: containerid: 54e7773be81acb5a26a282960c96a055d74cd105e55d09528d0a3fd50c4d4e13 podref: default/netshoot-deployment-6db7547555-s89pm - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: OverlappingRangeIPReservation metadata: creationTimestamp: "2023-03-15T12:58:22Z" generation: 1 name: 192.168.123.22 namespace: openshift-multus resourceVersion: "79342" uid: c7c2dfb9-18e7-4e2f-b2cf-49299c859774 spec: containerid: d075e6291dc82519f93205ce49eceed935dc3caa72196a4317684bb1276443d4 podref: default/netshoot-deployment-6db7547555-smhc8 kind: List metadata: resourceVersion: "" apiVersion: v1 items: - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: IPPool metadata: creationTimestamp: "2023-03-15T11:30:14Z" generation: 17 name: 192.168.123.0-24 namespace: openshift-multus resourceVersion: "79341" uid: c8a4f758-17fc-4177-8f0a-dd2894f9271f spec: allocations: "20": id: 87aead53324fc5d3346797a6c26ea438ccd0bbbf47bbb3e011d9b5522ab8926c podref: default/netshoot-deployment-6db7547555-xhfvr "21": id: 54e7773be81acb5a26a282960c96a055d74cd105e55d09528d0a3fd50c4d4e13 podref: default/netshoot-deployment-6db7547555-s89pm "22": id: d075e6291dc82519f93205ce49eceed935dc3caa72196a4317684bb1276443d4 podref: default/netshoot-deployment-6db7547555-smhc8 range: 192.168.123.0/24 kind: List metadata: resourceVersion: "" ================== Wed Mar 15 01:59:00 PM CET 2023 apiVersion: v1 items: - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: OverlappingRangeIPReservation metadata: creationTimestamp: "2023-03-15T12:58:20Z" generation: 1 name: 192.168.123.20 namespace: openshift-multus resourceVersion: "79302" uid: e052e83f-6e1c-4078-a6c0-a8febe80e49e spec: containerid: 87aead53324fc5d3346797a6c26ea438ccd0bbbf47bbb3e011d9b5522ab8926c podref: default/netshoot-deployment-6db7547555-xhfvr - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: OverlappingRangeIPReservation metadata: creationTimestamp: "2023-03-15T12:58:21Z" generation: 1 name: 192.168.123.21 namespace: openshift-multus resourceVersion: "79320" uid: 8c573e81-f81e-46b0-bdbd-f47a01f7b5d9 spec: containerid: 54e7773be81acb5a26a282960c96a055d74cd105e55d09528d0a3fd50c4d4e13 podref: default/netshoot-deployment-6db7547555-s89pm - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: OverlappingRangeIPReservation metadata: creationTimestamp: "2023-03-15T12:58:22Z" generation: 1 name: 192.168.123.22 namespace: openshift-multus resourceVersion: "79342" uid: c7c2dfb9-18e7-4e2f-b2cf-49299c859774 spec: containerid: d075e6291dc82519f93205ce49eceed935dc3caa72196a4317684bb1276443d4 podref: default/netshoot-deployment-6db7547555-smhc8 kind: List metadata: resourceVersion: "" apiVersion: v1 items: - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: IPPool metadata: creationTimestamp: "2023-03-15T11:30:14Z" generation: 17 name: 192.168.123.0-24 namespace: openshift-multus resourceVersion: "79341" uid: c8a4f758-17fc-4177-8f0a-dd2894f9271f spec: allocations: "20": id: 87aead53324fc5d3346797a6c26ea438ccd0bbbf47bbb3e011d9b5522ab8926c podref: default/netshoot-deployment-6db7547555-xhfvr "21": id: 54e7773be81acb5a26a282960c96a055d74cd105e55d09528d0a3fd50c4d4e13 podref: default/netshoot-deployment-6db7547555-s89pm "22": id: d075e6291dc82519f93205ce49eceed935dc3caa72196a4317684bb1276443d4 podref: default/netshoot-deployment-6db7547555-smhc8 range: 192.168.123.0/24 kind: List metadata: resourceVersion: "" ================== Wed Mar 15 01:59:06 PM CET 2023 apiVersion: v1 items: [] kind: List metadata: resourceVersion: "" apiVersion: v1 items: - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: IPPool metadata: creationTimestamp: "2023-03-15T11:30:14Z" generation: 18 name: 192.168.123.0-24 namespace: openshift-multus resourceVersion: "79579" uid: c8a4f758-17fc-4177-8f0a-dd2894f9271f spec: allocations: {} range: 192.168.123.0/24 kind: List metadata: resourceVersion: "" ================== Wed Mar 15 01:59:12 PM CET 2023 apiVersion: v1 items: [] kind: List metadata: resourceVersion: "" apiVersion: v1 items: - apiVersion: whereabouts.cni.cncf.io/v1alpha1 kind: IPPool metadata: creationTimestamp: "2023-03-15T11:30:14Z" generation: 18 name: 192.168.123.0-24 namespace: openshift-multus resourceVersion: "79579" uid: c8a4f758-17fc-4177-8f0a-dd2894f9271f spec: allocations: {} range: 192.168.123.0/24 kind: List metadata: resourceVersion: ""
[akaris@linux whereabouts-clusterbot]$ oc logs -n openshift-multus whereabouts-reconciler-n4m8j --tail=0 -f 2023-03-15T12:59:00Z [verbose] starting reconciler run 2023-03-15T12:59:00Z [debug] NewReconcileLooper - inferred connection data 2023-03-15T12:59:00Z [debug] listing IP pools 2023-03-15T12:59:02Z [debug] the IP reservation: IP: 192.168.123.20 is reserved for pod: default/netshoot-deployment-6db7547555-xhfvr 2023-03-15T12:59:02Z [debug] pod reference default/netshoot-deployment-6db7547555-xhfvr matches allocation; Allocation IP: 192.168.123.20; PodIPs: map[] 2023-03-15T12:59:02Z [debug] pod ref default/netshoot-deployment-6db7547555-xhfvr is not listed in the live pods list 2023-03-15T12:59:02Z [debug] the IP reservation: IP: 192.168.123.21 is reserved for pod: default/netshoot-deployment-6db7547555-s89pm 2023-03-15T12:59:02Z [debug] pod reference default/netshoot-deployment-6db7547555-s89pm matches allocation; Allocation IP: 192.168.123.21; PodIPs: map[] 2023-03-15T12:59:02Z [debug] pod ref default/netshoot-deployment-6db7547555-s89pm is not listed in the live pods list 2023-03-15T12:59:02Z [debug] the IP reservation: IP: 192.168.123.22 is reserved for pod: default/netshoot-deployment-6db7547555-smhc8 2023-03-15T12:59:02Z [debug] pod reference default/netshoot-deployment-6db7547555-smhc8 matches allocation; Allocation IP: 192.168.123.22; PodIPs: map[] 2023-03-15T12:59:02Z [debug] pod ref default/netshoot-deployment-6db7547555-smhc8 is not listed in the live pods list 2023-03-15T12:59:02Z [debug] pod reference default/netshoot-deployment-6db7547555-xhfvr matches allocation; Allocation IP: 192.168.123.20; PodIPs: map[] 2023-03-15T12:59:02Z [debug] pod ref default/netshoot-deployment-6db7547555-xhfvr is not listed in the live pods list 2023-03-15T12:59:02Z [debug] pod reference default/netshoot-deployment-6db7547555-s89pm matches allocation; Allocation IP: 192.168.123.21; PodIPs: map[] 2023-03-15T12:59:02Z [debug] pod ref default/netshoot-deployment-6db7547555-s89pm is not listed in the live pods list 2023-03-15T12:59:02Z [debug] pod reference default/netshoot-deployment-6db7547555-smhc8 matches allocation; Allocation IP: 192.168.123.22; PodIPs: map[] 2023-03-15T12:59:02Z [debug] pod ref default/netshoot-deployment-6db7547555-smhc8 is not listed in the live pods list 2023-03-15T12:59:02Z [debug] Going to update the reserve list to: [] 2023-03-15T12:59:02Z [debug] successfully cleanup IPs: [192.168.123.22] 2023-03-15T12:59:02Z [verbose] removed stale overlappingIP allocation [192.168.123.20] 2023-03-15T12:59:02Z [verbose] removed stale overlappingIP allocation [192.168.123.21] 2023-03-15T12:59:02Z [verbose] removed stale overlappingIP allocation [192.168.123.22] 2023-03-15T12:59:02Z [verbose] reconciler success
Now, scale to 6 replicas:
oc scale deployment netshoot-deployment --replicas 6
And we can see now that due to the missing allocations, the next batch of pods is assigned the same IPs and we end up with duplicate IP assignments:
3: net1@if40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 52:58:1f:9d:50:ca brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.22/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::5058:1fff:fe9d:50ca/64 scope link valid_lft forever preferred_lft forever 3: net1@if24: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 22:28:95:11:2e:af brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.20/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::2028:95ff:fe11:2eaf/64 scope link valid_lft forever preferred_lft forever ===================== Wed Mar 15 01:59:20 PM CET 2023 NAME READY STATUS RESTARTS AGE netshoot-deployment-6db7547555-s89pm 1/1 Running 0 61s netshoot-deployment-6db7547555-smhc8 1/1 Running 0 61s netshoot-deployment-6db7547555-xhfvr 1/1 Running 0 61s .items[].status3: net1@if40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 26:8c:9a:e5:79:e6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.21/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::248c:9aff:fee5:79e6/64 scope link valid_lft forever preferred_lft forever 3: net1@if40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 52:58:1f:9d:50:ca brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.22/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::5058:1fff:fe9d:50ca/64 scope link valid_lft forever preferred_lft forever 3: net1@if24: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 22:28:95:11:2e:af brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.20/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::2028:95ff:fe11:2eaf/64 scope link valid_lft forever preferred_lft forever ===================== Wed Mar 15 01:59:27 PM CET 2023 NAME READY STATUS RESTARTS AGE netshoot-deployment-6db7547555-ffvt4 1/1 Running 0 4s netshoot-deployment-6db7547555-mkq28 1/1 Running 0 4s netshoot-deployment-6db7547555-pgtcg 1/1 Running 0 4s netshoot-deployment-6db7547555-s89pm 1/1 Running 0 68s netshoot-deployment-6db7547555-smhc8 1/1 Running 0 68s netshoot-deployment-6db7547555-xhfvr 1/1 Running 0 68s .items[].status3: net1@if42: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 86:75:87:e8:60:30 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.22/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::8475:87ff:fee8:6030/64 scope link valid_lft forever preferred_lft forever 3: net1@if26: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether ea:c7:8f:81:11:c3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.21/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::e8c7:8fff:fe81:11c3/64 scope link valid_lft forever preferred_lft forever 3: net1@if42: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether c6:49:fa:0f:9a:e4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.20/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::c449:faff:fe0f:9ae4/64 scope link valid_lft forever preferred_lft forever 3: net1@if40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 26:8c:9a:e5:79:e6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.21/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::248c:9aff:fee5:79e6/64 scope link valid_lft forever preferred_lft forever 3: net1@if40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 52:58:1f:9d:50:ca brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.22/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::5058:1fff:fe9d:50ca/64 scope link valid_lft forever preferred_lft forever 3: net1@if24: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 22:28:95:11:2e:af brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.20/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::2028:95ff:fe11:2eaf/64 scope link valid_lft forever preferred_lft forever ===================== Wed Mar 15 01:59:38 PM CET 2023 NAME READY STATUS RESTARTS AGE netshoot-deployment-6db7547555-ffvt4 1/1 Running 0 15s netshoot-deployment-6db7547555-mkq28 1/1 Running 0 15s netshoot-deployment-6db7547555-pgtcg 1/1 Running 0 15s netshoot-deployment-6db7547555-s89pm 1/1 Running 0 79s netshoot-deployment-6db7547555-smhc8 1/1 Running 0 79s netshoot-deployment-6db7547555-xhfvr 1/1 Running 0 79s .items[].status3: net1@if42: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 86:75:87:e8:60:30 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.22/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::8475:87ff:fee8:6030/64 scope link valid_lft forever preferred_lft forever 3: net1@if26: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether ea:c7:8f:81:11:c3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.21/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::e8c7:8fff:fe81:11c3/64 scope link valid_lft forever preferred_lft forever 3: net1@if42: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether c6:49:fa:0f:9a:e4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.20/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::c449:faff:fe0f:9ae4/64 scope link valid_lft forever preferred_lft forever 3: net1@if40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 26:8c:9a:e5:79:e6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.21/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::248c:9aff:fee5:79e6/64 scope link valid_lft forever preferred_lft forever 3: net1@if40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 52:58:1f:9d:50:ca brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.22/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::5058:1fff:fe9d:50ca/64 scope link valid_lft forever preferred_lft forever 3: net1@if24: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 22:28:95:11:2e:af brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.20/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::2028:95ff:fe11:2eaf/64 scope link valid_lft forever preferred_lft forever ===================== Wed Mar 15 01:59:49 PM CET 2023 NAME READY STATUS RESTARTS AGE netshoot-deployment-6db7547555-ffvt4 1/1 Running 0 26s netshoot-deployment-6db7547555-mkq28 1/1 Running 0 26s netshoot-deployment-6db7547555-pgtcg 1/1 Running 0 26s netshoot-deployment-6db7547555-s89pm 1/1 Running 0 90s netshoot-deployment-6db7547555-smhc8 1/1 Running 0 90s netshoot-deployment-6db7547555-xhfvr 1/1 Running 0 90s .items[].status3: net1@if42: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 86:75:87:e8:60:30 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.22/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::8475:87ff:fee8:6030/64 scope link valid_lft forever preferred_lft forever 3: net1@if26: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether ea:c7:8f:81:11:c3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.21/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::e8c7:8fff:fe81:11c3/64 scope link valid_lft forever preferred_lft forever 3: net1@if42: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether c6:49:fa:0f:9a:e4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.20/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::c449:faff:fe0f:9ae4/64 scope link valid_lft forever preferred_lft forever 3: net1@if40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 26:8c:9a:e5:79:e6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.21/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::248c:9aff:fee5:79e6/64 scope link valid_lft forever preferred_lft forever 3: net1@if40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 52:58:1f:9d:50:ca brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.22/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::5058:1fff:fe9d:50ca/64 scope link valid_lft forever preferred_lft forever 3: net1@if24: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 22:28:95:11:2e:af brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.20/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::2028:95ff:fe11:2eaf/64 scope link valid_lft forever preferred_lft forever ===================== Wed Mar 15 02:00:01 PM CET 2023 NAME READY STATUS RESTARTS AGE netshoot-deployment-6db7547555-ffvt4 1/1 Running 0 38s netshoot-deployment-6db7547555-mkq28 1/1 Running 0 38s netshoot-deployment-6db7547555-pgtcg 1/1 Running 0 38s netshoot-deployment-6db7547555-s89pm 1/1 Running 0 102s netshoot-deployment-6db7547555-smhc8 1/1 Running 0 102s netshoot-deployment-6db7547555-xhfvr 1/1 Running 0 102s .items[].status3: net1@if42: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 86:75:87:e8:60:30 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 192.168.123.22/24 brd 192.168.123.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::8475:87ff:fee8:6030/64 scope link valid_lft forever preferred_lft forever ^C [akaris@linux whereabouts-clusterbot]$
Root cause
This happens because the pods annotation is:
annotations:
k8s.ovn.org/pod-networks: '{"default":{"ip_addresses":["10.131.0.26/23"],"mac_address":"0a:58:0a:83:00:1a","gateway_ips":["10.131.0.1"],"ip_address":"10.131.0.26/23","gateway_ip":"10.131.0.1"}}'
k8s.v1.cni.cncf.io/network-status: |-
[{
"name": "ovn-kubernetes",
"interface": "eth0",
"ips": [
"10.131.0.26"
],
"mac": "0a:58:0a:83:00:1a",
"default": true,
"dns": {}
},{
"name": "default/br1-net",
"interface": "net1",
"ips": [
"192.168.123.22"
],
"mac": "86:75:87:e8:60:30",
"dns": {}
}]
But this code here:
92 func networkStatusFromPod(pod v1.Pod) string {
93 networkStatusAnnotationValue, isStatusAnnotationPresent := pod.Annotations[MultusNetworkStatusAnnotation]
94 if !isStatusAnnotationPresent || len(networkStatusAnnotationValue) == 0 {
95 return "[]"
96 }
97 return networkStatusAnnotationValue
98 }
Is looking for:
MultusNetworkStatusAnnotation = "k8s.v1.cni.cncf.io/networks-status"
Note network-status vs networks-status
Attachments
Issue Links
- is cloned by
-
-
- Closed
-
- is depended on by
-
-
- Closed
-
- links to
-
RHEA-2023:5006
rpm
