Loading...

Details

Type: Bug
Resolution: Done-Errata
Priority: Major
Fix Version/s: CNV v4.13.0
Affects Version/s: None
Component/s: CNV Network
Labels:
- blocker+
- cnv-4+
- cnvbugsm
- devel_ack+
- pm_ack+
- qa_ack+

Blocked:
False
Blocked Reason:

Hide

None

Show
None
Ready:
False
BZ Status:
CLOSED
BZ URL:
https://bugzilla.redhat.com/show_bug.cgi?id=2167012
Bugzilla Bug:
RHBZ: 2167012
[QE] How to address?:
---
[QE] Why QE missed?:
---

Severity:
High

SFDC Cases Links:
SFDC Cases Counter:

Description

Description of problem: Created a network bridge, nad. On attempting to create a vm with the same, it is failing to come up.

Version-Release number of selected component (if applicable):
4.13.0

How reproducible:
100%

Steps to Reproduce:
1. Create a nncp and nad, create a vm using the same
2. Vm is stuck in starting state
3.

Actual results:
=================
[centos@ci-vm-10-0-96-58 ~]$ kubectl get nodenetworkconfigurationpolicy -A -o yaml
apiVersion: v1
items:

apiVersion: nmstate.io/v1
kind: NodeNetworkConfigurationPolicy
metadata:
annotations:
nmstate.io/webhook-mutating-timestamp: "1675458303349988570"
creationTimestamp: "2023-02-03T21:05:03Z"
generation: 1
labels:
created-by-dynamic-class-creator: "Yes"
name: must-gather-br
resourceVersion: "13517946"
uid: 18f94152-5c69-49ee-8c38-98c4d326a0c8
spec:
desiredState:
interfaces:
bridge:
options:
stp:
enabled: false
port: []
ipv4:
auto-dns: true
dhcp: false
enabled: false
ipv6:
auto-dns: true
autoconf: false
dhcp: false
enabled: false
name: mg-br1
state: up
type: linux-bridge
status:
conditions:
lastHeartbeatTime: "2023-02-03T21:05:28Z"
lastTransitionTime: "2023-02-03T21:05:28Z"
message: 6/6 nodes successfully configured
reason: SuccessfullyConfigured
status: "True"
type: Available
lastHeartbeatTime: "2023-02-03T21:05:28Z"
lastTransitionTime: "2023-02-03T21:05:28Z"
reason: SuccessfullyConfigured
status: "False"
type: Degraded
lastHeartbeatTime: "2023-02-03T21:05:28Z"
lastTransitionTime: "2023-02-03T21:05:28Z"
reason: ConfigurationProgressing
status: "False"
type: Progressing
lastUnavailableNodeCountUpdate: "2023-02-03T21:05:28Z"
kind: List
metadata:
resourceVersion: ""
[centos@ci-vm-10-0-96-58 ~]$
==================
[centos@ci-vm-10-0-96-58 ~]$ oc get network-attachment-definition -A -o yaml
apiVersion: v1
items:
apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
annotations:
k8s.v1.cni.cncf.io/resourceName: bridge.network.kubevirt.io/mg-br1
creationTimestamp: "2023-02-03T21:05:28Z"
generation: 1
labels:
created-by-dynamic-class-creator: "Yes"
name: mg-br1
namespace: node-gather-unprivileged
resourceVersion: "13517948"
uid: ea51e09f-2b9e-4890-b2f0-5b31c29de800
spec:
config: ' {"cniVersion": "0.3.1", "type": "cnv-bridge", "name": "mg-br1", "bridge": "mg-br1"}
'
kind: List
metadata:
resourceVersion: ""
[centos@ci-vm-10-0-96-58 ~]$
================
[centos@ci-vm-10-0-96-58 ~]$ oc get vm -A -o yaml
apiVersion: v1
items:
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
annotations:
kubemacpool.io/transaction-timestamp: "2023-02-03T21:28:00.983243675Z"
kubevirt.io/latest-observed-api-version: v1
kubevirt.io/storage-observed-api-version: v1alpha3
creationTimestamp: "2023-02-03T21:05:29Z"
generation: 2
labels:
created-by-dynamic-class-creator: "Yes"
kubevirt.io/vm: must-gather-vm-2
name: must-gather-vm-2-1675458386-176019
namespace: node-gather-unprivileged
resourceVersion: "13543787"
uid: d98cc51e-83b8-41b6-810c-be6fde37f79c
spec:
running: true
template:
metadata:
creationTimestamp: null
labels:
kubevirt.io/domain: must-gather-vm-2-1675458386-176019
kubevirt.io/vm: must-gather-vm-2-1675458386-176019
spec:
domain:
cpu:
cores: 1
devices:
disks:
disk:
bus: virtio
name: containerdisk
disk:
bus: virtio
name: cloudinitdisk
interfaces:
macAddress: 02:86:35:00:00:08
masquerade: {}
name: default
bridge: {}
macAddress: 02:86:35:00:00:09
name: mg-br1
rng: {}
machine:
type: pc-q35-rhel9.0.0
resources:
requests:
memory: 1Gi
networks:
name: default
pod: {}
multus:
networkName: mg-br1
name: mg-br1
terminationGracePeriodSeconds: 30
volumes:
containerDisk:
image: quay.io/openshift-cnv/qe-cnv-tests-fedora:35@sha256:cc9f7458a2c9ae8c644953b1068c54d5267c6241ae1b3c3379e39ca3196c7046
name: containerdisk
cloudInitNoCloud:
userData: |-
#cloud-config
chpasswd:
expire: false
password: password
user: fedora

ssh_authorized_keys:
[ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj47ubVnxR16JU7ZfDli3N5QVBAwJBRh2xMryyjk5dtfugo5JIPGB2cyXTqEDdzuRmI+Vkb/A5duJyBRlA+9RndGGmhhMnj8and3wu5/cEb7DkF6ZJ25QV4LQx3K/i57LStUHXRTvruHOZ2nCuVXWqi7wSvz5YcvEv7O8pNF5uGmqHlShBdxQxcjurXACZ1YY0YDJDr3AJai1KF9zehVJODuSbrnOYpThVWGjFuFAnNxbtuZ8EOSougN2aYTf2qr/KFGDHtewIkzZmP6cjzKO5bN3pVbXxmb2Gces/BYHntY4MXBTUqwsmsCRC5SAz14bEP/vsLtrNhjq9vCS+BjMT root@exec1.rdocloud]
runcmd: ['grep ssh-rsa /etc/crypto-policies/back-ends/opensshserver.config || sudo update-crypto-policies --set LEGACY || true', "sudo sed -i 's/^#\\?PasswordAuthentication no/PasswordAuthentication yes/g' /etc/ssh/sshd_config", 'sudo systemctl enable sshd', 'sudo systemctl restart sshd']
name: cloudinitdisk
status:
conditions:

lastProbeTime: "2023-02-03T21:28:01Z"
lastTransitionTime: "2023-02-03T21:28:01Z"
message: Guest VM is not reported as running
reason: GuestNotRunning
status: "False"
type: Ready
created: true
printableStatus: Starting
volumeSnapshotStatuses:
enabled: false
name: containerdisk
reason: Snapshot is not supported for this volumeSource type [containerdisk]
enabled: false
name: cloudinitdisk
reason: Snapshot is not supported for this volumeSource type [cloudinitdisk]
kind: List
metadata:
resourceVersion: ""
[centos@ci-vm-10-0-96-58 ~]$

Following events are found:
=================
node-gather-unprivileged 26m Normal SuccessfulCreate virtualmachine/must-gather-vm-2-1675458386-176019 Started the virtual machine by creating the new virtual machine instance must-gather-vm-2-1675458386-176019
node-gather-unprivileged 26m Normal SuccessfulCreate virtualmachineinstance/must-gather-vm-2-1675458386-176019 Created virtual machine pod virt-launcher-must-gather-vm-2-1675458386-176019-95tsr
node-gather-unprivileged 26m Normal Scheduled pod/virt-launcher-must-gather-vm-2-1675458386-176019-95tsr Successfully assigned node-gather-unprivileged/virt-launcher-must-gather-vm-2-1675458386-176019-95tsr to c01-dbn-413-tdp54-worker-0-hvbzd
node-gather-unprivileged 26m Normal AddedInterface pod/virt-launcher-must-gather-vm-2-1675458386-176019-95tsr Add eth0 [10.128.2.100/23] from ovn-kubernetes
node-gather-unprivileged 26m Warning FailedCreatePodSandBox pod/virt-launcher-must-gather-vm-2-1675458386-176019-95tsr Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_virt-launcher-must-gather-vm-2-1675458386-176019-95tsr_node-gather-unprivileged_04a6e6dc-a484-4bc7-8133-39c14f8c5eab_0(6cf7b15d5343799f96b705e8d671854540c1a600388add86a3f2235ed840baa2): error adding pod node-gather-unprivileged_virt-launcher-must-gather-vm-2-1675458386-176019-95tsr to CNI network "multus-cni-network": plugin type="multus" name="multus-cni-network" failed (add): [node-gather-unprivileged/virt-launcher-must-gather-vm-2-1675458386-176019-95tsr/04a6e6dc-a484-4bc7-8133-39c14f8c5eab:mg-br1]: error adding container to network "mg-br1": netplugin failed: "/var/lib/cni/bin/cnv-bridge: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by /var/lib/cni/bin/cnv-bridge)\n/var/lib/cni/bin/cnv-bridge: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by /var/lib/cni/bin/cnv-bridge)\n"
node-gather-unprivileged 26m Normal AddedInterface pod/virt-launcher-must-gather-vm-2-1675458386-176019-95tsr Add eth0 [10.128.2.100/23] from ovn-kubernetes
node-gather-unprivileged 26m Warning FailedCreatePodSandBox pod/virt-launcher-must-gather-vm-2-1675458386-176019-95tsr Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_virt-launcher-must-gather-vm-2-1675458386-176019-95tsr_node-gather-unprivileged_04a6e6dc-a484-4bc7-8133-39c14f8c5eab_0(c39a8ac72a300bd8545f27955a71dd1a90adf3cc582f02aef0c5b1a63549d2eb): error adding pod node-gather-unprivileged_virt-launcher-must-gather-vm-2-1675458386-176019-95tsr to CNI network "multus-cni-network": plugin type="multus" name="multus-cni-network" failed (add): [node-gather-unprivileged/virt-launcher-must-gather-vm-2-1675458386-176019-95tsr/04a6e6dc-a484-4bc7-8133-39c14f8c5eab:mg-br1]: error adding container to network "mg-br1": netplugin failed: "/var/lib/cni/bin/cnv-bridge: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by /var/lib/cni/bin/cnv-bridge)\n/var/lib/cni/bin/cnv-bridge: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by /var/lib/cni/bin/cnv-bridge)\n"
Expected results:
Vms should be in running state.

Additional info: