-
Bug
-
Resolution: Unresolved
-
Normal
-
None
-
None
-
4
-
False
-
-
False
-
?
-
?
-
?
-
?
-
None
-
-
-
Important
When loading an AnsibleTest CR like this:
apiVersion: test.openstack.org/v1beta1 kind: AnsibleTest metadata: creationTimestamp: "2024-11-04T17:34:57Z" generation: 1 name: ansibletest namespace: openstack resourceVersion: "6215831" uid: 71534195-6d0f-4db8-be79-9b1e8fcaf195 spec: ansibleCollections: "" ansibleExtraVars: "" ansibleGitRepo: https://github.com/NFV18/ansible-nfv ansibleInventory: | localhost ansible_connection=local ansible_python_interpreter=python3 undercloud-0 ansible_connection=local ansible_python_interpreter=python3 compute-0 ansible_host=192.168.122.100 ansible_user=cloud-admin ansible_ssh_private_key_file=/var/lib/ansible/.ssh/compute_id compute-1 ansible_host=192.168.122.101 ansible_user=cloud-admin ansible_ssh_private_key_file=/var/lib/ansible/.ssh/compute_id [compute] compute-0 compute-1 [local] localhost [undercloud] undercloud-0 ansiblePlaybookPath: playbooks/packet_gen/trex/performance_scenario.yml ansibleVarFiles: | --- # Use exist cloud resources cloud_resources: external # DUT compute dut_compute: compute-1 # Trex config trex_lcores: "2-11" # testpmd path (for rhel 8.4 vm) testpmd_bin: "/root/dpdk/build/app/dpdk-testpmd" # Disable search rates higher than the specified binary_search_disable_upward_search: true # Used to balance queues binary_search_warmup_trial_runtime: 120 discover_instance_external_ip: true ssh_key: test_keypair.key dynamic_instances: - name: trex group: trex user: cloud-user - name: testpmd-dpdk-dut group: dpdk_dut user: cloud-user - name: testpmd-sriov-vf-dut group: sriov_dut user: cloud-user backoffLimit: 0 computeSSHKeySecretName: dataplane-ansible-ssh-private-key-secret containerImage: quay.io/podified-master-centos9/openstack-ansible-tests:current-podified debug: true openStackConfigMap: openstack-config openStackConfigSecret: openstack-config-secret privileged: true storageClass: lvms-local-storage workflow: - ansibleExtraVars: ' -e manual_run=false -e binary_perf_log=/var/lib/AnsibleTests/external_files/dpdk_performance.log -e dut_group=dpdk_dut -e dut_type=dpdk -e testpmd_lcores=3,4,5 -e trex_rate=2 -e emc_insert_inv_prob=100 -e clone_traffic_gen_repo=false -e private_key_fetch_location=~/ -e python_interperter=/usr/bin/python3 ' backoffLimit: 0 computeSSHKeySecretName: "" containerImage: "" openStackConfigMap: openstack-config openStackConfigSecret: openstack-config-secret stepName: dpdk storageClass: local-storage workloadSSHKeySecretName: "" - ansibleExtraVars: ' -e manual_run=false -e binary_perf_log=/var/lib/AnsibleTests/external_files/sriov_performance.log -e dut_group=sriov_dut -e dut_type=sriov -e testpmd_lcores=1,2,3 -e trex_rate=15 -e clone_traffic_gen_repo=false -e private_key_fetch_location=~/ -e python_interperter=/usr/bin/python3 ' backoffLimit: 0 computeSSHKeySecretName: "" containerImage: "" openStackConfigMap: openstack-config openStackConfigSecret: openstack-config-secret stepName: sriov storageClass: local-storage workloadSSHKeySecretName: "" workloadSSHKeySecretName: open-tofu-ssh-keys
We hit this error:
[zuul@controller-0 ~]$ oc logs -f pod/test-operator-controller-manager-bd7d49858-wgvwt ... 2024-11-04T17:41:03Z INFO Starting workers {"controller": "ansibletest", "controllerGroup": "test.openstack.org", "controllerKind": "AnsibleTest", "worker count": 1} 2024-11-04T17:41:04Z INFO Controllers.AnsibleTest Can not acquire lock {"controller": "ansibletest", "controllerGroup": "test.openstack.org", "controllerKind": "AnsibleTest", "AnsibleTest": {"name":"ansibletest","namespace":"openstack"}, "namespace": "openstack", "name": "ansibletest", "reconcileID": "a3409fcd-92cc-4223-bc56-bec66c179a9c"} 2024-11-04T17:42:04Z INFO Controllers.AnsibleTest Can not acquire lock {"controller": "ansibletest", "controllerGroup": "test.openstack.org", "controllerKind": "AnsibleTest", "AnsibleTest": {"name":"ansibletest","namespace":"openstack"}, "namespace": "openstack", "name": "ansibletest", "reconcileID": "b987f984-23ce-4b8d-a261-c4004df61474"} 2024-11-04T17:42:59Z INFO ansibletest-resource default {"name": "ansibletest"} 2024-11-04T17:42:59Z INFO ansibletest-resource validate create {"name": "ansibletest"} 2024-11-04T17:42:59Z INFO Controllers.AnsibleTest Lock acquired {"controller": "ansibletest", "controllerGroup": "test.openstack.org", "controllerKind": "AnsibleTest", "AnsibleTest": {"name":"ansibletest","namespace":"openstack"}, "namespace": "openstack", "name": "ansibletest", "reconcileID": "11971c22-136a-48b3-876b-61af8fa12467"} 2024-11-04T17:42:59Z INFO Observed a panic in reconciler: runtime error: invalid memory address or nil pointer dereference {"controller": "ansibletest", "controllerGroup": "test.openstack.org", "controllerKind": "AnsibleTest", "AnsibleTest": {"name":"ansibletest","namespace":"openstack"}, "namespace": "openstack", "name": "ansibletest", "reconcileID": "11971c22-136a-48b3-876b-61af8fa12467"} panic: runtime error: invalid memory address or nil pointer dereference [recovered] panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x170ba9c] goroutine 316 [running]: sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile.func1() /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.6/pkg/internal/controller/controller.go:116 +0x1e5 panic({0x19076c0?, 0x2b0bba0?}) /usr/lib/golang/src/runtime/panic.go:914 +0x21f github.com/openstack-k8s-operators/test-operator/pkg/ansibletest.GetVolumeMounts(0x1, 0xc000cbd180, 0x0) /remote-source/app/pkg/ansibletest/volumes.go:208 +0x6fc github.com/openstack-k8s-operators/test-operator/pkg/ansibletest.Job(0xc000cbd180, 0xc0012830b0, {0xc0009e32a0, 0x20}, {0xc0009cc9c0, 0x13}, 0x0?, 0x0?, 0x0?, 0x0, ...) /remote-source/app/pkg/ansibletest/job.go:65 +0x412 github.com/openstack-k8s-operators/test-operator/controllers.(*AnsibleTestReconciler).Reconcile(0xc000128f40, {0x1e043c0?, 0xc000cde9c0}, {{{0xc000c97ae0?, 0x5?}, {0xc000c97ad0?, 0xc002dcbd08?}}}) /remote-source/app/controllers/ansibletest_controller.go:274 +0x1665 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x1e07720?, {0x1e043c0?, 0xc000cde9c0?}, {{{0xc000c97ae0?, 0xb?}, {0xc000c97ad0?, 0x0?}}}) /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.6/pkg/internal/controller/controller.go:119 +0xb7 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc00068ef00, {0x1e043f8, 0xc000250000}, {0x19b8180?, 0xc00068b3a0?}) /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.6/pkg/internal/controller/controller.go:316 +0x3cc sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc00068ef00, {0x1e043f8, 0xc000250000}) /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.6/pkg/internal/controller/controller.go:266 +0x1af sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2() /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.6/pkg/internal/controller/controller.go:227 +0x79 created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2 in goroutine 147 /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.6/pkg/internal/controller/controller.go:223 +0x565
As a result the ansibletest resource keeps waiting forever:
[zuul@controller-0 ci-framework]$ oc get ansibletests.test.openstack.org ansibletest NAME STATUS MESSAGE ansibletest Unknown Deployment not started
The problem seems to be here: ExtraConfigmapsMounts is expected. So if the AnsibleTest doesn't have extraConfigmapsMounts we hit the above error.
Adding this to workflow items as a workaround seems to work:
extraConfigmapsMounts: - name: dummy-ansibletest-cm mountPath: /mnt subPath: dummy
- links to
- mentioned on
(1 mentioned on)