Extract /home/jenkins/oadp-e2e-qe.tar.gz to /alabama/cspi Extract /home/jenkins/oadp-apps-deployer.tar.gz to /alabama/oadpApps Extract /home/jenkins/mtc-python-client.tar.gz to /alabama/pyclient Create and populate /tmp/test-settings... Login as Kubeadmin to the test cluster at https://api.ci-op-98rbwkjv-2c502.cspilp.interop.ccitredhat.com:6443... WARNING: Using insecure TLS client config. Setting this option is not supported! Login successful. You have access to 78 projects, the list has been suppressed. You can list all projects with 'oc projects' Using project "default". Create virtual environment and install required packages... Collecting ansible_runner Downloading ansible_runner-2.4.2-py3-none-any.whl.metadata (3.2 kB) Collecting pexpect>=4.5 (from ansible_runner) Downloading pexpect-4.9.0-py2.py3-none-any.whl.metadata (2.5 kB) Collecting packaging (from ansible_runner) Downloading packaging-25.0-py3-none-any.whl.metadata (3.3 kB) Collecting python-daemon (from ansible_runner) Downloading python_daemon-3.1.2-py3-none-any.whl.metadata (4.8 kB) Collecting pyyaml (from ansible_runner) Downloading pyyaml-6.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (2.4 kB) Collecting ptyprocess>=0.5 (from pexpect>=4.5->ansible_runner) Downloading ptyprocess-0.7.0-py2.py3-none-any.whl.metadata (1.3 kB) Collecting lockfile>=0.10 (from python-daemon->ansible_runner) Downloading lockfile-0.12.2-py2.py3-none-any.whl.metadata (2.4 kB) Downloading ansible_runner-2.4.2-py3-none-any.whl (79 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 79.8/79.8 kB 3.9 MB/s eta 0:00:00 Downloading pexpect-4.9.0-py2.py3-none-any.whl (63 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 63.8/63.8 kB 7.4 MB/s eta 0:00:00 Downloading packaging-25.0-py3-none-any.whl (66 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 66.5/66.5 kB 8.5 MB/s eta 0:00:00 Downloading python_daemon-3.1.2-py3-none-any.whl (30 kB) Downloading pyyaml-6.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (807 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 807.9/807.9 kB 28.8 MB/s eta 0:00:00 Downloading lockfile-0.12.2-py2.py3-none-any.whl (13 kB) Downloading ptyprocess-0.7.0-py2.py3-none-any.whl (13 kB) Installing collected packages: ptyprocess, lockfile, pyyaml, python-daemon, pexpect, packaging, ansible_runner Successfully installed ansible_runner-2.4.2 lockfile-0.12.2 packaging-25.0 pexpect-4.9.0 ptyprocess-0.7.0 python-daemon-3.1.2 pyyaml-6.0.3 [notice] A new release of pip is available: 23.3.2 -> 25.3 [notice] To update, run: pip install --upgrade pip Processing /alabama/oadpApps Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' Preparing metadata (pyproject.toml): started Preparing metadata (pyproject.toml): finished with status 'done' Building wheels for collected packages: ocpdeployer Building wheel for ocpdeployer (pyproject.toml): started Building wheel for ocpdeployer (pyproject.toml): finished with status 'done' Created wheel for ocpdeployer: filename=ocpdeployer-0.0.1-py2.py3-none-any.whl size=241503 sha256=bff902331294be43cbdd866e9bb3e11de47596f22bc2befc07a36e15c9e92da9 Stored in directory: /tmp/pip-ephem-wheel-cache-p35abxo5/wheels/55/c3/15/eb89266a7928fafe53678a24892891bbfb18405fbd475eb4c6 Successfully built ocpdeployer Installing collected packages: ocpdeployer Successfully installed ocpdeployer-0.0.1 [notice] A new release of pip is available: 23.3.2 -> 25.3 [notice] To update, run: pip install --upgrade pip Processing /alabama/pyclient Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' Preparing metadata (pyproject.toml): started Preparing metadata (pyproject.toml): finished with status 'done' Collecting suds-py3 (from mtc==0.0.1) Downloading suds_py3-1.4.5.0-py3-none-any.whl.metadata (778 bytes) Collecting requests (from mtc==0.0.1) Downloading requests-2.32.5-py3-none-any.whl.metadata (4.9 kB) Collecting jinja2 (from mtc==0.0.1) Downloading jinja2-3.1.6-py3-none-any.whl.metadata (2.9 kB) Collecting kubernetes==11.0.0 (from mtc==0.0.1) Downloading kubernetes-11.0.0-py3-none-any.whl.metadata (1.5 kB) Collecting openshift==0.11.2 (from mtc==0.0.1) Downloading openshift-0.11.2.tar.gz (19 kB) Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' Preparing metadata (pyproject.toml): started Preparing metadata (pyproject.toml): finished with status 'done' Collecting certifi>=14.05.14 (from kubernetes==11.0.0->mtc==0.0.1) Downloading certifi-2026.1.4-py3-none-any.whl.metadata (2.5 kB) Collecting six>=1.9.0 (from kubernetes==11.0.0->mtc==0.0.1) Downloading six-1.17.0-py2.py3-none-any.whl.metadata (1.7 kB) Collecting python-dateutil>=2.5.3 (from kubernetes==11.0.0->mtc==0.0.1) Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) Collecting setuptools>=21.0.0 (from kubernetes==11.0.0->mtc==0.0.1) Using cached setuptools-80.9.0-py3-none-any.whl.metadata (6.6 kB) Requirement already satisfied: pyyaml>=3.12 in /alabama/venv/lib64/python3.12/site-packages (from kubernetes==11.0.0->mtc==0.0.1) (6.0.3) Collecting google-auth>=1.0.1 (from kubernetes==11.0.0->mtc==0.0.1) Downloading google_auth-2.47.0-py3-none-any.whl.metadata (6.4 kB) Collecting websocket-client!=0.40.0,!=0.41.*,!=0.42.*,>=0.32.0 (from kubernetes==11.0.0->mtc==0.0.1) Downloading websocket_client-1.9.0-py3-none-any.whl.metadata (8.3 kB) Collecting requests-oauthlib (from kubernetes==11.0.0->mtc==0.0.1) Downloading requests_oauthlib-2.0.0-py2.py3-none-any.whl.metadata (11 kB) Collecting urllib3>=1.24.2 (from kubernetes==11.0.0->mtc==0.0.1) Downloading urllib3-2.6.3-py3-none-any.whl.metadata (6.9 kB) Collecting python-string-utils (from openshift==0.11.2->mtc==0.0.1) Downloading python_string_utils-1.0.0-py3-none-any.whl.metadata (12 kB) Collecting ruamel.yaml>=0.15 (from openshift==0.11.2->mtc==0.0.1) Downloading ruamel_yaml-0.19.1-py3-none-any.whl.metadata (16 kB) Collecting MarkupSafe>=2.0 (from jinja2->mtc==0.0.1) Downloading markupsafe-3.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (2.7 kB) Collecting charset_normalizer<4,>=2 (from requests->mtc==0.0.1) Downloading charset_normalizer-3.4.4-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (37 kB) Collecting idna<4,>=2.5 (from requests->mtc==0.0.1) Downloading idna-3.11-py3-none-any.whl.metadata (8.4 kB) Collecting pyasn1-modules>=0.2.1 (from google-auth>=1.0.1->kubernetes==11.0.0->mtc==0.0.1) Downloading pyasn1_modules-0.4.2-py3-none-any.whl.metadata (3.5 kB) Collecting rsa<5,>=3.1.4 (from google-auth>=1.0.1->kubernetes==11.0.0->mtc==0.0.1) Downloading rsa-4.9.1-py3-none-any.whl.metadata (5.6 kB) Collecting oauthlib>=3.0.0 (from requests-oauthlib->kubernetes==11.0.0->mtc==0.0.1) Downloading oauthlib-3.3.1-py3-none-any.whl.metadata (7.9 kB) Collecting pyasn1<0.7.0,>=0.6.1 (from pyasn1-modules>=0.2.1->google-auth>=1.0.1->kubernetes==11.0.0->mtc==0.0.1) Downloading pyasn1-0.6.1-py3-none-any.whl.metadata (8.4 kB) Downloading kubernetes-11.0.0-py3-none-any.whl (1.5 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.5/1.5 MB 32.1 MB/s eta 0:00:00 Downloading jinja2-3.1.6-py3-none-any.whl (134 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 134.9/134.9 kB 14.5 MB/s eta 0:00:00 Downloading requests-2.32.5-py3-none-any.whl (64 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.7/64.7 kB 7.0 MB/s eta 0:00:00 Downloading suds_py3-1.4.5.0-py3-none-any.whl (298 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 298.8/298.8 kB 30.0 MB/s eta 0:00:00 Downloading certifi-2026.1.4-py3-none-any.whl (152 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 152.9/152.9 kB 16.4 MB/s eta 0:00:00 Downloading charset_normalizer-3.4.4-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (153 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 153.5/153.5 kB 15.6 MB/s eta 0:00:00 Downloading google_auth-2.47.0-py3-none-any.whl (234 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 234.9/234.9 kB 25.4 MB/s eta 0:00:00 Downloading idna-3.11-py3-none-any.whl (71 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 71.0/71.0 kB 7.9 MB/s eta 0:00:00 Downloading markupsafe-3.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (22 kB) Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 229.9/229.9 kB 28.0 MB/s eta 0:00:00 Downloading ruamel_yaml-0.19.1-py3-none-any.whl (118 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 118.1/118.1 kB 13.0 MB/s eta 0:00:00 Using cached setuptools-80.9.0-py3-none-any.whl (1.2 MB) Downloading six-1.17.0-py2.py3-none-any.whl (11 kB) Downloading urllib3-2.6.3-py3-none-any.whl (131 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 131.6/131.6 kB 15.6 MB/s eta 0:00:00 Downloading websocket_client-1.9.0-py3-none-any.whl (82 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 82.6/82.6 kB 11.4 MB/s eta 0:00:00 Downloading python_string_utils-1.0.0-py3-none-any.whl (26 kB) Downloading requests_oauthlib-2.0.0-py2.py3-none-any.whl (24 kB) Downloading oauthlib-3.3.1-py3-none-any.whl (160 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 160.1/160.1 kB 18.4 MB/s eta 0:00:00 Downloading pyasn1_modules-0.4.2-py3-none-any.whl (181 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 181.3/181.3 kB 23.3 MB/s eta 0:00:00 Downloading rsa-4.9.1-py3-none-any.whl (34 kB) Downloading pyasn1-0.6.1-py3-none-any.whl (83 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 83.1/83.1 kB 9.7 MB/s eta 0:00:00 Building wheels for collected packages: mtc, openshift Building wheel for mtc (pyproject.toml): started Building wheel for mtc (pyproject.toml): finished with status 'done' Created wheel for mtc: filename=mtc-0.0.1-py3-none-any.whl size=31146 sha256=260d255f28db8f589ab5cc9f5f859a740d22516bc4dfdd83b491113c8b8a0b93 Stored in directory: /tmp/pip-ephem-wheel-cache-5zb2njcj/wheels/f1/2c/83/c09cb54cb0e821a8186cf5320758c27e7227ec862045210509 Building wheel for openshift (pyproject.toml): started Building wheel for openshift (pyproject.toml): finished with status 'done' Created wheel for openshift: filename=openshift-0.11.2-py3-none-any.whl size=19881 sha256=6f6865ae1874f6b69f4bffb266af79a81f9a5fb9fefb0c66239b74f7e78e2a87 Stored in directory: /alabama/.cache/pip/wheels/34/b7/02/4eb142942314b119c5fb3d4e595ac59486c1f3d79ff665397d Successfully built mtc openshift Installing collected packages: suds-py3, websocket-client, urllib3, six, setuptools, ruamel.yaml, python-string-utils, pyasn1, oauthlib, MarkupSafe, idna, charset_normalizer, certifi, rsa, requests, python-dateutil, pyasn1-modules, jinja2, requests-oauthlib, google-auth, kubernetes, openshift, mtc Successfully installed MarkupSafe-3.0.3 certifi-2026.1.4 charset_normalizer-3.4.4 google-auth-2.47.0 idna-3.11 jinja2-3.1.6 kubernetes-11.0.0 mtc-0.0.1 oauthlib-3.3.1 openshift-0.11.2 pyasn1-0.6.1 pyasn1-modules-0.4.2 python-dateutil-2.9.0.post0 python-string-utils-1.0.0 requests-2.32.5 requests-oauthlib-2.0.0 rsa-4.9.1 ruamel.yaml-0.19.1 setuptools-80.9.0 six-1.17.0 suds-py3-1.4.5.0 urllib3-2.6.3 websocket-client-1.9.0 [notice] A new release of pip is available: 23.3.2 -> 25.3 [notice] To update, run: pip install --upgrade pip go: downloading go1.25.0 (linux/amd64) go: downloading github.com/openshift/velero v0.10.2-0.20251209203558-3b4c38bb2b79 go: downloading k8s.io/api v0.33.3 go: downloading github.com/onsi/gomega v1.36.3 go: downloading github.com/onsi/ginkgo/v2 v2.23.4 go: downloading k8s.io/apimachinery v0.33.3 go: downloading github.com/migtools/oadp-non-admin v0.0.0-20250409143544-08533a6c302d go: downloading k8s.io/utils v0.0.0-20241104100929-3ea5e8cea738 go: downloading github.com/openshift/oadp-operator v1.0.2-0.20260109185244-876ff3ff7ffe go: downloading github.com/google/uuid v1.6.0 go: downloading github.com/aws/aws-sdk-go v1.44.253 go: downloading k8s.io/client-go v0.33.3 go: downloading github.com/kubernetes-csi/external-snapshotter/client/v4 v4.2.0 go: downloading sigs.k8s.io/controller-runtime v0.21.0 go: downloading gopkg.in/yaml.v2 v2.4.0 go: downloading github.com/operator-framework/api v0.14.1-0.20220413143725-33310d6154f3 go: downloading github.com/andygrunwald/go-jira v1.16.0 go: downloading github.com/apenella/go-ansible v1.1.5 go: downloading github.com/openshift/api v0.0.0-20240524162738-d899f8877d22 go: downloading k8s.io/kubectl v0.33.3 go: downloading github.com/openshift/client-go v0.0.0-20211209144617-7385dd6338e3 go: downloading sigs.k8s.io/yaml v1.4.0 go: downloading github.com/google/go-cmp v0.7.0 go: downloading github.com/go-logr/logr v1.4.3 go: downloading github.com/gogo/protobuf v1.3.2 go: downloading sigs.k8s.io/randfill v1.0.0 go: downloading gopkg.in/inf.v0 v0.9.1 go: downloading github.com/spf13/pflag v1.0.6-0.20210604193023-d5e0c0615ace go: downloading k8s.io/klog/v2 v2.130.1 go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.6.0 go: downloading github.com/evanphx/json-patch/v5 v5.9.11 go: downloading github.com/fatih/structs v1.1.0 go: downloading github.com/golang-jwt/jwt/v4 v4.5.2 go: downloading github.com/google/go-querystring v1.1.0 go: downloading github.com/pkg/errors v0.9.1 go: downloading github.com/trivago/tgo v1.0.7 go: downloading github.com/sirupsen/logrus v1.9.3 go: downloading golang.org/x/term v0.37.0 go: downloading github.com/apenella/go-common-utils/error v0.0.0-20210528133155-34ba915e28c8 go: downloading github.com/apenella/go-common-utils/data v0.0.0-20210528133155-34ba915e28c8 go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 go: downloading golang.org/x/net v0.47.0 go: downloading github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674 go: downloading github.com/spf13/cobra v1.8.1 go: downloading k8s.io/cli-runtime v0.33.3 go: downloading k8s.io/component-base v0.33.3 go: downloading gopkg.in/yaml.v3 v3.0.1 go: downloading go.uber.org/automaxprocs v1.6.0 go: downloading golang.org/x/sys v0.38.0 go: downloading sigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3 go: downloading github.com/json-iterator/go v1.1.12 go: downloading k8s.io/kube-openapi v0.0.0-20250318190949-c8a335a9a2ff go: downloading k8s.io/apiextensions-apiserver v0.33.3 go: downloading gopkg.in/evanphx/json-patch.v4 v4.12.0 go: downloading github.com/blang/semver/v4 v4.0.0 go: downloading github.com/google/gnostic-models v0.6.9 go: downloading google.golang.org/protobuf v1.36.6 go: downloading golang.org/x/time v0.12.0 go: downloading github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc go: downloading github.com/fxamacker/cbor/v2 v2.7.0 go: downloading golang.org/x/oauth2 v0.30.0 go: downloading github.com/moby/spdystream v0.5.0 go: downloading github.com/inconshreveable/mousetrap v1.1.0 go: downloading github.com/jonboulle/clockwork v0.4.0 go: downloading k8s.io/component-helpers v0.33.3 go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de go: downloading github.com/distribution/reference v0.6.0 go: downloading github.com/moby/term v0.5.0 go: downloading sigs.k8s.io/kustomize/kustomize/v5 v5.6.0 go: downloading sigs.k8s.io/kustomize/kyaml v0.19.0 go: downloading github.com/exponent-io/jsonpath v0.0.0-20210407135951-1de76d718b3f go: downloading github.com/lithammer/dedent v1.1.0 go: downloading k8s.io/metrics v0.33.3 go: downloading github.com/chai2010/gettext-go v1.0.2 go: downloading github.com/MakeNowJust/heredoc v1.0.0 go: downloading github.com/mitchellh/go-wordwrap v1.0.1 go: downloading github.com/russross/blackfriday/v2 v2.1.0 go: downloading golang.org/x/text v0.31.0 go: downloading golang.org/x/tools v0.38.0 go: downloading github.com/go-task/slim-sprig/v3 v3.0.0 go: downloading github.com/kubernetes-csi/external-snapshotter/client/v8 v8.2.0 go: downloading github.com/aws/aws-sdk-go-v2/config v1.26.3 go: downloading github.com/stretchr/testify v1.10.0 go: downloading gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c go: downloading github.com/go-logr/zapr v1.3.0 go: downloading go.uber.org/zap v1.27.0 go: downloading go.uber.org/goleak v1.3.0 go: downloading github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 go: downloading github.com/aws/aws-sdk-go-v2/credentials v1.17.26 go: downloading github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.15.11 go: downloading github.com/aws/aws-sdk-go-v2 v1.30.3 go: downloading github.com/aws/aws-sdk-go-v2/service/s3 v1.48.0 go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd go: downloading github.com/modern-go/reflect2 v1.0.2 go: downloading github.com/go-openapi/jsonreference v0.20.2 go: downloading github.com/go-openapi/swag v0.23.0 go: downloading github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f go: downloading github.com/x448/float16 v0.8.4 go: downloading golang.org/x/sync v0.18.0 go: downloading sigs.k8s.io/kustomize/api v0.19.0 go: downloading github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 go: downloading github.com/peterbourgon/diskv v2.0.1+incompatible go: downloading github.com/fatih/camelcase v1.0.0 go: downloading github.com/opencontainers/go-digest v1.0.0 go: downloading github.com/creack/pty v1.1.18 go: downloading github.com/google/pprof v0.0.0-20250403155104-27863c87afa6 go: downloading github.com/spf13/afero v1.10.0 go: downloading github.com/kr/pretty v0.3.1 go: downloading github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 go: downloading github.com/stretchr/objx v0.5.2 go: downloading github.com/go-openapi/jsonpointer v0.21.0 go: downloading go.uber.org/multierr v1.11.0 go: downloading github.com/emicklei/go-restful/v3 v3.11.0 go: downloading github.com/jmespath/go-jmespath v0.4.0 go: downloading github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.11 go: downloading github.com/aws/aws-sdk-go-v2/internal/ini v1.8.0 go: downloading github.com/aws/aws-sdk-go-v2/service/sso v1.22.3 go: downloading github.com/aws/aws-sdk-go-v2/service/ssooidc v1.26.4 go: downloading github.com/aws/aws-sdk-go-v2/service/sts v1.30.3 go: downloading github.com/aws/smithy-go v1.20.3 go: downloading github.com/mailru/easyjson v0.7.7 go: downloading github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.5.4 go: downloading github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.15 go: downloading github.com/aws/aws-sdk-go-v2/internal/v4a v1.2.10 go: downloading github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.11.3 go: downloading github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.2.10 go: downloading github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.11.17 go: downloading github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.16.10 go: downloading github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161 go: downloading github.com/google/btree v1.1.3 go: downloading github.com/go-errors/errors v1.4.2 go: downloading github.com/google/gofuzz v1.2.0 go: downloading github.com/prometheus/client_golang v1.22.0 go: downloading github.com/prashantv/gostub v1.1.0 go: downloading github.com/rogpeppe/go-internal v1.13.1 go: downloading github.com/kr/text v0.2.0 go: downloading github.com/jmespath/go-jmespath/internal/testify v1.5.1 go: downloading github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.15 go: downloading gomodules.xyz/jsonpatch/v2 v2.4.0 go: downloading github.com/prometheus/client_model v0.6.2 go: downloading github.com/sergi/go-diff v1.2.0 go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 go: downloading github.com/xlab/treeprint v1.2.0 go: downloading github.com/josharian/intern v1.0.0 go: downloading github.com/fsnotify/fsnotify v1.7.0 go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: downloading github.com/evanphx/json-patch v5.6.0+incompatible go: downloading github.com/prometheus/procfs v0.15.1 go: downloading github.com/klauspost/compress v1.18.0 go: downloading github.com/beorn7/perks v1.0.1 go: downloading github.com/cespare/xxhash/v2 v2.3.0 go: downloading github.com/prometheus/common v0.65.0 go: downloading github.com/kylelemons/godebug v1.1.0 storageclass.storage.k8s.io/gp2-csi annotated storageclass.storage.k8s.io/gp3-csi annotated storageclass.storage.k8s.io/odf-operator-ceph-rbd annotated storageclass.storage.k8s.io/odf-operator-ceph-rbd-virtualization annotated storageclass.storage.k8s.io/odf-operator-cephfs annotated storageclass.storage.k8s.io/openshift-storage.noobaa.io annotated storageclass.storage.k8s.io/odf-operator-ceph-rbd annotated + readonly 'RED=\e[31m' + RED='\e[31m' + readonly 'BLUE=\033[34m' + BLUE='\033[34m' + readonly 'CLEAR=\e[39m' + CLEAR='\e[39m' ++ oc get infrastructures cluster -o 'jsonpath={.status.platform}' ++ awk '{print tolower($0)}' + CLOUD_PROVIDER=aws + [[ '' == \t\r\u\e ]] + echo /home/jenkins/.kube/config /home/jenkins/.kube/config + [[ aws == *-arm* ]] + [[ aws == *-fips* ]] + E2E_TIMEOUT_MULTIPLIER=2 + export NAMESPACE=openshift-adp + NAMESPACE=openshift-adp + export PROVIDER=aws + PROVIDER=aws ++ echo aws ++ awk '{print tolower($0)}' + BACKUP_LOCATION=aws + export BACKUP_LOCATION=aws + BACKUP_LOCATION=aws + export BUCKET=ci-op-98rbwkjv-interopoadp + BUCKET=ci-op-98rbwkjv-interopoadp + OADP_CREDS_FILE=/tmp/test-settings/credentials + OADP_VSL_CREDS_FILE=/tmp/test-settings/aws_vsl_creds +++ readlink -f /alabama/cspi/test_settings/scripts/test_runner.sh ++ dirname /alabama/cspi/test_settings/scripts/test_runner.sh + readonly SCRIPT_DIR=/alabama/cspi/test_settings/scripts + SCRIPT_DIR=/alabama/cspi/test_settings/scripts ++ cd /alabama/cspi/test_settings/scripts ++ git rev-parse --show-toplevel + readonly TOP_DIR=/alabama/cspi + TOP_DIR=/alabama/cspi + echo /alabama/cspi /alabama/cspi + TESTS_FOLDER=/alabama/cspi/e2e/kubevirt-plugin ++ oc get nodes -o 'jsonpath={.items[*].metadata.labels.kubernetes\.io/arch}' ++ tr ' ' '\n' ++ sort -u ++ xargs + export NODES_ARCHITECTURE=amd64 + NODES_ARCHITECTURE=amd64 + export OADP_REPOSITORY=redhat + OADP_REPOSITORY=redhat + SKIP_DPA_CREATION=false ++ oc get ns openshift-storage ++ echo true + OPENSHIFT_STORAGE=true + '[' redhat == upstream-velero ']' + '[' true == true ']' ++ oc get sc ++ awk '$1 ~ /^.+ceph-rbd$/ {print $1}' ++ tail -1 + CEPH_RBD_STORAGE_CLASS=odf-operator-ceph-rbd + '[' -n odf-operator-ceph-rbd ']' + export CEPH_RBD_STORAGE_CLASS + echo 'ceph-rbd StorageClass found: odf-operator-ceph-rbd' ceph-rbd StorageClass found: odf-operator-ceph-rbd ++ oc get storageclass -o 'jsonpath={range .items[*]}{@.metadata.name}{" "}{@.metadata.annotations.storageclass\.kubernetes\.io/is-default-class}{"\n"}{end}' ++ awk '$2=="true"{print $1}' ++ wc -l + NUM_DEFAULT_STORAGE_CLASS=1 + '[' 1 -ne 1 ']' ++ oc get storageclass -o 'jsonpath={.items[?(@.metadata.annotations.storageclass\.kubernetes\.io/is-default-class=='\''true'\'')].metadata.name}' + DEFAULT_SC=odf-operator-ceph-rbd + export STORAGE_CLASS=odf-operator-ceph-rbd + STORAGE_CLASS=odf-operator-ceph-rbd + '[' -n odf-operator-ceph-rbd ']' + '[' odf-operator-ceph-rbd '!=' odf-operator-ceph-rbd ']' + export STORAGE_CLASS_OPENSHIFT_STORAGE=odf-operator-ceph-rbd + STORAGE_CLASS_OPENSHIFT_STORAGE=odf-operator-ceph-rbd + echo 'Using the StorageClass for openshift-storage: odf-operator-ceph-rbd' Using the StorageClass for openshift-storage: odf-operator-ceph-rbd + [[ amd64 != \a\m\d\6\4 ]] + TEST_FILTER='!// || (// && !exclude_aws && (!/target/ || target_aws) ) ' + [[ aws =~ ^(osp|vsphere|gcp-wif|ibmcloud|azure-sts|aro-4) ]] + [[ aws =~ ^azure_sak ]] ++ oc config current-context ++ awk -F / '{print $2}' + SETTINGS_TMP=/alabama/cspi/output_files/api-ci-op-98rbwkjv-2c502-cspilp-interop-ccitredhat-com:6443 + mkdir -p /alabama/cspi/output_files/api-ci-op-98rbwkjv-2c502-cspilp-interop-ccitredhat-com:6443 ++ oc get authentication cluster -o 'jsonpath={.spec.serviceAccountIssuer}' + IS_OIDC= + '[' '!' -z ']' + [[ aws == \a\w\s ]] + export PROVIDER=aws + PROVIDER=aws + export CREDS_SECRET_REF=cloud-credentials + CREDS_SECRET_REF=cloud-credentials ++ oc get infrastructures cluster -o 'jsonpath={.status.platformStatus.aws.region}' --allow-missing-template-keys=false + export REGION=us-east-2 + REGION=us-east-2 + settings_script=aws_settings.sh + '[' aws == aws-sts ']' + BUCKET=ci-op-98rbwkjv-interopoadp + TMP_DIR=/alabama/cspi/output_files/api-ci-op-98rbwkjv-2c502-cspilp-interop-ccitredhat-com:6443 + source /alabama/cspi/test_settings/scripts/aws_settings.sh ++ cat ++ [[ aws == *aws* ]] ++ cat ++ echo -e '\n }\n}' +++ cat /alabama/cspi/output_files/api-ci-op-98rbwkjv-2c502-cspilp-interop-ccitredhat-com:6443/settings.json ++ x='{ "metadata": { "namespace": "openshift-adp" }, "spec": { "configuration":{ "velero":{ "defaultPlugins": [ "openshift", "aws" ] } }, "backupLocations": [ { "velero": { "provider": "aws", "default": true, "config": { "region": "us-east-2" }, "credential":{ "name": "cloud-credentials", "key": "cloud" }, "objectStorage":{ "bucket": "ci-op-98rbwkjv-interopoadp" } } } ] , "snapshotLocations": [ { "velero": { "provider": "aws", "config": { "profile": "default", "region": "us-east-2" } } } ] } }' ++ echo '{ "metadata": { "namespace": "openshift-adp" }, "spec": { "configuration":{ "velero":{ "defaultPlugins": [ "openshift", "aws" ] } }, "backupLocations": [ { "velero": { "provider": "aws", "default": true, "config": { "region": "us-east-2" }, "credential":{ "name": "cloud-credentials", "key": "cloud" }, "objectStorage":{ "bucket": "ci-op-98rbwkjv-interopoadp" } } } ] , "snapshotLocations": [ { "velero": { "provider": "aws", "config": { "profile": "default", "region": "us-east-2" } } } ] } }' ++ grep -o '^[^#]*' + FILE_SETTINGS_NAME=settings.json + printf '\033[34mGenerated settings file under /alabama/cspi/output_files/api-ci-op-98rbwkjv-2c502-cspilp-interop-ccitredhat-com:6443/settings.json\e[39m\n' Generated settings file under /alabama/cspi/output_files/api-ci-op-98rbwkjv-2c502-cspilp-interop-ccitredhat-com:6443/settings.json + cat /alabama/cspi/output_files/api-ci-op-98rbwkjv-2c502-cspilp-interop-ccitredhat-com:6443/settings.json ++ oc get volumesnapshotclass -o name + for i in $(oc get volumesnapshotclass -o name) + oc annotate volumesnapshotclass.snapshot.storage.k8s.io/csi-aws-vsc snapshot.storage.kubernetes.io/is-default-class- volumesnapshotclass.snapshot.storage.k8s.io/csi-aws-vsc annotated + for i in $(oc get volumesnapshotclass -o name) + oc annotate volumesnapshotclass.snapshot.storage.k8s.io/odf-operator-cephfsplugin-snapclass snapshot.storage.kubernetes.io/is-default-class- volumesnapshotclass.snapshot.storage.k8s.io/odf-operator-cephfsplugin-snapclass annotated + for i in $(oc get volumesnapshotclass -o name) + oc annotate volumesnapshotclass.snapshot.storage.k8s.io/odf-operator-rbdplugin-snapclass snapshot.storage.kubernetes.io/is-default-class- volumesnapshotclass.snapshot.storage.k8s.io/odf-operator-rbdplugin-snapclass annotated ++ ./e2e/must-gather/get-latest-build.sh + oc get configmaps -n default must-gather-image + UPSTREAM_VERSION=99.0.0 ++ oc get OperatorCondition -n openshift-adp -o 'jsonpath={.items[*].metadata.name}' ++ awk -F v '{print $2}' + OADP_VERSION=1.5.3 + '[' -z 1.5.3 ']' + '[' 1.5.3 == 99.0.0 ']' ++ oc get sub redhat-oadp-operator -n openshift-adp -o 'jsonpath={.spec.source}' + OADP_REPO=redhat-operators + '[' -z redhat-operators ']' + '[' redhat-operators == redhat-operators ']' + REGISTRY_PATH=registry.redhat.io/oadp/oadp-mustgather-rhel9: + TAG=1.5.3 + export MUST_GATHER_BUILD=registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 + MUST_GATHER_BUILD=registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 + echo registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 + export MUST_GATHER_BUILD=registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 + MUST_GATHER_BUILD=registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 + '[' -z registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 ']' + export NUM_OF_OADP_INSTANCES=1 + NUM_OF_OADP_INSTANCES=1 ++ echo --skip=tc-id:OADP-555 ++ tr ' ' '\n' ++ grep '^--' ++ tr '\n' ' ' + GINKO_PARAM='--skip=tc-id:OADP-555 ' ++ echo --skip=tc-id:OADP-555 ++ tr ' ' '\n' ++ grep '^-' ++ grep -v '^--' ++ tr '\n' ' ' + TEST_PARAM= + ginkgo run --nodes=1 -mod=mod --show-node-events --flake-attempts 3 --junit-report=/logs/artifacts/junit_oadp_cnv_results.xml '--label-filter=!// || (// && !exclude_aws && (!/target/ || target_aws) ) ' --skip=tc-id:OADP-555 -p /alabama/cspi/e2e/kubevirt-plugin/ -- -credentials_file=/tmp/test-settings/credentials -vsl_credentials_file=/tmp/test-settings/aws_vsl_creds -oadp_namespace=openshift-adp -settings=/alabama/cspi/output_files/api-ci-op-98rbwkjv-2c502-cspilp-interop-ccitredhat-com:6443/settings.json -must_gather_image=registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 -timeout_multiplier=2 -skip_dpa_creation=false Ginkgo detected a version mismatch between the Ginkgo CLI and the version of Ginkgo imported by your packages: Ginkgo CLI Version: 2.27.5 Mismatched package versions found: 2.23.4 used by kubevirt-plugin Ginkgo will continue to attempt to run but you may see errors (including flag parsing errors) and should either update your go.mod or your version of the Ginkgo CLI to match. To install the matching version of the CLI run go install github.com/onsi/ginkgo/v2/ginkgo from a path that contains a go.mod file. Alternatively you can use go run github.com/onsi/ginkgo/v2/ginkgo from a path that contains a go.mod file to invoke the matching version of the Ginkgo CLI. If you are attempting to test multiple packages that each have a different version of the Ginkgo library with a single Ginkgo CLI that is currently unsupported.  2026/01/14 16:22:02 Setting up clients 2026/01/14 16:22:02 Getting default StorageClass... 2026/01/14 16:22:02 Checking default storage class count Run the command: oc get sc 2026/01/14 16:22:02 Got default StorageClass odf-operator-ceph-rbd 2026/01/14 16:22:02 oc get sc NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE gp2-csi ebs.csi.aws.com Delete WaitForFirstConsumer true 58m gp3-csi ebs.csi.aws.com Delete WaitForFirstConsumer true 58m odf-operator-ceph-rbd (default) openshift-storage.rbd.csi.ceph.com Delete Immediate true 6m29s odf-operator-ceph-rbd-virtualization openshift-storage.rbd.csi.ceph.com Delete Immediate true 6m29s odf-operator-cephfs openshift-storage.cephfs.csi.ceph.com Delete Immediate true 6m29s openshift-storage.noobaa.io openshift-storage.noobaa.io/obc Delete Immediate false 2m7s 2026/01/14 16:22:02 Using velero prefix: velero-e2e-kubevirt-28bae2e0-f165-11f0-b728-0a58ac1877e4 Running Suite: OADP E2E Virtualization Workloads Suite - /alabama/cspi/e2e/kubevirt-plugin ========================================================================================== Random Seed: 1768407675 Will run 4 of 5 specs ------------------------------ [BeforeSuite]  /alabama/cspi/e2e/kubevirt-plugin/kubevirt_suite_test.go:62 > Enter [BeforeSuite] TOP-LEVEL @ 01/14/26 16:22:02.582 < Exit [BeforeSuite] TOP-LEVEL @ 01/14/26 16:22:02.617 (35ms) [BeforeSuite] PASSED [0.035 seconds] ------------------------------ CSI: Backup/Restore Openshift Virtualization Workloads  [tc-id:OADP-185] [kubevirt] [Tier0] [vf:vm] Backing up started VM should succeed /alabama/cspi/e2e/kubevirt-plugin/backup_restore_csi.go:34 > Enter [BeforeEach] CSI: Backup/Restore Openshift Virtualization Workloads @ 01/14/26 16:22:02.618 < Exit [BeforeEach] CSI: Backup/Restore Openshift Virtualization Workloads @ 01/14/26 16:22:02.629 (11ms) > Enter [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:22:02.629 < Exit [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:22:02.629 (0s) > Enter [It] [tc-id:OADP-185] [kubevirt] [Tier0] [vf:vm] Backing up started VM should succeed @ 01/14/26 16:22:02.629 2026/01/14 16:22:02 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/14/26 16:22:02.633 2026/01/14 16:22:02 csi 2026/01/14 16:22:02 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "47bb13a7-70c5-4d59-80a7-4d52e5449271", "resourceVersion": "68137", "generation": 1, "creationTimestamp": "2026-01-14T16:22:02Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:22:02Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/14/26 16:22:02.67 2026/01/14 16:22:02 Waiting for velero pod to be running 2026/01/14 16:22:07 pod: velero-5944c96584-q2z6r is not yet running with status: {1 Pending [{PodReadyToStartContainers 1 False 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:22:02 +0000 UTC } {Initialized 1 False 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:22:02 +0000 UTC ContainersNotInitialized containers with incomplete status: [openshift-velero-plugin velero-plugin-for-aws kubevirt-velero-plugin hypershift-oadp-plugin]} {Ready 1 False 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:22:02 +0000 UTC ContainersNotReady containers with unready status: [velero]} {ContainersReady 1 False 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:22:02 +0000 UTC ContainersNotReady containers with unready status: [velero]} {PodScheduled 1 True 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:22:02 +0000 UTC }] 10.0.52.28 [{10.0.52.28}] [] 2026-01-14 16:22:02 +0000 UTC [{openshift-velero-plugin {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:9b888c75a1ddf274b1f0aa053ddbbc9acc3270417712456ffa29ef81a883b38a 0xc0009cf818 map[] nil [{plugins /target false } {kube-api-access-62k7g /var/run/secrets/kubernetes.io/serviceaccount true 0xc000dc9ea0}] nil [] } {velero-plugin-for-aws {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:2a099649d3d7101a0b6dff9f6372bc76903ba6ecb6578dbeef8a8b3853f48026 0xc0009cf85f map[] nil [{plugins /target false } {kube-api-access-62k7g /var/run/secrets/kubernetes.io/serviceaccount true 0xc000dc9eb0}] nil [] } {kubevirt-velero-plugin {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:77d6057b1783872ec65d7d6a88d8a1212c38aacc982c7efd46ed2eb77b5e9e70 0xc0009cf878 map[] nil [{plugins /target false } {kube-api-access-62k7g /var/run/secrets/kubernetes.io/serviceaccount true 0xc000dc9ec0}] nil [] } {hypershift-oadp-plugin {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-hypershift-velero-plugin-rhel9@sha256:f8798364f94bb1bf4db264b0439c5a280b40cafa408c47b4160d9cc9c5d87e53 0xc0009cf8af map[] nil [{plugins /target false } {kube-api-access-62k7g /var/run/secrets/kubernetes.io/serviceaccount true 0xc000dc9ed0}] nil [] }] [{velero {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-rhel9@sha256:dd5897b1579599cfa8d8a03a1c8d3cf68e3927c4ba5dee4e33c2a2d4c9e6ed89 0xc0009cf7bf map[] nil [{plugins /plugins false } {scratch /scratch false } {certs /etc/ssl/certs false } {bound-sa-token /var/run/secrets/openshift/serviceaccount true 0xc000dc9e80} {kube-api-access-62k7g /var/run/secrets/kubernetes.io/serviceaccount true 0xc000dc9e90}] nil [] }] Burstable [] []} 2026/01/14 16:22:12 pod: velero-5944c96584-q2z6r is not yet running with status: {1 Pending [{PodReadyToStartContainers 1 True 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:22:09 +0000 UTC } {Initialized 1 False 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:22:02 +0000 UTC ContainersNotInitialized containers with incomplete status: [kubevirt-velero-plugin hypershift-oadp-plugin]} {Ready 1 False 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:22:02 +0000 UTC ContainersNotReady containers with unready status: [velero]} {ContainersReady 1 False 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:22:02 +0000 UTC ContainersNotReady containers with unready status: [velero]} {PodScheduled 1 True 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:22:02 +0000 UTC }] 10.0.52.28 [{10.0.52.28}] 10.128.2.56 [{10.128.2.56}] 2026-01-14 16:22:02 +0000 UTC [{openshift-velero-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2026-01-14 16:22:08 +0000 UTC,FinishedAt:2026-01-14 16:22:08 +0000 UTC,ContainerID:cri-o://b64bfc0a1e450fbb084ba1ab21c6575f50fd103f44e6cf2691476520a109abbe,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:9b888c75a1ddf274b1f0aa053ddbbc9acc3270417712456ffa29ef81a883b38a registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:212589b9f5b75a043c9a7094cb681a39687795c5e78e625beb6d00960ea57e70 cri-o://b64bfc0a1e450fbb084ba1ab21c6575f50fd103f44e6cf2691476520a109abbe 0xc000b26048 map[cpu:{{500 -3} {} 500m DecimalSI} memory:{{134217728 0} {} BinarySI}] &ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{134217728 0} {} BinarySI},},Claims:[]ResourceClaim{},} [{plugins /target false } {kube-api-access-62k7g /var/run/secrets/kubernetes.io/serviceaccount true 0xc000ae6b00}] &ContainerUser{Linux:&LinuxContainerUser{UID:1000740000,GID:0,SupplementalGroups:[0 1000740000],},} [] } {velero-plugin-for-aws {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2026-01-14 16:22:11 +0000 UTC,FinishedAt:2026-01-14 16:22:11 +0000 UTC,ContainerID:cri-o://88ace4b461f21bdc77ee199b2079aca07f8fe67467ccca2f1cdf06755e24c61d,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:2a099649d3d7101a0b6dff9f6372bc76903ba6ecb6578dbeef8a8b3853f48026 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:2a099649d3d7101a0b6dff9f6372bc76903ba6ecb6578dbeef8a8b3853f48026 cri-o://88ace4b461f21bdc77ee199b2079aca07f8fe67467ccca2f1cdf06755e24c61d 0xc000b260c9 map[cpu:{{500 -3} {} 500m DecimalSI} memory:{{134217728 0} {} BinarySI}] &ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{134217728 0} {} BinarySI},},Claims:[]ResourceClaim{},} [{plugins /target false } {kube-api-access-62k7g /var/run/secrets/kubernetes.io/serviceaccount true 0xc000ae6b30}] &ContainerUser{Linux:&LinuxContainerUser{UID:1000740000,GID:0,SupplementalGroups:[0 1000740000],},} [] } {kubevirt-velero-plugin {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:77d6057b1783872ec65d7d6a88d8a1212c38aacc982c7efd46ed2eb77b5e9e70 0xc000b26118 map[] nil [{plugins /target false } {kube-api-access-62k7g /var/run/secrets/kubernetes.io/serviceaccount true 0xc000ae6b40}] nil [] } {hypershift-oadp-plugin {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-hypershift-velero-plugin-rhel9@sha256:f8798364f94bb1bf4db264b0439c5a280b40cafa408c47b4160d9cc9c5d87e53 0xc000b2615f map[] nil [{plugins /target false } {kube-api-access-62k7g /var/run/secrets/kubernetes.io/serviceaccount true 0xc000ae6b50}] nil [] }] [{velero {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-rhel9@sha256:dd5897b1579599cfa8d8a03a1c8d3cf68e3927c4ba5dee4e33c2a2d4c9e6ed89 0xc0009cffef map[] nil [{plugins /plugins false } {scratch /scratch false } {certs /etc/ssl/certs false } {bound-sa-token /var/run/secrets/openshift/serviceaccount true 0xc000ae6ac0} {kube-api-access-62k7g /var/run/secrets/kubernetes.io/serviceaccount true 0xc000ae6ad0}] nil [] }] Burstable [] []} 2026/01/14 16:22:17 pod: velero-5944c96584-q2z6r is not yet running with status: {1 Pending [{PodReadyToStartContainers 1 True 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:22:09 +0000 UTC } {Initialized 1 False 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:22:02 +0000 UTC ContainersNotInitialized containers with incomplete status: [kubevirt-velero-plugin hypershift-oadp-plugin]} {Ready 1 False 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:22:02 +0000 UTC ContainersNotReady containers with unready status: [velero]} {ContainersReady 1 False 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:22:02 +0000 UTC ContainersNotReady containers with unready status: [velero]} {PodScheduled 1 True 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:22:02 +0000 UTC }] 10.0.52.28 [{10.0.52.28}] 10.128.2.56 [{10.128.2.56}] 2026-01-14 16:22:02 +0000 UTC [{openshift-velero-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2026-01-14 16:22:08 +0000 UTC,FinishedAt:2026-01-14 16:22:08 +0000 UTC,ContainerID:cri-o://b64bfc0a1e450fbb084ba1ab21c6575f50fd103f44e6cf2691476520a109abbe,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:9b888c75a1ddf274b1f0aa053ddbbc9acc3270417712456ffa29ef81a883b38a registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:212589b9f5b75a043c9a7094cb681a39687795c5e78e625beb6d00960ea57e70 cri-o://b64bfc0a1e450fbb084ba1ab21c6575f50fd103f44e6cf2691476520a109abbe 0xc00094e008 map[cpu:{{500 -3} {} 500m DecimalSI} memory:{{134217728 0} {} BinarySI}] &ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{134217728 0} {} BinarySI},},Claims:[]ResourceClaim{},} [{plugins /target false } {kube-api-access-62k7g /var/run/secrets/kubernetes.io/serviceaccount true 0xc000d05050}] &ContainerUser{Linux:&LinuxContainerUser{UID:1000740000,GID:0,SupplementalGroups:[0 1000740000],},} [] } {velero-plugin-for-aws {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2026-01-14 16:22:11 +0000 UTC,FinishedAt:2026-01-14 16:22:11 +0000 UTC,ContainerID:cri-o://88ace4b461f21bdc77ee199b2079aca07f8fe67467ccca2f1cdf06755e24c61d,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:2a099649d3d7101a0b6dff9f6372bc76903ba6ecb6578dbeef8a8b3853f48026 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:2a099649d3d7101a0b6dff9f6372bc76903ba6ecb6578dbeef8a8b3853f48026 cri-o://88ace4b461f21bdc77ee199b2079aca07f8fe67467ccca2f1cdf06755e24c61d 0xc00094e089 map[cpu:{{500 -3} {} 500m DecimalSI} memory:{{134217728 0} {} BinarySI}] &ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{134217728 0} {} BinarySI},},Claims:[]ResourceClaim{},} [{plugins /target false } {kube-api-access-62k7g /var/run/secrets/kubernetes.io/serviceaccount true 0xc000d05060}] &ContainerUser{Linux:&LinuxContainerUser{UID:1000740000,GID:0,SupplementalGroups:[0 1000740000],},} [] } {kubevirt-velero-plugin {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:77d6057b1783872ec65d7d6a88d8a1212c38aacc982c7efd46ed2eb77b5e9e70 0xc00094e0d8 map[] nil [{plugins /target false } {kube-api-access-62k7g /var/run/secrets/kubernetes.io/serviceaccount true 0xc000d05070}] nil [] } {hypershift-oadp-plugin {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-hypershift-velero-plugin-rhel9@sha256:f8798364f94bb1bf4db264b0439c5a280b40cafa408c47b4160d9cc9c5d87e53 0xc00094e11f map[] nil [{plugins /target false } {kube-api-access-62k7g /var/run/secrets/kubernetes.io/serviceaccount true 0xc000d05080}] nil [] }] [{velero {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-rhel9@sha256:dd5897b1579599cfa8d8a03a1c8d3cf68e3927c4ba5dee4e33c2a2d4c9e6ed89 0xc000013f6f map[] nil [{plugins /plugins false } {scratch /scratch false } {certs /etc/ssl/certs false } {bound-sa-token /var/run/secrets/openshift/serviceaccount true 0xc000d05030} {kube-api-access-62k7g /var/run/secrets/kubernetes.io/serviceaccount true 0xc000d05040}] nil [] }] Burstable [] []} 2026/01/14 16:22:22 pod: velero-5944c96584-q2z6r is not yet running with status: {1 Pending [{PodReadyToStartContainers 1 True 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:22:09 +0000 UTC } {Initialized 1 False 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:22:02 +0000 UTC ContainersNotInitialized containers with incomplete status: [hypershift-oadp-plugin]} {Ready 1 False 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:22:02 +0000 UTC ContainersNotReady containers with unready status: [velero]} {ContainersReady 1 False 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:22:02 +0000 UTC ContainersNotReady containers with unready status: [velero]} {PodScheduled 1 True 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:22:02 +0000 UTC }] 10.0.52.28 [{10.0.52.28}] 10.128.2.56 [{10.128.2.56}] 2026-01-14 16:22:02 +0000 UTC [{openshift-velero-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2026-01-14 16:22:08 +0000 UTC,FinishedAt:2026-01-14 16:22:08 +0000 UTC,ContainerID:cri-o://b64bfc0a1e450fbb084ba1ab21c6575f50fd103f44e6cf2691476520a109abbe,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:9b888c75a1ddf274b1f0aa053ddbbc9acc3270417712456ffa29ef81a883b38a registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:212589b9f5b75a043c9a7094cb681a39687795c5e78e625beb6d00960ea57e70 cri-o://b64bfc0a1e450fbb084ba1ab21c6575f50fd103f44e6cf2691476520a109abbe 0xc000b26a48 map[cpu:{{500 -3} {} 500m DecimalSI} memory:{{134217728 0} {} BinarySI}] &ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{134217728 0} {} BinarySI},},Claims:[]ResourceClaim{},} [{plugins /target false } {kube-api-access-62k7g /var/run/secrets/kubernetes.io/serviceaccount true 0xc000ae74c0}] &ContainerUser{Linux:&LinuxContainerUser{UID:1000740000,GID:0,SupplementalGroups:[0 1000740000],},} [] } {velero-plugin-for-aws {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2026-01-14 16:22:11 +0000 UTC,FinishedAt:2026-01-14 16:22:11 +0000 UTC,ContainerID:cri-o://88ace4b461f21bdc77ee199b2079aca07f8fe67467ccca2f1cdf06755e24c61d,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:2a099649d3d7101a0b6dff9f6372bc76903ba6ecb6578dbeef8a8b3853f48026 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:2a099649d3d7101a0b6dff9f6372bc76903ba6ecb6578dbeef8a8b3853f48026 cri-o://88ace4b461f21bdc77ee199b2079aca07f8fe67467ccca2f1cdf06755e24c61d 0xc000b26ac9 map[cpu:{{500 -3} {} 500m DecimalSI} memory:{{134217728 0} {} BinarySI}] &ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{134217728 0} {} BinarySI},},Claims:[]ResourceClaim{},} [{plugins /target false } {kube-api-access-62k7g /var/run/secrets/kubernetes.io/serviceaccount true 0xc000ae74d0}] &ContainerUser{Linux:&LinuxContainerUser{UID:1000740000,GID:0,SupplementalGroups:[0 1000740000],},} [] } {kubevirt-velero-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2026-01-14 16:22:17 +0000 UTC,FinishedAt:2026-01-14 16:22:17 +0000 UTC,ContainerID:cri-o://948af1259bb15930761f9a71aacaf0f74976cf5e495c7b34d67b601f67799d25,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:77d6057b1783872ec65d7d6a88d8a1212c38aacc982c7efd46ed2eb77b5e9e70 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:684020e409da37fd3a41935b51ac22e506fb7a45e787ae8c0cfd6ac0dfcbff09 cri-o://948af1259bb15930761f9a71aacaf0f74976cf5e495c7b34d67b601f67799d25 0xc000b26b18 map[cpu:{{500 -3} {} 500m DecimalSI} memory:{{134217728 0} {} BinarySI}] &ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{134217728 0} {} BinarySI},},Claims:[]ResourceClaim{},} [{plugins /target false } {kube-api-access-62k7g /var/run/secrets/kubernetes.io/serviceaccount true 0xc000ae74e0}] &ContainerUser{Linux:&LinuxContainerUser{UID:1000740000,GID:0,SupplementalGroups:[0 1000740000],},} [] } {hypershift-oadp-plugin {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-hypershift-velero-plugin-rhel9@sha256:f8798364f94bb1bf4db264b0439c5a280b40cafa408c47b4160d9cc9c5d87e53 0xc000b26b9f map[] nil [{plugins /target false } {kube-api-access-62k7g /var/run/secrets/kubernetes.io/serviceaccount true 0xc000ae74f0}] nil [] }] [{velero {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-rhel9@sha256:dd5897b1579599cfa8d8a03a1c8d3cf68e3927c4ba5dee4e33c2a2d4c9e6ed89 0xc000b269ef map[] nil [{plugins /plugins false } {scratch /scratch false } {certs /etc/ssl/certs false } {bound-sa-token /var/run/secrets/openshift/serviceaccount true 0xc000ae74a0} {kube-api-access-62k7g /var/run/secrets/kubernetes.io/serviceaccount true 0xc000ae74b0}] nil [] }] Burstable [] []} 2026/01/14 16:22:27 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2026/01/14 16:22:27 DPA successfully reconciled STEP: Prepare backup resources, depending on the volumes backup type @ 01/14/26 16:22:27.813 Run the command: oc get ns openshift-storage &> /dev/null && echo true || echo false 2026/01/14 16:22:27 The 'openshift-storage' namespace exists 2026/01/14 16:22:27 Checking default storage class count 2026/01/14 16:22:27 Using the CSI driver: openshift-storage.rbd.csi.ceph.com 2026/01/14 16:22:27 Snapclass 'example-snapclass' doesn't exist, creating 2026/01/14 16:22:28 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:22:28 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd STEP: Installing application for case ocp-kubevirt @ 01/14/26 16:22:28.06 2026/01/14 16:22:28 🔄 [Attempt 1/3] Executing operation: Deploy application ocp-kubevirt 2026/01/14 16:22:28 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: WorkerProcess for [localhost/TASK: Extract Kubernetes minor version from cluster] errantly sent data directly to stderr instead of using Display:  /usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py:1013: InsecureRequestWarning: Unverified HTTPS request is being made to host 'api.ci-op-98rbwkjv-2c502.cspilp.interop.ccitredhat.com'. Adding certificate verification is strongly advised. Se  [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:22:31 2026-01-14 16:22:29,842 p=20155 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:22:29,842 p=20155 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:22:30,177 p=20155 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:22:30,177 p=20155 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:22:30,521 p=20155 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:22:30,521 p=20155 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:22:30,864 p=20155 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:22:30,864 p=20155 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:22:30,883 p=20155 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:22:30,883 p=20155 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:22:30,883 p=20155 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:22:30,904 p=20155 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:22:30,904 p=20155 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:22:30,920 p=20155 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:22:30,920 p=20155 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:22:31,393 p=20155 u=1004100000 n=ansible WARNING| [WARNING]: WorkerProcess for [localhost/TASK: Extract Kubernetes minor version from cluster] errantly sent data directly to stderr instead of using Display: /usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py:1013: InsecureRequestWarning: Unverified HTTPS request is being made to host 'api.ci-op-98rbwkjv-2c502.cspilp.interop.ccitredhat.com'. Adding certificate verification is strongly advised. Se 2026-01-14 16:22:31,395 p=20155 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:22:31,395 p=20155 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:22:31,395 p=20155 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:22:31,395 p=20155 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:22:31,418 p=20155 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:22:31,419 p=20155 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:22:31,419 p=20155 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:22:31,419 p=20155 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:22:31,419 p=20155 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-185 2026/01/14 16:22:31 2026/01/14 16:22:31 Found server error pattern in ansible logs: '502' 2026/01/14 16:22:31 Detected server errors in ansible logs - treating as retryable 2026/01/14 16:22:31 ⚠️ Operation 'Deploy application ocp-kubevirt' failed with retryable server error (attempt 1/3): Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-185","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt","user_kubeconfig":"/home/jenkins/.kube/config","vm_running":true,"with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 2026/01/14 16:22:31 🕐 Waiting 1m0s before retry... 2026/01/14 16:23:31 🔄 [Attempt 2/3] Executing operation: Deploy application ocp-kubevirt 2026/01/14 16:23:31 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:23:34 2026-01-14 16:23:33,318 p=20335 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:23:33,318 p=20335 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:23:33,646 p=20335 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:23:33,647 p=20335 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:23:33,972 p=20335 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:23:33,973 p=20335 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:23:34,296 p=20335 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:23:34,296 p=20335 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:23:34,315 p=20335 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:23:34,315 p=20335 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:23:34,315 p=20335 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:23:34,335 p=20335 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:23:34,335 p=20335 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:23:34,351 p=20335 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:23:34,351 p=20335 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:23:34,748 p=20335 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:23:34,748 p=20335 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:23:34,748 p=20335 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:23:34,748 p=20335 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:23:34,770 p=20335 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:23:34,771 p=20335 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:23:34,771 p=20335 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:23:34,772 p=20335 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:23:34,772 p=20335 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-185 2026/01/14 16:23:34 2026/01/14 16:23:34 ❌ Operation 'Deploy application ocp-kubevirt' failed with non-retryable error: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-185","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt","user_kubeconfig":"/home/jenkins/.kube/config","vm_running":true,"with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 [FAILED] in [It] - /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:23:34.975 < Exit [It] [tc-id:OADP-185] [kubevirt] [Tier0] [vf:vm] Backing up started VM should succeed @ 01/14/26 16:23:34.975 (1m32.346s) > Enter [JustAfterEach] TOP-LEVEL @ 01/14/26 16:23:34.975 2026/01/14 16:23:34 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 STEP: Get the failed spec name @ 01/14/26 16:23:34.975 2026/01/14 16:23:34 The failed spec name is: CSI: Backup/Restore Openshift Virtualization Workloads [tc-id:OADP-185] [kubevirt] [Tier0] [vf:vm] Backing up started VM should succeed STEP: Create a folder for all must-gather files if it doesn't exists already @ 01/14/26 16:23:34.975 2026/01/14 16:23:34 The folder logs does not exists, creating new folder with the name: logs STEP: Create a folder for the failed spec if it doesn't exists already @ 01/14/26 16:23:34.975 2026/01/14 16:23:34 The folder logs/It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-185_kubevirt_Tier0_vf_vm_Backing_up_started_VM_should_succeed does not exists, creating new folder with the name: logs/It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-185_kubevirt_Tier0_vf_vm_Backing_up_started_VM_should_succeed STEP: Run must-gather because the spec failed @ 01/14/26 16:23:34.975 2026/01/14 16:23:34 Log the present working directory path:- /alabama/cspi/e2e/kubevirt-plugin 2026/01/14 16:23:34 [adm must-gather --dest-dir /alabama/cspi/e2e/kubevirt-plugin/logs/It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-185_kubevirt_Tier0_vf_vm_Backing_up_started_VM_should_succeed --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3] 2026/01/14 16:23:59 Log all the files present in /alabama/cspi/e2e/kubevirt-plugin/logs directory 2026/01/14 16:23:59 It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-185_kubevirt_Tier0_vf_vm_Backing_up_started_VM_should_succeed STEP: Find must-gather folder and rename it to a shorter more readable name @ 01/14/26 16:23:59.736 < Exit [JustAfterEach] TOP-LEVEL @ 01/14/26 16:23:59.737 (24.762s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:23:59.737 2026/01/14 16:23:59 Cleaning app 2026/01/14 16:23:59 Using admin kubeconfig for with_cleanup operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:24:02 2026-01-14 16:24:01,402 p=20558 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:24:01,402 p=20558 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:24:01,723 p=20558 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:24:01,723 p=20558 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:24:02,045 p=20558 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:24:02,045 p=20558 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:24:02,373 p=20558 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:24:02,373 p=20558 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:24:02,392 p=20558 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:24:02,393 p=20558 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:24:02,393 p=20558 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:24:02,413 p=20558 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:24:02,413 p=20558 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:24:02,427 p=20558 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:24:02,428 p=20558 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:24:02,826 p=20558 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:24:02,826 p=20558 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:24:02,826 p=20558 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:24:02,826 p=20558 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:24:02,849 p=20558 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:24:02,849 p=20558 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:24:02,849 p=20558 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:24:02,850 p=20558 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:24:02,850 p=20558 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-185 2026/01/14 16:24:03 < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:24:03.016 (3.28s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:24:03.016 2026/01/14 16:24:03 Cleaning setup resources for the backup 2026/01/14 16:24:03 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:24:03 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2026/01/14 16:24:03 Deleting VolumeSnapshotClass 'example-snapclass' < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:24:03.059 (43ms) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:24:03.059 2026/01/14 16:24:03 Deleting DPA CR < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:24:03.069 (10ms) Attempt #1 Failed. Retrying ↺ @ 01/14/26 16:24:03.069 > Enter [BeforeEach] CSI: Backup/Restore Openshift Virtualization Workloads @ 01/14/26 16:24:03.069 < Exit [BeforeEach] CSI: Backup/Restore Openshift Virtualization Workloads @ 01/14/26 16:24:03.094 (25ms) > Enter [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:24:03.094 < Exit [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:24:03.094 (0s) > Enter [It] [tc-id:OADP-185] [kubevirt] [Tier0] [vf:vm] Backing up started VM should succeed @ 01/14/26 16:24:03.094 2026/01/14 16:24:03 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/14/26 16:24:03.107 2026/01/14 16:24:03 csi 2026/01/14 16:24:03 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "1581b91c-b104-46fe-b064-f332463727d8", "resourceVersion": "70453", "generation": 1, "creationTimestamp": "2026-01-14T16:24:03Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:24:03Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/14/26 16:24:03.214 2026/01/14 16:24:03 Waiting for velero pod to be running 2026/01/14 16:24:03 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2026/01/14 16:24:03 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "1581b91c-b104-46fe-b064-f332463727d8", "resourceVersion": "70453", "generation": 1, "creationTimestamp": "2026-01-14T16:24:03Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:24:03Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } 2026/01/14 16:24:08 DPA successfully reconciled STEP: Prepare backup resources, depending on the volumes backup type @ 01/14/26 16:24:08.256 2026/01/14 16:24:08 Snapclass 'example-snapclass' doesn't exist, creating 2026/01/14 16:24:08 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:24:08 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd STEP: Installing application for case ocp-kubevirt @ 01/14/26 16:24:08.303 2026/01/14 16:24:08 🔄 [Attempt 1/3] Executing operation: Deploy application ocp-kubevirt 2026/01/14 16:24:08 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:24:11 2026-01-14 16:24:09,973 p=20739 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:24:09,973 p=20739 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:24:10,295 p=20739 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:24:10,295 p=20739 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:24:10,616 p=20739 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:24:10,616 p=20739 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:24:10,941 p=20739 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:24:10,941 p=20739 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:24:10,959 p=20739 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:24:10,960 p=20739 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:24:10,960 p=20739 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:24:10,979 p=20739 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:24:10,980 p=20739 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:24:10,995 p=20739 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:24:10,995 p=20739 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:24:11,397 p=20739 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:24:11,398 p=20739 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:24:11,398 p=20739 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:24:11,398 p=20739 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:24:11,422 p=20739 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:24:11,422 p=20739 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:24:11,423 p=20739 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:24:11,423 p=20739 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:24:11,423 p=20739 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-185 2026/01/14 16:24:11 2026/01/14 16:24:11 ❌ Operation 'Deploy application ocp-kubevirt' failed with non-retryable error: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-185","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt","user_kubeconfig":"/home/jenkins/.kube/config","vm_running":true,"with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 [FAILED] in [It] - /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:24:11.599 < Exit [It] [tc-id:OADP-185] [kubevirt] [Tier0] [vf:vm] Backing up started VM should succeed @ 01/14/26 16:24:11.599 (8.505s) > Enter [JustAfterEach] TOP-LEVEL @ 01/14/26 16:24:11.599 2026/01/14 16:24:11 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 STEP: Get the failed spec name @ 01/14/26 16:24:11.599 2026/01/14 16:24:11 The failed spec name is: CSI: Backup/Restore Openshift Virtualization Workloads [tc-id:OADP-185] [kubevirt] [Tier0] [vf:vm] Backing up started VM should succeed STEP: Create a folder for all must-gather files if it doesn't exists already @ 01/14/26 16:24:11.599 STEP: Create a folder for the failed spec if it doesn't exists already @ 01/14/26 16:24:11.599 STEP: Run must-gather because the spec failed @ 01/14/26 16:24:11.599 2026/01/14 16:24:11 Log the present working directory path:- /alabama/cspi/e2e/kubevirt-plugin 2026/01/14 16:24:11 [adm must-gather --dest-dir /alabama/cspi/e2e/kubevirt-plugin/logs/It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-185_kubevirt_Tier0_vf_vm_Backing_up_started_VM_should_succeed --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3] 2026/01/14 16:24:22 Log all the files present in /alabama/cspi/e2e/kubevirt-plugin/logs directory 2026/01/14 16:24:22 It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-185_kubevirt_Tier0_vf_vm_Backing_up_started_VM_should_succeed STEP: Find must-gather folder and rename it to a shorter more readable name @ 01/14/26 16:24:22.343 The folder logs/It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-185_kubevirt_Tier0_vf_vm_Backing_up_started_VM_should_succeed/must-gather already exists, skipping renaming the folder < Exit [JustAfterEach] TOP-LEVEL @ 01/14/26 16:24:22.343 (10.744s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:24:22.344 2026/01/14 16:24:22 Cleaning app 2026/01/14 16:24:22 Using admin kubeconfig for with_cleanup operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:24:25 2026-01-14 16:24:24,015 p=20961 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:24:24,015 p=20961 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:24:24,332 p=20961 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:24:24,332 p=20961 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:24:24,661 p=20961 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:24:24,661 p=20961 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:24:24,979 p=20961 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:24:24,980 p=20961 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:24:24,998 p=20961 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:24:24,998 p=20961 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:24:24,998 p=20961 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:24:25,020 p=20961 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:24:25,021 p=20961 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:24:25,035 p=20961 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:24:25,036 p=20961 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:24:25,431 p=20961 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:24:25,432 p=20961 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:24:25,432 p=20961 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:24:25,432 p=20961 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:24:25,454 p=20961 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:24:25,455 p=20961 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:24:25,455 p=20961 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:24:25,456 p=20961 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:24:25,456 p=20961 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-185 2026/01/14 16:24:25 < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:24:25.636 (3.293s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:24:25.636 2026/01/14 16:24:25 Cleaning setup resources for the backup 2026/01/14 16:24:25 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:24:25 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2026/01/14 16:24:25 Deleting VolumeSnapshotClass 'example-snapclass' < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:24:25.66 (24ms) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:24:25.661 2026/01/14 16:24:25 Deleting DPA CR < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:24:25.687 (27ms) Attempt #2 Failed. Retrying ↺ @ 01/14/26 16:24:25.687 > Enter [BeforeEach] CSI: Backup/Restore Openshift Virtualization Workloads @ 01/14/26 16:24:25.687 < Exit [BeforeEach] CSI: Backup/Restore Openshift Virtualization Workloads @ 01/14/26 16:24:25.699 (11ms) > Enter [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:24:25.699 < Exit [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:24:25.699 (0s) > Enter [It] [tc-id:OADP-185] [kubevirt] [Tier0] [vf:vm] Backing up started VM should succeed @ 01/14/26 16:24:25.699 2026/01/14 16:24:25 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/14/26 16:24:25.705 2026/01/14 16:24:25 csi 2026/01/14 16:24:25 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "1cc7ae11-6fdc-4a4d-870c-558ff37f1673", "resourceVersion": "70968", "generation": 1, "creationTimestamp": "2026-01-14T16:24:25Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:24:25Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/14/26 16:24:25.784 2026/01/14 16:24:25 Waiting for velero pod to be running 2026/01/14 16:24:25 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2026/01/14 16:24:25 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "1cc7ae11-6fdc-4a4d-870c-558ff37f1673", "resourceVersion": "70968", "generation": 1, "creationTimestamp": "2026-01-14T16:24:25Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:24:25Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } 2026/01/14 16:24:30 DPA successfully reconciled STEP: Prepare backup resources, depending on the volumes backup type @ 01/14/26 16:24:30.819 2026/01/14 16:24:30 Snapclass 'example-snapclass' doesn't exist, creating 2026/01/14 16:24:30 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:24:30 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd STEP: Installing application for case ocp-kubevirt @ 01/14/26 16:24:30.952 2026/01/14 16:24:30 🔄 [Attempt 1/3] Executing operation: Deploy application ocp-kubevirt 2026/01/14 16:24:30 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:24:34 2026-01-14 16:24:32,645 p=21140 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:24:32,646 p=21140 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:24:32,967 p=21140 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:24:32,967 p=21140 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:24:33,289 p=21140 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:24:33,289 p=21140 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:24:33,611 p=21140 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:24:33,611 p=21140 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:24:33,629 p=21140 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:24:33,630 p=21140 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:24:33,630 p=21140 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:24:33,649 p=21140 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:24:33,649 p=21140 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:24:33,664 p=21140 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:24:33,665 p=21140 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:24:34,056 p=21140 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:24:34,056 p=21140 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:24:34,057 p=21140 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:24:34,057 p=21140 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:24:34,079 p=21140 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:24:34,080 p=21140 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:24:34,080 p=21140 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:24:34,081 p=21140 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:24:34,081 p=21140 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-185 2026/01/14 16:24:34 2026/01/14 16:24:34 ❌ Operation 'Deploy application ocp-kubevirt' failed with non-retryable error: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-185","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt","user_kubeconfig":"/home/jenkins/.kube/config","vm_running":true,"with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 [FAILED] in [It] - /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:24:34.244 < Exit [It] [tc-id:OADP-185] [kubevirt] [Tier0] [vf:vm] Backing up started VM should succeed @ 01/14/26 16:24:34.244 (8.546s) > Enter [JustAfterEach] TOP-LEVEL @ 01/14/26 16:24:34.244 2026/01/14 16:24:34 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 STEP: Get the failed spec name @ 01/14/26 16:24:34.244 2026/01/14 16:24:34 The failed spec name is: CSI: Backup/Restore Openshift Virtualization Workloads [tc-id:OADP-185] [kubevirt] [Tier0] [vf:vm] Backing up started VM should succeed STEP: Create a folder for all must-gather files if it doesn't exists already @ 01/14/26 16:24:34.244 STEP: Create a folder for the failed spec if it doesn't exists already @ 01/14/26 16:24:34.244 STEP: Run must-gather because the spec failed @ 01/14/26 16:24:34.244 2026/01/14 16:24:34 Log the present working directory path:- /alabama/cspi/e2e/kubevirt-plugin 2026/01/14 16:24:34 [adm must-gather --dest-dir /alabama/cspi/e2e/kubevirt-plugin/logs/It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-185_kubevirt_Tier0_vf_vm_Backing_up_started_VM_should_succeed --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3] 2026/01/14 16:24:45 Log all the files present in /alabama/cspi/e2e/kubevirt-plugin/logs directory 2026/01/14 16:24:45 It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-185_kubevirt_Tier0_vf_vm_Backing_up_started_VM_should_succeed STEP: Find must-gather folder and rename it to a shorter more readable name @ 01/14/26 16:24:45.008 The folder logs/It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-185_kubevirt_Tier0_vf_vm_Backing_up_started_VM_should_succeed/must-gather already exists, skipping renaming the folder < Exit [JustAfterEach] TOP-LEVEL @ 01/14/26 16:24:45.008 (10.764s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:24:45.008 2026/01/14 16:24:45 Cleaning app 2026/01/14 16:24:45 Using admin kubeconfig for with_cleanup operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:24:48 2026-01-14 16:24:46,685 p=21363 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:24:46,685 p=21363 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:24:46,996 p=21363 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:24:46,996 p=21363 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:24:47,315 p=21363 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:24:47,315 p=21363 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:24:47,635 p=21363 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:24:47,636 p=21363 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:24:47,654 p=21363 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:24:47,654 p=21363 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:24:47,654 p=21363 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:24:47,675 p=21363 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:24:47,675 p=21363 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:24:47,689 p=21363 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:24:47,690 p=21363 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:24:48,085 p=21363 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:24:48,086 p=21363 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:24:48,086 p=21363 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:24:48,086 p=21363 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:24:48,108 p=21363 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:24:48,109 p=21363 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:24:48,109 p=21363 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:24:48,110 p=21363 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:24:48,110 p=21363 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-185 2026/01/14 16:24:48 < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:24:48.276 (3.268s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:24:48.276 2026/01/14 16:24:48 Cleaning setup resources for the backup 2026/01/14 16:24:48 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:24:48 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2026/01/14 16:24:48 Deleting VolumeSnapshotClass 'example-snapclass' < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:24:48.401 (124ms) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:24:48.401 2026/01/14 16:24:48 Deleting DPA CR < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:24:48.412 (11ms) • [FAILED] [165.794 seconds] CSI: Backup/Restore Openshift Virtualization Workloads  [It] [tc-id:OADP-185] [kubevirt] [Tier0] [vf:vm] Backing up started VM should succeed /alabama/cspi/e2e/kubevirt-plugin/backup_restore_csi.go:34 [FAILED] Unexpected error: <*errors.Error | 0xc00029a1c0>: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-185","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt","user_kubeconfig":"/home/jenkins/.kube/config","vm_running":true,"with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 { context: "(DefaultExecute::Execute)", message: "Error during command execution: ansible-playbook error: one or more host failed\n\nCommand executed: /usr/local/bin/ansible-playbook --extra-vars {\"admin_kubeconfig\":\"/home/jenkins/.kube/config\",\"namespace\":\"test-oadp-185\",\"non_admin_user\":false,\"use_role\":\"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt\",\"user_kubeconfig\":\"/home/jenkins/.kube/config\",\"vm_running\":true,\"with_deploy\":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml\n\nexit status 2", wrappedErrors: nil, } occurred In [It] at: /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:24:34.244 There were additional failures detected. To view them in detail run ginkgo -vv ------------------------------ CSI: Backup/Restore Openshift Virtualization Workloads  [tc-id:OADP-186] [kubevirt] [Tier0] [vf:vm] Stopped VM should be restored /alabama/cspi/e2e/kubevirt-plugin/backup_restore_csi.go:51 > Enter [BeforeEach] CSI: Backup/Restore Openshift Virtualization Workloads @ 01/14/26 16:24:48.412 < Exit [BeforeEach] CSI: Backup/Restore Openshift Virtualization Workloads @ 01/14/26 16:24:48.428 (16ms) > Enter [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:24:48.428 < Exit [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:24:48.428 (0s) > Enter [It] [tc-id:OADP-186] [kubevirt] [Tier0] [vf:vm] Stopped VM should be restored @ 01/14/26 16:24:48.428 2026/01/14 16:24:48 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/14/26 16:24:48.433 2026/01/14 16:24:48 csi 2026/01/14 16:24:48 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "36719bab-ddf9-4c64-a1ea-fb3936cb1f0d", "resourceVersion": "71468", "generation": 1, "creationTimestamp": "2026-01-14T16:24:48Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:24:48Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/14/26 16:24:48.497 2026/01/14 16:24:48 Waiting for velero pod to be running 2026/01/14 16:24:48 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2026/01/14 16:24:48 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "36719bab-ddf9-4c64-a1ea-fb3936cb1f0d", "resourceVersion": "71468", "generation": 1, "creationTimestamp": "2026-01-14T16:24:48Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:24:48Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } 2026/01/14 16:24:53 DPA successfully reconciled STEP: Prepare backup resources, depending on the volumes backup type @ 01/14/26 16:24:53.529 Run the command: oc get ns openshift-storage &> /dev/null && echo true || echo false 2026/01/14 16:24:53 The 'openshift-storage' namespace exists 2026/01/14 16:24:53 Checking default storage class count 2026/01/14 16:24:53 Using the CSI driver: openshift-storage.rbd.csi.ceph.com 2026/01/14 16:24:53 Snapclass 'example-snapclass' doesn't exist, creating 2026/01/14 16:24:53 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:24:53 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd STEP: Installing application for case ocp-kubevirt @ 01/14/26 16:24:53.759 2026/01/14 16:24:53 🔄 [Attempt 1/3] Executing operation: Deploy application ocp-kubevirt 2026/01/14 16:24:53 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:24:56 2026-01-14 16:24:55,434 p=21564 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:24:55,434 p=21564 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:24:55,761 p=21564 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:24:55,761 p=21564 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:24:56,076 p=21564 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:24:56,077 p=21564 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:24:56,397 p=21564 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:24:56,397 p=21564 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:24:56,415 p=21564 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:24:56,415 p=21564 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:24:56,416 p=21564 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:24:56,436 p=21564 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:24:56,436 p=21564 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:24:56,450 p=21564 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:24:56,451 p=21564 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:24:56,856 p=21564 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:24:56,857 p=21564 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:24:56,857 p=21564 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:24:56,857 p=21564 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:24:56,879 p=21564 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:24:56,880 p=21564 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:24:56,880 p=21564 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:24:56,881 p=21564 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:24:56,881 p=21564 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-186 2026/01/14 16:24:57 2026/01/14 16:24:57 ❌ Operation 'Deploy application ocp-kubevirt' failed with non-retryable error: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-186","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt","user_kubeconfig":"/home/jenkins/.kube/config","vm_running":false,"with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 [FAILED] in [It] - /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:24:57.045 < Exit [It] [tc-id:OADP-186] [kubevirt] [Tier0] [vf:vm] Stopped VM should be restored @ 01/14/26 16:24:57.045 (8.617s) > Enter [JustAfterEach] TOP-LEVEL @ 01/14/26 16:24:57.045 2026/01/14 16:24:57 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 STEP: Get the failed spec name @ 01/14/26 16:24:57.046 2026/01/14 16:24:57 The failed spec name is: CSI: Backup/Restore Openshift Virtualization Workloads [tc-id:OADP-186] [kubevirt] [Tier0] [vf:vm] Stopped VM should be restored STEP: Create a folder for all must-gather files if it doesn't exists already @ 01/14/26 16:24:57.046 STEP: Create a folder for the failed spec if it doesn't exists already @ 01/14/26 16:24:57.046 2026/01/14 16:24:57 The folder logs/It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-186_kubevirt_Tier0_vf_vm_Stopped_VM_should_be_restored does not exists, creating new folder with the name: logs/It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-186_kubevirt_Tier0_vf_vm_Stopped_VM_should_be_restored STEP: Run must-gather because the spec failed @ 01/14/26 16:24:57.046 2026/01/14 16:24:57 Log the present working directory path:- /alabama/cspi/e2e/kubevirt-plugin 2026/01/14 16:24:57 [adm must-gather --dest-dir /alabama/cspi/e2e/kubevirt-plugin/logs/It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-186_kubevirt_Tier0_vf_vm_Stopped_VM_should_be_restored --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3] 2026/01/14 16:25:07 Log all the files present in /alabama/cspi/e2e/kubevirt-plugin/logs directory 2026/01/14 16:25:07 It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-185_kubevirt_Tier0_vf_vm_Backing_up_started_VM_should_succeed 2026/01/14 16:25:07 It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-186_kubevirt_Tier0_vf_vm_Stopped_VM_should_be_restored STEP: Find must-gather folder and rename it to a shorter more readable name @ 01/14/26 16:25:07.767 < Exit [JustAfterEach] TOP-LEVEL @ 01/14/26 16:25:07.767 (10.721s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:25:07.767 2026/01/14 16:25:07 Cleaning app 2026/01/14 16:25:07 Using admin kubeconfig for with_cleanup operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:25:10 2026-01-14 16:25:09,456 p=21788 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:25:09,457 p=21788 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:25:09,780 p=21788 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:25:09,780 p=21788 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:25:10,106 p=21788 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:25:10,107 p=21788 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:25:10,427 p=21788 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:25:10,428 p=21788 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:25:10,446 p=21788 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:25:10,446 p=21788 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:25:10,446 p=21788 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:25:10,466 p=21788 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:25:10,466 p=21788 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:25:10,480 p=21788 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:25:10,480 p=21788 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:25:10,882 p=21788 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:25:10,883 p=21788 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:25:10,883 p=21788 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:25:10,883 p=21788 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:25:10,906 p=21788 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:25:10,907 p=21788 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:25:10,907 p=21788 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:25:10,908 p=21788 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:25:10,908 p=21788 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-186 2026/01/14 16:25:11 < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:25:11.087 (3.32s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:25:11.087 2026/01/14 16:25:11 Cleaning setup resources for the backup 2026/01/14 16:25:11 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:25:11 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2026/01/14 16:25:11 Deleting VolumeSnapshotClass 'example-snapclass' < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:25:11.116 (28ms) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:25:11.116 2026/01/14 16:25:11 Deleting DPA CR < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:25:11.124 (8ms) Attempt #1 Failed. Retrying ↺ @ 01/14/26 16:25:11.124 > Enter [BeforeEach] CSI: Backup/Restore Openshift Virtualization Workloads @ 01/14/26 16:25:11.124 < Exit [BeforeEach] CSI: Backup/Restore Openshift Virtualization Workloads @ 01/14/26 16:25:11.138 (14ms) > Enter [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:25:11.138 < Exit [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:25:11.138 (0s) > Enter [It] [tc-id:OADP-186] [kubevirt] [Tier0] [vf:vm] Stopped VM should be restored @ 01/14/26 16:25:11.138 2026/01/14 16:25:11 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/14/26 16:25:11.153 2026/01/14 16:25:11 csi 2026/01/14 16:25:11 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "02bd45b8-eff4-48b3-a8d6-da9bcb6ffa74", "resourceVersion": "72025", "generation": 1, "creationTimestamp": "2026-01-14T16:25:11Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:25:11Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/14/26 16:25:11.226 2026/01/14 16:25:11 Waiting for velero pod to be running 2026/01/14 16:25:11 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2026/01/14 16:25:11 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "02bd45b8-eff4-48b3-a8d6-da9bcb6ffa74", "resourceVersion": "72025", "generation": 1, "creationTimestamp": "2026-01-14T16:25:11Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:25:11Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } 2026/01/14 16:25:16 DPA successfully reconciled STEP: Prepare backup resources, depending on the volumes backup type @ 01/14/26 16:25:16.252 2026/01/14 16:25:16 Snapclass 'example-snapclass' doesn't exist, creating 2026/01/14 16:25:16 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:25:16 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd STEP: Installing application for case ocp-kubevirt @ 01/14/26 16:25:16.382 2026/01/14 16:25:16 🔄 [Attempt 1/3] Executing operation: Deploy application ocp-kubevirt 2026/01/14 16:25:16 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:25:19 2026-01-14 16:25:18,068 p=21968 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:25:18,068 p=21968 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:25:18,391 p=21968 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:25:18,392 p=21968 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:25:18,714 p=21968 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:25:18,714 p=21968 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:25:19,035 p=21968 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:25:19,036 p=21968 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:25:19,054 p=21968 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:25:19,054 p=21968 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:25:19,054 p=21968 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:25:19,074 p=21968 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:25:19,074 p=21968 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:25:19,089 p=21968 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:25:19,089 p=21968 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:25:19,487 p=21968 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:25:19,488 p=21968 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:25:19,488 p=21968 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:25:19,488 p=21968 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:25:19,511 p=21968 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:25:19,511 p=21968 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:25:19,511 p=21968 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:25:19,512 p=21968 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:25:19,512 p=21968 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-186 2026/01/14 16:25:19 2026/01/14 16:25:19 ❌ Operation 'Deploy application ocp-kubevirt' failed with non-retryable error: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-186","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt","user_kubeconfig":"/home/jenkins/.kube/config","vm_running":false,"with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 [FAILED] in [It] - /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:25:19.694 < Exit [It] [tc-id:OADP-186] [kubevirt] [Tier0] [vf:vm] Stopped VM should be restored @ 01/14/26 16:25:19.694 (8.556s) > Enter [JustAfterEach] TOP-LEVEL @ 01/14/26 16:25:19.694 2026/01/14 16:25:19 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 STEP: Get the failed spec name @ 01/14/26 16:25:19.694 2026/01/14 16:25:19 The failed spec name is: CSI: Backup/Restore Openshift Virtualization Workloads [tc-id:OADP-186] [kubevirt] [Tier0] [vf:vm] Stopped VM should be restored STEP: Create a folder for all must-gather files if it doesn't exists already @ 01/14/26 16:25:19.694 STEP: Create a folder for the failed spec if it doesn't exists already @ 01/14/26 16:25:19.694 STEP: Run must-gather because the spec failed @ 01/14/26 16:25:19.694 2026/01/14 16:25:19 Log the present working directory path:- /alabama/cspi/e2e/kubevirt-plugin 2026/01/14 16:25:19 [adm must-gather --dest-dir /alabama/cspi/e2e/kubevirt-plugin/logs/It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-186_kubevirt_Tier0_vf_vm_Stopped_VM_should_be_restored --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3] 2026/01/14 16:25:30 Log all the files present in /alabama/cspi/e2e/kubevirt-plugin/logs directory 2026/01/14 16:25:30 It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-185_kubevirt_Tier0_vf_vm_Backing_up_started_VM_should_succeed 2026/01/14 16:25:30 It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-186_kubevirt_Tier0_vf_vm_Stopped_VM_should_be_restored STEP: Find must-gather folder and rename it to a shorter more readable name @ 01/14/26 16:25:30.42 The folder logs/It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-186_kubevirt_Tier0_vf_vm_Stopped_VM_should_be_restored/must-gather already exists, skipping renaming the folder < Exit [JustAfterEach] TOP-LEVEL @ 01/14/26 16:25:30.42 (10.726s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:25:30.42 2026/01/14 16:25:30 Cleaning app 2026/01/14 16:25:30 Using admin kubeconfig for with_cleanup operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:25:33 2026-01-14 16:25:32,106 p=22192 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:25:32,107 p=22192 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:25:32,452 p=22192 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:25:32,452 p=22192 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:25:32,778 p=22192 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:25:32,778 p=22192 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:25:33,108 p=22192 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:25:33,108 p=22192 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:25:33,126 p=22192 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:25:33,127 p=22192 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:25:33,127 p=22192 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:25:33,149 p=22192 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:25:33,149 p=22192 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:25:33,164 p=22192 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:25:33,164 p=22192 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:25:33,565 p=22192 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:25:33,566 p=22192 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:25:33,566 p=22192 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:25:33,566 p=22192 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:25:33,589 p=22192 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:25:33,590 p=22192 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:25:33,590 p=22192 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:25:33,590 p=22192 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:25:33,591 p=22192 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-186 2026/01/14 16:25:33 < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:25:33.764 (3.344s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:25:33.764 2026/01/14 16:25:33 Cleaning setup resources for the backup 2026/01/14 16:25:33 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:25:33 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2026/01/14 16:25:33 Deleting VolumeSnapshotClass 'example-snapclass' < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:25:33.792 (28ms) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:25:33.792 2026/01/14 16:25:33 Deleting DPA CR < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:25:33.804 (13ms) Attempt #2 Failed. Retrying ↺ @ 01/14/26 16:25:33.804 > Enter [BeforeEach] CSI: Backup/Restore Openshift Virtualization Workloads @ 01/14/26 16:25:33.805 < Exit [BeforeEach] CSI: Backup/Restore Openshift Virtualization Workloads @ 01/14/26 16:25:33.819 (15ms) > Enter [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:25:33.819 < Exit [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:25:33.819 (0s) > Enter [It] [tc-id:OADP-186] [kubevirt] [Tier0] [vf:vm] Stopped VM should be restored @ 01/14/26 16:25:33.819 2026/01/14 16:25:33 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/14/26 16:25:33.83 2026/01/14 16:25:33 csi 2026/01/14 16:25:33 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "a8161bed-46c4-4a01-a331-08fa7a7832e7", "resourceVersion": "72540", "generation": 1, "creationTimestamp": "2026-01-14T16:25:33Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:25:33Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/14/26 16:25:33.89 2026/01/14 16:25:33 Waiting for velero pod to be running 2026/01/14 16:25:33 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2026/01/14 16:25:33 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "a8161bed-46c4-4a01-a331-08fa7a7832e7", "resourceVersion": "72540", "generation": 1, "creationTimestamp": "2026-01-14T16:25:33Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:25:33Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } 2026/01/14 16:25:38 DPA successfully reconciled STEP: Prepare backup resources, depending on the volumes backup type @ 01/14/26 16:25:38.916 2026/01/14 16:25:38 Snapclass 'example-snapclass' doesn't exist, creating 2026/01/14 16:25:38 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:25:38 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd STEP: Installing application for case ocp-kubevirt @ 01/14/26 16:25:39.051 2026/01/14 16:25:39 🔄 [Attempt 1/3] Executing operation: Deploy application ocp-kubevirt 2026/01/14 16:25:39 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:25:42 2026-01-14 16:25:40,725 p=22371 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:25:40,725 p=22371 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:25:41,047 p=22371 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:25:41,048 p=22371 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:25:41,366 p=22371 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:25:41,366 p=22371 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:25:41,686 p=22371 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:25:41,687 p=22371 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:25:41,705 p=22371 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:25:41,705 p=22371 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:25:41,705 p=22371 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:25:41,725 p=22371 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:25:41,725 p=22371 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:25:41,740 p=22371 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:25:41,741 p=22371 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:25:42,150 p=22371 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:25:42,151 p=22371 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:25:42,151 p=22371 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:25:42,151 p=22371 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:25:42,173 p=22371 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:25:42,173 p=22371 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:25:42,173 p=22371 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:25:42,174 p=22371 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:25:42,174 p=22371 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-186 2026/01/14 16:25:42 2026/01/14 16:25:42 ❌ Operation 'Deploy application ocp-kubevirt' failed with non-retryable error: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-186","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt","user_kubeconfig":"/home/jenkins/.kube/config","vm_running":false,"with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 [FAILED] in [It] - /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:25:42.383 < Exit [It] [tc-id:OADP-186] [kubevirt] [Tier0] [vf:vm] Stopped VM should be restored @ 01/14/26 16:25:42.383 (8.564s) > Enter [JustAfterEach] TOP-LEVEL @ 01/14/26 16:25:42.383 2026/01/14 16:25:42 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 STEP: Get the failed spec name @ 01/14/26 16:25:42.384 2026/01/14 16:25:42 The failed spec name is: CSI: Backup/Restore Openshift Virtualization Workloads [tc-id:OADP-186] [kubevirt] [Tier0] [vf:vm] Stopped VM should be restored STEP: Create a folder for all must-gather files if it doesn't exists already @ 01/14/26 16:25:42.384 STEP: Create a folder for the failed spec if it doesn't exists already @ 01/14/26 16:25:42.384 STEP: Run must-gather because the spec failed @ 01/14/26 16:25:42.384 2026/01/14 16:25:42 Log the present working directory path:- /alabama/cspi/e2e/kubevirt-plugin 2026/01/14 16:25:42 [adm must-gather --dest-dir /alabama/cspi/e2e/kubevirt-plugin/logs/It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-186_kubevirt_Tier0_vf_vm_Stopped_VM_should_be_restored --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3] 2026/01/14 16:25:53 Log all the files present in /alabama/cspi/e2e/kubevirt-plugin/logs directory 2026/01/14 16:25:53 It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-185_kubevirt_Tier0_vf_vm_Backing_up_started_VM_should_succeed 2026/01/14 16:25:53 It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-186_kubevirt_Tier0_vf_vm_Stopped_VM_should_be_restored STEP: Find must-gather folder and rename it to a shorter more readable name @ 01/14/26 16:25:53.344 The folder logs/It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-186_kubevirt_Tier0_vf_vm_Stopped_VM_should_be_restored/must-gather already exists, skipping renaming the folder < Exit [JustAfterEach] TOP-LEVEL @ 01/14/26 16:25:53.344 (10.961s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:25:53.344 2026/01/14 16:25:53 Cleaning app 2026/01/14 16:25:53 Using admin kubeconfig for with_cleanup operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:25:56 2026-01-14 16:25:55,027 p=22597 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:25:55,027 p=22597 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:25:55,348 p=22597 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:25:55,348 p=22597 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:25:55,675 p=22597 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:25:55,675 p=22597 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:25:55,997 p=22597 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:25:55,997 p=22597 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:25:56,015 p=22597 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:25:56,015 p=22597 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:25:56,016 p=22597 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:25:56,036 p=22597 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:25:56,036 p=22597 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:25:56,051 p=22597 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:25:56,051 p=22597 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:25:56,445 p=22597 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:25:56,446 p=22597 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:25:56,446 p=22597 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:25:56,446 p=22597 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:25:56,468 p=22597 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:25:56,468 p=22597 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:25:56,469 p=22597 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:25:56,469 p=22597 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:25:56,469 p=22597 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-186 2026/01/14 16:25:56 < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:25:56.642 (3.298s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:25:56.642 2026/01/14 16:25:56 Cleaning setup resources for the backup 2026/01/14 16:25:56 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:25:56 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2026/01/14 16:25:56 Deleting VolumeSnapshotClass 'example-snapclass' < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:25:56.671 (28ms) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:25:56.671 2026/01/14 16:25:56 Deleting DPA CR < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:25:56.68 (10ms) • [FAILED] [68.268 seconds] CSI: Backup/Restore Openshift Virtualization Workloads  [It] [tc-id:OADP-186] [kubevirt] [Tier0] [vf:vm] Stopped VM should be restored /alabama/cspi/e2e/kubevirt-plugin/backup_restore_csi.go:51 [FAILED] Unexpected error: <*errors.Error | 0xc000c2ef80>: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-186","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt","user_kubeconfig":"/home/jenkins/.kube/config","vm_running":false,"with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 { context: "(DefaultExecute::Execute)", message: "Error during command execution: ansible-playbook error: one or more host failed\n\nCommand executed: /usr/local/bin/ansible-playbook --extra-vars {\"admin_kubeconfig\":\"/home/jenkins/.kube/config\",\"namespace\":\"test-oadp-186\",\"non_admin_user\":false,\"use_role\":\"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt\",\"user_kubeconfig\":\"/home/jenkins/.kube/config\",\"vm_running\":false,\"with_deploy\":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml\n\nexit status 2", wrappedErrors: nil, } occurred In [It] at: /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:25:42.383 There were additional failures detected. To view them in detail run ginkgo -vv ------------------------------ CSI: Backup/Restore Openshift Virtualization Workloads  [tc-id:OADP-187] [kubevirt] [Tier0] [vf:vm] Backup-restore data volume /alabama/cspi/e2e/kubevirt-plugin/backup_restore_csi.go:68 > Enter [BeforeEach] CSI: Backup/Restore Openshift Virtualization Workloads @ 01/14/26 16:25:56.681 < Exit [BeforeEach] CSI: Backup/Restore Openshift Virtualization Workloads @ 01/14/26 16:25:56.695 (14ms) > Enter [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:25:56.695 < Exit [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:25:56.695 (0s) > Enter [It] [tc-id:OADP-187] [kubevirt] [Tier0] [vf:vm] Backup-restore data volume @ 01/14/26 16:25:56.695 2026/01/14 16:25:56 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/14/26 16:25:56.706 2026/01/14 16:25:56 csi 2026/01/14 16:25:56 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "d87050f1-362b-4663-906a-2bf376d52cb8", "resourceVersion": "73058", "generation": 1, "creationTimestamp": "2026-01-14T16:25:56Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:25:56Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/14/26 16:25:56.767 2026/01/14 16:25:56 Waiting for velero pod to be running 2026/01/14 16:25:56 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2026/01/14 16:25:56 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "d87050f1-362b-4663-906a-2bf376d52cb8", "resourceVersion": "73058", "generation": 1, "creationTimestamp": "2026-01-14T16:25:56Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:25:56Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } 2026/01/14 16:26:01 DPA successfully reconciled STEP: Prepare backup resources, depending on the volumes backup type @ 01/14/26 16:26:01.789 Run the command: oc get ns openshift-storage &> /dev/null && echo true || echo false 2026/01/14 16:26:01 The 'openshift-storage' namespace exists 2026/01/14 16:26:01 Checking default storage class count 2026/01/14 16:26:01 Using the CSI driver: openshift-storage.rbd.csi.ceph.com 2026/01/14 16:26:01 Snapclass 'example-snapclass' doesn't exist, creating 2026/01/14 16:26:02 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:26:02 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd STEP: Installing application for case ocp-datavolume @ 01/14/26 16:26:02.117 2026/01/14 16:26:02 🔄 [Attempt 1/3] Executing operation: Deploy application ocp-datavolume 2026/01/14 16:26:02 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:26:05 2026-01-14 16:26:03,824 p=22798 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:26:03,824 p=22798 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:26:04,146 p=22798 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:26:04,146 p=22798 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:26:04,466 p=22798 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:26:04,467 p=22798 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:26:04,786 p=22798 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:26:04,786 p=22798 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:26:04,804 p=22798 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:26:04,805 p=22798 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:26:04,805 p=22798 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:26:04,825 p=22798 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:26:04,825 p=22798 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:26:04,839 p=22798 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:26:04,840 p=22798 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:26:05,225 p=22798 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:26:05,226 p=22798 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:26:05,226 p=22798 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:26:05,226 p=22798 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:26:05,248 p=22798 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:26:05,249 p=22798 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:26:05,249 p=22798 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:26:05,249 p=22798 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:26:05,250 p=22798 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-187 2026/01/14 16:26:05 2026/01/14 16:26:05 ❌ Operation 'Deploy application ocp-datavolume' failed with non-retryable error: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-187","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-datavolume","user_kubeconfig":"/home/jenkins/.kube/config","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 [FAILED] in [It] - /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:26:05.423 < Exit [It] [tc-id:OADP-187] [kubevirt] [Tier0] [vf:vm] Backup-restore data volume @ 01/14/26 16:26:05.423 (8.728s) > Enter [JustAfterEach] TOP-LEVEL @ 01/14/26 16:26:05.423 2026/01/14 16:26:05 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 STEP: Get the failed spec name @ 01/14/26 16:26:05.423 2026/01/14 16:26:05 The failed spec name is: CSI: Backup/Restore Openshift Virtualization Workloads [tc-id:OADP-187] [kubevirt] [Tier0] [vf:vm] Backup-restore data volume STEP: Create a folder for all must-gather files if it doesn't exists already @ 01/14/26 16:26:05.423 STEP: Create a folder for the failed spec if it doesn't exists already @ 01/14/26 16:26:05.423 2026/01/14 16:26:05 The folder logs/It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-187_kubevirt_Tier0_vf_vm_Backup-restore_data_volume does not exists, creating new folder with the name: logs/It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-187_kubevirt_Tier0_vf_vm_Backup-restore_data_volume STEP: Run must-gather because the spec failed @ 01/14/26 16:26:05.423 2026/01/14 16:26:05 Log the present working directory path:- /alabama/cspi/e2e/kubevirt-plugin 2026/01/14 16:26:05 [adm must-gather --dest-dir /alabama/cspi/e2e/kubevirt-plugin/logs/It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-187_kubevirt_Tier0_vf_vm_Backup-restore_data_volume --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3] 2026/01/14 16:26:16 Log all the files present in /alabama/cspi/e2e/kubevirt-plugin/logs directory 2026/01/14 16:26:16 It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-185_kubevirt_Tier0_vf_vm_Backing_up_started_VM_should_succeed 2026/01/14 16:26:16 It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-186_kubevirt_Tier0_vf_vm_Stopped_VM_should_be_restored 2026/01/14 16:26:16 It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-187_kubevirt_Tier0_vf_vm_Backup-restore_data_volume STEP: Find must-gather folder and rename it to a shorter more readable name @ 01/14/26 16:26:16.139 < Exit [JustAfterEach] TOP-LEVEL @ 01/14/26 16:26:16.139 (10.717s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:26:16.139 2026/01/14 16:26:16 Cleaning app 2026/01/14 16:26:16 Using admin kubeconfig for with_cleanup operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:26:19 2026-01-14 16:26:17,838 p=23021 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:26:17,838 p=23021 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:26:18,163 p=23021 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:26:18,163 p=23021 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:26:18,490 p=23021 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:26:18,490 p=23021 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:26:18,822 p=23021 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:26:18,822 p=23021 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:26:18,840 p=23021 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:26:18,841 p=23021 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:26:18,841 p=23021 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:26:18,860 p=23021 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:26:18,860 p=23021 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:26:18,875 p=23021 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:26:18,876 p=23021 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:26:19,280 p=23021 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:26:19,280 p=23021 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:26:19,281 p=23021 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:26:19,281 p=23021 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:26:19,303 p=23021 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:26:19,304 p=23021 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:26:19,304 p=23021 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:26:19,305 p=23021 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:26:19,305 p=23021 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-187 2026/01/14 16:26:19 < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:26:19.474 (3.334s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:26:19.474 2026/01/14 16:26:19 Cleaning setup resources for the backup 2026/01/14 16:26:19 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:26:19 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2026/01/14 16:26:19 Deleting VolumeSnapshotClass 'example-snapclass' < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:26:19.499 (25ms) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:26:19.499 2026/01/14 16:26:19 Deleting DPA CR < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:26:19.509 (10ms) Attempt #1 Failed. Retrying ↺ @ 01/14/26 16:26:19.509 > Enter [BeforeEach] CSI: Backup/Restore Openshift Virtualization Workloads @ 01/14/26 16:26:19.509 < Exit [BeforeEach] CSI: Backup/Restore Openshift Virtualization Workloads @ 01/14/26 16:26:19.524 (15ms) > Enter [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:26:19.524 < Exit [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:26:19.524 (0s) > Enter [It] [tc-id:OADP-187] [kubevirt] [Tier0] [vf:vm] Backup-restore data volume @ 01/14/26 16:26:19.524 2026/01/14 16:26:19 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/14/26 16:26:19.531 2026/01/14 16:26:19 csi 2026/01/14 16:26:19 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "2bc02426-1425-454a-83f8-5c1c0fc32c31", "resourceVersion": "73576", "generation": 1, "creationTimestamp": "2026-01-14T16:26:19Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:26:19Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/14/26 16:26:19.594 2026/01/14 16:26:19 Waiting for velero pod to be running 2026/01/14 16:26:19 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2026/01/14 16:26:19 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "2bc02426-1425-454a-83f8-5c1c0fc32c31", "resourceVersion": "73576", "generation": 1, "creationTimestamp": "2026-01-14T16:26:19Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:26:19Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } 2026/01/14 16:26:24 DPA successfully reconciled STEP: Prepare backup resources, depending on the volumes backup type @ 01/14/26 16:26:24.616 2026/01/14 16:26:24 Snapclass 'example-snapclass' doesn't exist, creating 2026/01/14 16:26:24 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:26:24 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd STEP: Installing application for case ocp-datavolume @ 01/14/26 16:26:24.745 2026/01/14 16:26:24 🔄 [Attempt 1/3] Executing operation: Deploy application ocp-datavolume 2026/01/14 16:26:24 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:26:27 2026-01-14 16:26:26,446 p=23199 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:26:26,447 p=23199 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:26:26,779 p=23199 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:26:26,779 p=23199 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:26:27,109 p=23199 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:26:27,109 p=23199 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:26:27,434 p=23199 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:26:27,434 p=23199 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:26:27,452 p=23199 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:26:27,453 p=23199 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:26:27,453 p=23199 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:26:27,473 p=23199 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:26:27,473 p=23199 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:26:27,488 p=23199 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:26:27,489 p=23199 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:26:27,887 p=23199 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:26:27,888 p=23199 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:26:27,888 p=23199 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:26:27,888 p=23199 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:26:27,910 p=23199 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:26:27,911 p=23199 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:26:27,911 p=23199 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:26:27,912 p=23199 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:26:27,912 p=23199 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-187 2026/01/14 16:26:28 2026/01/14 16:26:28 ❌ Operation 'Deploy application ocp-datavolume' failed with non-retryable error: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-187","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-datavolume","user_kubeconfig":"/home/jenkins/.kube/config","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 [FAILED] in [It] - /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:26:28.092 < Exit [It] [tc-id:OADP-187] [kubevirt] [Tier0] [vf:vm] Backup-restore data volume @ 01/14/26 16:26:28.092 (8.568s) > Enter [JustAfterEach] TOP-LEVEL @ 01/14/26 16:26:28.092 2026/01/14 16:26:28 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 STEP: Get the failed spec name @ 01/14/26 16:26:28.092 2026/01/14 16:26:28 The failed spec name is: CSI: Backup/Restore Openshift Virtualization Workloads [tc-id:OADP-187] [kubevirt] [Tier0] [vf:vm] Backup-restore data volume STEP: Create a folder for all must-gather files if it doesn't exists already @ 01/14/26 16:26:28.092 STEP: Create a folder for the failed spec if it doesn't exists already @ 01/14/26 16:26:28.092 STEP: Run must-gather because the spec failed @ 01/14/26 16:26:28.092 2026/01/14 16:26:28 Log the present working directory path:- /alabama/cspi/e2e/kubevirt-plugin 2026/01/14 16:26:28 [adm must-gather --dest-dir /alabama/cspi/e2e/kubevirt-plugin/logs/It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-187_kubevirt_Tier0_vf_vm_Backup-restore_data_volume --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3] 2026/01/14 16:26:38 Log all the files present in /alabama/cspi/e2e/kubevirt-plugin/logs directory 2026/01/14 16:26:38 It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-185_kubevirt_Tier0_vf_vm_Backing_up_started_VM_should_succeed 2026/01/14 16:26:38 It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-186_kubevirt_Tier0_vf_vm_Stopped_VM_should_be_restored 2026/01/14 16:26:38 It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-187_kubevirt_Tier0_vf_vm_Backup-restore_data_volume STEP: Find must-gather folder and rename it to a shorter more readable name @ 01/14/26 16:26:38.862 The folder logs/It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-187_kubevirt_Tier0_vf_vm_Backup-restore_data_volume/must-gather already exists, skipping renaming the folder < Exit [JustAfterEach] TOP-LEVEL @ 01/14/26 16:26:38.862 (10.77s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:26:38.862 2026/01/14 16:26:38 Cleaning app 2026/01/14 16:26:38 Using admin kubeconfig for with_cleanup operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:26:42 2026-01-14 16:26:40,546 p=23422 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:26:40,546 p=23422 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:26:40,868 p=23422 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:26:40,868 p=23422 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:26:41,193 p=23422 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:26:41,194 p=23422 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:26:41,519 p=23422 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:26:41,519 p=23422 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:26:41,538 p=23422 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:26:41,538 p=23422 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:26:41,538 p=23422 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:26:41,559 p=23422 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:26:41,559 p=23422 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:26:41,574 p=23422 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:26:41,574 p=23422 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:26:41,974 p=23422 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:26:41,974 p=23422 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:26:41,974 p=23422 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:26:41,975 p=23422 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:26:41,997 p=23422 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:26:41,998 p=23422 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:26:41,998 p=23422 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:26:41,999 p=23422 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:26:41,999 p=23422 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-187 2026/01/14 16:26:42 < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:26:42.17 (3.307s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:26:42.17 2026/01/14 16:26:42 Cleaning setup resources for the backup 2026/01/14 16:26:42 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:26:42 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2026/01/14 16:26:42 Deleting VolumeSnapshotClass 'example-snapclass' < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:26:42.197 (27ms) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:26:42.197 2026/01/14 16:26:42 Deleting DPA CR < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:26:42.21 (13ms) Attempt #2 Failed. Retrying ↺ @ 01/14/26 16:26:42.21 > Enter [BeforeEach] CSI: Backup/Restore Openshift Virtualization Workloads @ 01/14/26 16:26:42.21 < Exit [BeforeEach] CSI: Backup/Restore Openshift Virtualization Workloads @ 01/14/26 16:26:42.231 (21ms) > Enter [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:26:42.231 < Exit [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:26:42.231 (0s) > Enter [It] [tc-id:OADP-187] [kubevirt] [Tier0] [vf:vm] Backup-restore data volume @ 01/14/26 16:26:42.231 2026/01/14 16:26:42 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/14/26 16:26:42.237 2026/01/14 16:26:42 csi 2026/01/14 16:26:42 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "195e6403-a2a5-4b2a-aca8-6d75c121217c", "resourceVersion": "74040", "generation": 1, "creationTimestamp": "2026-01-14T16:26:42Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:26:42Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/14/26 16:26:42.302 2026/01/14 16:26:42 Waiting for velero pod to be running 2026/01/14 16:26:42 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2026/01/14 16:26:42 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "195e6403-a2a5-4b2a-aca8-6d75c121217c", "resourceVersion": "74040", "generation": 1, "creationTimestamp": "2026-01-14T16:26:42Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:26:42Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } 2026/01/14 16:26:47 DPA successfully reconciled STEP: Prepare backup resources, depending on the volumes backup type @ 01/14/26 16:26:47.325 2026/01/14 16:26:47 Snapclass 'example-snapclass' doesn't exist, creating 2026/01/14 16:26:47 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:26:47 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd STEP: Installing application for case ocp-datavolume @ 01/14/26 16:26:47.466 2026/01/14 16:26:47 🔄 [Attempt 1/3] Executing operation: Deploy application ocp-datavolume 2026/01/14 16:26:47 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:26:50 2026-01-14 16:26:49,147 p=23601 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:26:49,148 p=23601 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:26:49,468 p=23601 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:26:49,468 p=23601 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:26:49,791 p=23601 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:26:49,792 p=23601 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:26:50,113 p=23601 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:26:50,114 p=23601 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:26:50,132 p=23601 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:26:50,132 p=23601 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:26:50,132 p=23601 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:26:50,152 p=23601 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:26:50,153 p=23601 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:26:50,167 p=23601 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:26:50,168 p=23601 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:26:50,572 p=23601 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:26:50,573 p=23601 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:26:50,573 p=23601 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:26:50,573 p=23601 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:26:50,596 p=23601 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:26:50,596 p=23601 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:26:50,597 p=23601 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:26:50,597 p=23601 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:26:50,597 p=23601 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-187 2026/01/14 16:26:50 2026/01/14 16:26:50 ❌ Operation 'Deploy application ocp-datavolume' failed with non-retryable error: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-187","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-datavolume","user_kubeconfig":"/home/jenkins/.kube/config","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 [FAILED] in [It] - /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:26:50.767 < Exit [It] [tc-id:OADP-187] [kubevirt] [Tier0] [vf:vm] Backup-restore data volume @ 01/14/26 16:26:50.767 (8.536s) > Enter [JustAfterEach] TOP-LEVEL @ 01/14/26 16:26:50.767 2026/01/14 16:26:50 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 STEP: Get the failed spec name @ 01/14/26 16:26:50.767 2026/01/14 16:26:50 The failed spec name is: CSI: Backup/Restore Openshift Virtualization Workloads [tc-id:OADP-187] [kubevirt] [Tier0] [vf:vm] Backup-restore data volume STEP: Create a folder for all must-gather files if it doesn't exists already @ 01/14/26 16:26:50.767 STEP: Create a folder for the failed spec if it doesn't exists already @ 01/14/26 16:26:50.767 STEP: Run must-gather because the spec failed @ 01/14/26 16:26:50.767 2026/01/14 16:26:50 Log the present working directory path:- /alabama/cspi/e2e/kubevirt-plugin 2026/01/14 16:26:50 [adm must-gather --dest-dir /alabama/cspi/e2e/kubevirt-plugin/logs/It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-187_kubevirt_Tier0_vf_vm_Backup-restore_data_volume --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3] 2026/01/14 16:27:01 Log all the files present in /alabama/cspi/e2e/kubevirt-plugin/logs directory 2026/01/14 16:27:01 It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-185_kubevirt_Tier0_vf_vm_Backing_up_started_VM_should_succeed 2026/01/14 16:27:01 It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-186_kubevirt_Tier0_vf_vm_Stopped_VM_should_be_restored 2026/01/14 16:27:01 It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-187_kubevirt_Tier0_vf_vm_Backup-restore_data_volume STEP: Find must-gather folder and rename it to a shorter more readable name @ 01/14/26 16:27:01.577 The folder logs/It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-187_kubevirt_Tier0_vf_vm_Backup-restore_data_volume/must-gather already exists, skipping renaming the folder < Exit [JustAfterEach] TOP-LEVEL @ 01/14/26 16:27:01.577 (10.81s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:27:01.577 2026/01/14 16:27:01 Cleaning app 2026/01/14 16:27:01 Using admin kubeconfig for with_cleanup operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:27:04 2026-01-14 16:27:03,279 p=23828 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:27:03,280 p=23828 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:27:03,605 p=23828 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:27:03,606 p=23828 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:27:03,927 p=23828 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:27:03,928 p=23828 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:27:04,253 p=23828 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:27:04,253 p=23828 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:27:04,271 p=23828 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:27:04,271 p=23828 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:27:04,271 p=23828 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:27:04,291 p=23828 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:27:04,291 p=23828 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:27:04,305 p=23828 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:27:04,306 p=23828 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:27:04,702 p=23828 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:27:04,702 p=23828 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:27:04,702 p=23828 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:27:04,703 p=23828 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:27:04,725 p=23828 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:27:04,726 p=23828 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:27:04,726 p=23828 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:27:04,726 p=23828 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:27:04,727 p=23828 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-187 2026/01/14 16:27:04 < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:27:04.901 (3.324s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:27:04.901 2026/01/14 16:27:04 Cleaning setup resources for the backup 2026/01/14 16:27:04 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:27:04 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2026/01/14 16:27:04 Deleting VolumeSnapshotClass 'example-snapclass' < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:27:04.931 (29ms) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:27:04.931 2026/01/14 16:27:04 Deleting DPA CR < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:27:04.942 (12ms) • [FAILED] [68.262 seconds] CSI: Backup/Restore Openshift Virtualization Workloads  [It] [tc-id:OADP-187] [kubevirt] [Tier0] [vf:vm] Backup-restore data volume /alabama/cspi/e2e/kubevirt-plugin/backup_restore_csi.go:68 [FAILED] Unexpected error: <*errors.Error | 0xc000ae3880>: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-187","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-datavolume","user_kubeconfig":"/home/jenkins/.kube/config","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 { context: "(DefaultExecute::Execute)", message: "Error during command execution: ansible-playbook error: one or more host failed\n\nCommand executed: /usr/local/bin/ansible-playbook --extra-vars {\"admin_kubeconfig\":\"/home/jenkins/.kube/config\",\"namespace\":\"test-oadp-187\",\"non_admin_user\":false,\"use_role\":\"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-datavolume\",\"user_kubeconfig\":\"/home/jenkins/.kube/config\",\"with_deploy\":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml\n\nexit status 2", wrappedErrors: nil, } occurred In [It] at: /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:26:50.767 There were additional failures detected. To view them in detail run ginkgo -vv ------------------------------ S ------------------------------ Native CSI Data Mover: Backup/Restore Openshift Virtualization Workloads  [tc-id:OADP-401] [kubevirt] [Tier1] Started VM should over ceph filesytem mode /alabama/cspi/e2e/kubevirt-plugin/backup_restore_datamover.go:129 > Enter [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:27:04.943 < Exit [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:27:04.943 (0s) > Enter [It] [tc-id:OADP-401] [kubevirt] [Tier1] Started VM should over ceph filesytem mode @ 01/14/26 16:27:04.943 2026/01/14 16:27:04 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/14/26 16:27:04.949 2026/01/14 16:27:04 native-datamover 2026/01/14 16:27:05 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "7f5721cd-b8b6-41ed-ac70-28ac576f1b17", "resourceVersion": "74571", "generation": 1, "creationTimestamp": "2026-01-14T16:27:04Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:27:04Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:nodeAgent": { ".": {}, "f:enable": {}, "f:podConfig": { ".": {}, "f:resourceAllocations": {} }, "f:uploaderType": {} }, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false }, "nodeAgent": { "enable": true, "podConfig": { "resourceAllocations": {} }, "uploaderType": "kopia" } }, "features": null, "logFormat": "text" }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/14/26 16:27:05.028 2026/01/14 16:27:05 Waiting for velero pod to be running 2026/01/14 16:27:05 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2026/01/14 16:27:05 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "7f5721cd-b8b6-41ed-ac70-28ac576f1b17", "resourceVersion": "74571", "generation": 1, "creationTimestamp": "2026-01-14T16:27:04Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:27:04Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:nodeAgent": { ".": {}, "f:enable": {}, "f:podConfig": { ".": {}, "f:resourceAllocations": {} }, "f:uploaderType": {} }, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false }, "nodeAgent": { "enable": true, "podConfig": { "resourceAllocations": {} }, "uploaderType": "kopia" } }, "features": null, "logFormat": "text" }, "status": {} } 2026/01/14 16:27:10 DPA successfully reconciled STEP: Prepare backup resources, depending on the volumes backup type @ 01/14/26 16:27:10.054 Run the command: oc get ns openshift-storage &> /dev/null && echo true || echo false 2026/01/14 16:27:10 The 'openshift-storage' namespace exists 2026/01/14 16:27:10 Checking default storage class count 2026/01/14 16:27:10 Using the CSI driver: openshift-storage.rbd.csi.ceph.com 2026/01/14 16:27:10 Snapclass 'example-snapclass' doesn't exist, creating 2026/01/14 16:27:10 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:27:10 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2026/01/14 16:27:10 Checking for correct number of running NodeAgent pods... 2026/01/14 16:27:10 pod: node-agent-5lfs8 is not yet running with status: {1 Pending [{PodReadyToStartContainers 1 False 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:27:05 +0000 UTC } {Initialized 1 True 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:27:05 +0000 UTC } {Ready 1 False 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:27:05 +0000 UTC ContainersNotReady containers with unready status: [node-agent]} {ContainersReady 1 False 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:27:05 +0000 UTC ContainersNotReady containers with unready status: [node-agent]} {PodScheduled 1 True 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:27:05 +0000 UTC }] 10.0.3.222 [{10.0.3.222}] [] 2026-01-14 16:27:05 +0000 UTC [] [{node-agent {&ContainerStateWaiting{Reason:ContainerCreating,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-rhel9@sha256:dd5897b1579599cfa8d8a03a1c8d3cf68e3927c4ba5dee4e33c2a2d4c9e6ed89 0xc000b2634a map[] nil [{host-pods /host_pods false } {host-plugins /var/lib/kubelet/plugins false } {scratch /scratch false } {certs /etc/ssl/certs false } {bound-sa-token /var/run/secrets/openshift/serviceaccount true 0xc000dc9c20} {credentials /tmp/credentials false } {home-velero /home/velero false } {tmp /tmp false } {kube-api-access-tjgcw /var/run/secrets/kubernetes.io/serviceaccount true 0xc000dc9c30}] nil [] }] Burstable [] []} STEP: Installing application for case ocp-kubevirt @ 01/14/26 16:27:15.317 2026/01/14 16:27:15 🔄 [Attempt 1/3] Executing operation: Deploy application ocp-kubevirt 2026/01/14 16:27:15 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:27:18 2026-01-14 16:27:16,990 p=24028 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:27:16,990 p=24028 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:27:17,312 p=24028 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:27:17,312 p=24028 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:27:17,635 p=24028 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:27:17,635 p=24028 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:27:17,954 p=24028 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:27:17,954 p=24028 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:27:17,972 p=24028 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:27:17,972 p=24028 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:27:17,972 p=24028 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:27:17,993 p=24028 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:27:17,993 p=24028 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:27:18,008 p=24028 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:27:18,008 p=24028 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:27:18,406 p=24028 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:27:18,406 p=24028 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:27:18,406 p=24028 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:27:18,406 p=24028 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:27:18,429 p=24028 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:27:18,430 p=24028 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:27:18,430 p=24028 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:27:18,431 p=24028 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:27:18,431 p=24028 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-401 2026/01/14 16:27:18 2026/01/14 16:27:18 ⚠️ Operation 'Deploy application ocp-kubevirt' failed with retryable server error (attempt 1/3): Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-401","non_admin_user":false,"storage_class":"odf-operator-cephfs","use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt","user_kubeconfig":"/home/jenkins/.kube/config","vm_disk_pvc_volume_mode":"Filesystem","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 2026/01/14 16:27:18 🕐 Waiting 1m0s before retry... 2026/01/14 16:28:18 🔄 [Attempt 2/3] Executing operation: Deploy application ocp-kubevirt 2026/01/14 16:28:18 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:28:21 2026-01-14 16:28:20,305 p=24208 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:28:20,305 p=24208 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:28:20,628 p=24208 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:28:20,629 p=24208 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:28:20,951 p=24208 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:28:20,951 p=24208 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:28:21,273 p=24208 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:28:21,273 p=24208 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:28:21,290 p=24208 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:28:21,291 p=24208 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:28:21,291 p=24208 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:28:21,311 p=24208 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:28:21,311 p=24208 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:28:21,326 p=24208 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:28:21,327 p=24208 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:28:21,721 p=24208 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:28:21,721 p=24208 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:28:21,721 p=24208 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:28:21,722 p=24208 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:28:21,744 p=24208 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:28:21,744 p=24208 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:28:21,744 p=24208 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:28:21,745 p=24208 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:28:21,745 p=24208 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-401 2026/01/14 16:28:21 2026/01/14 16:28:21 ⚠️ Operation 'Deploy application ocp-kubevirt' failed with retryable server error (attempt 2/3): Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-401","non_admin_user":false,"storage_class":"odf-operator-cephfs","use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt","user_kubeconfig":"/home/jenkins/.kube/config","vm_disk_pvc_volume_mode":"Filesystem","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 2026/01/14 16:28:21 🕐 Waiting 2m0s before retry... 2026/01/14 16:30:21 🔄 [Attempt 3/3] Executing operation: Deploy application ocp-kubevirt 2026/01/14 16:30:21 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:30:25 2026-01-14 16:30:23,667 p=24389 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:30:23,667 p=24389 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:30:23,990 p=24389 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:30:23,990 p=24389 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:30:24,311 p=24389 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:30:24,312 p=24389 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:30:24,634 p=24389 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:30:24,635 p=24389 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:30:24,653 p=24389 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:30:24,653 p=24389 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:30:24,653 p=24389 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:30:24,674 p=24389 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:30:24,674 p=24389 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:30:24,689 p=24389 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:30:24,690 p=24389 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:30:25,084 p=24389 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:30:25,084 p=24389 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:30:25,084 p=24389 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:30:25,084 p=24389 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:30:25,107 p=24389 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:30:25,107 p=24389 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:30:25,108 p=24389 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:30:25,108 p=24389 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:30:25,108 p=24389 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-401 2026/01/14 16:30:25 2026/01/14 16:30:25 ❌ Operation 'Deploy application ocp-kubevirt' failed after 3 attempts. Final error: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-401","non_admin_user":false,"storage_class":"odf-operator-cephfs","use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt","user_kubeconfig":"/home/jenkins/.kube/config","vm_disk_pvc_volume_mode":"Filesystem","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 [FAILED] in [It] - /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:30:25.311 < Exit [It] [tc-id:OADP-401] [kubevirt] [Tier1] Started VM should over ceph filesytem mode @ 01/14/26 16:30:25.311 (3m20.368s) > Enter [JustAfterEach] TOP-LEVEL @ 01/14/26 16:30:25.311 2026/01/14 16:30:25 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 STEP: Get the failed spec name @ 01/14/26 16:30:25.311 2026/01/14 16:30:25 The failed spec name is: Native CSI Data Mover: Backup/Restore Openshift Virtualization Workloads [tc-id:OADP-401] [kubevirt] [Tier1] Started VM should over ceph filesytem mode STEP: Create a folder for all must-gather files if it doesn't exists already @ 01/14/26 16:30:25.311 STEP: Create a folder for the failed spec if it doesn't exists already @ 01/14/26 16:30:25.311 2026/01/14 16:30:25 The folder logs/It_Native_CSI_Data_Mover_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-401_kubevirt_Tier1_Started_VM_should_over_ceph_filesytem_mode does not exists, creating new folder with the name: logs/It_Native_CSI_Data_Mover_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-401_kubevirt_Tier1_Started_VM_should_over_ceph_filesytem_mode STEP: Run must-gather because the spec failed @ 01/14/26 16:30:25.311 2026/01/14 16:30:25 Log the present working directory path:- /alabama/cspi/e2e/kubevirt-plugin 2026/01/14 16:30:25 [adm must-gather --dest-dir /alabama/cspi/e2e/kubevirt-plugin/logs/It_Native_CSI_Data_Mover_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-401_kubevirt_Tier1_Started_VM_should_over_ceph_filesytem_mode --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3] 2026/01/14 16:30:36 Log all the files present in /alabama/cspi/e2e/kubevirt-plugin/logs directory 2026/01/14 16:30:36 It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-185_kubevirt_Tier0_vf_vm_Backing_up_started_VM_should_succeed 2026/01/14 16:30:36 It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-186_kubevirt_Tier0_vf_vm_Stopped_VM_should_be_restored 2026/01/14 16:30:36 It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-187_kubevirt_Tier0_vf_vm_Backup-restore_data_volume 2026/01/14 16:30:36 It_Native_CSI_Data_Mover_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-401_kubevirt_Tier1_Started_VM_should_over_ceph_filesytem_mode STEP: Find must-gather folder and rename it to a shorter more readable name @ 01/14/26 16:30:36.073 < Exit [JustAfterEach] TOP-LEVEL @ 01/14/26 16:30:36.073 (10.762s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:30:36.073 2026/01/14 16:30:36 Cleaning app 2026/01/14 16:30:36 Using admin kubeconfig for with_cleanup operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:30:39 2026-01-14 16:30:37,745 p=24612 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:30:37,745 p=24612 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:30:38,070 p=24612 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:30:38,070 p=24612 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:30:38,407 p=24612 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:30:38,407 p=24612 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:30:38,728 p=24612 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:30:38,728 p=24612 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:30:38,746 p=24612 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:30:38,746 p=24612 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:30:38,746 p=24612 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:30:38,766 p=24612 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:30:38,766 p=24612 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:30:38,782 p=24612 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:30:38,782 p=24612 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:30:39,190 p=24612 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:30:39,190 p=24612 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:30:39,190 p=24612 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:30:39,190 p=24612 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:30:39,213 p=24612 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:30:39,213 p=24612 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:30:39,214 p=24612 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:30:39,214 p=24612 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:30:39,214 p=24612 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-401 2026/01/14 16:30:39 < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:30:39.384 (3.311s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:30:39.384 2026/01/14 16:30:39 Cleaning setup resources for the backup 2026/01/14 16:30:39 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:30:39 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2026/01/14 16:30:39 Deleting VolumeSnapshotClass 'example-snapclass' < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:30:39.421 (37ms) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:30:39.421 2026/01/14 16:30:39 Deleting DPA CR < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:30:39.437 (16ms) Attempt #1 Failed. Retrying ↺ @ 01/14/26 16:30:39.437 > Enter [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:30:39.438 < Exit [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:30:39.438 (0s) > Enter [It] [tc-id:OADP-401] [kubevirt] [Tier1] Started VM should over ceph filesytem mode @ 01/14/26 16:30:39.438 2026/01/14 16:30:39 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/14/26 16:30:39.447 2026/01/14 16:30:39 native-datamover 2026/01/14 16:30:39 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "94745a30-9db2-43b2-adf3-5d75f51c7cbc", "resourceVersion": "77878", "generation": 1, "creationTimestamp": "2026-01-14T16:30:39Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:30:39Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:nodeAgent": { ".": {}, "f:enable": {}, "f:podConfig": { ".": {}, "f:resourceAllocations": {} }, "f:uploaderType": {} }, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false }, "nodeAgent": { "enable": true, "podConfig": { "resourceAllocations": {} }, "uploaderType": "kopia" } }, "features": null, "logFormat": "text" }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/14/26 16:30:39.509 2026/01/14 16:30:39 Waiting for velero pod to be running 2026/01/14 16:30:39 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2026/01/14 16:30:39 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "94745a30-9db2-43b2-adf3-5d75f51c7cbc", "resourceVersion": "77878", "generation": 1, "creationTimestamp": "2026-01-14T16:30:39Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:30:39Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:nodeAgent": { ".": {}, "f:enable": {}, "f:podConfig": { ".": {}, "f:resourceAllocations": {} }, "f:uploaderType": {} }, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false }, "nodeAgent": { "enable": true, "podConfig": { "resourceAllocations": {} }, "uploaderType": "kopia" } }, "features": null, "logFormat": "text" }, "status": {} } 2026/01/14 16:30:44 DPA successfully reconciled STEP: Prepare backup resources, depending on the volumes backup type @ 01/14/26 16:30:44.555 2026/01/14 16:30:44 Snapclass 'example-snapclass' doesn't exist, creating 2026/01/14 16:30:44 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:30:44 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2026/01/14 16:30:44 Checking for correct number of running NodeAgent pods... STEP: Installing application for case ocp-kubevirt @ 01/14/26 16:30:44.695 2026/01/14 16:30:44 🔄 [Attempt 1/3] Executing operation: Deploy application ocp-kubevirt 2026/01/14 16:30:44 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:30:47 2026-01-14 16:30:46,359 p=24791 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:30:46,359 p=24791 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:30:46,681 p=24791 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:30:46,682 p=24791 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:30:47,001 p=24791 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:30:47,002 p=24791 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:30:47,320 p=24791 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:30:47,320 p=24791 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:30:47,338 p=24791 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:30:47,339 p=24791 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:30:47,339 p=24791 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:30:47,358 p=24791 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:30:47,359 p=24791 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:30:47,373 p=24791 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:30:47,374 p=24791 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:30:47,771 p=24791 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:30:47,772 p=24791 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:30:47,772 p=24791 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:30:47,772 p=24791 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:30:47,794 p=24791 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:30:47,795 p=24791 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:30:47,795 p=24791 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:30:47,796 p=24791 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:30:47,796 p=24791 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-401 2026/01/14 16:30:47 2026/01/14 16:30:47 ⚠️ Operation 'Deploy application ocp-kubevirt' failed with retryable server error (attempt 1/3): Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-401","non_admin_user":false,"storage_class":"odf-operator-cephfs","use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt","user_kubeconfig":"/home/jenkins/.kube/config","vm_disk_pvc_volume_mode":"Filesystem","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 2026/01/14 16:30:47 🕐 Waiting 1m0s before retry... 2026/01/14 16:31:47 🔄 [Attempt 2/3] Executing operation: Deploy application ocp-kubevirt 2026/01/14 16:31:47 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:31:51 2026-01-14 16:31:49,629 p=24972 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:31:49,630 p=24972 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:31:49,952 p=24972 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:31:49,952 p=24972 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:31:50,274 p=24972 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:31:50,275 p=24972 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:31:50,594 p=24972 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:31:50,594 p=24972 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:31:50,613 p=24972 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:31:50,613 p=24972 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:31:50,613 p=24972 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:31:50,633 p=24972 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:31:50,634 p=24972 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:31:50,648 p=24972 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:31:50,648 p=24972 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:31:51,059 p=24972 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:31:51,059 p=24972 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:31:51,060 p=24972 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:31:51,060 p=24972 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:31:51,082 p=24972 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:31:51,083 p=24972 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:31:51,083 p=24972 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:31:51,083 p=24972 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:31:51,084 p=24972 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-401 2026/01/14 16:31:51 2026/01/14 16:31:51 ⚠️ Operation 'Deploy application ocp-kubevirt' failed with retryable server error (attempt 2/3): Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-401","non_admin_user":false,"storage_class":"odf-operator-cephfs","use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt","user_kubeconfig":"/home/jenkins/.kube/config","vm_disk_pvc_volume_mode":"Filesystem","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 2026/01/14 16:31:51 🕐 Waiting 2m0s before retry... 2026/01/14 16:33:51 🔄 [Attempt 3/3] Executing operation: Deploy application ocp-kubevirt 2026/01/14 16:33:51 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:33:54 2026-01-14 16:33:52,938 p=25152 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:33:52,938 p=25152 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:33:53,260 p=25152 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:33:53,260 p=25152 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:33:53,585 p=25152 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:33:53,585 p=25152 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:33:53,906 p=25152 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:33:53,906 p=25152 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:33:53,924 p=25152 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:33:53,925 p=25152 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:33:53,925 p=25152 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:33:53,944 p=25152 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:33:53,944 p=25152 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:33:53,959 p=25152 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:33:53,959 p=25152 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:33:54,357 p=25152 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:33:54,357 p=25152 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:33:54,357 p=25152 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:33:54,357 p=25152 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:33:54,380 p=25152 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:33:54,381 p=25152 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:33:54,381 p=25152 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:33:54,382 p=25152 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:33:54,382 p=25152 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-401 2026/01/14 16:33:54 2026/01/14 16:33:54 ❌ Operation 'Deploy application ocp-kubevirt' failed after 3 attempts. Final error: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-401","non_admin_user":false,"storage_class":"odf-operator-cephfs","use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt","user_kubeconfig":"/home/jenkins/.kube/config","vm_disk_pvc_volume_mode":"Filesystem","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 [FAILED] in [It] - /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:33:54.596 < Exit [It] [tc-id:OADP-401] [kubevirt] [Tier1] Started VM should over ceph filesytem mode @ 01/14/26 16:33:54.596 (3m15.158s) > Enter [JustAfterEach] TOP-LEVEL @ 01/14/26 16:33:54.596 2026/01/14 16:33:54 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 STEP: Get the failed spec name @ 01/14/26 16:33:54.596 2026/01/14 16:33:54 The failed spec name is: Native CSI Data Mover: Backup/Restore Openshift Virtualization Workloads [tc-id:OADP-401] [kubevirt] [Tier1] Started VM should over ceph filesytem mode STEP: Create a folder for all must-gather files if it doesn't exists already @ 01/14/26 16:33:54.596 STEP: Create a folder for the failed spec if it doesn't exists already @ 01/14/26 16:33:54.596 STEP: Run must-gather because the spec failed @ 01/14/26 16:33:54.596 2026/01/14 16:33:54 Log the present working directory path:- /alabama/cspi/e2e/kubevirt-plugin 2026/01/14 16:33:54 [adm must-gather --dest-dir /alabama/cspi/e2e/kubevirt-plugin/logs/It_Native_CSI_Data_Mover_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-401_kubevirt_Tier1_Started_VM_should_over_ceph_filesytem_mode --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3] 2026/01/14 16:34:05 Log all the files present in /alabama/cspi/e2e/kubevirt-plugin/logs directory 2026/01/14 16:34:05 It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-185_kubevirt_Tier0_vf_vm_Backing_up_started_VM_should_succeed 2026/01/14 16:34:05 It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-186_kubevirt_Tier0_vf_vm_Stopped_VM_should_be_restored 2026/01/14 16:34:05 It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-187_kubevirt_Tier0_vf_vm_Backup-restore_data_volume 2026/01/14 16:34:05 It_Native_CSI_Data_Mover_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-401_kubevirt_Tier1_Started_VM_should_over_ceph_filesytem_mode STEP: Find must-gather folder and rename it to a shorter more readable name @ 01/14/26 16:34:05.479 The folder logs/It_Native_CSI_Data_Mover_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-401_kubevirt_Tier1_Started_VM_should_over_ceph_filesytem_mode/must-gather already exists, skipping renaming the folder < Exit [JustAfterEach] TOP-LEVEL @ 01/14/26 16:34:05.479 (10.883s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:34:05.479 2026/01/14 16:34:05 Cleaning app 2026/01/14 16:34:05 Using admin kubeconfig for with_cleanup operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:34:08 2026-01-14 16:34:07,154 p=25376 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:34:07,155 p=25376 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:34:07,473 p=25376 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:34:07,474 p=25376 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:34:07,795 p=25376 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:34:07,795 p=25376 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:34:08,114 p=25376 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:34:08,114 p=25376 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:34:08,133 p=25376 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:34:08,133 p=25376 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:34:08,133 p=25376 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:34:08,154 p=25376 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:34:08,154 p=25376 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:34:08,170 p=25376 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:34:08,171 p=25376 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:34:08,569 p=25376 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:34:08,570 p=25376 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:34:08,570 p=25376 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:34:08,570 p=25376 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:34:08,593 p=25376 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:34:08,593 p=25376 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:34:08,593 p=25376 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:34:08,594 p=25376 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:34:08,594 p=25376 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-401 2026/01/14 16:34:08 < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:34:08.762 (3.283s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:34:08.762 2026/01/14 16:34:08 Cleaning setup resources for the backup 2026/01/14 16:34:08 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:34:08 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2026/01/14 16:34:08 Deleting VolumeSnapshotClass 'example-snapclass' < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:34:08.798 (36ms) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:34:08.798 2026/01/14 16:34:08 Deleting DPA CR < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:34:08.808 (10ms) Attempt #2 Failed. Retrying ↺ @ 01/14/26 16:34:08.808 > Enter [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:34:08.808 < Exit [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:34:08.808 (0s) > Enter [It] [tc-id:OADP-401] [kubevirt] [Tier1] Started VM should over ceph filesytem mode @ 01/14/26 16:34:08.808 2026/01/14 16:34:08 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/14/26 16:34:08.815 2026/01/14 16:34:08 native-datamover 2026/01/14 16:34:08 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "44cfe3f9-4781-4025-81e5-32ad16b04b63", "resourceVersion": "81228", "generation": 1, "creationTimestamp": "2026-01-14T16:34:08Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:34:08Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:nodeAgent": { ".": {}, "f:enable": {}, "f:podConfig": { ".": {}, "f:resourceAllocations": {} }, "f:uploaderType": {} }, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false }, "nodeAgent": { "enable": true, "podConfig": { "resourceAllocations": {} }, "uploaderType": "kopia" } }, "features": null, "logFormat": "text" }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/14/26 16:34:08.905 2026/01/14 16:34:08 Waiting for velero pod to be running 2026/01/14 16:34:08 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2026/01/14 16:34:08 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "44cfe3f9-4781-4025-81e5-32ad16b04b63", "resourceVersion": "81228", "generation": 1, "creationTimestamp": "2026-01-14T16:34:08Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:34:08Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:nodeAgent": { ".": {}, "f:enable": {}, "f:podConfig": { ".": {}, "f:resourceAllocations": {} }, "f:uploaderType": {} }, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false }, "nodeAgent": { "enable": true, "podConfig": { "resourceAllocations": {} }, "uploaderType": "kopia" } }, "features": null, "logFormat": "text" }, "status": {} } 2026/01/14 16:34:13 DPA successfully reconciled STEP: Prepare backup resources, depending on the volumes backup type @ 01/14/26 16:34:13.934 2026/01/14 16:34:13 Snapclass 'example-snapclass' doesn't exist, creating 2026/01/14 16:34:13 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:34:13 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2026/01/14 16:34:14 Checking for correct number of running NodeAgent pods... STEP: Installing application for case ocp-kubevirt @ 01/14/26 16:34:14.075 2026/01/14 16:34:14 🔄 [Attempt 1/3] Executing operation: Deploy application ocp-kubevirt 2026/01/14 16:34:14 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:34:17 2026-01-14 16:34:15,739 p=25556 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:34:15,739 p=25556 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:34:16,063 p=25556 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:34:16,063 p=25556 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:34:16,387 p=25556 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:34:16,387 p=25556 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:34:16,705 p=25556 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:34:16,705 p=25556 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:34:16,723 p=25556 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:34:16,724 p=25556 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:34:16,724 p=25556 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:34:16,743 p=25556 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:34:16,744 p=25556 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:34:16,759 p=25556 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:34:16,759 p=25556 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:34:17,146 p=25556 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:34:17,147 p=25556 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:34:17,147 p=25556 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:34:17,147 p=25556 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:34:17,170 p=25556 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:34:17,170 p=25556 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:34:17,171 p=25556 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:34:17,171 p=25556 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:34:17,171 p=25556 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-401 2026/01/14 16:34:17 2026/01/14 16:34:17 ⚠️ Operation 'Deploy application ocp-kubevirt' failed with retryable server error (attempt 1/3): Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-401","non_admin_user":false,"storage_class":"odf-operator-cephfs","use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt","user_kubeconfig":"/home/jenkins/.kube/config","vm_disk_pvc_volume_mode":"Filesystem","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 2026/01/14 16:34:17 🕐 Waiting 1m0s before retry... 2026/01/14 16:35:17 🔄 [Attempt 2/3] Executing operation: Deploy application ocp-kubevirt 2026/01/14 16:35:17 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:35:20 2026-01-14 16:35:19,031 p=25736 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:35:19,031 p=25736 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:35:19,353 p=25736 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:35:19,353 p=25736 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:35:19,674 p=25736 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:35:19,674 p=25736 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:35:19,991 p=25736 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:35:19,991 p=25736 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:35:20,010 p=25736 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:35:20,010 p=25736 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:35:20,010 p=25736 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:35:20,030 p=25736 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:35:20,030 p=25736 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:35:20,046 p=25736 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:35:20,046 p=25736 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:35:20,447 p=25736 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:35:20,447 p=25736 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:35:20,448 p=25736 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:35:20,448 p=25736 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:35:20,470 p=25736 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:35:20,470 p=25736 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:35:20,471 p=25736 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:35:20,471 p=25736 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:35:20,471 p=25736 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-401 2026/01/14 16:35:20 2026/01/14 16:35:20 ⚠️ Operation 'Deploy application ocp-kubevirt' failed with retryable server error (attempt 2/3): Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-401","non_admin_user":false,"storage_class":"odf-operator-cephfs","use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt","user_kubeconfig":"/home/jenkins/.kube/config","vm_disk_pvc_volume_mode":"Filesystem","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 2026/01/14 16:35:20 🕐 Waiting 2m0s before retry... 2026/01/14 16:37:20 🔄 [Attempt 3/3] Executing operation: Deploy application ocp-kubevirt 2026/01/14 16:37:20 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:37:23 2026-01-14 16:37:22,358 p=25915 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:37:22,359 p=25915 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:37:22,681 p=25915 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:37:22,681 p=25915 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:37:23,006 p=25915 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:37:23,007 p=25915 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:37:23,335 p=25915 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:37:23,335 p=25915 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:37:23,353 p=25915 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:37:23,353 p=25915 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:37:23,353 p=25915 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:37:23,374 p=25915 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:37:23,374 p=25915 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:37:23,389 p=25915 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:37:23,390 p=25915 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:37:23,784 p=25915 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:37:23,784 p=25915 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:37:23,785 p=25915 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:37:23,785 p=25915 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:37:23,806 p=25915 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:37:23,807 p=25915 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:37:23,807 p=25915 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:37:23,808 p=25915 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:37:23,808 p=25915 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-401 2026/01/14 16:37:24 2026/01/14 16:37:24 ❌ Operation 'Deploy application ocp-kubevirt' failed after 3 attempts. Final error: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-401","non_admin_user":false,"storage_class":"odf-operator-cephfs","use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt","user_kubeconfig":"/home/jenkins/.kube/config","vm_disk_pvc_volume_mode":"Filesystem","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 [FAILED] in [It] - /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:37:24.013 < Exit [It] [tc-id:OADP-401] [kubevirt] [Tier1] Started VM should over ceph filesytem mode @ 01/14/26 16:37:24.013 (3m15.205s) > Enter [JustAfterEach] TOP-LEVEL @ 01/14/26 16:37:24.013 2026/01/14 16:37:24 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 STEP: Get the failed spec name @ 01/14/26 16:37:24.013 2026/01/14 16:37:24 The failed spec name is: Native CSI Data Mover: Backup/Restore Openshift Virtualization Workloads [tc-id:OADP-401] [kubevirt] [Tier1] Started VM should over ceph filesytem mode STEP: Create a folder for all must-gather files if it doesn't exists already @ 01/14/26 16:37:24.013 STEP: Create a folder for the failed spec if it doesn't exists already @ 01/14/26 16:37:24.013 STEP: Run must-gather because the spec failed @ 01/14/26 16:37:24.013 2026/01/14 16:37:24 Log the present working directory path:- /alabama/cspi/e2e/kubevirt-plugin 2026/01/14 16:37:24 [adm must-gather --dest-dir /alabama/cspi/e2e/kubevirt-plugin/logs/It_Native_CSI_Data_Mover_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-401_kubevirt_Tier1_Started_VM_should_over_ceph_filesytem_mode --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3] 2026/01/14 16:37:34 Log all the files present in /alabama/cspi/e2e/kubevirt-plugin/logs directory 2026/01/14 16:37:34 It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-185_kubevirt_Tier0_vf_vm_Backing_up_started_VM_should_succeed 2026/01/14 16:37:34 It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-186_kubevirt_Tier0_vf_vm_Stopped_VM_should_be_restored 2026/01/14 16:37:34 It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-187_kubevirt_Tier0_vf_vm_Backup-restore_data_volume 2026/01/14 16:37:34 It_Native_CSI_Data_Mover_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-401_kubevirt_Tier1_Started_VM_should_over_ceph_filesytem_mode STEP: Find must-gather folder and rename it to a shorter more readable name @ 01/14/26 16:37:34.965 The folder logs/It_Native_CSI_Data_Mover_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-401_kubevirt_Tier1_Started_VM_should_over_ceph_filesytem_mode/must-gather already exists, skipping renaming the folder < Exit [JustAfterEach] TOP-LEVEL @ 01/14/26 16:37:34.965 (10.951s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:37:34.965 2026/01/14 16:37:34 Cleaning app 2026/01/14 16:37:34 Using admin kubeconfig for with_cleanup operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:37:38 2026-01-14 16:37:36,640 p=26138 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:37:36,640 p=26138 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:37:36,956 p=26138 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:37:36,956 p=26138 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:37:37,274 p=26138 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:37:37,274 p=26138 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:37:37,599 p=26138 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:37:37,599 p=26138 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:37:37,617 p=26138 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:37:37,618 p=26138 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:37:37,618 p=26138 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:37:37,637 p=26138 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:37:37,637 p=26138 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:37:37,652 p=26138 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:37:37,652 p=26138 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:37:38,051 p=26138 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:37:38,051 p=26138 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:37:38,052 p=26138 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:37:38,052 p=26138 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:37:38,077 p=26138 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:37:38,078 p=26138 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:37:38,078 p=26138 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:37:38,079 p=26138 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:37:38,079 p=26138 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-401 2026/01/14 16:37:38 < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:37:38.266 (3.301s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:37:38.266 2026/01/14 16:37:38 Cleaning setup resources for the backup 2026/01/14 16:37:38 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:37:38 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2026/01/14 16:37:38 Deleting VolumeSnapshotClass 'example-snapclass' < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:37:38.3 (35ms) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:37:38.3 2026/01/14 16:37:38 Deleting DPA CR < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:37:38.311 (10ms) • [FAILED] [633.368 seconds] Native CSI Data Mover: Backup/Restore Openshift Virtualization Workloads  [It] [tc-id:OADP-401] [kubevirt] [Tier1] Started VM should over ceph filesytem mode /alabama/cspi/e2e/kubevirt-plugin/backup_restore_datamover.go:129 [FAILED] Unexpected error: <*fmt.wrapError | 0xc000776000>: operation 'Deploy application ocp-kubevirt' failed after 3 attempts: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-401","non_admin_user":false,"storage_class":"odf-operator-cephfs","use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt","user_kubeconfig":"/home/jenkins/.kube/config","vm_disk_pvc_volume_mode":"Filesystem","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 { msg: "operation 'Deploy application ocp-kubevirt' failed after 3 attempts: Error during command execution: ansible-playbook error: one or more host failed\n\nCommand executed: /usr/local/bin/ansible-playbook --extra-vars {\"admin_kubeconfig\":\"/home/jenkins/.kube/config\",\"namespace\":\"test-oadp-401\",\"non_admin_user\":false,\"storage_class\":\"odf-operator-cephfs\",\"use_role\":\"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt\",\"user_kubeconfig\":\"/home/jenkins/.kube/config\",\"vm_disk_pvc_volume_mode\":\"Filesystem\",\"with_deploy\":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml\n\nexit status 2", err: <*errors.Error | 0xc000ea2000>{ context: "(DefaultExecute::Execute)", message: "Error during command execution: ansible-playbook error: one or more host failed\n\nCommand executed: /usr/local/bin/ansible-playbook --extra-vars {\"admin_kubeconfig\":\"/home/jenkins/.kube/config\",\"namespace\":\"test-oadp-401\",\"non_admin_user\":false,\"storage_class\":\"odf-operator-cephfs\",\"use_role\":\"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt\",\"user_kubeconfig\":\"/home/jenkins/.kube/config\",\"vm_disk_pvc_volume_mode\":\"Filesystem\",\"with_deploy\":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml\n\nexit status 2", wrappedErrors: nil, }, } occurred In [It] at: /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:37:24.013 There were additional failures detected. To view them in detail run ginkgo -vv ------------------------------ [AfterSuite]  /alabama/cspi/e2e/kubevirt-plugin/kubevirt_suite_test.go:105 > Enter [AfterSuite] TOP-LEVEL @ 01/14/26 16:37:38.311 < Exit [AfterSuite] TOP-LEVEL @ 01/14/26 16:37:38.325 (14ms) [AfterSuite] PASSED [0.014 seconds] ------------------------------ [ReportAfterSuite] Autogenerated ReportAfterSuite for --junit-report autogenerated by Ginkgo > Enter [ReportAfterSuite] TOP-LEVEL @ 01/14/26 16:37:38.325 < Exit [ReportAfterSuite] TOP-LEVEL @ 01/14/26 16:37:38.33 (5ms) [ReportAfterSuite] PASSED [0.005 seconds] ------------------------------ Summarizing 4 Failures: [FAIL] CSI: Backup/Restore Openshift Virtualization Workloads  [It] [tc-id:OADP-185] [kubevirt] [Tier0] [vf:vm] Backing up started VM should succeed /alabama/cspi/test_common/backup_restore_app_case.go:49 [FAIL] CSI: Backup/Restore Openshift Virtualization Workloads  [It] [tc-id:OADP-186] [kubevirt] [Tier0] [vf:vm] Stopped VM should be restored /alabama/cspi/test_common/backup_restore_app_case.go:49 [FAIL] CSI: Backup/Restore Openshift Virtualization Workloads  [It] [tc-id:OADP-187] [kubevirt] [Tier0] [vf:vm] Backup-restore data volume /alabama/cspi/test_common/backup_restore_app_case.go:49 [FAIL] Native CSI Data Mover: Backup/Restore Openshift Virtualization Workloads  [It] [tc-id:OADP-401] [kubevirt] [Tier1] Started VM should over ceph filesytem mode /alabama/cspi/test_common/backup_restore_app_case.go:49 Ran 4 of 5 Specs in 935.742 seconds FAIL! -- 0 Passed | 4 Failed | 0 Pending | 1 Skipped --- FAIL: TestOADPE2E (935.75s) FAIL Ginkgo ran 1 suite in 16m22.896809211s Test Suite Failed + readonly 'RED=\e[31m' + RED='\e[31m' + readonly 'BLUE=\033[34m' + BLUE='\033[34m' + readonly 'CLEAR=\e[39m' + CLEAR='\e[39m' ++ oc get infrastructures cluster -o 'jsonpath={.status.platform}' ++ awk '{print tolower($0)}' + CLOUD_PROVIDER=aws + [[ '' == \t\r\u\e ]] + echo /home/jenkins/.kube/config /home/jenkins/.kube/config + [[ aws == *-arm* ]] + [[ aws == *-fips* ]] + E2E_TIMEOUT_MULTIPLIER=2 + export NAMESPACE=openshift-adp + NAMESPACE=openshift-adp + export PROVIDER=aws + PROVIDER=aws ++ echo aws ++ awk '{print tolower($0)}' + BACKUP_LOCATION=aws + export BACKUP_LOCATION=aws + BACKUP_LOCATION=aws + export BUCKET=ci-op-98rbwkjv-interopoadp + BUCKET=ci-op-98rbwkjv-interopoadp + OADP_CREDS_FILE=/tmp/test-settings/credentials + OADP_VSL_CREDS_FILE=/tmp/test-settings/aws_vsl_creds +++ readlink -f /alabama/cspi/test_settings/scripts/test_runner.sh ++ dirname /alabama/cspi/test_settings/scripts/test_runner.sh + readonly SCRIPT_DIR=/alabama/cspi/test_settings/scripts + SCRIPT_DIR=/alabama/cspi/test_settings/scripts ++ cd /alabama/cspi/test_settings/scripts ++ git rev-parse --show-toplevel + readonly TOP_DIR=/alabama/cspi + TOP_DIR=/alabama/cspi + echo /alabama/cspi /alabama/cspi + TESTS_FOLDER=/alabama/cspi/e2e ++ oc get nodes -o 'jsonpath={.items[*].metadata.labels.kubernetes\.io/arch}' ++ tr ' ' '\n' ++ sort -u ++ xargs + export NODES_ARCHITECTURE=amd64 + NODES_ARCHITECTURE=amd64 + export OADP_REPOSITORY=redhat + OADP_REPOSITORY=redhat + SKIP_DPA_CREATION=false ++ oc get ns openshift-storage ++ echo true + OPENSHIFT_STORAGE=true + '[' redhat == upstream-velero ']' + '[' true == true ']' ++ oc get sc ++ awk '$1 ~ /^.+ceph-rbd$/ {print $1}' ++ tail -1 + CEPH_RBD_STORAGE_CLASS=odf-operator-ceph-rbd + '[' -n odf-operator-ceph-rbd ']' + export CEPH_RBD_STORAGE_CLASS + echo 'ceph-rbd StorageClass found: odf-operator-ceph-rbd' ceph-rbd StorageClass found: odf-operator-ceph-rbd ++ oc get storageclass -o 'jsonpath={range .items[*]}{@.metadata.name}{" "}{@.metadata.annotations.storageclass\.kubernetes\.io/is-default-class}{"\n"}{end}' ++ awk '$2=="true"{print $1}' ++ wc -l + NUM_DEFAULT_STORAGE_CLASS=1 + '[' 1 -ne 1 ']' ++ oc get storageclass -o 'jsonpath={.items[?(@.metadata.annotations.storageclass\.kubernetes\.io/is-default-class=='\''true'\'')].metadata.name}' + DEFAULT_SC=odf-operator-ceph-rbd + export STORAGE_CLASS=odf-operator-ceph-rbd + STORAGE_CLASS=odf-operator-ceph-rbd + '[' -n odf-operator-ceph-rbd ']' + '[' odf-operator-ceph-rbd '!=' odf-operator-ceph-rbd ']' + export STORAGE_CLASS_OPENSHIFT_STORAGE=odf-operator-ceph-rbd + STORAGE_CLASS_OPENSHIFT_STORAGE=odf-operator-ceph-rbd + echo 'Using the StorageClass for openshift-storage: odf-operator-ceph-rbd' Using the StorageClass for openshift-storage: odf-operator-ceph-rbd + [[ amd64 != \a\m\d\6\4 ]] + TEST_FILTER='!// || (// && !exclude_aws && (!/target/ || target_aws) ) ' + [[ aws =~ ^(osp|vsphere|gcp-wif|ibmcloud|azure-sts|aro-4) ]] + [[ aws =~ ^azure_sak ]] ++ oc config current-context ++ awk -F / '{print $2}' + SETTINGS_TMP=/alabama/cspi/output_files/api-ci-op-98rbwkjv-2c502-cspilp-interop-ccitredhat-com:6443 + mkdir -p /alabama/cspi/output_files/api-ci-op-98rbwkjv-2c502-cspilp-interop-ccitredhat-com:6443 ++ oc get authentication cluster -o 'jsonpath={.spec.serviceAccountIssuer}' + IS_OIDC= + '[' '!' -z ']' + [[ aws == \a\w\s ]] + export PROVIDER=aws + PROVIDER=aws + export CREDS_SECRET_REF=cloud-credentials + CREDS_SECRET_REF=cloud-credentials ++ oc get infrastructures cluster -o 'jsonpath={.status.platformStatus.aws.region}' --allow-missing-template-keys=false + export REGION=us-east-2 + REGION=us-east-2 + settings_script=aws_settings.sh + '[' aws == aws-sts ']' + BUCKET=ci-op-98rbwkjv-interopoadp + TMP_DIR=/alabama/cspi/output_files/api-ci-op-98rbwkjv-2c502-cspilp-interop-ccitredhat-com:6443 + source /alabama/cspi/test_settings/scripts/aws_settings.sh ++ cat ++ [[ aws == *aws* ]] ++ cat ++ echo -e '\n }\n}' +++ cat /alabama/cspi/output_files/api-ci-op-98rbwkjv-2c502-cspilp-interop-ccitredhat-com:6443/settings.json ++ x='{ "metadata": { "namespace": "openshift-adp" }, "spec": { "configuration":{ "velero":{ "defaultPlugins": [ "openshift", "aws" ] } }, "backupLocations": [ { "velero": { "provider": "aws", "default": true, "config": { "region": "us-east-2" }, "credential":{ "name": "cloud-credentials", "key": "cloud" }, "objectStorage":{ "bucket": "ci-op-98rbwkjv-interopoadp" } } } ] , "snapshotLocations": [ { "velero": { "provider": "aws", "config": { "profile": "default", "region": "us-east-2" } } } ] } }' ++ echo '{ "metadata": { "namespace": "openshift-adp" }, "spec": { "configuration":{ "velero":{ "defaultPlugins": [ "openshift", "aws" ] } }, "backupLocations": [ { "velero": { "provider": "aws", "default": true, "config": { "region": "us-east-2" }, "credential":{ "name": "cloud-credentials", "key": "cloud" }, "objectStorage":{ "bucket": "ci-op-98rbwkjv-interopoadp" } } } ] , "snapshotLocations": [ { "velero": { "provider": "aws", "config": { "profile": "default", "region": "us-east-2" } } } ] } }' ++ grep -o '^[^#]*' + FILE_SETTINGS_NAME=settings.json + printf '\033[34mGenerated settings file under /alabama/cspi/output_files/api-ci-op-98rbwkjv-2c502-cspilp-interop-ccitredhat-com:6443/settings.json\e[39m\n' Generated settings file under /alabama/cspi/output_files/api-ci-op-98rbwkjv-2c502-cspilp-interop-ccitredhat-com:6443/settings.json + cat /alabama/cspi/output_files/api-ci-op-98rbwkjv-2c502-cspilp-interop-ccitredhat-com:6443/settings.json ++ oc get volumesnapshotclass -o name + for i in $(oc get volumesnapshotclass -o name) + oc annotate volumesnapshotclass.snapshot.storage.k8s.io/csi-aws-vsc snapshot.storage.kubernetes.io/is-default-class- volumesnapshotclass.snapshot.storage.k8s.io/csi-aws-vsc annotated + for i in $(oc get volumesnapshotclass -o name) + oc annotate volumesnapshotclass.snapshot.storage.k8s.io/odf-operator-cephfsplugin-snapclass snapshot.storage.kubernetes.io/is-default-class- volumesnapshotclass.snapshot.storage.k8s.io/odf-operator-cephfsplugin-snapclass annotated + for i in $(oc get volumesnapshotclass -o name) + oc annotate volumesnapshotclass.snapshot.storage.k8s.io/odf-operator-rbdplugin-snapclass snapshot.storage.kubernetes.io/is-default-class- volumesnapshotclass.snapshot.storage.k8s.io/odf-operator-rbdplugin-snapclass annotated ++ ./e2e/must-gather/get-latest-build.sh + oc get configmaps -n default must-gather-image + UPSTREAM_VERSION=99.0.0 ++ oc get OperatorCondition -n openshift-adp -o 'jsonpath={.items[*].metadata.name}' ++ awk -F v '{print $2}' + OADP_VERSION=1.5.3 + '[' -z 1.5.3 ']' + '[' 1.5.3 == 99.0.0 ']' ++ oc get sub redhat-oadp-operator -n openshift-adp -o 'jsonpath={.spec.source}' + OADP_REPO=redhat-operators + '[' -z redhat-operators ']' + '[' redhat-operators == redhat-operators ']' + REGISTRY_PATH=registry.redhat.io/oadp/oadp-mustgather-rhel9: + TAG=1.5.3 + export MUST_GATHER_BUILD=registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 + MUST_GATHER_BUILD=registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 + echo registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 + export MUST_GATHER_BUILD=registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 + MUST_GATHER_BUILD=registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 + '[' -z registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 ']' + export NUM_OF_OADP_INSTANCES=1 + NUM_OF_OADP_INSTANCES=1 ++ echo --focus=interop ++ tr ' ' '\n' ++ grep '^--' ++ tr '\n' ' ' + GINKO_PARAM='--focus=interop ' ++ echo --focus=interop ++ tr ' ' '\n' ++ grep '^-' ++ grep -v '^--' ++ tr '\n' ' ' + TEST_PARAM= + ginkgo run --nodes=1 -mod=mod --show-node-events --flake-attempts 3 --junit-report=/logs/artifacts/junit_oadp_interop_results.xml '--label-filter=!// || (// && !exclude_aws && (!/target/ || target_aws) ) ' --focus=interop -p /alabama/cspi/e2e/ -- -credentials_file=/tmp/test-settings/credentials -vsl_credentials_file=/tmp/test-settings/aws_vsl_creds -oadp_namespace=openshift-adp -settings=/alabama/cspi/output_files/api-ci-op-98rbwkjv-2c502-cspilp-interop-ccitredhat-com:6443/settings.json -must_gather_image=registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 -timeout_multiplier=2 -skip_dpa_creation=false Ginkgo detected a version mismatch between the Ginkgo CLI and the version of Ginkgo imported by your packages: Ginkgo CLI Version: 2.27.5 Mismatched package versions found: 2.23.4 used by e2e Ginkgo will continue to attempt to run but you may see errors (including flag parsing errors) and should either update your go.mod or your version of the Ginkgo CLI to match. To install the matching version of the CLI run go install github.com/onsi/ginkgo/v2/ginkgo from a path that contains a go.mod file. Alternatively you can use go run github.com/onsi/ginkgo/v2/ginkgo from a path that contains a go.mod file to invoke the matching version of the Ginkgo CLI. If you are attempting to test multiple packages that each have a different version of the Ginkgo library with a single Ginkgo CLI that is currently unsupported.  2026/01/14 16:37:45 Setting up clients 2026/01/14 16:37:45 Getting default StorageClass... 2026/01/14 16:37:45 Checking default storage class count Run the command: oc get sc 2026/01/14 16:37:45 Got default StorageClass odf-operator-ceph-rbd 2026/01/14 16:37:45 oc get sc NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE gp2-csi ebs.csi.aws.com Delete WaitForFirstConsumer true 73m gp3-csi ebs.csi.aws.com Delete WaitForFirstConsumer true 73m odf-operator-ceph-rbd (default) openshift-storage.rbd.csi.ceph.com Delete Immediate true 22m odf-operator-ceph-rbd-virtualization openshift-storage.rbd.csi.ceph.com Delete Immediate true 22m odf-operator-cephfs openshift-storage.cephfs.csi.ceph.com Delete Immediate true 22m openshift-storage.noobaa.io openshift-storage.noobaa.io/obc Delete Immediate false 17m 2026/01/14 16:37:45 Using velero prefix: velero-e2e-5adbdce9-f167-11f0-950a-0a58ac1877e4 2026/01/14 16:37:45 Checking default storage class count Running Suite: OADP E2E Suite - /alabama/cspi/e2e ================================================= Random Seed: 1768408660 Will run 6 of 193 specs ------------------------------ [SynchronizedBeforeSuite]  /alabama/cspi/e2e/e2e_suite_test.go:85 > Enter [SynchronizedBeforeSuite] TOP-LEVEL @ 01/14/26 16:37:45.696 < Exit [SynchronizedBeforeSuite] TOP-LEVEL @ 01/14/26 16:37:45.7 (5ms) > Enter [SynchronizedBeforeSuite] TOP-LEVEL @ 01/14/26 16:37:45.701 2026/01/14 16:37:45 The VSL credentials file: /tmp/test-settings/aws_vsl_creds doesn't exists 2026/01/14 16:37:45 The error message is: stat /tmp/test-settings/aws_vsl_creds: no such file or directory < Exit [SynchronizedBeforeSuite] TOP-LEVEL @ 01/14/26 16:37:45.729 (28ms) [SynchronizedBeforeSuite] PASSED [0.033 seconds] ------------------------------ SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ Backup restore tests Application backup [tc-id:OADP-437][interop][smoke] [Tier0] [vf:fsb-kopia] MySQL application with filesystem, Kopia [mr-check] /alabama/cspi/e2e/app_backup/backup_restore.go:48 > Enter [BeforeEach] TOP-LEVEL @ 01/14/26 16:37:45.733 < Exit [BeforeEach] TOP-LEVEL @ 01/14/26 16:37:45.733 (0s) > Enter [BeforeEach] Backup restore tests @ 01/14/26 16:37:45.733 < Exit [BeforeEach] Backup restore tests @ 01/14/26 16:37:45.74 (6ms) > Enter [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:37:45.74 < Exit [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:37:45.74 (0s) > Enter [It] [tc-id:OADP-437][interop][smoke] [Tier0] [vf:fsb-kopia] MySQL application with filesystem, Kopia @ 01/14/26 16:37:45.74 2026/01/14 16:37:45 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/14/26 16:37:45.746 2026/01/14 16:37:45 kopia 2026/01/14 16:37:45 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "028c55d1-eee0-45d0-8329-df45798139f4", "resourceVersion": "84640", "generation": 1, "creationTimestamp": "2026-01-14T16:37:45Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:37:45Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:nodeAgent": { ".": {}, "f:enable": {}, "f:podConfig": { ".": {}, "f:resourceAllocations": {} }, "f:uploaderType": {} }, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift" ], "disableFsBackup": false }, "nodeAgent": { "enable": true, "podConfig": { "resourceAllocations": {} }, "uploaderType": "kopia" } }, "features": null, "logFormat": "text" }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/14/26 16:37:45.774 2026/01/14 16:37:45 Waiting for velero pod to be running 2026/01/14 16:37:50 pod: velero-54467cd776-mflfr is not yet running with status: {1 Pending [{PodReadyToStartContainers 1 True 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:37:47 +0000 UTC } {Initialized 1 True 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:37:50 +0000 UTC } {Ready 1 False 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:37:45 +0000 UTC ContainersNotReady containers with unready status: [velero]} {ContainersReady 1 False 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:37:45 +0000 UTC ContainersNotReady containers with unready status: [velero]} {PodScheduled 1 True 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:37:45 +0000 UTC }] 10.0.52.28 [{10.0.52.28}] 10.128.2.82 [{10.128.2.82}] 2026-01-14 16:37:45 +0000 UTC [{openshift-velero-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2026-01-14 16:37:46 +0000 UTC,FinishedAt:2026-01-14 16:37:46 +0000 UTC,ContainerID:cri-o://84dbac8412b44e468dd3e946067ba18108f01df1d475a3fd056fd9cd5fbad197,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:9b888c75a1ddf274b1f0aa053ddbbc9acc3270417712456ffa29ef81a883b38a registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:212589b9f5b75a043c9a7094cb681a39687795c5e78e625beb6d00960ea57e70 cri-o://84dbac8412b44e468dd3e946067ba18108f01df1d475a3fd056fd9cd5fbad197 0xc00131c678 map[cpu:{{500 -3} {} 500m DecimalSI} memory:{{134217728 0} {} BinarySI}] &ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{134217728 0} {} BinarySI},},Claims:[]ResourceClaim{},} [{plugins /target false } {kube-api-access-92spz /var/run/secrets/kubernetes.io/serviceaccount true 0xc000d823f0}] &ContainerUser{Linux:&LinuxContainerUser{UID:1000740000,GID:0,SupplementalGroups:[0 1000740000],},} [] } {velero-plugin-for-aws {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2026-01-14 16:37:47 +0000 UTC,FinishedAt:2026-01-14 16:37:47 +0000 UTC,ContainerID:cri-o://75ac510f42806463744a86711163595e4a99a25ba8406199e81579970aa313f7,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:2a099649d3d7101a0b6dff9f6372bc76903ba6ecb6578dbeef8a8b3853f48026 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:2a099649d3d7101a0b6dff9f6372bc76903ba6ecb6578dbeef8a8b3853f48026 cri-o://75ac510f42806463744a86711163595e4a99a25ba8406199e81579970aa313f7 0xc00131c6f9 map[cpu:{{500 -3} {} 500m DecimalSI} memory:{{134217728 0} {} BinarySI}] &ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{134217728 0} {} BinarySI},},Claims:[]ResourceClaim{},} [{plugins /target false } {kube-api-access-92spz /var/run/secrets/kubernetes.io/serviceaccount true 0xc000d82400}] &ContainerUser{Linux:&LinuxContainerUser{UID:1000740000,GID:0,SupplementalGroups:[0 1000740000],},} [] } {kubevirt-velero-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2026-01-14 16:37:48 +0000 UTC,FinishedAt:2026-01-14 16:37:48 +0000 UTC,ContainerID:cri-o://73cea675b2b2ff0aadfb86c85bfd3bf91783264fb5a278d705a127311ddaa99f,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:77d6057b1783872ec65d7d6a88d8a1212c38aacc982c7efd46ed2eb77b5e9e70 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:684020e409da37fd3a41935b51ac22e506fb7a45e787ae8c0cfd6ac0dfcbff09 cri-o://73cea675b2b2ff0aadfb86c85bfd3bf91783264fb5a278d705a127311ddaa99f 0xc00131c748 map[cpu:{{500 -3} {} 500m DecimalSI} memory:{{134217728 0} {} BinarySI}] &ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{134217728 0} {} BinarySI},},Claims:[]ResourceClaim{},} [{plugins /target false } {kube-api-access-92spz /var/run/secrets/kubernetes.io/serviceaccount true 0xc000d82410}] &ContainerUser{Linux:&LinuxContainerUser{UID:1000740000,GID:0,SupplementalGroups:[0 1000740000],},} [] } {hypershift-oadp-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2026-01-14 16:37:49 +0000 UTC,FinishedAt:2026-01-14 16:37:49 +0000 UTC,ContainerID:cri-o://0483b490c0ac9112a7f45a7badbe7d234d522ccd4119953d66f92a0643b895f5,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-hypershift-velero-plugin-rhel9@sha256:f8798364f94bb1bf4db264b0439c5a280b40cafa408c47b4160d9cc9c5d87e53 registry.redhat.io/oadp/oadp-hypershift-velero-plugin-rhel9@sha256:5d9c2ff91b5851f486fdcadcf9991007cdbe7a81f57774c6d0b3798091316af6 cri-o://0483b490c0ac9112a7f45a7badbe7d234d522ccd4119953d66f92a0643b895f5 0xc00131c7c9 map[cpu:{{500 -3} {} 500m DecimalSI} memory:{{134217728 0} {} BinarySI}] &ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{134217728 0} {} BinarySI},},Claims:[]ResourceClaim{},} [{plugins /target false } {kube-api-access-92spz /var/run/secrets/kubernetes.io/serviceaccount true 0xc000d82420}] &ContainerUser{Linux:&LinuxContainerUser{UID:1000740000,GID:0,SupplementalGroups:[0 1000740000],},} [] }] [{velero {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-rhel9@sha256:dd5897b1579599cfa8d8a03a1c8d3cf68e3927c4ba5dee4e33c2a2d4c9e6ed89 0xc00131c61f map[] nil [{plugins /plugins false } {scratch /scratch false } {certs /etc/ssl/certs false } {bound-sa-token /var/run/secrets/openshift/serviceaccount true 0xc000d823d0} {kube-api-access-92spz /var/run/secrets/kubernetes.io/serviceaccount true 0xc000d823e0}] nil [] }] Burstable [] []} 2026/01/14 16:37:55 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2026/01/14 16:37:55 DPA successfully reconciled STEP: Prepare backup resources, depending on the volumes backup type @ 01/14/26 16:37:55.805 2026/01/14 16:37:55 Checking for correct number of running NodeAgent pods... STEP: Installing application for case mysql @ 01/14/26 16:37:55.816 2026/01/14 16:37:55 🔄 [Attempt 1/3] Executing operation: Deploy application ocp-mysql 2026/01/14 16:37:55 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:37:59 2026-01-14 16:37:57,683 p=27261 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:37:57,683 p=27261 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:37:58,030 p=27261 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:37:58,030 p=27261 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:37:58,379 p=27261 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:37:58,380 p=27261 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:37:58,749 p=27261 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:37:58,750 p=27261 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:37:58,770 p=27261 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:37:58,771 p=27261 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:37:58,771 p=27261 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:37:58,794 p=27261 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:37:58,794 p=27261 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:37:58,812 p=27261 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:37:58,812 p=27261 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:37:59,242 p=27261 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:37:59,243 p=27261 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:37:59,243 p=27261 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:37:59,243 p=27261 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:37:59,268 p=27261 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:37:59,268 p=27261 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:37:59,269 p=27261 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:37:59,269 p=27261 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:37:59,269 p=27261 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-437-kopia 2026/01/14 16:37:59 2026/01/14 16:37:59 ❌ Operation 'Deploy application ocp-mysql' failed with non-retryable error: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-437-kopia","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql","user_kubeconfig":"/home/jenkins/.kube/config","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 [FAILED] in [It] - /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:37:59.453 < Exit [It] [tc-id:OADP-437][interop][smoke] [Tier0] [vf:fsb-kopia] MySQL application with filesystem, Kopia @ 01/14/26 16:37:59.453 (13.713s) > Enter [JustAfterEach] TOP-LEVEL @ 01/14/26 16:37:59.453 2026/01/14 16:37:59 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 STEP: Get the failed spec name @ 01/14/26 16:37:59.453 2026/01/14 16:37:59 The failed spec name is: Backup restore tests Application backup [tc-id:OADP-437][interop][smoke] [Tier0] [vf:fsb-kopia] MySQL application with filesystem, Kopia STEP: Create a folder for all must-gather files if it doesn't exists already @ 01/14/26 16:37:59.453 2026/01/14 16:37:59 The folder logs does not exists, creating new folder with the name: logs STEP: Create a folder for the failed spec if it doesn't exists already @ 01/14/26 16:37:59.453 2026/01/14 16:37:59 The folder logs/It_Backup_restore_tests_Application_backup_tc-id_OADP-437_interop_smoke_Tier0_vf_fsb-kopia_MySQL_application_with_filesystem_Kopia_mr-check does not exists, creating new folder with the name: logs/It_Backup_restore_tests_Application_backup_tc-id_OADP-437_interop_smoke_Tier0_vf_fsb-kopia_MySQL_application_with_filesystem_Kopia_mr-check STEP: Run must-gather because the spec failed @ 01/14/26 16:37:59.453 2026/01/14 16:37:59 Log the present working directory path:- /alabama/cspi/e2e 2026/01/14 16:37:59 [adm must-gather --dest-dir /alabama/cspi/e2e/logs/It_Backup_restore_tests_Application_backup_tc-id_OADP-437_interop_smoke_Tier0_vf_fsb-kopia_MySQL_application_with_filesystem_Kopia_mr-check --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3] 2026/01/14 16:38:10 Log all the files present in /alabama/cspi/e2e/logs directory 2026/01/14 16:38:10 It_Backup_restore_tests_Application_backup_tc-id_OADP-437_interop_smoke_Tier0_vf_fsb-kopia_MySQL_application_with_filesystem_Kopia_mr-check STEP: Find must-gather folder and rename it to a shorter more readable name @ 01/14/26 16:38:10.25 Secret deletion flag: true Deleting cloud credentials secret cloud-credentials in namespace openshift-adp < Exit [JustAfterEach] TOP-LEVEL @ 01/14/26 16:38:10.402 (10.949s) > Enter [DeferCleanup (Each)] Application backup @ 01/14/26 16:38:10.402 2026/01/14 16:38:10 Cleaning app 2026/01/14 16:38:10 Using admin kubeconfig for with_cleanup operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:38:14 2026-01-14 16:38:12,620 p=27485 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:38:12,620 p=27485 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:38:12,978 p=27485 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:38:12,979 p=27485 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:38:13,335 p=27485 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:38:13,336 p=27485 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:38:13,840 p=27485 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:38:13,841 p=27485 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:38:13,860 p=27485 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:38:13,861 p=27485 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:38:13,861 p=27485 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:38:13,883 p=27485 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:38:13,883 p=27485 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:38:13,900 p=27485 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:38:13,901 p=27485 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:38:14,338 p=27485 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:38:14,338 p=27485 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:38:14,339 p=27485 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:38:14,339 p=27485 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:38:14,363 p=27485 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:38:14,363 p=27485 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:38:14,364 p=27485 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:38:14,364 p=27485 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:38:14,364 p=27485 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-437-kopia 2026/01/14 16:38:14 < Exit [DeferCleanup (Each)] Application backup @ 01/14/26 16:38:14.555 (4.153s) > Enter [DeferCleanup (Each)] Application backup @ 01/14/26 16:38:14.555 2026/01/14 16:38:14 Cleaning setup resources for the backup < Exit [DeferCleanup (Each)] Application backup @ 01/14/26 16:38:14.555 (0s) > Enter [DeferCleanup (Each)] Application backup @ 01/14/26 16:38:14.555 2026/01/14 16:38:14 Deleting DPA CR < Exit [DeferCleanup (Each)] Application backup @ 01/14/26 16:38:14.565 (10ms) Attempt #1 Failed. Retrying ↺ @ 01/14/26 16:38:14.565 > Enter [BeforeEach] TOP-LEVEL @ 01/14/26 16:38:14.565 < Exit [BeforeEach] TOP-LEVEL @ 01/14/26 16:38:14.565 (0s) > Enter [BeforeEach] Backup restore tests @ 01/14/26 16:38:14.565 < Exit [BeforeEach] Backup restore tests @ 01/14/26 16:38:14.571 (6ms) > Enter [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:38:14.571 < Exit [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:38:14.571 (0s) > Enter [It] [tc-id:OADP-437][interop][smoke] [Tier0] [vf:fsb-kopia] MySQL application with filesystem, Kopia @ 01/14/26 16:38:14.571 2026/01/14 16:38:14 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/14/26 16:38:14.577 2026/01/14 16:38:14 kopia 2026/01/14 16:38:14 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "56ebea02-e41e-4daf-b89d-6d34157bfeaa", "resourceVersion": "85274", "generation": 1, "creationTimestamp": "2026-01-14T16:38:14Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:38:14Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:nodeAgent": { ".": {}, "f:enable": {}, "f:podConfig": { ".": {}, "f:resourceAllocations": {} }, "f:uploaderType": {} }, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift" ], "disableFsBackup": false }, "nodeAgent": { "enable": true, "podConfig": { "resourceAllocations": {} }, "uploaderType": "kopia" } }, "features": null, "logFormat": "text" }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/14/26 16:38:14.625 2026/01/14 16:38:14 Waiting for velero pod to be running 2026/01/14 16:38:14 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2026/01/14 16:38:14 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "56ebea02-e41e-4daf-b89d-6d34157bfeaa", "resourceVersion": "85274", "generation": 1, "creationTimestamp": "2026-01-14T16:38:14Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:38:14Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:nodeAgent": { ".": {}, "f:enable": {}, "f:podConfig": { ".": {}, "f:resourceAllocations": {} }, "f:uploaderType": {} }, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift" ], "disableFsBackup": false }, "nodeAgent": { "enable": true, "podConfig": { "resourceAllocations": {} }, "uploaderType": "kopia" } }, "features": null, "logFormat": "text" }, "status": {} } 2026/01/14 16:38:19 DPA successfully reconciled STEP: Prepare backup resources, depending on the volumes backup type @ 01/14/26 16:38:19.682 2026/01/14 16:38:19 Checking for correct number of running NodeAgent pods... STEP: Installing application for case mysql @ 01/14/26 16:38:19.695 2026/01/14 16:38:19 🔄 [Attempt 1/3] Executing operation: Deploy application ocp-mysql 2026/01/14 16:38:19 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:38:23 2026-01-14 16:38:21,894 p=27658 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:38:21,894 p=27658 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:38:22,247 p=27658 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:38:22,248 p=27658 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:38:22,612 p=27658 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:38:22,612 p=27658 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:38:23,108 p=27658 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:38:23,108 p=27658 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:38:23,130 p=27658 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:38:23,131 p=27658 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:38:23,131 p=27658 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:38:23,154 p=27658 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:38:23,154 p=27658 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:38:23,171 p=27658 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:38:23,171 p=27658 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:38:23,674 p=27658 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:38:23,674 p=27658 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:38:23,674 p=27658 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:38:23,675 p=27658 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:38:23,699 p=27658 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:38:23,700 p=27658 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:38:23,700 p=27658 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:38:23,701 p=27658 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:38:23,701 p=27658 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-437-kopia 2026/01/14 16:38:23 2026/01/14 16:38:23 ❌ Operation 'Deploy application ocp-mysql' failed with non-retryable error: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-437-kopia","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql","user_kubeconfig":"/home/jenkins/.kube/config","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 [FAILED] in [It] - /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:38:23.906 < Exit [It] [tc-id:OADP-437][interop][smoke] [Tier0] [vf:fsb-kopia] MySQL application with filesystem, Kopia @ 01/14/26 16:38:23.906 (9.335s) > Enter [JustAfterEach] TOP-LEVEL @ 01/14/26 16:38:23.906 2026/01/14 16:38:23 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 STEP: Get the failed spec name @ 01/14/26 16:38:23.906 2026/01/14 16:38:23 The failed spec name is: Backup restore tests Application backup [tc-id:OADP-437][interop][smoke] [Tier0] [vf:fsb-kopia] MySQL application with filesystem, Kopia STEP: Create a folder for all must-gather files if it doesn't exists already @ 01/14/26 16:38:23.906 STEP: Create a folder for the failed spec if it doesn't exists already @ 01/14/26 16:38:23.906 STEP: Run must-gather because the spec failed @ 01/14/26 16:38:23.906 2026/01/14 16:38:23 Log the present working directory path:- /alabama/cspi/e2e 2026/01/14 16:38:23 [adm must-gather --dest-dir /alabama/cspi/e2e/logs/It_Backup_restore_tests_Application_backup_tc-id_OADP-437_interop_smoke_Tier0_vf_fsb-kopia_MySQL_application_with_filesystem_Kopia_mr-check --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3] 2026/01/14 16:38:34 Log all the files present in /alabama/cspi/e2e/logs directory 2026/01/14 16:38:34 It_Backup_restore_tests_Application_backup_tc-id_OADP-437_interop_smoke_Tier0_vf_fsb-kopia_MySQL_application_with_filesystem_Kopia_mr-check STEP: Find must-gather folder and rename it to a shorter more readable name @ 01/14/26 16:38:34.984 The folder logs/It_Backup_restore_tests_Application_backup_tc-id_OADP-437_interop_smoke_Tier0_vf_fsb-kopia_MySQL_application_with_filesystem_Kopia_mr-check/must-gather already exists, skipping renaming the folder Secret deletion flag: true Deleting cloud credentials secret cloud-credentials in namespace openshift-adp < Exit [JustAfterEach] TOP-LEVEL @ 01/14/26 16:38:34.993 (11.087s) > Enter [DeferCleanup (Each)] Application backup @ 01/14/26 16:38:34.993 2026/01/14 16:38:34 Cleaning app 2026/01/14 16:38:34 Using admin kubeconfig for with_cleanup operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:38:39 2026-01-14 16:38:37,026 p=27870 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:38:37,026 p=27870 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:38:37,496 p=27870 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:38:37,496 p=27870 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:38:37,938 p=27870 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:38:37,938 p=27870 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:38:38,299 p=27870 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:38:38,300 p=27870 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:38:38,319 p=27870 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:38:38,319 p=27870 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:38:38,319 p=27870 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:38:38,342 p=27870 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:38:38,342 p=27870 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:38:38,359 p=27870 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:38:38,359 p=27870 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:38:38,942 p=27870 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:38:38,943 p=27870 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:38:38,944 p=27870 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:38:38,944 p=27870 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:38:38,986 p=27870 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:38:38,987 p=27870 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:38:38,988 p=27870 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:38:38,989 p=27870 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:38:38,989 p=27870 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-437-kopia 2026/01/14 16:38:39 < Exit [DeferCleanup (Each)] Application backup @ 01/14/26 16:38:39.226 (4.233s) > Enter [DeferCleanup (Each)] Application backup @ 01/14/26 16:38:39.226 2026/01/14 16:38:39 Cleaning setup resources for the backup < Exit [DeferCleanup (Each)] Application backup @ 01/14/26 16:38:39.226 (0s) > Enter [DeferCleanup (Each)] Application backup @ 01/14/26 16:38:39.226 2026/01/14 16:38:39 Deleting DPA CR < Exit [DeferCleanup (Each)] Application backup @ 01/14/26 16:38:39.237 (11ms) Attempt #2 Failed. Retrying ↺ @ 01/14/26 16:38:39.237 > Enter [BeforeEach] TOP-LEVEL @ 01/14/26 16:38:39.237 < Exit [BeforeEach] TOP-LEVEL @ 01/14/26 16:38:39.237 (0s) > Enter [BeforeEach] Backup restore tests @ 01/14/26 16:38:39.237 < Exit [BeforeEach] Backup restore tests @ 01/14/26 16:38:39.244 (6ms) > Enter [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:38:39.244 < Exit [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:38:39.244 (0s) > Enter [It] [tc-id:OADP-437][interop][smoke] [Tier0] [vf:fsb-kopia] MySQL application with filesystem, Kopia @ 01/14/26 16:38:39.244 2026/01/14 16:38:39 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/14/26 16:38:39.25 2026/01/14 16:38:39 kopia 2026/01/14 16:38:39 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "71372971-44e3-4737-bf33-b7a68daa135a", "resourceVersion": "85819", "generation": 1, "creationTimestamp": "2026-01-14T16:38:39Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:38:39Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:nodeAgent": { ".": {}, "f:enable": {}, "f:podConfig": { ".": {}, "f:resourceAllocations": {} }, "f:uploaderType": {} }, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift" ], "disableFsBackup": false }, "nodeAgent": { "enable": true, "podConfig": { "resourceAllocations": {} }, "uploaderType": "kopia" } }, "features": null, "logFormat": "text" }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/14/26 16:38:39.36 2026/01/14 16:38:39 Waiting for velero pod to be running 2026/01/14 16:38:39 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2026/01/14 16:38:39 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "71372971-44e3-4737-bf33-b7a68daa135a", "resourceVersion": "85819", "generation": 1, "creationTimestamp": "2026-01-14T16:38:39Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:38:39Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:nodeAgent": { ".": {}, "f:enable": {}, "f:podConfig": { ".": {}, "f:resourceAllocations": {} }, "f:uploaderType": {} }, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift" ], "disableFsBackup": false }, "nodeAgent": { "enable": true, "podConfig": { "resourceAllocations": {} }, "uploaderType": "kopia" } }, "features": null, "logFormat": "text" }, "status": {} } 2026/01/14 16:38:44 DPA successfully reconciled STEP: Prepare backup resources, depending on the volumes backup type @ 01/14/26 16:38:44.38 2026/01/14 16:38:44 Checking for correct number of running NodeAgent pods... STEP: Installing application for case mysql @ 01/14/26 16:38:44.392 2026/01/14 16:38:44 🔄 [Attempt 1/3] Executing operation: Deploy application ocp-mysql 2026/01/14 16:38:44 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:38:48 2026-01-14 16:38:46,342 p=28041 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:38:46,343 p=28041 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:38:46,824 p=28041 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:38:46,824 p=28041 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:38:47,184 p=28041 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:38:47,185 p=28041 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:38:47,543 p=28041 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:38:47,544 p=28041 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:38:47,563 p=28041 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:38:47,563 p=28041 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:38:47,563 p=28041 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:38:47,585 p=28041 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:38:47,585 p=28041 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:38:47,602 p=28041 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:38:47,602 p=28041 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:38:48,144 p=28041 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:38:48,144 p=28041 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:38:48,145 p=28041 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:38:48,145 p=28041 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:38:48,170 p=28041 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:38:48,171 p=28041 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:38:48,171 p=28041 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:38:48,172 p=28041 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:38:48,172 p=28041 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-437-kopia 2026/01/14 16:38:48 2026/01/14 16:38:48 ❌ Operation 'Deploy application ocp-mysql' failed with non-retryable error: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-437-kopia","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql","user_kubeconfig":"/home/jenkins/.kube/config","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 [FAILED] in [It] - /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:38:48.364 < Exit [It] [tc-id:OADP-437][interop][smoke] [Tier0] [vf:fsb-kopia] MySQL application with filesystem, Kopia @ 01/14/26 16:38:48.364 (9.12s) > Enter [JustAfterEach] TOP-LEVEL @ 01/14/26 16:38:48.364 2026/01/14 16:38:48 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 STEP: Get the failed spec name @ 01/14/26 16:38:48.364 2026/01/14 16:38:48 The failed spec name is: Backup restore tests Application backup [tc-id:OADP-437][interop][smoke] [Tier0] [vf:fsb-kopia] MySQL application with filesystem, Kopia STEP: Create a folder for all must-gather files if it doesn't exists already @ 01/14/26 16:38:48.364 STEP: Create a folder for the failed spec if it doesn't exists already @ 01/14/26 16:38:48.364 STEP: Run must-gather because the spec failed @ 01/14/26 16:38:48.364 2026/01/14 16:38:48 Log the present working directory path:- /alabama/cspi/e2e 2026/01/14 16:38:48 [adm must-gather --dest-dir /alabama/cspi/e2e/logs/It_Backup_restore_tests_Application_backup_tc-id_OADP-437_interop_smoke_Tier0_vf_fsb-kopia_MySQL_application_with_filesystem_Kopia_mr-check --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3] 2026/01/14 16:38:59 Log all the files present in /alabama/cspi/e2e/logs directory 2026/01/14 16:38:59 It_Backup_restore_tests_Application_backup_tc-id_OADP-437_interop_smoke_Tier0_vf_fsb-kopia_MySQL_application_with_filesystem_Kopia_mr-check STEP: Find must-gather folder and rename it to a shorter more readable name @ 01/14/26 16:38:59.393 The folder logs/It_Backup_restore_tests_Application_backup_tc-id_OADP-437_interop_smoke_Tier0_vf_fsb-kopia_MySQL_application_with_filesystem_Kopia_mr-check/must-gather already exists, skipping renaming the folder Secret deletion flag: true Deleting cloud credentials secret cloud-credentials in namespace openshift-adp < Exit [JustAfterEach] TOP-LEVEL @ 01/14/26 16:38:59.402 (11.038s) > Enter [DeferCleanup (Each)] Application backup @ 01/14/26 16:38:59.402 2026/01/14 16:38:59 Cleaning app 2026/01/14 16:38:59 Using admin kubeconfig for with_cleanup operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:39:03 2026-01-14 16:39:01,535 p=28251 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:39:01,535 p=28251 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:39:01,877 p=28251 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:39:01,877 p=28251 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:39:02,215 p=28251 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:39:02,215 p=28251 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:39:02,626 p=28251 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:39:02,626 p=28251 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:39:02,655 p=28251 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:39:02,656 p=28251 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:39:02,656 p=28251 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:39:02,689 p=28251 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:39:02,689 p=28251 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:39:02,716 p=28251 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:39:02,718 p=28251 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:39:03,210 p=28251 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:39:03,211 p=28251 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:39:03,211 p=28251 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:39:03,211 p=28251 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:39:03,234 p=28251 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:39:03,235 p=28251 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:39:03,235 p=28251 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:39:03,236 p=28251 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:39:03,236 p=28251 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-437-kopia 2026/01/14 16:39:03 < Exit [DeferCleanup (Each)] Application backup @ 01/14/26 16:39:03.411 (4.009s) > Enter [DeferCleanup (Each)] Application backup @ 01/14/26 16:39:03.411 2026/01/14 16:39:03 Cleaning setup resources for the backup < Exit [DeferCleanup (Each)] Application backup @ 01/14/26 16:39:03.411 (0s) > Enter [DeferCleanup (Each)] Application backup @ 01/14/26 16:39:03.411 2026/01/14 16:39:03 Deleting DPA CR < Exit [DeferCleanup (Each)] Application backup @ 01/14/26 16:39:03.42 (10ms) • [FAILED] [77.687 seconds] Backup restore tests Application backup [It] [tc-id:OADP-437][interop][smoke] [Tier0] [vf:fsb-kopia] MySQL application with filesystem, Kopia [mr-check] /alabama/cspi/e2e/app_backup/backup_restore.go:48 [FAILED] Unexpected error: <*errors.Error | 0xc000c5f400>: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-437-kopia","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql","user_kubeconfig":"/home/jenkins/.kube/config","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 { context: "(DefaultExecute::Execute)", message: "Error during command execution: ansible-playbook error: one or more host failed\n\nCommand executed: /usr/local/bin/ansible-playbook --extra-vars {\"admin_kubeconfig\":\"/home/jenkins/.kube/config\",\"namespace\":\"test-oadp-437-kopia\",\"non_admin_user\":false,\"use_role\":\"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql\",\"user_kubeconfig\":\"/home/jenkins/.kube/config\",\"with_deploy\":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml\n\nexit status 2", wrappedErrors: nil, } occurred In [It] at: /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:38:48.364 There were additional failures detected. To view them in detail run ginkgo -vv ------------------------------ S ------------------------------ Backup restore tests Application backup [tc-id:OADP-122] [interop] [skip-disconnected] [Tier0] [vf:csi-snapshots] Django application with BSL&CSI [exclude_aro-4] /alabama/cspi/e2e/app_backup/backup_restore.go:78 > Enter [BeforeEach] TOP-LEVEL @ 01/14/26 16:39:03.421 < Exit [BeforeEach] TOP-LEVEL @ 01/14/26 16:39:03.421 (0s) > Enter [BeforeEach] Backup restore tests @ 01/14/26 16:39:03.421 < Exit [BeforeEach] Backup restore tests @ 01/14/26 16:39:03.427 (6ms) > Enter [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:39:03.427 < Exit [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:39:03.427 (0s) > Enter [It] [tc-id:OADP-122] [interop] [skip-disconnected] [Tier0] [vf:csi-snapshots] Django application with BSL&CSI @ 01/14/26 16:39:03.427 2026/01/14 16:39:03 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/14/26 16:39:03.434 2026/01/14 16:39:03 csi 2026/01/14 16:39:03 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "365057ed-d9cd-4bf1-9f27-151139a9b962", "resourceVersion": "86385", "generation": 1, "creationTimestamp": "2026-01-14T16:39:03Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:39:03Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/14/26 16:39:03.51 2026/01/14 16:39:03 Waiting for velero pod to be running 2026/01/14 16:39:03 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2026/01/14 16:39:03 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "365057ed-d9cd-4bf1-9f27-151139a9b962", "resourceVersion": "86385", "generation": 1, "creationTimestamp": "2026-01-14T16:39:03Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:39:03Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } 2026/01/14 16:39:08 DPA successfully reconciled STEP: Prepare backup resources, depending on the volumes backup type @ 01/14/26 16:39:08.554 Run the command: oc get ns openshift-storage &> /dev/null && echo true || echo false 2026/01/14 16:39:08 The 'openshift-storage' namespace exists 2026/01/14 16:39:08 Checking default storage class count 2026/01/14 16:39:08 Using the CSI driver: openshift-storage.rbd.csi.ceph.com 2026/01/14 16:39:08 Snapclass 'example-snapclass' doesn't exist, creating 2026/01/14 16:39:08 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:39:08 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd STEP: Installing application for case django-persistent @ 01/14/26 16:39:08.794 2026/01/14 16:39:08 🔄 [Attempt 1/3] Executing operation: Deploy application ocp-django 2026/01/14 16:39:08 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:39:12 2026-01-14 16:39:10,669 p=28442 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:39:10,669 p=28442 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:39:11,004 p=28442 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:39:11,004 p=28442 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:39:11,335 p=28442 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:39:11,335 p=28442 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:39:11,672 p=28442 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:39:11,673 p=28442 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:39:11,691 p=28442 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:39:11,691 p=28442 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:39:11,691 p=28442 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:39:11,712 p=28442 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:39:11,712 p=28442 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:39:11,727 p=28442 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:39:11,728 p=28442 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:39:12,143 p=28442 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:39:12,144 p=28442 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:39:12,144 p=28442 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:39:12,144 p=28442 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:39:12,167 p=28442 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:39:12,168 p=28442 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:39:12,168 p=28442 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:39:12,168 p=28442 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:39:12,168 p=28442 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-122 2026/01/14 16:39:12 2026/01/14 16:39:12 ❌ Operation 'Deploy application ocp-django' failed with non-retryable error: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-122","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django","user_kubeconfig":"/home/jenkins/.kube/config","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 [FAILED] in [It] - /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:39:12.39 < Exit [It] [tc-id:OADP-122] [interop] [skip-disconnected] [Tier0] [vf:csi-snapshots] Django application with BSL&CSI @ 01/14/26 16:39:12.39 (8.963s) > Enter [JustAfterEach] TOP-LEVEL @ 01/14/26 16:39:12.39 2026/01/14 16:39:12 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 STEP: Get the failed spec name @ 01/14/26 16:39:12.391 2026/01/14 16:39:12 The failed spec name is: Backup restore tests Application backup [tc-id:OADP-122] [interop] [skip-disconnected] [Tier0] [vf:csi-snapshots] Django application with BSL&CSI STEP: Create a folder for all must-gather files if it doesn't exists already @ 01/14/26 16:39:12.391 STEP: Create a folder for the failed spec if it doesn't exists already @ 01/14/26 16:39:12.391 2026/01/14 16:39:12 The folder logs/It_Backup_restore_tests_Application_backup_tc-id_OADP-122_interop_skip-disconnected_Tier0_vf_csi-snapshots_Django_application_with_BSL_CSI_exclude_aro-4 does not exists, creating new folder with the name: logs/It_Backup_restore_tests_Application_backup_tc-id_OADP-122_interop_skip-disconnected_Tier0_vf_csi-snapshots_Django_application_with_BSL_CSI_exclude_aro-4 STEP: Run must-gather because the spec failed @ 01/14/26 16:39:12.391 2026/01/14 16:39:12 Log the present working directory path:- /alabama/cspi/e2e 2026/01/14 16:39:12 [adm must-gather --dest-dir /alabama/cspi/e2e/logs/It_Backup_restore_tests_Application_backup_tc-id_OADP-122_interop_skip-disconnected_Tier0_vf_csi-snapshots_Django_application_with_BSL_CSI_exclude_aro-4 --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3] 2026/01/14 16:39:23 Log all the files present in /alabama/cspi/e2e/logs directory 2026/01/14 16:39:23 It_Backup_restore_tests_Application_backup_tc-id_OADP-122_interop_skip-disconnected_Tier0_vf_csi-snapshots_Django_application_with_BSL_CSI_exclude_aro-4 2026/01/14 16:39:23 It_Backup_restore_tests_Application_backup_tc-id_OADP-437_interop_smoke_Tier0_vf_fsb-kopia_MySQL_application_with_filesystem_Kopia_mr-check STEP: Find must-gather folder and rename it to a shorter more readable name @ 01/14/26 16:39:23.153 Secret deletion flag: true Deleting cloud credentials secret cloud-credentials in namespace openshift-adp < Exit [JustAfterEach] TOP-LEVEL @ 01/14/26 16:39:23.163 (10.773s) > Enter [DeferCleanup (Each)] Application backup @ 01/14/26 16:39:23.163 2026/01/14 16:39:23 Reset number of visits to 0 2026/01/14 16:39:23 Cleaning app 2026/01/14 16:39:23 Using admin kubeconfig for with_cleanup operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:39:26 2026-01-14 16:39:24,926 p=28666 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:39:24,927 p=28666 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:39:25,268 p=28666 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:39:25,268 p=28666 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:39:25,601 p=28666 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:39:25,601 p=28666 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:39:25,926 p=28666 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:39:25,926 p=28666 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:39:25,944 p=28666 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:39:25,944 p=28666 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:39:25,944 p=28666 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:39:25,965 p=28666 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:39:25,965 p=28666 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:39:25,980 p=28666 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:39:25,980 p=28666 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:39:26,380 p=28666 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:39:26,380 p=28666 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:39:26,381 p=28666 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:39:26,381 p=28666 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:39:26,403 p=28666 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:39:26,404 p=28666 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:39:26,404 p=28666 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:39:26,404 p=28666 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:39:26,404 p=28666 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-122 2026/01/14 16:39:26 < Exit [DeferCleanup (Each)] Application backup @ 01/14/26 16:39:26.583 (3.419s) > Enter [DeferCleanup (Each)] Application backup @ 01/14/26 16:39:26.583 2026/01/14 16:39:26 Cleaning setup resources for the backup 2026/01/14 16:39:26 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:39:26 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2026/01/14 16:39:26 Deleting VolumeSnapshotClass 'example-snapclass' < Exit [DeferCleanup (Each)] Application backup @ 01/14/26 16:39:26.608 (26ms) > Enter [DeferCleanup (Each)] Application backup @ 01/14/26 16:39:26.608 2026/01/14 16:39:26 Deleting DPA CR < Exit [DeferCleanup (Each)] Application backup @ 01/14/26 16:39:26.619 (11ms) Attempt #1 Failed. Retrying ↺ @ 01/14/26 16:39:26.619 > Enter [BeforeEach] TOP-LEVEL @ 01/14/26 16:39:26.619 < Exit [BeforeEach] TOP-LEVEL @ 01/14/26 16:39:26.619 (0s) > Enter [BeforeEach] Backup restore tests @ 01/14/26 16:39:26.619 < Exit [BeforeEach] Backup restore tests @ 01/14/26 16:39:26.626 (7ms) > Enter [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:39:26.626 < Exit [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:39:26.626 (0s) > Enter [It] [tc-id:OADP-122] [interop] [skip-disconnected] [Tier0] [vf:csi-snapshots] Django application with BSL&CSI @ 01/14/26 16:39:26.626 2026/01/14 16:39:26 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/14/26 16:39:26.637 2026/01/14 16:39:26 csi 2026/01/14 16:39:26 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "0b46acaf-0f1f-4d5c-ba54-ca7b4bc347b4", "resourceVersion": "86908", "generation": 1, "creationTimestamp": "2026-01-14T16:39:26Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:39:26Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/14/26 16:39:26.684 2026/01/14 16:39:26 Waiting for velero pod to be running 2026/01/14 16:39:26 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2026/01/14 16:39:26 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "0b46acaf-0f1f-4d5c-ba54-ca7b4bc347b4", "resourceVersion": "86908", "generation": 1, "creationTimestamp": "2026-01-14T16:39:26Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:39:26Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } 2026/01/14 16:39:31 DPA successfully reconciled STEP: Prepare backup resources, depending on the volumes backup type @ 01/14/26 16:39:31.718 2026/01/14 16:39:31 Snapclass 'example-snapclass' doesn't exist, creating 2026/01/14 16:39:31 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:39:31 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd STEP: Installing application for case django-persistent @ 01/14/26 16:39:31.949 2026/01/14 16:39:31 🔄 [Attempt 1/3] Executing operation: Deploy application ocp-django 2026/01/14 16:39:31 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:39:35 2026-01-14 16:39:33,701 p=28847 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:39:33,701 p=28847 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:39:34,036 p=28847 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:39:34,036 p=28847 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:39:34,361 p=28847 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:39:34,361 p=28847 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:39:34,684 p=28847 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:39:34,684 p=28847 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:39:34,702 p=28847 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:39:34,703 p=28847 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:39:34,703 p=28847 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:39:34,723 p=28847 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:39:34,723 p=28847 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:39:34,739 p=28847 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:39:34,739 p=28847 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:39:35,142 p=28847 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:39:35,143 p=28847 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:39:35,143 p=28847 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:39:35,143 p=28847 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:39:35,166 p=28847 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:39:35,167 p=28847 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:39:35,167 p=28847 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:39:35,168 p=28847 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:39:35,168 p=28847 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-122 2026/01/14 16:39:35 2026/01/14 16:39:35 ❌ Operation 'Deploy application ocp-django' failed with non-retryable error: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-122","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django","user_kubeconfig":"/home/jenkins/.kube/config","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 [FAILED] in [It] - /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:39:35.374 < Exit [It] [tc-id:OADP-122] [interop] [skip-disconnected] [Tier0] [vf:csi-snapshots] Django application with BSL&CSI @ 01/14/26 16:39:35.374 (8.748s) > Enter [JustAfterEach] TOP-LEVEL @ 01/14/26 16:39:35.374 2026/01/14 16:39:35 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 STEP: Get the failed spec name @ 01/14/26 16:39:35.374 2026/01/14 16:39:35 The failed spec name is: Backup restore tests Application backup [tc-id:OADP-122] [interop] [skip-disconnected] [Tier0] [vf:csi-snapshots] Django application with BSL&CSI STEP: Create a folder for all must-gather files if it doesn't exists already @ 01/14/26 16:39:35.374 STEP: Create a folder for the failed spec if it doesn't exists already @ 01/14/26 16:39:35.374 STEP: Run must-gather because the spec failed @ 01/14/26 16:39:35.374 2026/01/14 16:39:35 Log the present working directory path:- /alabama/cspi/e2e 2026/01/14 16:39:35 [adm must-gather --dest-dir /alabama/cspi/e2e/logs/It_Backup_restore_tests_Application_backup_tc-id_OADP-122_interop_skip-disconnected_Tier0_vf_csi-snapshots_Django_application_with_BSL_CSI_exclude_aro-4 --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3] 2026/01/14 16:39:46 Log all the files present in /alabama/cspi/e2e/logs directory 2026/01/14 16:39:46 It_Backup_restore_tests_Application_backup_tc-id_OADP-122_interop_skip-disconnected_Tier0_vf_csi-snapshots_Django_application_with_BSL_CSI_exclude_aro-4 2026/01/14 16:39:46 It_Backup_restore_tests_Application_backup_tc-id_OADP-437_interop_smoke_Tier0_vf_fsb-kopia_MySQL_application_with_filesystem_Kopia_mr-check STEP: Find must-gather folder and rename it to a shorter more readable name @ 01/14/26 16:39:46.34 The folder logs/It_Backup_restore_tests_Application_backup_tc-id_OADP-122_interop_skip-disconnected_Tier0_vf_csi-snapshots_Django_application_with_BSL_CSI_exclude_aro-4/must-gather already exists, skipping renaming the folder Secret deletion flag: true Deleting cloud credentials secret cloud-credentials in namespace openshift-adp < Exit [JustAfterEach] TOP-LEVEL @ 01/14/26 16:39:46.349 (10.975s) > Enter [DeferCleanup (Each)] Application backup @ 01/14/26 16:39:46.349 2026/01/14 16:39:46 Reset number of visits to 0 2026/01/14 16:39:46 Cleaning app 2026/01/14 16:39:46 Using admin kubeconfig for with_cleanup operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:39:49 2026-01-14 16:39:48,101 p=29071 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:39:48,101 p=29071 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:39:48,433 p=29071 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:39:48,433 p=29071 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:39:48,764 p=29071 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:39:48,765 p=29071 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:39:49,098 p=29071 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:39:49,098 p=29071 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:39:49,117 p=29071 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:39:49,117 p=29071 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:39:49,117 p=29071 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:39:49,137 p=29071 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:39:49,137 p=29071 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:39:49,152 p=29071 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:39:49,153 p=29071 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:39:49,564 p=29071 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:39:49,565 p=29071 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:39:49,565 p=29071 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:39:49,565 p=29071 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:39:49,587 p=29071 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:39:49,588 p=29071 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:39:49,588 p=29071 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:39:49,589 p=29071 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:39:49,589 p=29071 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-122 2026/01/14 16:39:49 < Exit [DeferCleanup (Each)] Application backup @ 01/14/26 16:39:49.759 (3.41s) > Enter [DeferCleanup (Each)] Application backup @ 01/14/26 16:39:49.759 2026/01/14 16:39:49 Cleaning setup resources for the backup 2026/01/14 16:39:49 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:39:49 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2026/01/14 16:39:49 Deleting VolumeSnapshotClass 'example-snapclass' < Exit [DeferCleanup (Each)] Application backup @ 01/14/26 16:39:49.782 (23ms) > Enter [DeferCleanup (Each)] Application backup @ 01/14/26 16:39:49.782 2026/01/14 16:39:49 Deleting DPA CR < Exit [DeferCleanup (Each)] Application backup @ 01/14/26 16:39:49.795 (13ms) Attempt #2 Failed. Retrying ↺ @ 01/14/26 16:39:49.795 > Enter [BeforeEach] TOP-LEVEL @ 01/14/26 16:39:49.795 < Exit [BeforeEach] TOP-LEVEL @ 01/14/26 16:39:49.795 (0s) > Enter [BeforeEach] Backup restore tests @ 01/14/26 16:39:49.795 < Exit [BeforeEach] Backup restore tests @ 01/14/26 16:39:49.806 (11ms) > Enter [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:39:49.806 < Exit [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:39:49.806 (0s) > Enter [It] [tc-id:OADP-122] [interop] [skip-disconnected] [Tier0] [vf:csi-snapshots] Django application with BSL&CSI @ 01/14/26 16:39:49.806 2026/01/14 16:39:49 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/14/26 16:39:49.818 2026/01/14 16:39:49 csi 2026/01/14 16:39:49 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "14557cb3-411b-400c-a41d-d5123793b1be", "resourceVersion": "87391", "generation": 1, "creationTimestamp": "2026-01-14T16:39:49Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:39:49Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/14/26 16:39:49.901 2026/01/14 16:39:49 Waiting for velero pod to be running 2026/01/14 16:39:49 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2026/01/14 16:39:49 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "14557cb3-411b-400c-a41d-d5123793b1be", "resourceVersion": "87391", "generation": 1, "creationTimestamp": "2026-01-14T16:39:49Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:39:49Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } 2026/01/14 16:39:54 DPA successfully reconciled STEP: Prepare backup resources, depending on the volumes backup type @ 01/14/26 16:39:54.923 2026/01/14 16:39:54 Snapclass 'example-snapclass' doesn't exist, creating 2026/01/14 16:39:54 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:39:54 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd STEP: Installing application for case django-persistent @ 01/14/26 16:39:55.055 2026/01/14 16:39:55 🔄 [Attempt 1/3] Executing operation: Deploy application ocp-django 2026/01/14 16:39:55 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:39:58 2026-01-14 16:39:56,777 p=29250 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:39:56,778 p=29250 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:39:57,108 p=29250 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:39:57,108 p=29250 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:39:57,431 p=29250 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:39:57,432 p=29250 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:39:57,758 p=29250 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:39:57,758 p=29250 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:39:57,776 p=29250 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:39:57,777 p=29250 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:39:57,777 p=29250 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:39:57,797 p=29250 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:39:57,797 p=29250 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:39:57,812 p=29250 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:39:57,812 p=29250 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:39:58,227 p=29250 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:39:58,228 p=29250 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:39:58,228 p=29250 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:39:58,228 p=29250 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:39:58,252 p=29250 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:39:58,252 p=29250 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:39:58,253 p=29250 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:39:58,253 p=29250 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:39:58,253 p=29250 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-122 2026/01/14 16:39:58 2026/01/14 16:39:58 ❌ Operation 'Deploy application ocp-django' failed with non-retryable error: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-122","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django","user_kubeconfig":"/home/jenkins/.kube/config","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 [FAILED] in [It] - /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:39:58.431 < Exit [It] [tc-id:OADP-122] [interop] [skip-disconnected] [Tier0] [vf:csi-snapshots] Django application with BSL&CSI @ 01/14/26 16:39:58.431 (8.625s) > Enter [JustAfterEach] TOP-LEVEL @ 01/14/26 16:39:58.431 2026/01/14 16:39:58 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 STEP: Get the failed spec name @ 01/14/26 16:39:58.431 2026/01/14 16:39:58 The failed spec name is: Backup restore tests Application backup [tc-id:OADP-122] [interop] [skip-disconnected] [Tier0] [vf:csi-snapshots] Django application with BSL&CSI STEP: Create a folder for all must-gather files if it doesn't exists already @ 01/14/26 16:39:58.431 STEP: Create a folder for the failed spec if it doesn't exists already @ 01/14/26 16:39:58.431 STEP: Run must-gather because the spec failed @ 01/14/26 16:39:58.431 2026/01/14 16:39:58 Log the present working directory path:- /alabama/cspi/e2e 2026/01/14 16:39:58 [adm must-gather --dest-dir /alabama/cspi/e2e/logs/It_Backup_restore_tests_Application_backup_tc-id_OADP-122_interop_skip-disconnected_Tier0_vf_csi-snapshots_Django_application_with_BSL_CSI_exclude_aro-4 --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3] 2026/01/14 16:40:09 Log all the files present in /alabama/cspi/e2e/logs directory 2026/01/14 16:40:09 It_Backup_restore_tests_Application_backup_tc-id_OADP-122_interop_skip-disconnected_Tier0_vf_csi-snapshots_Django_application_with_BSL_CSI_exclude_aro-4 2026/01/14 16:40:09 It_Backup_restore_tests_Application_backup_tc-id_OADP-437_interop_smoke_Tier0_vf_fsb-kopia_MySQL_application_with_filesystem_Kopia_mr-check STEP: Find must-gather folder and rename it to a shorter more readable name @ 01/14/26 16:40:09.322 The folder logs/It_Backup_restore_tests_Application_backup_tc-id_OADP-122_interop_skip-disconnected_Tier0_vf_csi-snapshots_Django_application_with_BSL_CSI_exclude_aro-4/must-gather already exists, skipping renaming the folder Secret deletion flag: true Deleting cloud credentials secret cloud-credentials in namespace openshift-adp < Exit [JustAfterEach] TOP-LEVEL @ 01/14/26 16:40:09.331 (10.9s) > Enter [DeferCleanup (Each)] Application backup @ 01/14/26 16:40:09.331 2026/01/14 16:40:09 Reset number of visits to 0 2026/01/14 16:40:09 Cleaning app 2026/01/14 16:40:09 Using admin kubeconfig for with_cleanup operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:40:12 2026-01-14 16:40:11,058 p=29473 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:40:11,058 p=29473 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:40:11,382 p=29473 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:40:11,382 p=29473 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:40:11,707 p=29473 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:40:11,707 p=29473 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:40:12,029 p=29473 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:40:12,029 p=29473 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:40:12,048 p=29473 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:40:12,049 p=29473 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:40:12,049 p=29473 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:40:12,069 p=29473 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:40:12,070 p=29473 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:40:12,085 p=29473 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:40:12,085 p=29473 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:40:12,501 p=29473 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:40:12,502 p=29473 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:40:12,502 p=29473 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:40:12,502 p=29473 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:40:12,525 p=29473 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:40:12,525 p=29473 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:40:12,525 p=29473 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:40:12,526 p=29473 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:40:12,526 p=29473 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-122 2026/01/14 16:40:12 < Exit [DeferCleanup (Each)] Application backup @ 01/14/26 16:40:12.703 (3.372s) > Enter [DeferCleanup (Each)] Application backup @ 01/14/26 16:40:12.703 2026/01/14 16:40:12 Cleaning setup resources for the backup 2026/01/14 16:40:12 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:40:12 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2026/01/14 16:40:12 Deleting VolumeSnapshotClass 'example-snapclass' < Exit [DeferCleanup (Each)] Application backup @ 01/14/26 16:40:12.823 (120ms) > Enter [DeferCleanup (Each)] Application backup @ 01/14/26 16:40:12.823 2026/01/14 16:40:12 Deleting DPA CR < Exit [DeferCleanup (Each)] Application backup @ 01/14/26 16:40:12.836 (13ms) • [FAILED] [69.415 seconds] Backup restore tests Application backup [It] [tc-id:OADP-122] [interop] [skip-disconnected] [Tier0] [vf:csi-snapshots] Django application with BSL&CSI [exclude_aro-4] /alabama/cspi/e2e/app_backup/backup_restore.go:78 [FAILED] Unexpected error: <*errors.Error | 0xc0013a14c0>: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-122","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django","user_kubeconfig":"/home/jenkins/.kube/config","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 { context: "(DefaultExecute::Execute)", message: "Error during command execution: ansible-playbook error: one or more host failed\n\nCommand executed: /usr/local/bin/ansible-playbook --extra-vars {\"admin_kubeconfig\":\"/home/jenkins/.kube/config\",\"namespace\":\"test-oadp-122\",\"non_admin_user\":false,\"use_role\":\"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django\",\"user_kubeconfig\":\"/home/jenkins/.kube/config\",\"with_deploy\":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml\n\nexit status 2", wrappedErrors: nil, } occurred In [It] at: /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:39:58.431 There were additional failures detected. To view them in detail run ginkgo -vv ------------------------------ S ------------------------------ Backup restore tests Application backup [tc-id:OADP-352][interop][skip-disconnected][smoke] [Tier0] [vf:volume-snapshot-location] Django application with BSL&VSL [vsl] /alabama/cspi/e2e/app_backup/backup_restore.go:112 > Enter [BeforeEach] TOP-LEVEL @ 01/14/26 16:40:12.836 < Exit [BeforeEach] TOP-LEVEL @ 01/14/26 16:40:12.836 (0s) > Enter [BeforeEach] Backup restore tests @ 01/14/26 16:40:12.836 < Exit [BeforeEach] Backup restore tests @ 01/14/26 16:40:12.845 (9ms) > Enter [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:40:12.845 < Exit [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:40:12.845 (0s) > Enter [It] [tc-id:OADP-352][interop][skip-disconnected][smoke] [Tier0] [vf:volume-snapshot-location] Django application with BSL&VSL @ 01/14/26 16:40:12.845 2026/01/14 16:40:12 Check if VSL custom credentials exist 2026/01/14 16:40:12 Check if the cloud provider is AWS 2026/01/14 16:40:12 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/14/26 16:40:12.864 2026/01/14 16:40:12 vsl 2026/01/14 16:40:12 Check if VSL custom credentials exist 2026/01/14 16:40:12 Check if the cloud provider is AWS 2026/01/14 16:40:12 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "831381fb-a2b0-4622-beb8-f09851328f30", "resourceVersion": "87935", "generation": 1, "creationTimestamp": "2026-01-14T16:40:12Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:40:12Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [ { "velero": { "provider": "aws", "config": { "profile": "default", "region": "us-east-2" } } } ], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/14/26 16:40:12.943 2026/01/14 16:40:12 Waiting for velero pod to be running 2026/01/14 16:40:12 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2026/01/14 16:40:12 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "831381fb-a2b0-4622-beb8-f09851328f30", "resourceVersion": "87935", "generation": 1, "creationTimestamp": "2026-01-14T16:40:12Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:40:12Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [ { "velero": { "provider": "aws", "config": { "profile": "default", "region": "us-east-2" } } } ], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } 2026/01/14 16:40:17 DPA successfully reconciled STEP: Prepare backup resources, depending on the volumes backup type @ 01/14/26 16:40:17.964 2026/01/14 16:40:17 Checking default storage class count [SKIPPED] in [It] - /alabama/cspi/lib/backup.go:422 @ 01/14/26 16:40:17.973 < Exit [It] [tc-id:OADP-352][interop][skip-disconnected][smoke] [Tier0] [vf:volume-snapshot-location] Django application with BSL&VSL @ 01/14/26 16:40:17.973 (5.128s) > Enter [JustAfterEach] TOP-LEVEL @ 01/14/26 16:40:17.973 2026/01/14 16:40:17 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 Secret deletion flag: true Deleting cloud credentials secret cloud-credentials in namespace openshift-adp < Exit [JustAfterEach] TOP-LEVEL @ 01/14/26 16:40:17.982 (9ms) > Enter [DeferCleanup (Each)] Application backup @ 01/14/26 16:40:17.982 2026/01/14 16:40:17 Deleting DPA CR < Exit [DeferCleanup (Each)] Application backup @ 01/14/26 16:40:17.993 (10ms) S [SKIPPED] [5.156 seconds] Backup restore tests Application backup [It] [tc-id:OADP-352][interop][skip-disconnected][smoke] [Tier0] [vf:volume-snapshot-location] Django application with BSL&VSL [vsl] /alabama/cspi/e2e/app_backup/backup_restore.go:112 [SKIPPED] Skipping VSL test because the default StorageClass provisioner openshift-storage.rbd.csi.ceph.com is not supported In [It] at: /alabama/cspi/lib/backup.go:422 @ 01/14/26 16:40:17.973 ------------------------------ SSSSSSSSSSSS ------------------------------ [datamover] DataMover: Backup/Restore stateful application with CSI  [tc-id:OADP-439][interop] [Tier0] [vf:data-mover] MySQL application /alabama/cspi/e2e/app_backup/backup_restore_datamover.go:34 > Enter [BeforeEach] TOP-LEVEL @ 01/14/26 16:40:17.993 < Exit [BeforeEach] TOP-LEVEL @ 01/14/26 16:40:17.993 (0s) > Enter [BeforeEach] [datamover] DataMover: Backup/Restore stateful application with CSI @ 01/14/26 16:40:17.993 < Exit [BeforeEach] [datamover] DataMover: Backup/Restore stateful application with CSI @ 01/14/26 16:40:18 (7ms) > Enter [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:40:18 < Exit [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:40:18 (0s) > Enter [It] [tc-id:OADP-439][interop] [Tier0] [vf:data-mover] MySQL application @ 01/14/26 16:40:18 2026/01/14 16:40:18 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/14/26 16:40:18.01 2026/01/14 16:40:18 native-datamover 2026/01/14 16:40:18 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "976dc178-b8d9-4f79-bfd4-6ac6ca7efc0f", "resourceVersion": "88106", "generation": 1, "creationTimestamp": "2026-01-14T16:40:18Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:40:18Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:nodeAgent": { ".": {}, "f:enable": {}, "f:podConfig": { ".": {}, "f:resourceAllocations": {} }, "f:uploaderType": {} }, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false }, "nodeAgent": { "enable": true, "podConfig": { "resourceAllocations": {} }, "uploaderType": "kopia" } }, "features": null, "logFormat": "text" }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/14/26 16:40:18.176 2026/01/14 16:40:18 Waiting for velero pod to be running 2026/01/14 16:40:18 pod: velero-7d4f678c7b-mgbkf is not yet running with status: {1 Pending [{PodReadyToStartContainers 1 True 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:40:14 +0000 UTC } {Initialized 1 True 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:40:17 +0000 UTC } {Ready 1 False 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:40:13 +0000 UTC ContainersNotReady containers with unready status: [velero]} {ContainersReady 1 False 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:40:13 +0000 UTC ContainersNotReady containers with unready status: [velero]} {PodScheduled 1 True 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:40:13 +0000 UTC }] 10.0.52.28 [{10.0.52.28}] 10.128.2.91 [{10.128.2.91}] 2026-01-14 16:40:13 +0000 UTC [{openshift-velero-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2026-01-14 16:40:13 +0000 UTC,FinishedAt:2026-01-14 16:40:13 +0000 UTC,ContainerID:cri-o://290cd8dafbeb0bb1039d63076735a93770f4526d6933cc49e06ed133ea7a99e4,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:9b888c75a1ddf274b1f0aa053ddbbc9acc3270417712456ffa29ef81a883b38a registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:212589b9f5b75a043c9a7094cb681a39687795c5e78e625beb6d00960ea57e70 cri-o://290cd8dafbeb0bb1039d63076735a93770f4526d6933cc49e06ed133ea7a99e4 0xc00118bb88 map[cpu:{{500 -3} {} 500m DecimalSI} memory:{{134217728 0} {} BinarySI}] &ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{134217728 0} {} BinarySI},},Claims:[]ResourceClaim{},} [{plugins /target false } {kube-api-access-rdndg /var/run/secrets/kubernetes.io/serviceaccount true 0xc000d06f90}] &ContainerUser{Linux:&LinuxContainerUser{UID:1000740000,GID:0,SupplementalGroups:[0 1000740000],},} [] } {velero-plugin-for-aws {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2026-01-14 16:40:14 +0000 UTC,FinishedAt:2026-01-14 16:40:14 +0000 UTC,ContainerID:cri-o://ef4257123d3f18a4c7ed5264199a5d348e313535a35ca5e90b147d626cc50d02,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:2a099649d3d7101a0b6dff9f6372bc76903ba6ecb6578dbeef8a8b3853f48026 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:2a099649d3d7101a0b6dff9f6372bc76903ba6ecb6578dbeef8a8b3853f48026 cri-o://ef4257123d3f18a4c7ed5264199a5d348e313535a35ca5e90b147d626cc50d02 0xc00118bc09 map[cpu:{{500 -3} {} 500m DecimalSI} memory:{{134217728 0} {} BinarySI}] &ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{134217728 0} {} BinarySI},},Claims:[]ResourceClaim{},} [{plugins /target false } {kube-api-access-rdndg /var/run/secrets/kubernetes.io/serviceaccount true 0xc000d06fa0}] &ContainerUser{Linux:&LinuxContainerUser{UID:1000740000,GID:0,SupplementalGroups:[0 1000740000],},} [] } {kubevirt-velero-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2026-01-14 16:40:15 +0000 UTC,FinishedAt:2026-01-14 16:40:15 +0000 UTC,ContainerID:cri-o://d235089941125453f3ad6ea1bbfa36d892f57f98a20a7aa830115f5dedc4de77,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:77d6057b1783872ec65d7d6a88d8a1212c38aacc982c7efd46ed2eb77b5e9e70 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:684020e409da37fd3a41935b51ac22e506fb7a45e787ae8c0cfd6ac0dfcbff09 cri-o://d235089941125453f3ad6ea1bbfa36d892f57f98a20a7aa830115f5dedc4de77 0xc00118bc58 map[cpu:{{500 -3} {} 500m DecimalSI} memory:{{134217728 0} {} BinarySI}] &ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{134217728 0} {} BinarySI},},Claims:[]ResourceClaim{},} [{plugins /target false } {kube-api-access-rdndg /var/run/secrets/kubernetes.io/serviceaccount true 0xc000d06fb0}] &ContainerUser{Linux:&LinuxContainerUser{UID:1000740000,GID:0,SupplementalGroups:[0 1000740000],},} [] } {hypershift-oadp-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2026-01-14 16:40:16 +0000 UTC,FinishedAt:2026-01-14 16:40:16 +0000 UTC,ContainerID:cri-o://3b9eafb677d884ecf68d91324da122ae7dc6856c549016e34292f39eee1a4bdc,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-hypershift-velero-plugin-rhel9@sha256:f8798364f94bb1bf4db264b0439c5a280b40cafa408c47b4160d9cc9c5d87e53 registry.redhat.io/oadp/oadp-hypershift-velero-plugin-rhel9@sha256:5d9c2ff91b5851f486fdcadcf9991007cdbe7a81f57774c6d0b3798091316af6 cri-o://3b9eafb677d884ecf68d91324da122ae7dc6856c549016e34292f39eee1a4bdc 0xc00118bcd9 map[cpu:{{500 -3} {} 500m DecimalSI} memory:{{134217728 0} {} BinarySI}] &ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{134217728 0} {} BinarySI},},Claims:[]ResourceClaim{},} [{plugins /target false } {kube-api-access-rdndg /var/run/secrets/kubernetes.io/serviceaccount true 0xc000d06fc0}] &ContainerUser{Linux:&LinuxContainerUser{UID:1000740000,GID:0,SupplementalGroups:[0 1000740000],},} [] }] [{velero {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-rhel9@sha256:dd5897b1579599cfa8d8a03a1c8d3cf68e3927c4ba5dee4e33c2a2d4c9e6ed89 0xc00118bb1f map[] nil [{plugins /plugins false } {scratch /scratch false } {certs /etc/ssl/certs false } {bound-sa-token /var/run/secrets/openshift/serviceaccount true 0xc000d06f70} {cloud-credentials /credentials false } {kube-api-access-rdndg /var/run/secrets/kubernetes.io/serviceaccount true 0xc000d06f80}] nil [] }] Burstable [] []} 2026/01/14 16:40:23 pod: velero-6c5c458f58-bhrws is not yet running with status: {1 Pending [{PodReadyToStartContainers 1 True 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:40:19 +0000 UTC } {Initialized 1 True 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:40:22 +0000 UTC } {Ready 1 False 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:40:18 +0000 UTC ContainersNotReady containers with unready status: [velero]} {ContainersReady 1 False 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:40:18 +0000 UTC ContainersNotReady containers with unready status: [velero]} {PodScheduled 1 True 0001-01-01 00:00:00 +0000 UTC 2026-01-14 16:40:18 +0000 UTC }] 10.0.52.28 [{10.0.52.28}] 10.128.2.92 [{10.128.2.92}] 2026-01-14 16:40:18 +0000 UTC [{openshift-velero-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2026-01-14 16:40:18 +0000 UTC,FinishedAt:2026-01-14 16:40:19 +0000 UTC,ContainerID:cri-o://a590843846f260263e6250dc3e24afdbe1a8c7cf07fd323637bbe9edd80c3bf6,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:9b888c75a1ddf274b1f0aa053ddbbc9acc3270417712456ffa29ef81a883b38a registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:212589b9f5b75a043c9a7094cb681a39687795c5e78e625beb6d00960ea57e70 cri-o://a590843846f260263e6250dc3e24afdbe1a8c7cf07fd323637bbe9edd80c3bf6 0xc001314ae8 map[cpu:{{500 -3} {} 500m DecimalSI} memory:{{134217728 0} {} BinarySI}] &ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{134217728 0} {} BinarySI},},Claims:[]ResourceClaim{},} [{plugins /target false } {kube-api-access-p7twn /var/run/secrets/kubernetes.io/serviceaccount true 0xc000d07a50}] &ContainerUser{Linux:&LinuxContainerUser{UID:1000740000,GID:0,SupplementalGroups:[0 1000740000],},} [] } {velero-plugin-for-aws {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2026-01-14 16:40:19 +0000 UTC,FinishedAt:2026-01-14 16:40:19 +0000 UTC,ContainerID:cri-o://4d400cc1ab085fa031b2b440e30be5898f9685a8dbdb459ba79dd9d721b5b933,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:2a099649d3d7101a0b6dff9f6372bc76903ba6ecb6578dbeef8a8b3853f48026 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:2a099649d3d7101a0b6dff9f6372bc76903ba6ecb6578dbeef8a8b3853f48026 cri-o://4d400cc1ab085fa031b2b440e30be5898f9685a8dbdb459ba79dd9d721b5b933 0xc001314b69 map[cpu:{{500 -3} {} 500m DecimalSI} memory:{{134217728 0} {} BinarySI}] &ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{134217728 0} {} BinarySI},},Claims:[]ResourceClaim{},} [{plugins /target false } {kube-api-access-p7twn /var/run/secrets/kubernetes.io/serviceaccount true 0xc000d07a60}] &ContainerUser{Linux:&LinuxContainerUser{UID:1000740000,GID:0,SupplementalGroups:[0 1000740000],},} [] } {kubevirt-velero-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2026-01-14 16:40:20 +0000 UTC,FinishedAt:2026-01-14 16:40:20 +0000 UTC,ContainerID:cri-o://8afaa16f9d567c0b73e05d904240f82a1e2ac39782f2cccdeb1629ec30bfc030,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:77d6057b1783872ec65d7d6a88d8a1212c38aacc982c7efd46ed2eb77b5e9e70 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:684020e409da37fd3a41935b51ac22e506fb7a45e787ae8c0cfd6ac0dfcbff09 cri-o://8afaa16f9d567c0b73e05d904240f82a1e2ac39782f2cccdeb1629ec30bfc030 0xc001314bb8 map[cpu:{{500 -3} {} 500m DecimalSI} memory:{{134217728 0} {} BinarySI}] &ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{134217728 0} {} BinarySI},},Claims:[]ResourceClaim{},} [{plugins /target false } {kube-api-access-p7twn /var/run/secrets/kubernetes.io/serviceaccount true 0xc000d07a70}] &ContainerUser{Linux:&LinuxContainerUser{UID:1000740000,GID:0,SupplementalGroups:[0 1000740000],},} [] } {hypershift-oadp-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2026-01-14 16:40:21 +0000 UTC,FinishedAt:2026-01-14 16:40:21 +0000 UTC,ContainerID:cri-o://f36d8a6d807292d13cd79d6835952c3a127250f14060e52636e4c1f0f8f353cc,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-hypershift-velero-plugin-rhel9@sha256:f8798364f94bb1bf4db264b0439c5a280b40cafa408c47b4160d9cc9c5d87e53 registry.redhat.io/oadp/oadp-hypershift-velero-plugin-rhel9@sha256:5d9c2ff91b5851f486fdcadcf9991007cdbe7a81f57774c6d0b3798091316af6 cri-o://f36d8a6d807292d13cd79d6835952c3a127250f14060e52636e4c1f0f8f353cc 0xc001314c39 map[cpu:{{500 -3} {} 500m DecimalSI} memory:{{134217728 0} {} BinarySI}] &ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{134217728 0} {} BinarySI},},Claims:[]ResourceClaim{},} [{plugins /target false } {kube-api-access-p7twn /var/run/secrets/kubernetes.io/serviceaccount true 0xc000d07a80}] &ContainerUser{Linux:&LinuxContainerUser{UID:1000740000,GID:0,SupplementalGroups:[0 1000740000],},} [] }] [{velero {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-rhel9@sha256:dd5897b1579599cfa8d8a03a1c8d3cf68e3927c4ba5dee4e33c2a2d4c9e6ed89 0xc001314a8f map[] nil [{plugins /plugins false } {scratch /scratch false } {certs /etc/ssl/certs false } {bound-sa-token /var/run/secrets/openshift/serviceaccount true 0xc000d07a30} {kube-api-access-p7twn /var/run/secrets/kubernetes.io/serviceaccount true 0xc000d07a40}] nil [] }] Burstable [] []} 2026/01/14 16:40:28 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2026/01/14 16:40:28 DPA successfully reconciled STEP: Prepare backup resources, depending on the volumes backup type @ 01/14/26 16:40:28.211 Run the command: oc get ns openshift-storage &> /dev/null && echo true || echo false 2026/01/14 16:40:28 The 'openshift-storage' namespace exists 2026/01/14 16:40:28 Checking default storage class count 2026/01/14 16:40:28 Using the CSI driver: openshift-storage.rbd.csi.ceph.com 2026/01/14 16:40:28 Snapclass 'example-snapclass' doesn't exist, creating 2026/01/14 16:40:28 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:40:28 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2026/01/14 16:40:28 Checking for correct number of running NodeAgent pods... STEP: Installing application for case mysql @ 01/14/26 16:40:28.472 2026/01/14 16:40:28 🔄 [Attempt 1/3] Executing operation: Deploy application ocp-mysql 2026/01/14 16:40:28 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:40:31 2026-01-14 16:40:30,186 p=29673 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:40:30,186 p=29673 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:40:30,518 p=29673 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:40:30,518 p=29673 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:40:30,845 p=29673 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:40:30,845 p=29673 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:40:31,170 p=29673 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:40:31,170 p=29673 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:40:31,188 p=29673 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:40:31,188 p=29673 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:40:31,188 p=29673 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:40:31,208 p=29673 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:40:31,208 p=29673 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:40:31,222 p=29673 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:40:31,223 p=29673 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:40:31,623 p=29673 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:40:31,624 p=29673 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:40:31,624 p=29673 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:40:31,624 p=29673 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:40:31,646 p=29673 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:40:31,647 p=29673 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:40:31,647 p=29673 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:40:31,648 p=29673 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:40:31,648 p=29673 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-439 2026/01/14 16:40:31 2026/01/14 16:40:31 ❌ Operation 'Deploy application ocp-mysql' failed with non-retryable error: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-439","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql","user_kubeconfig":"/home/jenkins/.kube/config","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 [FAILED] in [It] - /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:40:31.814 < Exit [It] [tc-id:OADP-439][interop] [Tier0] [vf:data-mover] MySQL application @ 01/14/26 16:40:31.814 (13.814s) > Enter [JustAfterEach] TOP-LEVEL @ 01/14/26 16:40:31.814 2026/01/14 16:40:31 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 STEP: Get the failed spec name @ 01/14/26 16:40:31.814 2026/01/14 16:40:31 The failed spec name is: [datamover] DataMover: Backup/Restore stateful application with CSI [tc-id:OADP-439][interop] [Tier0] [vf:data-mover] MySQL application STEP: Create a folder for all must-gather files if it doesn't exists already @ 01/14/26 16:40:31.814 STEP: Create a folder for the failed spec if it doesn't exists already @ 01/14/26 16:40:31.814 2026/01/14 16:40:31 The folder logs/It_datamover_DataMover_Backup_Restore_stateful_application_with_CSI_tc-id_OADP-439_interop_Tier0_vf_data-mover_MySQL_application does not exists, creating new folder with the name: logs/It_datamover_DataMover_Backup_Restore_stateful_application_with_CSI_tc-id_OADP-439_interop_Tier0_vf_data-mover_MySQL_application STEP: Run must-gather because the spec failed @ 01/14/26 16:40:31.815 2026/01/14 16:40:31 Log the present working directory path:- /alabama/cspi/e2e 2026/01/14 16:40:31 [adm must-gather --dest-dir /alabama/cspi/e2e/logs/It_datamover_DataMover_Backup_Restore_stateful_application_with_CSI_tc-id_OADP-439_interop_Tier0_vf_data-mover_MySQL_application --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3] 2026/01/14 16:40:42 Log all the files present in /alabama/cspi/e2e/logs directory 2026/01/14 16:40:42 It_Backup_restore_tests_Application_backup_tc-id_OADP-122_interop_skip-disconnected_Tier0_vf_csi-snapshots_Django_application_with_BSL_CSI_exclude_aro-4 2026/01/14 16:40:42 It_Backup_restore_tests_Application_backup_tc-id_OADP-437_interop_smoke_Tier0_vf_fsb-kopia_MySQL_application_with_filesystem_Kopia_mr-check 2026/01/14 16:40:42 It_datamover_DataMover_Backup_Restore_stateful_application_with_CSI_tc-id_OADP-439_interop_Tier0_vf_data-mover_MySQL_application STEP: Find must-gather folder and rename it to a shorter more readable name @ 01/14/26 16:40:42.573 Secret deletion flag: true Deleting cloud credentials secret cloud-credentials in namespace openshift-adp < Exit [JustAfterEach] TOP-LEVEL @ 01/14/26 16:40:42.583 (10.769s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:40:42.584 2026/01/14 16:40:42 Cleaning app 2026/01/14 16:40:42 Using admin kubeconfig for with_cleanup operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:40:45 2026-01-14 16:40:44,309 p=29896 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:40:44,309 p=29896 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:40:44,639 p=29896 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:40:44,639 p=29896 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:40:44,989 p=29896 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:40:44,989 p=29896 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:40:45,317 p=29896 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:40:45,317 p=29896 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:40:45,336 p=29896 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:40:45,336 p=29896 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:40:45,336 p=29896 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:40:45,356 p=29896 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:40:45,357 p=29896 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:40:45,371 p=29896 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:40:45,372 p=29896 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:40:45,773 p=29896 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:40:45,773 p=29896 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:40:45,773 p=29896 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:40:45,774 p=29896 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:40:45,795 p=29896 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:40:45,796 p=29896 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:40:45,796 p=29896 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:40:45,797 p=29896 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:40:45,797 p=29896 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-439 2026/01/14 16:40:45 < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:40:45.967 (3.384s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:40:45.967 2026/01/14 16:40:45 Cleaning setup resources for the backup 2026/01/14 16:40:45 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:40:45 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2026/01/14 16:40:45 Deleting VolumeSnapshotClass 'example-snapclass' < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:40:45.992 (25ms) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:40:45.992 2026/01/14 16:40:45 Deleting DPA CR < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:40:46.016 (24ms) Attempt #1 Failed. Retrying ↺ @ 01/14/26 16:40:46.016 > Enter [BeforeEach] TOP-LEVEL @ 01/14/26 16:40:46.016 < Exit [BeforeEach] TOP-LEVEL @ 01/14/26 16:40:46.016 (0s) > Enter [BeforeEach] [datamover] DataMover: Backup/Restore stateful application with CSI @ 01/14/26 16:40:46.016 < Exit [BeforeEach] [datamover] DataMover: Backup/Restore stateful application with CSI @ 01/14/26 16:40:46.023 (7ms) > Enter [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:40:46.023 < Exit [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:40:46.023 (0s) > Enter [It] [tc-id:OADP-439][interop] [Tier0] [vf:data-mover] MySQL application @ 01/14/26 16:40:46.023 2026/01/14 16:40:46 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/14/26 16:40:46.03 2026/01/14 16:40:46 native-datamover 2026/01/14 16:40:46 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "96045d1d-6f17-4680-8a87-c7658822f51c", "resourceVersion": "88670", "generation": 1, "creationTimestamp": "2026-01-14T16:40:46Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:40:46Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:nodeAgent": { ".": {}, "f:enable": {}, "f:podConfig": { ".": {}, "f:resourceAllocations": {} }, "f:uploaderType": {} }, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false }, "nodeAgent": { "enable": true, "podConfig": { "resourceAllocations": {} }, "uploaderType": "kopia" } }, "features": null, "logFormat": "text" }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/14/26 16:40:46.092 2026/01/14 16:40:46 Waiting for velero pod to be running 2026/01/14 16:40:46 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2026/01/14 16:40:46 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "96045d1d-6f17-4680-8a87-c7658822f51c", "resourceVersion": "88670", "generation": 1, "creationTimestamp": "2026-01-14T16:40:46Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:40:46Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:nodeAgent": { ".": {}, "f:enable": {}, "f:podConfig": { ".": {}, "f:resourceAllocations": {} }, "f:uploaderType": {} }, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false }, "nodeAgent": { "enable": true, "podConfig": { "resourceAllocations": {} }, "uploaderType": "kopia" } }, "features": null, "logFormat": "text" }, "status": {} } 2026/01/14 16:40:51 DPA successfully reconciled STEP: Prepare backup resources, depending on the volumes backup type @ 01/14/26 16:40:51.125 2026/01/14 16:40:51 Snapclass 'example-snapclass' doesn't exist, creating 2026/01/14 16:40:51 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:40:51 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2026/01/14 16:40:51 Checking for correct number of running NodeAgent pods... STEP: Installing application for case mysql @ 01/14/26 16:40:51.274 2026/01/14 16:40:51 🔄 [Attempt 1/3] Executing operation: Deploy application ocp-mysql 2026/01/14 16:40:51 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:40:54 2026-01-14 16:40:53,028 p=30075 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:40:53,028 p=30075 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:40:53,361 p=30075 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:40:53,361 p=30075 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:40:53,695 p=30075 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:40:53,695 p=30075 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:40:54,020 p=30075 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:40:54,020 p=30075 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:40:54,038 p=30075 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:40:54,038 p=30075 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:40:54,038 p=30075 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:40:54,059 p=30075 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:40:54,059 p=30075 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:40:54,074 p=30075 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:40:54,074 p=30075 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:40:54,481 p=30075 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:40:54,482 p=30075 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:40:54,482 p=30075 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:40:54,482 p=30075 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:40:54,506 p=30075 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:40:54,507 p=30075 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:40:54,507 p=30075 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:40:54,507 p=30075 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:40:54,507 p=30075 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-439 2026/01/14 16:40:54 2026/01/14 16:40:54 ❌ Operation 'Deploy application ocp-mysql' failed with non-retryable error: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-439","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql","user_kubeconfig":"/home/jenkins/.kube/config","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 [FAILED] in [It] - /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:40:54.681 < Exit [It] [tc-id:OADP-439][interop] [Tier0] [vf:data-mover] MySQL application @ 01/14/26 16:40:54.681 (8.658s) > Enter [JustAfterEach] TOP-LEVEL @ 01/14/26 16:40:54.681 2026/01/14 16:40:54 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 STEP: Get the failed spec name @ 01/14/26 16:40:54.681 2026/01/14 16:40:54 The failed spec name is: [datamover] DataMover: Backup/Restore stateful application with CSI [tc-id:OADP-439][interop] [Tier0] [vf:data-mover] MySQL application STEP: Create a folder for all must-gather files if it doesn't exists already @ 01/14/26 16:40:54.681 STEP: Create a folder for the failed spec if it doesn't exists already @ 01/14/26 16:40:54.682 STEP: Run must-gather because the spec failed @ 01/14/26 16:40:54.682 2026/01/14 16:40:54 Log the present working directory path:- /alabama/cspi/e2e 2026/01/14 16:40:54 [adm must-gather --dest-dir /alabama/cspi/e2e/logs/It_datamover_DataMover_Backup_Restore_stateful_application_with_CSI_tc-id_OADP-439_interop_Tier0_vf_data-mover_MySQL_application --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3] 2026/01/14 16:41:05 Log all the files present in /alabama/cspi/e2e/logs directory 2026/01/14 16:41:05 It_Backup_restore_tests_Application_backup_tc-id_OADP-122_interop_skip-disconnected_Tier0_vf_csi-snapshots_Django_application_with_BSL_CSI_exclude_aro-4 2026/01/14 16:41:05 It_Backup_restore_tests_Application_backup_tc-id_OADP-437_interop_smoke_Tier0_vf_fsb-kopia_MySQL_application_with_filesystem_Kopia_mr-check 2026/01/14 16:41:05 It_datamover_DataMover_Backup_Restore_stateful_application_with_CSI_tc-id_OADP-439_interop_Tier0_vf_data-mover_MySQL_application STEP: Find must-gather folder and rename it to a shorter more readable name @ 01/14/26 16:41:05.648 The folder logs/It_datamover_DataMover_Backup_Restore_stateful_application_with_CSI_tc-id_OADP-439_interop_Tier0_vf_data-mover_MySQL_application/must-gather already exists, skipping renaming the folder Secret deletion flag: true Deleting cloud credentials secret cloud-credentials in namespace openshift-adp < Exit [JustAfterEach] TOP-LEVEL @ 01/14/26 16:41:05.66 (10.979s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:41:05.66 2026/01/14 16:41:05 Cleaning app 2026/01/14 16:41:05 Using admin kubeconfig for with_cleanup operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:41:08 2026-01-14 16:41:07,391 p=30298 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:41:07,391 p=30298 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:41:07,716 p=30298 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:41:07,716 p=30298 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:41:08,047 p=30298 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:41:08,047 p=30298 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:41:08,377 p=30298 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:41:08,377 p=30298 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:41:08,395 p=30298 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:41:08,396 p=30298 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:41:08,396 p=30298 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:41:08,416 p=30298 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:41:08,417 p=30298 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:41:08,432 p=30298 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:41:08,432 p=30298 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:41:08,834 p=30298 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:41:08,835 p=30298 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:41:08,835 p=30298 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:41:08,835 p=30298 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:41:08,859 p=30298 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:41:08,859 p=30298 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:41:08,860 p=30298 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:41:08,860 p=30298 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:41:08,861 p=30298 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-439 2026/01/14 16:41:09 < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:41:09.036 (3.376s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:41:09.036 2026/01/14 16:41:09 Cleaning setup resources for the backup 2026/01/14 16:41:09 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:41:09 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2026/01/14 16:41:09 Deleting VolumeSnapshotClass 'example-snapclass' < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:41:09.061 (25ms) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:41:09.061 2026/01/14 16:41:09 Deleting DPA CR < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:41:09.074 (13ms) Attempt #2 Failed. Retrying ↺ @ 01/14/26 16:41:09.075 > Enter [BeforeEach] TOP-LEVEL @ 01/14/26 16:41:09.075 < Exit [BeforeEach] TOP-LEVEL @ 01/14/26 16:41:09.075 (0s) > Enter [BeforeEach] [datamover] DataMover: Backup/Restore stateful application with CSI @ 01/14/26 16:41:09.075 < Exit [BeforeEach] [datamover] DataMover: Backup/Restore stateful application with CSI @ 01/14/26 16:41:09.088 (13ms) > Enter [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:41:09.088 < Exit [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:41:09.088 (0s) > Enter [It] [tc-id:OADP-439][interop] [Tier0] [vf:data-mover] MySQL application @ 01/14/26 16:41:09.088 2026/01/14 16:41:09 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/14/26 16:41:09.095 2026/01/14 16:41:09 native-datamover 2026/01/14 16:41:09 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "46d6f568-4715-47bd-9031-9445d7020a86", "resourceVersion": "89285", "generation": 1, "creationTimestamp": "2026-01-14T16:41:09Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:41:09Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:nodeAgent": { ".": {}, "f:enable": {}, "f:podConfig": { ".": {}, "f:resourceAllocations": {} }, "f:uploaderType": {} }, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false }, "nodeAgent": { "enable": true, "podConfig": { "resourceAllocations": {} }, "uploaderType": "kopia" } }, "features": null, "logFormat": "text" }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/14/26 16:41:09.176 2026/01/14 16:41:09 Waiting for velero pod to be running 2026/01/14 16:41:09 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2026/01/14 16:41:09 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "46d6f568-4715-47bd-9031-9445d7020a86", "resourceVersion": "89285", "generation": 1, "creationTimestamp": "2026-01-14T16:41:09Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:41:09Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:nodeAgent": { ".": {}, "f:enable": {}, "f:podConfig": { ".": {}, "f:resourceAllocations": {} }, "f:uploaderType": {} }, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false }, "nodeAgent": { "enable": true, "podConfig": { "resourceAllocations": {} }, "uploaderType": "kopia" } }, "features": null, "logFormat": "text" }, "status": {} } 2026/01/14 16:41:14 DPA successfully reconciled STEP: Prepare backup resources, depending on the volumes backup type @ 01/14/26 16:41:14.199 2026/01/14 16:41:14 Snapclass 'example-snapclass' doesn't exist, creating 2026/01/14 16:41:14 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:41:14 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2026/01/14 16:41:14 Checking for correct number of running NodeAgent pods... STEP: Installing application for case mysql @ 01/14/26 16:41:14.343 2026/01/14 16:41:14 🔄 [Attempt 1/3] Executing operation: Deploy application ocp-mysql 2026/01/14 16:41:14 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:41:17 2026-01-14 16:41:16,056 p=30477 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:41:16,056 p=30477 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:41:16,387 p=30477 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:41:16,388 p=30477 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:41:16,712 p=30477 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:41:16,712 p=30477 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:41:17,041 p=30477 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:41:17,042 p=30477 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:41:17,060 p=30477 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:41:17,061 p=30477 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:41:17,061 p=30477 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:41:17,081 p=30477 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:41:17,081 p=30477 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:41:17,097 p=30477 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:41:17,097 p=30477 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:41:17,499 p=30477 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:41:17,500 p=30477 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:41:17,500 p=30477 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:41:17,500 p=30477 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:41:17,525 p=30477 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:41:17,525 p=30477 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:41:17,526 p=30477 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:41:17,526 p=30477 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:41:17,526 p=30477 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-439 2026/01/14 16:41:17 2026/01/14 16:41:17 ❌ Operation 'Deploy application ocp-mysql' failed with non-retryable error: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-439","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql","user_kubeconfig":"/home/jenkins/.kube/config","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 [FAILED] in [It] - /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:41:17.7 < Exit [It] [tc-id:OADP-439][interop] [Tier0] [vf:data-mover] MySQL application @ 01/14/26 16:41:17.7 (8.612s) > Enter [JustAfterEach] TOP-LEVEL @ 01/14/26 16:41:17.7 2026/01/14 16:41:17 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 STEP: Get the failed spec name @ 01/14/26 16:41:17.7 2026/01/14 16:41:17 The failed spec name is: [datamover] DataMover: Backup/Restore stateful application with CSI [tc-id:OADP-439][interop] [Tier0] [vf:data-mover] MySQL application STEP: Create a folder for all must-gather files if it doesn't exists already @ 01/14/26 16:41:17.7 STEP: Create a folder for the failed spec if it doesn't exists already @ 01/14/26 16:41:17.7 STEP: Run must-gather because the spec failed @ 01/14/26 16:41:17.7 2026/01/14 16:41:17 Log the present working directory path:- /alabama/cspi/e2e 2026/01/14 16:41:17 [adm must-gather --dest-dir /alabama/cspi/e2e/logs/It_datamover_DataMover_Backup_Restore_stateful_application_with_CSI_tc-id_OADP-439_interop_Tier0_vf_data-mover_MySQL_application --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3] 2026/01/14 16:41:28 Log all the files present in /alabama/cspi/e2e/logs directory 2026/01/14 16:41:28 It_Backup_restore_tests_Application_backup_tc-id_OADP-122_interop_skip-disconnected_Tier0_vf_csi-snapshots_Django_application_with_BSL_CSI_exclude_aro-4 2026/01/14 16:41:28 It_Backup_restore_tests_Application_backup_tc-id_OADP-437_interop_smoke_Tier0_vf_fsb-kopia_MySQL_application_with_filesystem_Kopia_mr-check 2026/01/14 16:41:28 It_datamover_DataMover_Backup_Restore_stateful_application_with_CSI_tc-id_OADP-439_interop_Tier0_vf_data-mover_MySQL_application STEP: Find must-gather folder and rename it to a shorter more readable name @ 01/14/26 16:41:28.71 The folder logs/It_datamover_DataMover_Backup_Restore_stateful_application_with_CSI_tc-id_OADP-439_interop_Tier0_vf_data-mover_MySQL_application/must-gather already exists, skipping renaming the folder Secret deletion flag: true Deleting cloud credentials secret cloud-credentials in namespace openshift-adp < Exit [JustAfterEach] TOP-LEVEL @ 01/14/26 16:41:28.719 (11.019s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:41:28.719 2026/01/14 16:41:28 Cleaning app 2026/01/14 16:41:28 Using admin kubeconfig for with_cleanup operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:41:31 2026-01-14 16:41:30,413 p=30702 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:41:30,413 p=30702 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:41:30,736 p=30702 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:41:30,736 p=30702 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:41:31,060 p=30702 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:41:31,060 p=30702 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:41:31,384 p=30702 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:41:31,385 p=30702 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:41:31,403 p=30702 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:41:31,403 p=30702 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:41:31,403 p=30702 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:41:31,423 p=30702 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:41:31,424 p=30702 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:41:31,438 p=30702 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:41:31,439 p=30702 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:41:31,844 p=30702 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:41:31,845 p=30702 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:41:31,845 p=30702 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:41:31,845 p=30702 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:41:31,867 p=30702 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:41:31,867 p=30702 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:41:31,868 p=30702 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:41:31,868 p=30702 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:41:31,868 p=30702 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-439 2026/01/14 16:41:32 < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:41:32.044 (3.325s) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:41:32.044 2026/01/14 16:41:32 Cleaning setup resources for the backup 2026/01/14 16:41:32 Setting new default StorageClass 'odf-operator-ceph-rbd' 2026/01/14 16:41:32 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2026/01/14 16:41:32 Deleting VolumeSnapshotClass 'example-snapclass' < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:41:32.066 (22ms) > Enter [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:41:32.066 2026/01/14 16:41:32 Deleting DPA CR < Exit [DeferCleanup (Each)] TOP-LEVEL @ 01/14/26 16:41:32.076 (10ms) • [FAILED] [74.083 seconds] [datamover] DataMover: Backup/Restore stateful application with CSI  [It] [tc-id:OADP-439][interop] [Tier0] [vf:data-mover] MySQL application /alabama/cspi/e2e/app_backup/backup_restore_datamover.go:34 [FAILED] Unexpected error: <*errors.Error | 0xc0012a1500>: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-439","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql","user_kubeconfig":"/home/jenkins/.kube/config","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 { context: "(DefaultExecute::Execute)", message: "Error during command execution: ansible-playbook error: one or more host failed\n\nCommand executed: /usr/local/bin/ansible-playbook --extra-vars {\"admin_kubeconfig\":\"/home/jenkins/.kube/config\",\"namespace\":\"test-oadp-439\",\"non_admin_user\":false,\"use_role\":\"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql\",\"user_kubeconfig\":\"/home/jenkins/.kube/config\",\"with_deploy\":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml\n\nexit status 2", wrappedErrors: nil, } occurred In [It] at: /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:41:17.7 There were additional failures detected. To view them in detail run ginkgo -vv ------------------------------ SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [skip-disconnected] Restore hooks tests Successful Init hook [tc-id:OADP-164][interop][smoke] [Tier0] MySQL app with Kopia /alabama/cspi/e2e/hooks/restore_hooks.go:134 > Enter [BeforeEach] TOP-LEVEL @ 01/14/26 16:41:32.077 < Exit [BeforeEach] TOP-LEVEL @ 01/14/26 16:41:32.077 (0s) > Enter [BeforeEach] [skip-disconnected] Restore hooks tests @ 01/14/26 16:41:32.077 < Exit [BeforeEach] [skip-disconnected] Restore hooks tests @ 01/14/26 16:41:32.087 (11ms) > Enter [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:41:32.087 < Exit [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:41:32.087 (0s) > Enter [It] [tc-id:OADP-164][interop][smoke] [Tier0] MySQL app with Kopia @ 01/14/26 16:41:32.087 2026/01/14 16:41:32 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/14/26 16:41:32.096 2026/01/14 16:41:32 kopia 2026/01/14 16:41:32 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "f48d8eff-1668-4aee-a96d-867579c3259e", "resourceVersion": "89852", "generation": 1, "creationTimestamp": "2026-01-14T16:41:32Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:41:32Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:nodeAgent": { ".": {}, "f:enable": {}, "f:podConfig": { ".": {}, "f:resourceAllocations": {} }, "f:uploaderType": {} }, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift" ], "disableFsBackup": false }, "nodeAgent": { "enable": true, "podConfig": { "resourceAllocations": {} }, "uploaderType": "kopia" } }, "features": null, "logFormat": "text" }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/14/26 16:41:32.191 2026/01/14 16:41:32 Waiting for velero pod to be running 2026/01/14 16:41:32 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2026/01/14 16:41:32 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "f48d8eff-1668-4aee-a96d-867579c3259e", "resourceVersion": "89852", "generation": 1, "creationTimestamp": "2026-01-14T16:41:32Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:41:32Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:nodeAgent": { ".": {}, "f:enable": {}, "f:podConfig": { ".": {}, "f:resourceAllocations": {} }, "f:uploaderType": {} }, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift" ], "disableFsBackup": false }, "nodeAgent": { "enable": true, "podConfig": { "resourceAllocations": {} }, "uploaderType": "kopia" } }, "features": null, "logFormat": "text" }, "status": {} } 2026/01/14 16:41:37 DPA successfully reconciled STEP: Prepare backup resources, depending on the volumes backup type @ 01/14/26 16:41:37.226 2026/01/14 16:41:37 Checking for correct number of running NodeAgent pods... STEP: Installing application for case mysql-hooks-e2e @ 01/14/26 16:41:37.243 2026/01/14 16:41:37 🔄 [Attempt 1/3] Executing operation: Deploy application ocp-mysql 2026/01/14 16:41:37 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:41:40 2026-01-14 16:41:38,946 p=30881 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:41:38,946 p=30881 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:41:39,270 p=30881 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:41:39,270 p=30881 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:41:39,594 p=30881 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:41:39,594 p=30881 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:41:39,917 p=30881 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:41:39,917 p=30881 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:41:39,935 p=30881 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:41:39,936 p=30881 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:41:39,936 p=30881 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:41:39,955 p=30881 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:41:39,956 p=30881 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:41:39,970 p=30881 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:41:39,971 p=30881 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:41:40,370 p=30881 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:41:40,371 p=30881 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:41:40,371 p=30881 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:41:40,371 p=30881 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:41:40,394 p=30881 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:41:40,395 p=30881 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:41:40,395 p=30881 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:41:40,395 p=30881 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:41:40,396 p=30881 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-164 2026/01/14 16:41:40 2026/01/14 16:41:40 ❌ Operation 'Deploy application ocp-mysql' failed with non-retryable error: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-164","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql","user_kubeconfig":"/home/jenkins/.kube/config","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 [FAILED] in [It] - /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:41:40.571 < Exit [It] [tc-id:OADP-164][interop][smoke] [Tier0] MySQL app with Kopia @ 01/14/26 16:41:40.571 (8.484s) > Enter [JustAfterEach] TOP-LEVEL @ 01/14/26 16:41:40.571 2026/01/14 16:41:40 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 STEP: Get the failed spec name @ 01/14/26 16:41:40.571 2026/01/14 16:41:40 The failed spec name is: [skip-disconnected] Restore hooks tests Successful Init hook [tc-id:OADP-164][interop][smoke] [Tier0] MySQL app with Kopia STEP: Create a folder for all must-gather files if it doesn't exists already @ 01/14/26 16:41:40.571 STEP: Create a folder for the failed spec if it doesn't exists already @ 01/14/26 16:41:40.571 2026/01/14 16:41:40 The folder logs/It_skip-disconnected_Restore_hooks_tests_Successful_Init_hook_tc-id_OADP-164_interop_smoke_Tier0_MySQL_app_with_Kopia does not exists, creating new folder with the name: logs/It_skip-disconnected_Restore_hooks_tests_Successful_Init_hook_tc-id_OADP-164_interop_smoke_Tier0_MySQL_app_with_Kopia STEP: Run must-gather because the spec failed @ 01/14/26 16:41:40.571 2026/01/14 16:41:40 Log the present working directory path:- /alabama/cspi/e2e 2026/01/14 16:41:40 [adm must-gather --dest-dir /alabama/cspi/e2e/logs/It_skip-disconnected_Restore_hooks_tests_Successful_Init_hook_tc-id_OADP-164_interop_smoke_Tier0_MySQL_app_with_Kopia --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3] 2026/01/14 16:41:51 Log all the files present in /alabama/cspi/e2e/logs directory 2026/01/14 16:41:51 It_Backup_restore_tests_Application_backup_tc-id_OADP-122_interop_skip-disconnected_Tier0_vf_csi-snapshots_Django_application_with_BSL_CSI_exclude_aro-4 2026/01/14 16:41:51 It_Backup_restore_tests_Application_backup_tc-id_OADP-437_interop_smoke_Tier0_vf_fsb-kopia_MySQL_application_with_filesystem_Kopia_mr-check 2026/01/14 16:41:51 It_datamover_DataMover_Backup_Restore_stateful_application_with_CSI_tc-id_OADP-439_interop_Tier0_vf_data-mover_MySQL_application 2026/01/14 16:41:51 It_skip-disconnected_Restore_hooks_tests_Successful_Init_hook_tc-id_OADP-164_interop_smoke_Tier0_MySQL_app_with_Kopia STEP: Find must-gather folder and rename it to a shorter more readable name @ 01/14/26 16:41:51.367 Secret deletion flag: true Deleting cloud credentials secret cloud-credentials in namespace openshift-adp < Exit [JustAfterEach] TOP-LEVEL @ 01/14/26 16:41:51.376 (10.805s) > Enter [DeferCleanup (Each)] Successful Init hook @ 01/14/26 16:41:51.376 2026/01/14 16:41:51 Cleaning app 2026/01/14 16:41:51 Using admin kubeconfig for with_cleanup operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:41:54 2026-01-14 16:41:53,098 p=31106 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:41:53,098 p=31106 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:41:53,423 p=31106 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:41:53,423 p=31106 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:41:53,741 p=31106 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:41:53,741 p=31106 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:41:54,057 p=31106 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:41:54,058 p=31106 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:41:54,076 p=31106 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:41:54,076 p=31106 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:41:54,076 p=31106 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:41:54,096 p=31106 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:41:54,096 p=31106 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:41:54,110 p=31106 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:41:54,111 p=31106 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:41:54,513 p=31106 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:41:54,514 p=31106 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:41:54,514 p=31106 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:41:54,514 p=31106 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:41:54,536 p=31106 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:41:54,537 p=31106 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:41:54,537 p=31106 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:41:54,538 p=31106 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:41:54,538 p=31106 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-164 2026/01/14 16:41:54 < Exit [DeferCleanup (Each)] Successful Init hook @ 01/14/26 16:41:54.691 (3.315s) > Enter [DeferCleanup (Each)] Successful Init hook @ 01/14/26 16:41:54.691 2026/01/14 16:41:54 Cleaning setup resources for the backup < Exit [DeferCleanup (Each)] Successful Init hook @ 01/14/26 16:41:54.691 (0s) > Enter [DeferCleanup (Each)] Successful Init hook @ 01/14/26 16:41:54.691 2026/01/14 16:41:54 Deleting DPA CR < Exit [DeferCleanup (Each)] Successful Init hook @ 01/14/26 16:41:54.701 (10ms) Attempt #1 Failed. Retrying ↺ @ 01/14/26 16:41:54.701 > Enter [BeforeEach] TOP-LEVEL @ 01/14/26 16:41:54.701 < Exit [BeforeEach] TOP-LEVEL @ 01/14/26 16:41:54.701 (0s) > Enter [BeforeEach] [skip-disconnected] Restore hooks tests @ 01/14/26 16:41:54.701 < Exit [BeforeEach] [skip-disconnected] Restore hooks tests @ 01/14/26 16:41:54.709 (8ms) > Enter [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:41:54.709 < Exit [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:41:54.709 (0s) > Enter [It] [tc-id:OADP-164][interop][smoke] [Tier0] MySQL app with Kopia @ 01/14/26 16:41:54.709 2026/01/14 16:41:54 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/14/26 16:41:54.715 2026/01/14 16:41:54 kopia 2026/01/14 16:41:54 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "7477d9a0-9a01-4d6f-86a0-f1b12c2ca933", "resourceVersion": "90384", "generation": 1, "creationTimestamp": "2026-01-14T16:41:54Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:41:54Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:nodeAgent": { ".": {}, "f:enable": {}, "f:podConfig": { ".": {}, "f:resourceAllocations": {} }, "f:uploaderType": {} }, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift" ], "disableFsBackup": false }, "nodeAgent": { "enable": true, "podConfig": { "resourceAllocations": {} }, "uploaderType": "kopia" } }, "features": null, "logFormat": "text" }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/14/26 16:41:54.791 2026/01/14 16:41:54 Waiting for velero pod to be running 2026/01/14 16:41:54 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2026/01/14 16:41:54 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "7477d9a0-9a01-4d6f-86a0-f1b12c2ca933", "resourceVersion": "90384", "generation": 1, "creationTimestamp": "2026-01-14T16:41:54Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:41:54Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:nodeAgent": { ".": {}, "f:enable": {}, "f:podConfig": { ".": {}, "f:resourceAllocations": {} }, "f:uploaderType": {} }, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift" ], "disableFsBackup": false }, "nodeAgent": { "enable": true, "podConfig": { "resourceAllocations": {} }, "uploaderType": "kopia" } }, "features": null, "logFormat": "text" }, "status": {} } 2026/01/14 16:41:59 DPA successfully reconciled STEP: Prepare backup resources, depending on the volumes backup type @ 01/14/26 16:41:59.833 2026/01/14 16:41:59 Checking for correct number of running NodeAgent pods... STEP: Installing application for case mysql-hooks-e2e @ 01/14/26 16:41:59.845 2026/01/14 16:41:59 🔄 [Attempt 1/3] Executing operation: Deploy application ocp-mysql 2026/01/14 16:41:59 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:42:03 2026-01-14 16:42:01,562 p=31284 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:42:01,562 p=31284 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:42:01,884 p=31284 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:42:01,884 p=31284 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:42:02,208 p=31284 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:42:02,208 p=31284 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:42:02,532 p=31284 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:42:02,532 p=31284 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:42:02,551 p=31284 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:42:02,551 p=31284 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:42:02,551 p=31284 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:42:02,572 p=31284 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:42:02,572 p=31284 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:42:02,586 p=31284 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:42:02,587 p=31284 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:42:02,992 p=31284 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:42:02,992 p=31284 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:42:02,993 p=31284 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:42:02,993 p=31284 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:42:03,015 p=31284 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:42:03,016 p=31284 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:42:03,016 p=31284 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:42:03,016 p=31284 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:42:03,016 p=31284 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-164 2026/01/14 16:42:03 2026/01/14 16:42:03 ❌ Operation 'Deploy application ocp-mysql' failed with non-retryable error: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-164","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql","user_kubeconfig":"/home/jenkins/.kube/config","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 [FAILED] in [It] - /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:42:03.215 < Exit [It] [tc-id:OADP-164][interop][smoke] [Tier0] MySQL app with Kopia @ 01/14/26 16:42:03.215 (8.506s) > Enter [JustAfterEach] TOP-LEVEL @ 01/14/26 16:42:03.215 2026/01/14 16:42:03 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 STEP: Get the failed spec name @ 01/14/26 16:42:03.215 2026/01/14 16:42:03 The failed spec name is: [skip-disconnected] Restore hooks tests Successful Init hook [tc-id:OADP-164][interop][smoke] [Tier0] MySQL app with Kopia STEP: Create a folder for all must-gather files if it doesn't exists already @ 01/14/26 16:42:03.215 STEP: Create a folder for the failed spec if it doesn't exists already @ 01/14/26 16:42:03.215 STEP: Run must-gather because the spec failed @ 01/14/26 16:42:03.215 2026/01/14 16:42:03 Log the present working directory path:- /alabama/cspi/e2e 2026/01/14 16:42:03 [adm must-gather --dest-dir /alabama/cspi/e2e/logs/It_skip-disconnected_Restore_hooks_tests_Successful_Init_hook_tc-id_OADP-164_interop_smoke_Tier0_MySQL_app_with_Kopia --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3] 2026/01/14 16:42:14 Log all the files present in /alabama/cspi/e2e/logs directory 2026/01/14 16:42:14 It_Backup_restore_tests_Application_backup_tc-id_OADP-122_interop_skip-disconnected_Tier0_vf_csi-snapshots_Django_application_with_BSL_CSI_exclude_aro-4 2026/01/14 16:42:14 It_Backup_restore_tests_Application_backup_tc-id_OADP-437_interop_smoke_Tier0_vf_fsb-kopia_MySQL_application_with_filesystem_Kopia_mr-check 2026/01/14 16:42:14 It_datamover_DataMover_Backup_Restore_stateful_application_with_CSI_tc-id_OADP-439_interop_Tier0_vf_data-mover_MySQL_application 2026/01/14 16:42:14 It_skip-disconnected_Restore_hooks_tests_Successful_Init_hook_tc-id_OADP-164_interop_smoke_Tier0_MySQL_app_with_Kopia STEP: Find must-gather folder and rename it to a shorter more readable name @ 01/14/26 16:42:14.166 The folder logs/It_skip-disconnected_Restore_hooks_tests_Successful_Init_hook_tc-id_OADP-164_interop_smoke_Tier0_MySQL_app_with_Kopia/must-gather already exists, skipping renaming the folder Secret deletion flag: true Deleting cloud credentials secret cloud-credentials in namespace openshift-adp < Exit [JustAfterEach] TOP-LEVEL @ 01/14/26 16:42:14.176 (10.961s) > Enter [DeferCleanup (Each)] Successful Init hook @ 01/14/26 16:42:14.176 2026/01/14 16:42:14 Cleaning app 2026/01/14 16:42:14 Using admin kubeconfig for with_cleanup operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:42:17 2026-01-14 16:42:15,873 p=31507 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:42:15,873 p=31507 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:42:16,189 p=31507 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:42:16,189 p=31507 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:42:16,510 p=31507 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:42:16,510 p=31507 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:42:16,830 p=31507 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:42:16,830 p=31507 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:42:16,848 p=31507 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:42:16,849 p=31507 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:42:16,849 p=31507 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:42:16,870 p=31507 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:42:16,871 p=31507 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:42:16,886 p=31507 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:42:16,887 p=31507 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:42:17,289 p=31507 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:42:17,289 p=31507 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:42:17,290 p=31507 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:42:17,290 p=31507 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:42:17,312 p=31507 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:42:17,313 p=31507 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:42:17,313 p=31507 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:42:17,313 p=31507 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:42:17,313 p=31507 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-164 2026/01/14 16:42:17 < Exit [DeferCleanup (Each)] Successful Init hook @ 01/14/26 16:42:17.471 (3.295s) > Enter [DeferCleanup (Each)] Successful Init hook @ 01/14/26 16:42:17.471 2026/01/14 16:42:17 Cleaning setup resources for the backup < Exit [DeferCleanup (Each)] Successful Init hook @ 01/14/26 16:42:17.471 (0s) > Enter [DeferCleanup (Each)] Successful Init hook @ 01/14/26 16:42:17.471 2026/01/14 16:42:17 Deleting DPA CR < Exit [DeferCleanup (Each)] Successful Init hook @ 01/14/26 16:42:17.48 (9ms) Attempt #2 Failed. Retrying ↺ @ 01/14/26 16:42:17.48 > Enter [BeforeEach] TOP-LEVEL @ 01/14/26 16:42:17.48 < Exit [BeforeEach] TOP-LEVEL @ 01/14/26 16:42:17.48 (0s) > Enter [BeforeEach] [skip-disconnected] Restore hooks tests @ 01/14/26 16:42:17.48 < Exit [BeforeEach] [skip-disconnected] Restore hooks tests @ 01/14/26 16:42:17.487 (6ms) > Enter [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:42:17.487 < Exit [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:42:17.487 (0s) > Enter [It] [tc-id:OADP-164][interop][smoke] [Tier0] MySQL app with Kopia @ 01/14/26 16:42:17.487 2026/01/14 16:42:17 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/14/26 16:42:17.493 2026/01/14 16:42:17 kopia 2026/01/14 16:42:17 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "801a844c-0f91-421b-bf04-81261b361b2c", "resourceVersion": "90979", "generation": 1, "creationTimestamp": "2026-01-14T16:42:17Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:42:17Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:nodeAgent": { ".": {}, "f:enable": {}, "f:podConfig": { ".": {}, "f:resourceAllocations": {} }, "f:uploaderType": {} }, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift" ], "disableFsBackup": false }, "nodeAgent": { "enable": true, "podConfig": { "resourceAllocations": {} }, "uploaderType": "kopia" } }, "features": null, "logFormat": "text" }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/14/26 16:42:17.553 2026/01/14 16:42:17 Waiting for velero pod to be running 2026/01/14 16:42:17 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2026/01/14 16:42:17 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "801a844c-0f91-421b-bf04-81261b361b2c", "resourceVersion": "90979", "generation": 1, "creationTimestamp": "2026-01-14T16:42:17Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:42:17Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:nodeAgent": { ".": {}, "f:enable": {}, "f:podConfig": { ".": {}, "f:resourceAllocations": {} }, "f:uploaderType": {} }, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift" ], "disableFsBackup": false }, "nodeAgent": { "enable": true, "podConfig": { "resourceAllocations": {} }, "uploaderType": "kopia" } }, "features": null, "logFormat": "text" }, "status": {} } 2026/01/14 16:42:22 DPA successfully reconciled STEP: Prepare backup resources, depending on the volumes backup type @ 01/14/26 16:42:22.589 2026/01/14 16:42:22 Checking for correct number of running NodeAgent pods... STEP: Installing application for case mysql-hooks-e2e @ 01/14/26 16:42:22.606 2026/01/14 16:42:22 🔄 [Attempt 1/3] Executing operation: Deploy application ocp-mysql 2026/01/14 16:42:22 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:42:25 2026-01-14 16:42:24,328 p=31686 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:42:24,328 p=31686 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:42:24,648 p=31686 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:42:24,648 p=31686 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:42:24,970 p=31686 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:42:24,971 p=31686 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:42:25,288 p=31686 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:42:25,288 p=31686 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:42:25,306 p=31686 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:42:25,306 p=31686 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:42:25,306 p=31686 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:42:25,326 p=31686 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:42:25,326 p=31686 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:42:25,341 p=31686 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:42:25,342 p=31686 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:42:25,740 p=31686 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:42:25,741 p=31686 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:42:25,741 p=31686 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:42:25,741 p=31686 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:42:25,763 p=31686 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:42:25,764 p=31686 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:42:25,764 p=31686 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:42:25,765 p=31686 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:42:25,765 p=31686 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-164 2026/01/14 16:42:25 2026/01/14 16:42:25 ❌ Operation 'Deploy application ocp-mysql' failed with non-retryable error: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-164","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql","user_kubeconfig":"/home/jenkins/.kube/config","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 [FAILED] in [It] - /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:42:25.935 < Exit [It] [tc-id:OADP-164][interop][smoke] [Tier0] MySQL app with Kopia @ 01/14/26 16:42:25.935 (8.448s) > Enter [JustAfterEach] TOP-LEVEL @ 01/14/26 16:42:25.935 2026/01/14 16:42:25 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 STEP: Get the failed spec name @ 01/14/26 16:42:25.935 2026/01/14 16:42:25 The failed spec name is: [skip-disconnected] Restore hooks tests Successful Init hook [tc-id:OADP-164][interop][smoke] [Tier0] MySQL app with Kopia STEP: Create a folder for all must-gather files if it doesn't exists already @ 01/14/26 16:42:25.935 STEP: Create a folder for the failed spec if it doesn't exists already @ 01/14/26 16:42:25.935 STEP: Run must-gather because the spec failed @ 01/14/26 16:42:25.935 2026/01/14 16:42:25 Log the present working directory path:- /alabama/cspi/e2e 2026/01/14 16:42:25 [adm must-gather --dest-dir /alabama/cspi/e2e/logs/It_skip-disconnected_Restore_hooks_tests_Successful_Init_hook_tc-id_OADP-164_interop_smoke_Tier0_MySQL_app_with_Kopia --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3] 2026/01/14 16:42:36 Log all the files present in /alabama/cspi/e2e/logs directory 2026/01/14 16:42:36 It_Backup_restore_tests_Application_backup_tc-id_OADP-122_interop_skip-disconnected_Tier0_vf_csi-snapshots_Django_application_with_BSL_CSI_exclude_aro-4 2026/01/14 16:42:36 It_Backup_restore_tests_Application_backup_tc-id_OADP-437_interop_smoke_Tier0_vf_fsb-kopia_MySQL_application_with_filesystem_Kopia_mr-check 2026/01/14 16:42:36 It_datamover_DataMover_Backup_Restore_stateful_application_with_CSI_tc-id_OADP-439_interop_Tier0_vf_data-mover_MySQL_application 2026/01/14 16:42:36 It_skip-disconnected_Restore_hooks_tests_Successful_Init_hook_tc-id_OADP-164_interop_smoke_Tier0_MySQL_app_with_Kopia STEP: Find must-gather folder and rename it to a shorter more readable name @ 01/14/26 16:42:36.909 The folder logs/It_skip-disconnected_Restore_hooks_tests_Successful_Init_hook_tc-id_OADP-164_interop_smoke_Tier0_MySQL_app_with_Kopia/must-gather already exists, skipping renaming the folder Secret deletion flag: true Deleting cloud credentials secret cloud-credentials in namespace openshift-adp < Exit [JustAfterEach] TOP-LEVEL @ 01/14/26 16:42:36.918 (10.983s) > Enter [DeferCleanup (Each)] Successful Init hook @ 01/14/26 16:42:36.918 2026/01/14 16:42:36 Cleaning app 2026/01/14 16:42:36 Using admin kubeconfig for with_cleanup operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:42:40 2026-01-14 16:42:38,614 p=31911 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:42:38,614 p=31911 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:42:38,931 p=31911 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:42:38,931 p=31911 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:42:39,241 p=31911 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:42:39,241 p=31911 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:42:39,560 p=31911 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:42:39,560 p=31911 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:42:39,578 p=31911 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:42:39,578 p=31911 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:42:39,579 p=31911 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:42:39,598 p=31911 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:42:39,598 p=31911 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:42:39,613 p=31911 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:42:39,614 p=31911 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:42:40,011 p=31911 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:42:40,011 p=31911 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:42:40,012 p=31911 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:42:40,012 p=31911 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:42:40,034 p=31911 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:42:40,034 p=31911 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:42:40,035 p=31911 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:42:40,035 p=31911 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:42:40,035 p=31911 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n test-oadp-164 2026/01/14 16:42:40 < Exit [DeferCleanup (Each)] Successful Init hook @ 01/14/26 16:42:40.204 (3.286s) > Enter [DeferCleanup (Each)] Successful Init hook @ 01/14/26 16:42:40.205 2026/01/14 16:42:40 Cleaning setup resources for the backup < Exit [DeferCleanup (Each)] Successful Init hook @ 01/14/26 16:42:40.205 (0s) > Enter [DeferCleanup (Each)] Successful Init hook @ 01/14/26 16:42:40.205 2026/01/14 16:42:40 Deleting DPA CR < Exit [DeferCleanup (Each)] Successful Init hook @ 01/14/26 16:42:40.215 (10ms) • [FAILED] [68.138 seconds] [skip-disconnected] Restore hooks tests Successful Init hook [It] [tc-id:OADP-164][interop][smoke] [Tier0] MySQL app with Kopia /alabama/cspi/e2e/hooks/restore_hooks.go:134 [FAILED] Unexpected error: <*errors.Error | 0xc0013a0000>: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"test-oadp-164","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql","user_kubeconfig":"/home/jenkins/.kube/config","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 { context: "(DefaultExecute::Execute)", message: "Error during command execution: ansible-playbook error: one or more host failed\n\nCommand executed: /usr/local/bin/ansible-playbook --extra-vars {\"admin_kubeconfig\":\"/home/jenkins/.kube/config\",\"namespace\":\"test-oadp-164\",\"non_admin_user\":false,\"use_role\":\"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql\",\"user_kubeconfig\":\"/home/jenkins/.kube/config\",\"with_deploy\":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml\n\nexit status 2", wrappedErrors: nil, } occurred In [It] at: /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:42:25.935 There were additional failures detected. To view them in detail run ginkgo -vv ------------------------------ SSSSSSSSSSSSSSS > Enter [ReportAfterEach] [upstream-velero] Credentials suite @ 01/14/26 16:42:40.215 < Exit [ReportAfterEach] [upstream-velero] Credentials suite @ 01/14/26 16:42:40.215 (0s) SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ Incremental backup restore tests Incremental restore pod count [tc-id:OADP-165][interop] [Tier0] [vf:restore-policies] Todolist app with CSI - policy: update /alabama/cspi/e2e/incremental_restore/backup_restore_incremental.go:94 > Enter [BeforeEach] TOP-LEVEL @ 01/14/26 16:42:40.216 < Exit [BeforeEach] TOP-LEVEL @ 01/14/26 16:42:40.216 (0s) > Enter [BeforeEach] Incremental backup restore tests @ 01/14/26 16:42:40.216 < Exit [BeforeEach] Incremental backup restore tests @ 01/14/26 16:42:40.222 (6ms) > Enter [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:42:40.222 < Exit [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:42:40.222 (0s) > Enter [It] [tc-id:OADP-165][interop] [Tier0] [vf:restore-policies] Todolist app with CSI - policy: update @ 01/14/26 16:42:40.222 2026/01/14 16:42:40 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/14/26 16:42:40.229 2026/01/14 16:42:40 csi 2026/01/14 16:42:40 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "89cc86cb-8204-4fdc-b589-3910f213ebf4", "resourceVersion": "91484", "generation": 1, "creationTimestamp": "2026-01-14T16:42:40Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:42:40Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/14/26 16:42:40.309 2026/01/14 16:42:40 Waiting for velero pod to be running 2026/01/14 16:42:40 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2026/01/14 16:42:40 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "89cc86cb-8204-4fdc-b589-3910f213ebf4", "resourceVersion": "91484", "generation": 1, "creationTimestamp": "2026-01-14T16:42:40Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:42:40Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } 2026/01/14 16:42:45 DPA successfully reconciled STEP: Installing application for case todolist-backup @ 01/14/26 16:42:45.338 2026/01/14 16:42:45 🔄 [Attempt 1/3] Executing operation: Deploy application ocp-todolist-mariadb 2026/01/14 16:42:45 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:42:48 2026-01-14 16:42:47,067 p=32090 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:42:47,067 p=32090 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:42:47,394 p=32090 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:42:47,394 p=32090 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:42:47,717 p=32090 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:42:47,717 p=32090 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:42:48,040 p=32090 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:42:48,040 p=32090 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:42:48,058 p=32090 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:42:48,059 p=32090 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:42:48,059 p=32090 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:42:48,078 p=32090 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:42:48,079 p=32090 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:42:48,094 p=32090 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:42:48,094 p=32090 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:42:48,483 p=32090 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:42:48,484 p=32090 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:42:48,484 p=32090 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:42:48,484 p=32090 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:42:48,506 p=32090 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:42:48,507 p=32090 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:42:48,507 p=32090 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:42:48,507 p=32090 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:42:48,508 p=32090 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n todolist-mariadb-csi-policy-update 2026/01/14 16:42:48 2026/01/14 16:42:48 ❌ Operation 'Deploy application ocp-todolist-mariadb' failed with non-retryable error: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"todolist-mariadb-csi-policy-update","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-todolist-mariadb","user_kubeconfig":"/home/jenkins/.kube/config","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 [FAILED] in [It] - /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:42:48.686 < Exit [It] [tc-id:OADP-165][interop] [Tier0] [vf:restore-policies] Todolist app with CSI - policy: update @ 01/14/26 16:42:48.686 (8.464s) > Enter [JustAfterEach] TOP-LEVEL @ 01/14/26 16:42:48.686 2026/01/14 16:42:48 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 STEP: Get the failed spec name @ 01/14/26 16:42:48.687 2026/01/14 16:42:48 The failed spec name is: Incremental backup restore tests Incremental restore pod count [tc-id:OADP-165][interop] [Tier0] [vf:restore-policies] Todolist app with CSI - policy: update STEP: Create a folder for all must-gather files if it doesn't exists already @ 01/14/26 16:42:48.687 STEP: Create a folder for the failed spec if it doesn't exists already @ 01/14/26 16:42:48.687 2026/01/14 16:42:48 The folder logs/It_Incremental_backup_restore_tests_Incremental_restore_pod_count_tc-id_OADP-165_interop_Tier0_vf_restore-policies_Todolist_app_with_CSI_-_policy_update does not exists, creating new folder with the name: logs/It_Incremental_backup_restore_tests_Incremental_restore_pod_count_tc-id_OADP-165_interop_Tier0_vf_restore-policies_Todolist_app_with_CSI_-_policy_update STEP: Run must-gather because the spec failed @ 01/14/26 16:42:48.687 2026/01/14 16:42:48 Log the present working directory path:- /alabama/cspi/e2e 2026/01/14 16:42:48 [adm must-gather --dest-dir /alabama/cspi/e2e/logs/It_Incremental_backup_restore_tests_Incremental_restore_pod_count_tc-id_OADP-165_interop_Tier0_vf_restore-policies_Todolist_app_with_CSI_-_policy_update --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3] 2026/01/14 16:42:59 Log all the files present in /alabama/cspi/e2e/logs directory 2026/01/14 16:42:59 It_Backup_restore_tests_Application_backup_tc-id_OADP-122_interop_skip-disconnected_Tier0_vf_csi-snapshots_Django_application_with_BSL_CSI_exclude_aro-4 2026/01/14 16:42:59 It_Backup_restore_tests_Application_backup_tc-id_OADP-437_interop_smoke_Tier0_vf_fsb-kopia_MySQL_application_with_filesystem_Kopia_mr-check 2026/01/14 16:42:59 It_Incremental_backup_restore_tests_Incremental_restore_pod_count_tc-id_OADP-165_interop_Tier0_vf_restore-policies_Todolist_app_with_CSI_-_policy_update 2026/01/14 16:42:59 It_datamover_DataMover_Backup_Restore_stateful_application_with_CSI_tc-id_OADP-439_interop_Tier0_vf_data-mover_MySQL_application 2026/01/14 16:42:59 It_skip-disconnected_Restore_hooks_tests_Successful_Init_hook_tc-id_OADP-164_interop_smoke_Tier0_MySQL_app_with_Kopia STEP: Find must-gather folder and rename it to a shorter more readable name @ 01/14/26 16:42:59.48 Secret deletion flag: true Deleting cloud credentials secret cloud-credentials in namespace openshift-adp < Exit [JustAfterEach] TOP-LEVEL @ 01/14/26 16:42:59.489 (10.803s) > Enter [DeferCleanup (Each)] Incremental restore pod count @ 01/14/26 16:42:59.489 2026/01/14 16:42:59 Cleaning app 2026/01/14 16:42:59 Using admin kubeconfig for with_cleanup operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:43:02 2026-01-14 16:43:01,155 p=32311 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:43:01,155 p=32311 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:43:01,467 p=32311 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:43:01,467 p=32311 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:43:01,780 p=32311 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:43:01,780 p=32311 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:43:02,097 p=32311 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:43:02,097 p=32311 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:43:02,114 p=32311 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:43:02,115 p=32311 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:43:02,115 p=32311 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:43:02,134 p=32311 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:43:02,134 p=32311 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:43:02,149 p=32311 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:43:02,149 p=32311 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:43:02,553 p=32311 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:43:02,554 p=32311 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:43:02,554 p=32311 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:43:02,554 p=32311 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:43:02,576 p=32311 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:43:02,577 p=32311 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:43:02,577 p=32311 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:43:02,578 p=32311 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:43:02,578 p=32311 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n todolist-mariadb-csi-policy-update 2026/01/14 16:43:02 < Exit [DeferCleanup (Each)] Incremental restore pod count @ 01/14/26 16:43:02.743 (3.254s) > Enter [DeferCleanup (Each)] Incremental restore pod count @ 01/14/26 16:43:02.743 2026/01/14 16:43:02 Deleting DPA CR < Exit [DeferCleanup (Each)] Incremental restore pod count @ 01/14/26 16:43:02.753 (10ms) Attempt #1 Failed. Retrying ↺ @ 01/14/26 16:43:02.753 > Enter [BeforeEach] TOP-LEVEL @ 01/14/26 16:43:02.753 < Exit [BeforeEach] TOP-LEVEL @ 01/14/26 16:43:02.753 (0s) > Enter [BeforeEach] Incremental backup restore tests @ 01/14/26 16:43:02.753 < Exit [BeforeEach] Incremental backup restore tests @ 01/14/26 16:43:02.76 (6ms) > Enter [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:43:02.76 < Exit [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:43:02.76 (0s) > Enter [It] [tc-id:OADP-165][interop] [Tier0] [vf:restore-policies] Todolist app with CSI - policy: update @ 01/14/26 16:43:02.76 2026/01/14 16:43:02 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/14/26 16:43:02.766 2026/01/14 16:43:02 csi 2026/01/14 16:43:02 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "d4b4cde0-a26a-4e93-a610-5ba9acfa423e", "resourceVersion": "91966", "generation": 1, "creationTimestamp": "2026-01-14T16:43:02Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:43:02Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/14/26 16:43:02.853 2026/01/14 16:43:02 Waiting for velero pod to be running 2026/01/14 16:43:02 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2026/01/14 16:43:02 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "d4b4cde0-a26a-4e93-a610-5ba9acfa423e", "resourceVersion": "91966", "generation": 1, "creationTimestamp": "2026-01-14T16:43:02Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:43:02Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } 2026/01/14 16:43:07 DPA successfully reconciled STEP: Installing application for case todolist-backup @ 01/14/26 16:43:07.875 2026/01/14 16:43:07 🔄 [Attempt 1/3] Executing operation: Deploy application ocp-todolist-mariadb 2026/01/14 16:43:07 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:43:11 2026-01-14 16:43:09,566 p=32490 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:43:09,567 p=32490 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:43:09,881 p=32490 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:43:09,881 p=32490 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:43:10,201 p=32490 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:43:10,201 p=32490 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:43:10,517 p=32490 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:43:10,517 p=32490 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:43:10,535 p=32490 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:43:10,536 p=32490 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:43:10,536 p=32490 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:43:10,556 p=32490 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:43:10,556 p=32490 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:43:10,570 p=32490 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:43:10,571 p=32490 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:43:10,973 p=32490 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:43:10,974 p=32490 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:43:10,974 p=32490 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:43:10,974 p=32490 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:43:10,997 p=32490 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:43:10,997 p=32490 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:43:10,998 p=32490 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:43:10,998 p=32490 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:43:10,998 p=32490 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n todolist-mariadb-csi-policy-update 2026/01/14 16:43:11 2026/01/14 16:43:11 ❌ Operation 'Deploy application ocp-todolist-mariadb' failed with non-retryable error: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"todolist-mariadb-csi-policy-update","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-todolist-mariadb","user_kubeconfig":"/home/jenkins/.kube/config","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 [FAILED] in [It] - /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:43:11.167 < Exit [It] [tc-id:OADP-165][interop] [Tier0] [vf:restore-policies] Todolist app with CSI - policy: update @ 01/14/26 16:43:11.167 (8.407s) > Enter [JustAfterEach] TOP-LEVEL @ 01/14/26 16:43:11.167 2026/01/14 16:43:11 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 STEP: Get the failed spec name @ 01/14/26 16:43:11.167 2026/01/14 16:43:11 The failed spec name is: Incremental backup restore tests Incremental restore pod count [tc-id:OADP-165][interop] [Tier0] [vf:restore-policies] Todolist app with CSI - policy: update STEP: Create a folder for all must-gather files if it doesn't exists already @ 01/14/26 16:43:11.167 STEP: Create a folder for the failed spec if it doesn't exists already @ 01/14/26 16:43:11.167 STEP: Run must-gather because the spec failed @ 01/14/26 16:43:11.167 2026/01/14 16:43:11 Log the present working directory path:- /alabama/cspi/e2e 2026/01/14 16:43:11 [adm must-gather --dest-dir /alabama/cspi/e2e/logs/It_Incremental_backup_restore_tests_Incremental_restore_pod_count_tc-id_OADP-165_interop_Tier0_vf_restore-policies_Todolist_app_with_CSI_-_policy_update --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3] 2026/01/14 16:43:22 Log all the files present in /alabama/cspi/e2e/logs directory 2026/01/14 16:43:22 It_Backup_restore_tests_Application_backup_tc-id_OADP-122_interop_skip-disconnected_Tier0_vf_csi-snapshots_Django_application_with_BSL_CSI_exclude_aro-4 2026/01/14 16:43:22 It_Backup_restore_tests_Application_backup_tc-id_OADP-437_interop_smoke_Tier0_vf_fsb-kopia_MySQL_application_with_filesystem_Kopia_mr-check 2026/01/14 16:43:22 It_Incremental_backup_restore_tests_Incremental_restore_pod_count_tc-id_OADP-165_interop_Tier0_vf_restore-policies_Todolist_app_with_CSI_-_policy_update 2026/01/14 16:43:22 It_datamover_DataMover_Backup_Restore_stateful_application_with_CSI_tc-id_OADP-439_interop_Tier0_vf_data-mover_MySQL_application 2026/01/14 16:43:22 It_skip-disconnected_Restore_hooks_tests_Successful_Init_hook_tc-id_OADP-164_interop_smoke_Tier0_MySQL_app_with_Kopia STEP: Find must-gather folder and rename it to a shorter more readable name @ 01/14/26 16:43:22.428 The folder logs/It_Incremental_backup_restore_tests_Incremental_restore_pod_count_tc-id_OADP-165_interop_Tier0_vf_restore-policies_Todolist_app_with_CSI_-_policy_update/must-gather already exists, skipping renaming the folder Secret deletion flag: true Deleting cloud credentials secret cloud-credentials in namespace openshift-adp < Exit [JustAfterEach] TOP-LEVEL @ 01/14/26 16:43:22.44 (11.273s) > Enter [DeferCleanup (Each)] Incremental restore pod count @ 01/14/26 16:43:22.44 2026/01/14 16:43:22 Cleaning app 2026/01/14 16:43:22 Using admin kubeconfig for with_cleanup operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:43:25 2026-01-14 16:43:24,125 p=32715 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:43:24,125 p=32715 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:43:24,445 p=32715 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:43:24,445 p=32715 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:43:24,765 p=32715 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:43:24,765 p=32715 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:43:25,089 p=32715 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:43:25,089 p=32715 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:43:25,107 p=32715 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:43:25,107 p=32715 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:43:25,107 p=32715 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:43:25,126 p=32715 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:43:25,126 p=32715 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:43:25,140 p=32715 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:43:25,141 p=32715 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:43:25,539 p=32715 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:43:25,540 p=32715 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:43:25,540 p=32715 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:43:25,540 p=32715 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:43:25,562 p=32715 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:43:25,563 p=32715 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:43:25,563 p=32715 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:43:25,563 p=32715 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:43:25,564 p=32715 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n todolist-mariadb-csi-policy-update 2026/01/14 16:43:25 < Exit [DeferCleanup (Each)] Incremental restore pod count @ 01/14/26 16:43:25.732 (3.292s) > Enter [DeferCleanup (Each)] Incremental restore pod count @ 01/14/26 16:43:25.732 2026/01/14 16:43:25 Deleting DPA CR < Exit [DeferCleanup (Each)] Incremental restore pod count @ 01/14/26 16:43:25.743 (11ms) Attempt #2 Failed. Retrying ↺ @ 01/14/26 16:43:25.743 > Enter [BeforeEach] TOP-LEVEL @ 01/14/26 16:43:25.743 < Exit [BeforeEach] TOP-LEVEL @ 01/14/26 16:43:25.743 (0s) > Enter [BeforeEach] Incremental backup restore tests @ 01/14/26 16:43:25.743 < Exit [BeforeEach] Incremental backup restore tests @ 01/14/26 16:43:25.75 (7ms) > Enter [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:43:25.75 < Exit [JustBeforeEach] TOP-LEVEL @ 01/14/26 16:43:25.75 (0s) > Enter [It] [tc-id:OADP-165][interop] [Tier0] [vf:restore-policies] Todolist app with CSI - policy: update @ 01/14/26 16:43:25.75 2026/01/14 16:43:25 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/14/26 16:43:25.756 2026/01/14 16:43:25 csi 2026/01/14 16:43:25 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "ef80841b-bb28-4a16-bf03-12df8c2c3737", "resourceVersion": "92430", "generation": 1, "creationTimestamp": "2026-01-14T16:43:25Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:43:25Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/14/26 16:43:25.831 2026/01/14 16:43:25 Waiting for velero pod to be running 2026/01/14 16:43:25 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2026/01/14 16:43:25 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "ef80841b-bb28-4a16-bf03-12df8c2c3737", "resourceVersion": "92430", "generation": 1, "creationTimestamp": "2026-01-14T16:43:25Z", "managedFields": [ { "manager": "e2e.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2026-01-14T16:43:25Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {}, "f:disableFsBackup": {} } }, "f:logFormat": {}, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-98rbwkjv-interopoadp", "prefix": "velero-e2e-5ae23593-f167-11f0-950a-0a58ac1877e4" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "hypershift", "csi" ], "disableFsBackup": false } }, "features": null, "logFormat": "text" }, "status": {} } 2026/01/14 16:43:30 DPA successfully reconciled STEP: Installing application for case todolist-backup @ 01/14/26 16:43:30.855 2026/01/14 16:43:30 🔄 [Attempt 1/3] Executing operation: Deploy application ocp-todolist-mariadb 2026/01/14 16:43:30 Using admin kubeconfig for with_deploy operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:43:34 2026-01-14 16:43:32,542 p=32894 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:43:32,542 p=32894 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:43:32,860 p=32894 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:43:32,861 p=32894 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:43:33,179 p=32894 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:43:33,180 p=32894 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:43:33,506 p=32894 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:43:33,507 p=32894 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:43:33,524 p=32894 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:43:33,525 p=32894 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:43:33,525 p=32894 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:43:33,545 p=32894 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:43:33,545 p=32894 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:43:33,560 p=32894 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:43:33,560 p=32894 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:43:33,957 p=32894 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:43:33,957 p=32894 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:43:33,957 p=32894 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:43:33,957 p=32894 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:43:33,979 p=32894 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:43:33,980 p=32894 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:43:33,980 p=32894 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:43:33,981 p=32894 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:43:33,981 p=32894 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n todolist-mariadb-csi-policy-update 2026/01/14 16:43:34 2026/01/14 16:43:34 ❌ Operation 'Deploy application ocp-todolist-mariadb' failed with non-retryable error: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"todolist-mariadb-csi-policy-update","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-todolist-mariadb","user_kubeconfig":"/home/jenkins/.kube/config","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 [FAILED] in [It] - /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:43:34.145 < Exit [It] [tc-id:OADP-165][interop] [Tier0] [vf:restore-policies] Todolist app with CSI - policy: update @ 01/14/26 16:43:34.145 (8.395s) > Enter [JustAfterEach] TOP-LEVEL @ 01/14/26 16:43:34.145 2026/01/14 16:43:34 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3 STEP: Get the failed spec name @ 01/14/26 16:43:34.146 2026/01/14 16:43:34 The failed spec name is: Incremental backup restore tests Incremental restore pod count [tc-id:OADP-165][interop] [Tier0] [vf:restore-policies] Todolist app with CSI - policy: update STEP: Create a folder for all must-gather files if it doesn't exists already @ 01/14/26 16:43:34.146 STEP: Create a folder for the failed spec if it doesn't exists already @ 01/14/26 16:43:34.146 STEP: Run must-gather because the spec failed @ 01/14/26 16:43:34.146 2026/01/14 16:43:34 Log the present working directory path:- /alabama/cspi/e2e 2026/01/14 16:43:34 [adm must-gather --dest-dir /alabama/cspi/e2e/logs/It_Incremental_backup_restore_tests_Incremental_restore_pod_count_tc-id_OADP-165_interop_Tier0_vf_restore-policies_Todolist_app_with_CSI_-_policy_update --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.5.3] 2026/01/14 16:43:45 Log all the files present in /alabama/cspi/e2e/logs directory 2026/01/14 16:43:45 It_Backup_restore_tests_Application_backup_tc-id_OADP-122_interop_skip-disconnected_Tier0_vf_csi-snapshots_Django_application_with_BSL_CSI_exclude_aro-4 2026/01/14 16:43:45 It_Backup_restore_tests_Application_backup_tc-id_OADP-437_interop_smoke_Tier0_vf_fsb-kopia_MySQL_application_with_filesystem_Kopia_mr-check 2026/01/14 16:43:45 It_Incremental_backup_restore_tests_Incremental_restore_pod_count_tc-id_OADP-165_interop_Tier0_vf_restore-policies_Todolist_app_with_CSI_-_policy_update 2026/01/14 16:43:45 It_datamover_DataMover_Backup_Restore_stateful_application_with_CSI_tc-id_OADP-439_interop_Tier0_vf_data-mover_MySQL_application 2026/01/14 16:43:45 It_skip-disconnected_Restore_hooks_tests_Successful_Init_hook_tc-id_OADP-164_interop_smoke_Tier0_MySQL_app_with_Kopia STEP: Find must-gather folder and rename it to a shorter more readable name @ 01/14/26 16:43:45.191 The folder logs/It_Incremental_backup_restore_tests_Incremental_restore_pod_count_tc-id_OADP-165_interop_Tier0_vf_restore-policies_Todolist_app_with_CSI_-_policy_update/must-gather already exists, skipping renaming the folder Secret deletion flag: true Deleting cloud credentials secret cloud-credentials in namespace openshift-adp < Exit [JustAfterEach] TOP-LEVEL @ 01/14/26 16:43:45.2 (11.054s) > Enter [DeferCleanup (Each)] Incremental restore pod count @ 01/14/26 16:43:45.2 2026/01/14 16:43:45 Cleaning app 2026/01/14 16:43:45 Using admin kubeconfig for with_cleanup operation: /home/jenkins/.kube/config [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name 'namespace'. Origin:   namespace  PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Print admin kubeconfig path] ********************************************* ok: [localhost] => {  "msg": "Admin KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Print user kubeconfig path] ********************************************** ok: [localhost] => {  "msg": "User KUBECONFIG path: /home/jenkins/.kube/config" } TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint (from admin kubeconfig)] **************************** changed: [localhost] TASK [Get admin token] ********************************************************* changed: [localhost] TASK [Get user token] ********************************************************** changed: [localhost] [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9  35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}"  ^ column 9  TASK [Set core facts (admin + user token)] ************************************* ok: [localhost] TASK [Choose token based on non_admin flag] ************************************ ok: [localhost] TASK [Print token] ************************************************************* ok: [localhost] => {  "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. TASK [Extract Kubernetes minor version from cluster] *************************** ok: [localhost] TASK [Map Kubernetes minor to OCP release] ************************************* [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage.  Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7  53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release  ^ column 7  <<< caused by >>>  Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7  54 55 - name: Map Kubernetes minor to OCP release 56 set_fact:  ^ column 7  <<< caused by >>>  Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26  55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >-  ^ column 26  <<< caused by >>>  Type 'Undefined' is unsupported for variable storage. Origin:   fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=1  skipped=0 rescued=0 ignored=0 2026/01/14 16:43:48 2026-01-14 16:43:46,885 p=33116 u=1004100000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2026-01-14 16:43:46,885 p=33116 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:43:47,208 p=33116 u=1004100000 n=ansible INFO| TASK [Get cluster endpoint (from admin kubeconfig)] **************************** 2026-01-14 16:43:47,208 p=33116 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:43:47,529 p=33116 u=1004100000 n=ansible INFO| TASK [Get admin token] ********************************************************* 2026-01-14 16:43:47,530 p=33116 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:43:47,852 p=33116 u=1004100000 n=ansible INFO| TASK [Get user token] ********************************************************** 2026-01-14 16:43:47,852 p=33116 u=1004100000 n=ansible INFO| changed: [localhost] 2026-01-14 16:43:47,870 p=33116 u=1004100000 n=ansible WARNING| [WARNING]: Found variable using reserved name 'namespace'. Origin: /alabama/cspi/sample-applications/ansible/main.yml:37:9 35 set_fact: 36 oc_binary: "oc" 37 namespace: "{{ use_role }}" ^ column 9 2026-01-14 16:43:47,870 p=33116 u=1004100000 n=ansible INFO| TASK [Set core facts (admin + user token)] ************************************* 2026-01-14 16:43:47,870 p=33116 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:43:47,890 p=33116 u=1004100000 n=ansible INFO| TASK [Choose token based on non_admin flag] ************************************ 2026-01-14 16:43:47,890 p=33116 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:43:47,905 p=33116 u=1004100000 n=ansible INFO| TASK [Print token] ************************************************************* 2026-01-14 16:43:47,905 p=33116 u=1004100000 n=ansible INFO| ok: [localhost] => { "msg": "Token: sha256~Yt-JUjpNesy3toncBTn7gXFx1x1iZJ-pBfApvYRBePs" } 2026-01-14 16:43:48,297 p=33116 u=1004100000 n=ansible INFO| TASK [Extract Kubernetes minor version from cluster] *************************** 2026-01-14 16:43:48,298 p=33116 u=1004100000 n=ansible WARNING| [WARNING]: Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. 2026-01-14 16:43:48,298 p=33116 u=1004100000 n=ansible INFO| [DEPRECATION WARNING]: The `ansible.module_utils.common._collections_compat` module is deprecated. This feature will be removed from ansible-core version 2.24. Use `collections.abc` from the Python standard library instead. 2026-01-14 16:43:48,298 p=33116 u=1004100000 n=ansible INFO| ok: [localhost] 2026-01-14 16:43:48,320 p=33116 u=1004100000 n=ansible INFO| TASK [Map Kubernetes minor to OCP release] ************************************* 2026-01-14 16:43:48,320 p=33116 u=1004100000 n=ansible ERROR| [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage. Task failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:55:7 53 {{ lookup('k8s', cluster_info='version', validate_certs=false, host=url, api_key=token).kubernetes.minor... 54 55 - name: Map Kubernetes minor to OCP release ^ column 7 <<< caused by >>> Finalization of task args for 'ansible.builtin.set_fact' failed. Origin: /alabama/cspi/sample-applications/ansible/main.yml:56:7 54 55 - name: Map Kubernetes minor to OCP release 56 set_fact: ^ column 7 <<< caused by >>> Error while resolving value for 'cluster_version': Error rendering template. Origin: /alabama/cspi/sample-applications/ansible/main.yml:57:26 55 - name: Map Kubernetes minor to OCP release 56 set_fact: 57 cluster_version: >- ^ column 26 <<< caused by >>> Type 'Undefined' is unsupported for variable storage. Origin: 2026-01-14 16:43:48,321 p=33116 u=1004100000 n=ansible INFO| fatal: [localhost]: FAILED! => {"changed": false, "msg": "Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'cluster_version': Error rendering template: Type 'Undefined' is unsupported for variable storage."} 2026-01-14 16:43:48,321 p=33116 u=1004100000 n=ansible INFO| PLAY RECAP ********************************************************************* 2026-01-14 16:43:48,321 p=33116 u=1004100000 n=ansible INFO| localhost : ok=12 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 Run the command: oc get event -n todolist-mariadb-csi-policy-update 2026/01/14 16:43:48 < Exit [DeferCleanup (Each)] Incremental restore pod count @ 01/14/26 16:43:48.484 (3.284s) > Enter [DeferCleanup (Each)] Incremental restore pod count @ 01/14/26 16:43:48.484 2026/01/14 16:43:48 Deleting DPA CR < Exit [DeferCleanup (Each)] Incremental restore pod count @ 01/14/26 16:43:48.495 (10ms) • [FAILED] [68.279 seconds] Incremental backup restore tests Incremental restore pod count [It] [tc-id:OADP-165][interop] [Tier0] [vf:restore-policies] Todolist app with CSI - policy: update /alabama/cspi/e2e/incremental_restore/backup_restore_incremental.go:94 [FAILED] Unexpected error: <*errors.Error | 0xc0013a1e40>: Error during command execution: ansible-playbook error: one or more host failed Command executed: /usr/local/bin/ansible-playbook --extra-vars {"admin_kubeconfig":"/home/jenkins/.kube/config","namespace":"todolist-mariadb-csi-policy-update","non_admin_user":false,"use_role":"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-todolist-mariadb","user_kubeconfig":"/home/jenkins/.kube/config","with_deploy":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml exit status 2 { context: "(DefaultExecute::Execute)", message: "Error during command execution: ansible-playbook error: one or more host failed\n\nCommand executed: /usr/local/bin/ansible-playbook --extra-vars {\"admin_kubeconfig\":\"/home/jenkins/.kube/config\",\"namespace\":\"todolist-mariadb-csi-policy-update\",\"non_admin_user\":false,\"use_role\":\"/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-todolist-mariadb\",\"user_kubeconfig\":\"/home/jenkins/.kube/config\",\"with_deploy\":true} --connection local /alabama/cspi/sample-applications/ansible/main.yml\n\nexit status 2", wrappedErrors: nil, } occurred In [It] at: /alabama/cspi/test_common/backup_restore_app_case.go:49 @ 01/14/26 16:43:34.145 There were additional failures detected. To view them in detail run ginkgo -vv ------------------------------ SSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [SynchronizedAfterSuite]  /alabama/cspi/e2e/e2e_suite_test.go:229 > Enter [SynchronizedAfterSuite] TOP-LEVEL @ 01/14/26 16:43:48.495 < Exit [SynchronizedAfterSuite] TOP-LEVEL @ 01/14/26 16:43:48.495 (0s) > Enter [SynchronizedAfterSuite] TOP-LEVEL @ 01/14/26 16:43:48.495 < Exit [SynchronizedAfterSuite] TOP-LEVEL @ 01/14/26 16:43:48.495 (0s) [SynchronizedAfterSuite] PASSED [0.000 seconds] ------------------------------ [ReportAfterSuite] Autogenerated ReportAfterSuite for --junit-report autogenerated by Ginkgo > Enter [ReportAfterSuite] TOP-LEVEL @ 01/14/26 16:43:48.495 < Exit [ReportAfterSuite] TOP-LEVEL @ 01/14/26 16:43:48.506 (11ms) [ReportAfterSuite] PASSED [0.011 seconds] ------------------------------ Summarizing 5 Failures: [FAIL] Backup restore tests Application backup [It] [tc-id:OADP-437][interop][smoke] [Tier0] [vf:fsb-kopia] MySQL application with filesystem, Kopia [mr-check] /alabama/cspi/test_common/backup_restore_app_case.go:49 [FAIL] Backup restore tests Application backup [It] [tc-id:OADP-122] [interop] [skip-disconnected] [Tier0] [vf:csi-snapshots] Django application with BSL&CSI [exclude_aro-4] /alabama/cspi/test_common/backup_restore_app_case.go:49 [FAIL] [datamover] DataMover: Backup/Restore stateful application with CSI  [It] [tc-id:OADP-439][interop] [Tier0] [vf:data-mover] MySQL application /alabama/cspi/test_common/backup_restore_app_case.go:49 [FAIL] [skip-disconnected] Restore hooks tests Successful Init hook [It] [tc-id:OADP-164][interop][smoke] [Tier0] MySQL app with Kopia /alabama/cspi/test_common/backup_restore_app_case.go:49 [FAIL] Incremental backup restore tests Incremental restore pod count [It] [tc-id:OADP-165][interop] [Tier0] [vf:restore-policies] Todolist app with CSI - policy: update /alabama/cspi/test_common/backup_restore_app_case.go:49 Ran 5 of 193 Specs in 362.800 seconds FAIL! -- 0 Passed | 5 Failed | 0 Pending | 188 Skipped --- FAIL: TestOADPE2E (362.83s) FAIL Ginkgo ran 1 suite in 6m8.173718646s Test Suite Failed [must-gather ] OUT 2026-01-14T16:44:18.610022356Z Using must-gather plug-in image: registry.redhat.io/oadp/oadp-mustgather-rhel9:v1.4 When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: 0cd423ed-2178-4c99-b8d7-354d4fc33d55 ClientVersion: 4.20.8 ClusterVersion: Stable at "4.21.0-0.nightly-2026-01-13-111112" ClusterOperators: clusteroperator/operator-lifecycle-manager is not upgradeable because ClusterServiceVersions blocking minor version upgrades to 4.22.0 or higher: - maximum supported OCP version for openshift-storage/odf-dependencies.v4.20.5-rhodf is 4.21 - maximum supported OCP version for openshift-storage/odf-operator.v4.20.5-rhodf is 4.21 [must-gather ] OUT 2026-01-14T16:44:18.655447036Z namespace/openshift-must-gather-q9fnk created [must-gather ] OUT 2026-01-14T16:44:18.668402058Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-8dq2f created [must-gather ] OUT 2026-01-14T16:44:18.705617229Z pod for plug-in image registry.redhat.io/oadp/oadp-mustgather-rhel9:v1.4 created [must-gather-vqqmv] POD 2026-01-14T16:44:33.416986926Z [disk usage checker] Started [must-gather-vqqmv] POD 2026-01-14T16:44:33.420959366Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70 [must-gather-vqqmv] POD 2026-01-14T16:44:36.900750664Z W0114 16:44:36.900694 3 warnings.go:70] apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather-vqqmv] POD 2026-01-14T16:44:36.974764410Z W0114 16:44:36.974724 3 warnings.go:70] kubevirt.io/v1 VirtualMachineInstancePresets is now deprecated and will be removed in v2. [must-gather-vqqmv] POD 2026-01-14T16:44:37.586805651Z W0114 16:44:37.586757 3 warnings.go:70] v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather-vqqmv] POD 2026-01-14T16:44:38.326446922Z W0114 16:44:38.326375 3 warnings.go:70] apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather-vqqmv] POD 2026-01-14T16:44:38.402441398Z W0114 16:44:38.402399 3 warnings.go:70] kubevirt.io/v1 VirtualMachineInstancePresets is now deprecated and will be removed in v2. [must-gather-vqqmv] POD 2026-01-14T16:44:38.426168713Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70 [must-gather-vqqmv] POD 2026-01-14T16:44:38.917242810Z W0114 16:44:38.917195 3 warnings.go:70] v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather-vqqmv] POD 2026-01-14T16:44:39.622468198Z Caches written to disk [must-gather-vqqmv] OUT 2026-01-14T16:44:40.640910761Z waiting for gather to complete [must-gather-vqqmv] OUT 2026-01-14T16:44:40.652816649Z downloading gather output [must-gather-vqqmv] OUT 2026-01-14T16:44:40.895293928Z receiving incremental file list [must-gather-vqqmv] OUT 2026-01-14T16:44:40.906771167Z ./ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.906866719Z version [must-gather-vqqmv] OUT 2026-01-14T16:44:40.924970802Z clusters/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.925012443Z clusters/0cd423ed/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.925115105Z clusters/0cd423ed/event-filter.html [must-gather-vqqmv] OUT 2026-01-14T16:44:40.927735792Z clusters/0cd423ed/oadp-must-gather-summary.md [must-gather-vqqmv] OUT 2026-01-14T16:44:40.927892855Z clusters/0cd423ed/timestamp [must-gather-vqqmv] OUT 2026-01-14T16:44:40.927944537Z clusters/0cd423ed/cluster-scoped-resources/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.927954807Z clusters/0cd423ed/cluster-scoped-resources/apiextensions.k8s.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.927960767Z clusters/0cd423ed/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.928038779Z clusters/0cd423ed/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/backuprepositories.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.928194572Z clusters/0cd423ed/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/backups.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.928452748Z clusters/0cd423ed/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/backupstoragelocations.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.928625471Z clusters/0cd423ed/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/cloudstorages.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.928825786Z clusters/0cd423ed/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterserviceversions.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.931375091Z clusters/0cd423ed/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/datadownloads.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.931601456Z clusters/0cd423ed/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dataprotectionapplications.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.932285721Z clusters/0cd423ed/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/datauploads.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.932461105Z clusters/0cd423ed/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/deletebackuprequests.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.932624608Z clusters/0cd423ed/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/downloadrequests.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.932810972Z clusters/0cd423ed/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/podvolumebackups.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.932989016Z clusters/0cd423ed/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/podvolumerestores.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.93315787Z clusters/0cd423ed/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/restores.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.933386605Z clusters/0cd423ed/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/schedules.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.93361035Z clusters/0cd423ed/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/serverstatusrequests.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.933785394Z clusters/0cd423ed/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/subscriptions.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.934616561Z clusters/0cd423ed/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/volumesnapshotlocations.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.934694423Z clusters/0cd423ed/cluster-scoped-resources/config.openshift.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.934793705Z clusters/0cd423ed/cluster-scoped-resources/config.openshift.io/clusterversions.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.934893547Z clusters/0cd423ed/cluster-scoped-resources/snapshot.storage.k8s.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.934905088Z clusters/0cd423ed/cluster-scoped-resources/snapshot.storage.k8s.io/volumesnapshotclasses/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.934969039Z clusters/0cd423ed/cluster-scoped-resources/snapshot.storage.k8s.io/volumesnapshotclasses/volumesnapshotclasses.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.935062101Z clusters/0cd423ed/cluster-scoped-resources/storage.k8s.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.935070421Z clusters/0cd423ed/cluster-scoped-resources/storage.k8s.io/csidrivers/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.935130703Z clusters/0cd423ed/cluster-scoped-resources/storage.k8s.io/csidrivers/csidrivers.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.935214864Z clusters/0cd423ed/cluster-scoped-resources/storage.k8s.io/storageclasses/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.935277186Z clusters/0cd423ed/cluster-scoped-resources/storage.k8s.io/storageclasses/storageclasses.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.935383428Z clusters/0cd423ed/namespaces/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.935391648Z clusters/0cd423ed/namespaces/openshift-adp/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.935450339Z clusters/0cd423ed/namespaces/openshift-adp/openshift-adp.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.935537041Z clusters/0cd423ed/namespaces/openshift-adp/apps.openshift.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.935613083Z clusters/0cd423ed/namespaces/openshift-adp/apps.openshift.io/deploymentconfigs.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.935690585Z clusters/0cd423ed/namespaces/openshift-adp/apps/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.935772757Z clusters/0cd423ed/namespaces/openshift-adp/apps/daemonsets.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.935904639Z clusters/0cd423ed/namespaces/openshift-adp/apps/deployments.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.936121654Z clusters/0cd423ed/namespaces/openshift-adp/apps/replicasets.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.936314048Z clusters/0cd423ed/namespaces/openshift-adp/apps/statefulsets.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.936365919Z clusters/0cd423ed/namespaces/openshift-adp/autoscaling/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.936448681Z clusters/0cd423ed/namespaces/openshift-adp/autoscaling/horizontalpodautoscalers.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.936503302Z clusters/0cd423ed/namespaces/openshift-adp/batch/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.936591134Z clusters/0cd423ed/namespaces/openshift-adp/batch/cronjobs.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.936717857Z clusters/0cd423ed/namespaces/openshift-adp/batch/jobs.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.936784288Z clusters/0cd423ed/namespaces/openshift-adp/build.openshift.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.936889471Z clusters/0cd423ed/namespaces/openshift-adp/build.openshift.io/buildconfigs.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.936991093Z clusters/0cd423ed/namespaces/openshift-adp/build.openshift.io/builds.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.937045134Z clusters/0cd423ed/namespaces/openshift-adp/cdi.kubevirt.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.937106075Z clusters/0cd423ed/namespaces/openshift-adp/cdi.kubevirt.io/dataimportcrons.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.937210968Z clusters/0cd423ed/namespaces/openshift-adp/cdi.kubevirt.io/datasources.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.93732853Z clusters/0cd423ed/namespaces/openshift-adp/cdi.kubevirt.io/datavolumes.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.937378481Z clusters/0cd423ed/namespaces/openshift-adp/clone.kubevirt.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.937440193Z clusters/0cd423ed/namespaces/openshift-adp/clone.kubevirt.io/virtualmachineclones.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.937492704Z clusters/0cd423ed/namespaces/openshift-adp/core/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.937552525Z clusters/0cd423ed/namespaces/openshift-adp/core/configmaps.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.937694488Z clusters/0cd423ed/namespaces/openshift-adp/core/endpoints.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.937828111Z clusters/0cd423ed/namespaces/openshift-adp/core/events.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.940726904Z clusters/0cd423ed/namespaces/openshift-adp/core/persistentvolumeclaims.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.940855847Z clusters/0cd423ed/namespaces/openshift-adp/core/pods.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.941045651Z clusters/0cd423ed/namespaces/openshift-adp/core/replicationcontrollers.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.941198884Z clusters/0cd423ed/namespaces/openshift-adp/core/secrets.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.941411189Z clusters/0cd423ed/namespaces/openshift-adp/core/services.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.94147678Z clusters/0cd423ed/namespaces/openshift-adp/discovery.k8s.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.941538242Z clusters/0cd423ed/namespaces/openshift-adp/discovery.k8s.io/endpointslices.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.941601323Z clusters/0cd423ed/namespaces/openshift-adp/export.kubevirt.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.941663674Z clusters/0cd423ed/namespaces/openshift-adp/export.kubevirt.io/virtualmachineexports.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.941731866Z clusters/0cd423ed/namespaces/openshift-adp/hco.kubevirt.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.941833008Z clusters/0cd423ed/namespaces/openshift-adp/hco.kubevirt.io/hyperconvergeds.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.941887899Z clusters/0cd423ed/namespaces/openshift-adp/image.openshift.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.941965701Z clusters/0cd423ed/namespaces/openshift-adp/image.openshift.io/imagestreams.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.942016202Z clusters/0cd423ed/namespaces/openshift-adp/instancetype.kubevirt.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.942090163Z clusters/0cd423ed/namespaces/openshift-adp/instancetype.kubevirt.io/virtualmachineinstancetypes.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.942189626Z clusters/0cd423ed/namespaces/openshift-adp/instancetype.kubevirt.io/virtualmachinepreferences.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.942241917Z clusters/0cd423ed/namespaces/openshift-adp/k8s.ovn.org/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.942302598Z clusters/0cd423ed/namespaces/openshift-adp/k8s.ovn.org/egressfirewalls.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.94239945Z clusters/0cd423ed/namespaces/openshift-adp/k8s.ovn.org/egressqoses.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.942443441Z clusters/0cd423ed/namespaces/openshift-adp/kubevirt.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.942514253Z clusters/0cd423ed/namespaces/openshift-adp/kubevirt.io/kubevirts.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.942622755Z clusters/0cd423ed/namespaces/openshift-adp/kubevirt.io/virtualmachineinstancemigrations.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.942718657Z clusters/0cd423ed/namespaces/openshift-adp/kubevirt.io/virtualmachineinstancepresets.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.94286002Z clusters/0cd423ed/namespaces/openshift-adp/kubevirt.io/virtualmachineinstancereplicasets.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.943046704Z clusters/0cd423ed/namespaces/openshift-adp/kubevirt.io/virtualmachineinstances.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.943181867Z clusters/0cd423ed/namespaces/openshift-adp/kubevirt.io/virtualmachines.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.943250099Z clusters/0cd423ed/namespaces/openshift-adp/monitoring.coreos.com/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.94332001Z clusters/0cd423ed/namespaces/openshift-adp/monitoring.coreos.com/servicemonitors.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.943369341Z clusters/0cd423ed/namespaces/openshift-adp/networking.k8s.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.943438393Z clusters/0cd423ed/namespaces/openshift-adp/networking.k8s.io/networkpolicies.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.943490384Z clusters/0cd423ed/namespaces/openshift-adp/operators.coreos.com/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.943498774Z clusters/0cd423ed/namespaces/openshift-adp/operators.coreos.com/clusterserviceversions/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.943560506Z clusters/0cd423ed/namespaces/openshift-adp/operators.coreos.com/clusterserviceversions/clusterserviceversions.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.943844892Z clusters/0cd423ed/namespaces/openshift-adp/operators.coreos.com/subscriptions/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.943908453Z clusters/0cd423ed/namespaces/openshift-adp/operators.coreos.com/subscriptions/subscriptions.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.943990625Z clusters/0cd423ed/namespaces/openshift-adp/pods/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.943999175Z clusters/0cd423ed/namespaces/openshift-adp/pods/openshift-adp-controller-manager-fd84cfc68-c8zpg/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.944060506Z clusters/0cd423ed/namespaces/openshift-adp/pods/openshift-adp-controller-manager-fd84cfc68-c8zpg/openshift-adp-controller-manager-fd84cfc68-c8zpg.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.944189669Z clusters/0cd423ed/namespaces/openshift-adp/pods/openshift-adp-controller-manager-fd84cfc68-c8zpg/manager/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.944198769Z clusters/0cd423ed/namespaces/openshift-adp/pods/openshift-adp-controller-manager-fd84cfc68-c8zpg/manager/manager/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.944202859Z clusters/0cd423ed/namespaces/openshift-adp/pods/openshift-adp-controller-manager-fd84cfc68-c8zpg/manager/manager/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.944295671Z clusters/0cd423ed/namespaces/openshift-adp/pods/openshift-adp-controller-manager-fd84cfc68-c8zpg/manager/manager/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.945418626Z clusters/0cd423ed/namespaces/openshift-adp/pods/openshift-adp-controller-manager-fd84cfc68-c8zpg/manager/manager/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.945530938Z clusters/0cd423ed/namespaces/openshift-adp/pods/openshift-adp-controller-manager-fd84cfc68-c8zpg/manager/manager/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.945575919Z clusters/0cd423ed/namespaces/openshift-adp/policy/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.94562362Z clusters/0cd423ed/namespaces/openshift-adp/policy/poddisruptionbudgets.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.945686152Z clusters/0cd423ed/namespaces/openshift-adp/pool.kubevirt.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.945748273Z clusters/0cd423ed/namespaces/openshift-adp/pool.kubevirt.io/virtualmachinepools.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.945828705Z clusters/0cd423ed/namespaces/openshift-adp/route.openshift.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.945916217Z clusters/0cd423ed/namespaces/openshift-adp/route.openshift.io/routes.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.945994358Z clusters/0cd423ed/namespaces/openshift-adp/snapshot.kubevirt.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.946043919Z clusters/0cd423ed/namespaces/openshift-adp/snapshot.kubevirt.io/virtualmachinerestores.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.946152972Z clusters/0cd423ed/namespaces/openshift-adp/snapshot.kubevirt.io/virtualmachinesnapshotcontents.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.946330015Z clusters/0cd423ed/namespaces/openshift-adp/snapshot.kubevirt.io/virtualmachinesnapshots.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.946380237Z clusters/0cd423ed/namespaces/openshift-cnv/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.946452948Z clusters/0cd423ed/namespaces/openshift-cnv/openshift-cnv.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.94653132Z clusters/0cd423ed/namespaces/openshift-cnv/apps.openshift.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.946591081Z clusters/0cd423ed/namespaces/openshift-cnv/apps.openshift.io/deploymentconfigs.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.946645852Z clusters/0cd423ed/namespaces/openshift-cnv/apps/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.946708164Z clusters/0cd423ed/namespaces/openshift-cnv/apps/daemonsets.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.946928448Z clusters/0cd423ed/namespaces/openshift-cnv/apps/deployments.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.947854248Z clusters/0cd423ed/namespaces/openshift-cnv/apps/replicasets.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.948632725Z clusters/0cd423ed/namespaces/openshift-cnv/apps/statefulsets.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.948684466Z clusters/0cd423ed/namespaces/openshift-cnv/autoscaling/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.948768148Z clusters/0cd423ed/namespaces/openshift-cnv/autoscaling/horizontalpodautoscalers.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.94882952Z clusters/0cd423ed/namespaces/openshift-cnv/batch/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.948909502Z clusters/0cd423ed/namespaces/openshift-cnv/batch/cronjobs.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.949032584Z clusters/0cd423ed/namespaces/openshift-cnv/batch/jobs.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.949082335Z clusters/0cd423ed/namespaces/openshift-cnv/build.openshift.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.949148317Z clusters/0cd423ed/namespaces/openshift-cnv/build.openshift.io/buildconfigs.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.949261459Z clusters/0cd423ed/namespaces/openshift-cnv/build.openshift.io/builds.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.94931579Z clusters/0cd423ed/namespaces/openshift-cnv/cdi.kubevirt.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.949379812Z clusters/0cd423ed/namespaces/openshift-cnv/cdi.kubevirt.io/dataimportcrons.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.949488674Z clusters/0cd423ed/namespaces/openshift-cnv/cdi.kubevirt.io/datasources.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.949586056Z clusters/0cd423ed/namespaces/openshift-cnv/cdi.kubevirt.io/datavolumes.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.949688399Z clusters/0cd423ed/namespaces/openshift-cnv/clone.kubevirt.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.949706079Z clusters/0cd423ed/namespaces/openshift-cnv/clone.kubevirt.io/virtualmachineclones.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.949794321Z clusters/0cd423ed/namespaces/openshift-cnv/core/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.949852512Z clusters/0cd423ed/namespaces/openshift-cnv/core/configmaps.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.951856605Z clusters/0cd423ed/namespaces/openshift-cnv/core/endpoints.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.95205464Z clusters/0cd423ed/namespaces/openshift-cnv/core/events.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.953495221Z clusters/0cd423ed/namespaces/openshift-cnv/core/persistentvolumeclaims.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.953624304Z clusters/0cd423ed/namespaces/openshift-cnv/core/pods.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.955248339Z clusters/0cd423ed/namespaces/openshift-cnv/core/replicationcontrollers.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.955406412Z clusters/0cd423ed/namespaces/openshift-cnv/core/secrets.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.955963944Z clusters/0cd423ed/namespaces/openshift-cnv/core/services.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.956136878Z clusters/0cd423ed/namespaces/openshift-cnv/discovery.k8s.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.956198069Z clusters/0cd423ed/namespaces/openshift-cnv/discovery.k8s.io/endpointslices.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.956405754Z clusters/0cd423ed/namespaces/openshift-cnv/export.kubevirt.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.956462915Z clusters/0cd423ed/namespaces/openshift-cnv/export.kubevirt.io/virtualmachineexports.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.956512876Z clusters/0cd423ed/namespaces/openshift-cnv/hco.kubevirt.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.956568128Z clusters/0cd423ed/namespaces/openshift-cnv/hco.kubevirt.io/hyperconvergeds.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.95669575Z clusters/0cd423ed/namespaces/openshift-cnv/image.openshift.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.956747252Z clusters/0cd423ed/namespaces/openshift-cnv/image.openshift.io/imagestreams.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.956844014Z clusters/0cd423ed/namespaces/openshift-cnv/instancetype.kubevirt.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.956912455Z clusters/0cd423ed/namespaces/openshift-cnv/instancetype.kubevirt.io/virtualmachineinstancetypes.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.957029418Z clusters/0cd423ed/namespaces/openshift-cnv/instancetype.kubevirt.io/virtualmachinepreferences.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.957074588Z clusters/0cd423ed/namespaces/openshift-cnv/k8s.ovn.org/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.95713194Z clusters/0cd423ed/namespaces/openshift-cnv/k8s.ovn.org/egressfirewalls.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.957252323Z clusters/0cd423ed/namespaces/openshift-cnv/k8s.ovn.org/egressqoses.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.957298904Z clusters/0cd423ed/namespaces/openshift-cnv/kubevirt.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.957365035Z clusters/0cd423ed/namespaces/openshift-cnv/kubevirt.io/kubevirts.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.957525549Z clusters/0cd423ed/namespaces/openshift-cnv/kubevirt.io/virtualmachineinstancemigrations.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.957631931Z clusters/0cd423ed/namespaces/openshift-cnv/kubevirt.io/virtualmachineinstancepresets.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.957733413Z clusters/0cd423ed/namespaces/openshift-cnv/kubevirt.io/virtualmachineinstancereplicasets.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.957927647Z clusters/0cd423ed/namespaces/openshift-cnv/kubevirt.io/virtualmachineinstances.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.95804495Z clusters/0cd423ed/namespaces/openshift-cnv/kubevirt.io/virtualmachines.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.958099071Z clusters/0cd423ed/namespaces/openshift-cnv/monitoring.coreos.com/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.958161262Z clusters/0cd423ed/namespaces/openshift-cnv/monitoring.coreos.com/servicemonitors.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.958251844Z clusters/0cd423ed/namespaces/openshift-cnv/networking.k8s.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.958317916Z clusters/0cd423ed/namespaces/openshift-cnv/networking.k8s.io/networkpolicies.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.958397707Z clusters/0cd423ed/namespaces/openshift-cnv/operators.coreos.com/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.958407717Z clusters/0cd423ed/namespaces/openshift-cnv/operators.coreos.com/clusterserviceversions/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.958461499Z clusters/0cd423ed/namespaces/openshift-cnv/operators.coreos.com/clusterserviceversions/clusterserviceversions.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.959188234Z clusters/0cd423ed/namespaces/openshift-cnv/operators.coreos.com/subscriptions/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.959266496Z clusters/0cd423ed/namespaces/openshift-cnv/operators.coreos.com/subscriptions/subscriptions.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.959347418Z clusters/0cd423ed/namespaces/openshift-cnv/pods/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.959354978Z clusters/0cd423ed/namespaces/openshift-cnv/pods/aaq-operator-76b56887f5-77ddf/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.959419799Z clusters/0cd423ed/namespaces/openshift-cnv/pods/aaq-operator-76b56887f5-77ddf/aaq-operator-76b56887f5-77ddf.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.959542532Z clusters/0cd423ed/namespaces/openshift-cnv/pods/aaq-operator-76b56887f5-77ddf/aaq-operator/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.959550182Z clusters/0cd423ed/namespaces/openshift-cnv/pods/aaq-operator-76b56887f5-77ddf/aaq-operator/aaq-operator/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.959554312Z clusters/0cd423ed/namespaces/openshift-cnv/pods/aaq-operator-76b56887f5-77ddf/aaq-operator/aaq-operator/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.959640584Z clusters/0cd423ed/namespaces/openshift-cnv/pods/aaq-operator-76b56887f5-77ddf/aaq-operator/aaq-operator/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.959808018Z clusters/0cd423ed/namespaces/openshift-cnv/pods/aaq-operator-76b56887f5-77ddf/aaq-operator/aaq-operator/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.959934891Z clusters/0cd423ed/namespaces/openshift-cnv/pods/aaq-operator-76b56887f5-77ddf/aaq-operator/aaq-operator/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.959991782Z clusters/0cd423ed/namespaces/openshift-cnv/pods/bridge-marker-9qmvg/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.960074904Z clusters/0cd423ed/namespaces/openshift-cnv/pods/bridge-marker-9qmvg/bridge-marker-9qmvg.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.960173766Z clusters/0cd423ed/namespaces/openshift-cnv/pods/bridge-marker-9qmvg/bridge-marker/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.960181226Z clusters/0cd423ed/namespaces/openshift-cnv/pods/bridge-marker-9qmvg/bridge-marker/bridge-marker/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.960185556Z clusters/0cd423ed/namespaces/openshift-cnv/pods/bridge-marker-9qmvg/bridge-marker/bridge-marker/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.960264388Z clusters/0cd423ed/namespaces/openshift-cnv/pods/bridge-marker-9qmvg/bridge-marker/bridge-marker/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.960400381Z clusters/0cd423ed/namespaces/openshift-cnv/pods/bridge-marker-9qmvg/bridge-marker/bridge-marker/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.960531654Z clusters/0cd423ed/namespaces/openshift-cnv/pods/bridge-marker-9qmvg/bridge-marker/bridge-marker/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.960577704Z clusters/0cd423ed/namespaces/openshift-cnv/pods/bridge-marker-j878j/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.960648396Z clusters/0cd423ed/namespaces/openshift-cnv/pods/bridge-marker-j878j/bridge-marker-j878j.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.960729248Z clusters/0cd423ed/namespaces/openshift-cnv/pods/bridge-marker-j878j/bridge-marker/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.960736878Z clusters/0cd423ed/namespaces/openshift-cnv/pods/bridge-marker-j878j/bridge-marker/bridge-marker/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.960741058Z clusters/0cd423ed/namespaces/openshift-cnv/pods/bridge-marker-j878j/bridge-marker/bridge-marker/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.960881271Z clusters/0cd423ed/namespaces/openshift-cnv/pods/bridge-marker-j878j/bridge-marker/bridge-marker/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.961000504Z clusters/0cd423ed/namespaces/openshift-cnv/pods/bridge-marker-j878j/bridge-marker/bridge-marker/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.961126896Z clusters/0cd423ed/namespaces/openshift-cnv/pods/bridge-marker-j878j/bridge-marker/bridge-marker/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.961169997Z clusters/0cd423ed/namespaces/openshift-cnv/pods/bridge-marker-zxrd8/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.961241519Z clusters/0cd423ed/namespaces/openshift-cnv/pods/bridge-marker-zxrd8/bridge-marker-zxrd8.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.961335911Z clusters/0cd423ed/namespaces/openshift-cnv/pods/bridge-marker-zxrd8/bridge-marker/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.961343751Z clusters/0cd423ed/namespaces/openshift-cnv/pods/bridge-marker-zxrd8/bridge-marker/bridge-marker/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.961347721Z clusters/0cd423ed/namespaces/openshift-cnv/pods/bridge-marker-zxrd8/bridge-marker/bridge-marker/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.961421523Z clusters/0cd423ed/namespaces/openshift-cnv/pods/bridge-marker-zxrd8/bridge-marker/bridge-marker/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.961542716Z clusters/0cd423ed/namespaces/openshift-cnv/pods/bridge-marker-zxrd8/bridge-marker/bridge-marker/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.961663338Z clusters/0cd423ed/namespaces/openshift-cnv/pods/bridge-marker-zxrd8/bridge-marker/bridge-marker/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.961712149Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-apiserver-7c7bf4df96-zbqvj/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.961808921Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-apiserver-7c7bf4df96-zbqvj/cdi-apiserver-7c7bf4df96-zbqvj.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.961906564Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-apiserver-7c7bf4df96-zbqvj/cdi-apiserver/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.961914654Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-apiserver-7c7bf4df96-zbqvj/cdi-apiserver/cdi-apiserver/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.961918914Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-apiserver-7c7bf4df96-zbqvj/cdi-apiserver/cdi-apiserver/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.962016016Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-apiserver-7c7bf4df96-zbqvj/cdi-apiserver/cdi-apiserver/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.962152109Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-apiserver-7c7bf4df96-zbqvj/cdi-apiserver/cdi-apiserver/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.962271901Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-apiserver-7c7bf4df96-zbqvj/cdi-apiserver/cdi-apiserver/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.962317982Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-deployment-7f6d468bc9-gjpk9/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.962404854Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-deployment-7f6d468bc9-gjpk9/cdi-deployment-7f6d468bc9-gjpk9.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.962515417Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-deployment-7f6d468bc9-gjpk9/cdi-deployment/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.962523537Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-deployment-7f6d468bc9-gjpk9/cdi-deployment/cdi-deployment/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.962527677Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-deployment-7f6d468bc9-gjpk9/cdi-deployment/cdi-deployment/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.962608559Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-deployment-7f6d468bc9-gjpk9/cdi-deployment/cdi-deployment/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.975612351Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-deployment-7f6d468bc9-gjpk9/cdi-deployment/cdi-deployment/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.975735723Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-deployment-7f6d468bc9-gjpk9/cdi-deployment/cdi-deployment/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.975808805Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-operator-6b95b6bd7f-nf2rk/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.975899537Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-operator-6b95b6bd7f-nf2rk/cdi-operator-6b95b6bd7f-nf2rk.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.9760339Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-operator-6b95b6bd7f-nf2rk/cdi-operator/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.97604677Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-operator-6b95b6bd7f-nf2rk/cdi-operator/cdi-operator/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.97605296Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-operator-6b95b6bd7f-nf2rk/cdi-operator/cdi-operator/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.976136612Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-operator-6b95b6bd7f-nf2rk/cdi-operator/cdi-operator/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.976875528Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-operator-6b95b6bd7f-nf2rk/cdi-operator/cdi-operator/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.976996291Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-operator-6b95b6bd7f-nf2rk/cdi-operator/cdi-operator/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.977039752Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-uploadproxy-6856464f6b-xpn7d/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.977130974Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-uploadproxy-6856464f6b-xpn7d/cdi-uploadproxy-6856464f6b-xpn7d.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.977238446Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-uploadproxy-6856464f6b-xpn7d/cdi-uploadproxy/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.977248946Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-uploadproxy-6856464f6b-xpn7d/cdi-uploadproxy/cdi-uploadproxy/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.977253146Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-uploadproxy-6856464f6b-xpn7d/cdi-uploadproxy/cdi-uploadproxy/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.977315258Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-uploadproxy-6856464f6b-xpn7d/cdi-uploadproxy/cdi-uploadproxy/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.977463951Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-uploadproxy-6856464f6b-xpn7d/cdi-uploadproxy/cdi-uploadproxy/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.977586354Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cdi-uploadproxy-6856464f6b-xpn7d/cdi-uploadproxy/cdi-uploadproxy/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.977643585Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cluster-network-addons-operator-658667f846-l676q/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.977720436Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cluster-network-addons-operator-658667f846-l676q/cluster-network-addons-operator-658667f846-l676q.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.977910891Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cluster-network-addons-operator-658667f846-l676q/cluster-network-addons-operator/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.977924661Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cluster-network-addons-operator-658667f846-l676q/cluster-network-addons-operator/cluster-network-addons-operator/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.977931481Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cluster-network-addons-operator-658667f846-l676q/cluster-network-addons-operator/cluster-network-addons-operator/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.977999682Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cluster-network-addons-operator-658667f846-l676q/cluster-network-addons-operator/cluster-network-addons-operator/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.979082716Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cluster-network-addons-operator-658667f846-l676q/cluster-network-addons-operator/cluster-network-addons-operator/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.979205989Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cluster-network-addons-operator-658667f846-l676q/cluster-network-addons-operator/cluster-network-addons-operator/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.9792552Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cluster-network-addons-operator-658667f846-l676q/kube-rbac-proxy/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.97926475Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cluster-network-addons-operator-658667f846-l676q/kube-rbac-proxy/kube-rbac-proxy/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.97928368Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cluster-network-addons-operator-658667f846-l676q/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.979362092Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cluster-network-addons-operator-658667f846-l676q/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.979511325Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cluster-network-addons-operator-658667f846-l676q/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.979633708Z clusters/0cd423ed/namespaces/openshift-cnv/pods/cluster-network-addons-operator-658667f846-l676q/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.979688069Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hco-operator-749c6bb784-4rpc4/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.979783581Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hco-operator-749c6bb784-4rpc4/hco-operator-749c6bb784-4rpc4.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.979910994Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hco-operator-749c6bb784-4rpc4/hyperconverged-cluster-operator/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.979926264Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hco-operator-749c6bb784-4rpc4/hyperconverged-cluster-operator/hyperconverged-cluster-operator/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.979932335Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hco-operator-749c6bb784-4rpc4/hyperconverged-cluster-operator/hyperconverged-cluster-operator/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.980004626Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hco-operator-749c6bb784-4rpc4/hyperconverged-cluster-operator/hyperconverged-cluster-operator/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.994998321Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hco-operator-749c6bb784-4rpc4/hyperconverged-cluster-operator/hyperconverged-cluster-operator/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.995118714Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hco-operator-749c6bb784-4rpc4/hyperconverged-cluster-operator/hyperconverged-cluster-operator/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.995150865Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hco-webhook-f795f78f6-wnphw/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.995227236Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hco-webhook-f795f78f6-wnphw/hco-webhook-f795f78f6-wnphw.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.995352159Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hco-webhook-f795f78f6-wnphw/hyperconverged-cluster-webhook/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.995366779Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hco-webhook-f795f78f6-wnphw/hyperconverged-cluster-webhook/hyperconverged-cluster-webhook/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.99537357Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hco-webhook-f795f78f6-wnphw/hyperconverged-cluster-webhook/hyperconverged-cluster-webhook/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.995421431Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hco-webhook-f795f78f6-wnphw/hyperconverged-cluster-webhook/hyperconverged-cluster-webhook/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.995739947Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hco-webhook-f795f78f6-wnphw/hyperconverged-cluster-webhook/hyperconverged-cluster-webhook/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.995881581Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hco-webhook-f795f78f6-wnphw/hyperconverged-cluster-webhook/hyperconverged-cluster-webhook/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.995941242Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hostpath-provisioner-operator-768bd5d579-rdsxz/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.996025053Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hostpath-provisioner-operator-768bd5d579-rdsxz/hostpath-provisioner-operator-768bd5d579-rdsxz.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.996135146Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hostpath-provisioner-operator-768bd5d579-rdsxz/hostpath-provisioner-operator/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.996144076Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hostpath-provisioner-operator-768bd5d579-rdsxz/hostpath-provisioner-operator/hostpath-provisioner-operator/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.996148296Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hostpath-provisioner-operator-768bd5d579-rdsxz/hostpath-provisioner-operator/hostpath-provisioner-operator/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.996217828Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hostpath-provisioner-operator-768bd5d579-rdsxz/hostpath-provisioner-operator/hostpath-provisioner-operator/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.996365531Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hostpath-provisioner-operator-768bd5d579-rdsxz/hostpath-provisioner-operator/hostpath-provisioner-operator/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.996480723Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hostpath-provisioner-operator-768bd5d579-rdsxz/hostpath-provisioner-operator/hostpath-provisioner-operator/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.996532895Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hyperconverged-cluster-cli-download-7d748cbbbd-nlbrw/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.996611106Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hyperconverged-cluster-cli-download-7d748cbbbd-nlbrw/hyperconverged-cluster-cli-download-7d748cbbbd-nlbrw.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.996716079Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hyperconverged-cluster-cli-download-7d748cbbbd-nlbrw/server/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.996723799Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hyperconverged-cluster-cli-download-7d748cbbbd-nlbrw/server/server/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.996727899Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hyperconverged-cluster-cli-download-7d748cbbbd-nlbrw/server/server/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.996835841Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hyperconverged-cluster-cli-download-7d748cbbbd-nlbrw/server/server/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.996993714Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hyperconverged-cluster-cli-download-7d748cbbbd-nlbrw/server/server/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.997114447Z clusters/0cd423ed/namespaces/openshift-cnv/pods/hyperconverged-cluster-cli-download-7d748cbbbd-nlbrw/server/server/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.997157078Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kube-cni-linux-bridge-plugin-4n52v/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.99726676Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kube-cni-linux-bridge-plugin-4n52v/kube-cni-linux-bridge-plugin-4n52v.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.997363033Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kube-cni-linux-bridge-plugin-4n52v/cni-plugins/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.997372953Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kube-cni-linux-bridge-plugin-4n52v/cni-plugins/cni-plugins/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.997382773Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kube-cni-linux-bridge-plugin-4n52v/cni-plugins/cni-plugins/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.997454815Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kube-cni-linux-bridge-plugin-4n52v/cni-plugins/cni-plugins/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.997591087Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kube-cni-linux-bridge-plugin-4n52v/cni-plugins/cni-plugins/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.99771418Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kube-cni-linux-bridge-plugin-4n52v/cni-plugins/cni-plugins/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.997781662Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kube-cni-linux-bridge-plugin-7bzd4/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.997863484Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kube-cni-linux-bridge-plugin-7bzd4/kube-cni-linux-bridge-plugin-7bzd4.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.997955995Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kube-cni-linux-bridge-plugin-7bzd4/cni-plugins/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.997963886Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kube-cni-linux-bridge-plugin-7bzd4/cni-plugins/cni-plugins/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.997967816Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kube-cni-linux-bridge-plugin-7bzd4/cni-plugins/cni-plugins/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.998047408Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kube-cni-linux-bridge-plugin-7bzd4/cni-plugins/cni-plugins/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.99817918Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kube-cni-linux-bridge-plugin-7bzd4/cni-plugins/cni-plugins/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.998303013Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kube-cni-linux-bridge-plugin-7bzd4/cni-plugins/cni-plugins/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.998353844Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kube-cni-linux-bridge-plugin-nh9br/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.998432286Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kube-cni-linux-bridge-plugin-nh9br/kube-cni-linux-bridge-plugin-nh9br.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.998515498Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kube-cni-linux-bridge-plugin-nh9br/cni-plugins/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.998523068Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kube-cni-linux-bridge-plugin-nh9br/cni-plugins/cni-plugins/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.998526968Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kube-cni-linux-bridge-plugin-nh9br/cni-plugins/cni-plugins/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.99861626Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kube-cni-linux-bridge-plugin-nh9br/cni-plugins/cni-plugins/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.998742012Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kube-cni-linux-bridge-plugin-nh9br/cni-plugins/cni-plugins/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.998881795Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kube-cni-linux-bridge-plugin-nh9br/cni-plugins/cni-plugins/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.998934607Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubemacpool-cert-manager-677779b56-vrmr7/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.999012898Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubemacpool-cert-manager-677779b56-vrmr7/kubemacpool-cert-manager-677779b56-vrmr7.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.999161952Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubemacpool-cert-manager-677779b56-vrmr7/manager/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.999169452Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubemacpool-cert-manager-677779b56-vrmr7/manager/manager/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.999179122Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubemacpool-cert-manager-677779b56-vrmr7/manager/manager/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.999259184Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubemacpool-cert-manager-677779b56-vrmr7/manager/manager/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.999445348Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubemacpool-cert-manager-677779b56-vrmr7/manager/manager/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.99956525Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubemacpool-cert-manager-677779b56-vrmr7/manager/manager/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:40.999619962Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubemacpool-mac-controller-manager-6468947cf5-r2rcv/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.999701073Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubemacpool-mac-controller-manager-6468947cf5-r2rcv/kubemacpool-mac-controller-manager-6468947cf5-r2rcv.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:40.999840036Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubemacpool-mac-controller-manager-6468947cf5-r2rcv/kube-rbac-proxy/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.999851047Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubemacpool-mac-controller-manager-6468947cf5-r2rcv/kube-rbac-proxy/kube-rbac-proxy/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.999855167Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubemacpool-mac-controller-manager-6468947cf5-r2rcv/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:40.999926868Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubemacpool-mac-controller-manager-6468947cf5-r2rcv/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.000079672Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubemacpool-mac-controller-manager-6468947cf5-r2rcv/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.000177884Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubemacpool-mac-controller-manager-6468947cf5-r2rcv/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.000219484Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubemacpool-mac-controller-manager-6468947cf5-r2rcv/manager/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.000229325Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubemacpool-mac-controller-manager-6468947cf5-r2rcv/manager/manager/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.000233335Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubemacpool-mac-controller-manager-6468947cf5-r2rcv/manager/manager/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.000301186Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubemacpool-mac-controller-manager-6468947cf5-r2rcv/manager/manager/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.000443259Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubemacpool-mac-controller-manager-6468947cf5-r2rcv/manager/manager/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.000530451Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-apiserver-proxy-5c5fc795c5-ccvtn/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.000606153Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-apiserver-proxy-5c5fc795c5-ccvtn/kubevirt-apiserver-proxy-5c5fc795c5-ccvtn.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:41.000702935Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-apiserver-proxy-5c5fc795c5-ccvtn/kubevirt-apiserver-proxy/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.000712715Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-apiserver-proxy-5c5fc795c5-ccvtn/kubevirt-apiserver-proxy/kubevirt-apiserver-proxy/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.000717865Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-apiserver-proxy-5c5fc795c5-ccvtn/kubevirt-apiserver-proxy/kubevirt-apiserver-proxy/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.000818008Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-apiserver-proxy-5c5fc795c5-ccvtn/kubevirt-apiserver-proxy/kubevirt-apiserver-proxy/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.000967231Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-apiserver-proxy-5c5fc795c5-ccvtn/kubevirt-apiserver-proxy/kubevirt-apiserver-proxy/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.001086883Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-apiserver-proxy-5c5fc795c5-ccvtn/kubevirt-apiserver-proxy/kubevirt-apiserver-proxy/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.001134514Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-apiserver-proxy-5c5fc795c5-lnl8h/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.001226446Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-apiserver-proxy-5c5fc795c5-lnl8h/kubevirt-apiserver-proxy-5c5fc795c5-lnl8h.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:41.001317368Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-apiserver-proxy-5c5fc795c5-lnl8h/kubevirt-apiserver-proxy/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.001325908Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-apiserver-proxy-5c5fc795c5-lnl8h/kubevirt-apiserver-proxy/kubevirt-apiserver-proxy/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.001330049Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-apiserver-proxy-5c5fc795c5-lnl8h/kubevirt-apiserver-proxy/kubevirt-apiserver-proxy/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.001411781Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-apiserver-proxy-5c5fc795c5-lnl8h/kubevirt-apiserver-proxy/kubevirt-apiserver-proxy/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.001551613Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-apiserver-proxy-5c5fc795c5-lnl8h/kubevirt-apiserver-proxy/kubevirt-apiserver-proxy/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.001674436Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-apiserver-proxy-5c5fc795c5-lnl8h/kubevirt-apiserver-proxy/kubevirt-apiserver-proxy/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.001729467Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-console-plugin-78d96d6dd4-pt6xs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.00183561Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-console-plugin-78d96d6dd4-pt6xs/kubevirt-console-plugin-78d96d6dd4-pt6xs.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:41.001938232Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-console-plugin-78d96d6dd4-pt6xs/kubevirt-console-plugin/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.001946222Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-console-plugin-78d96d6dd4-pt6xs/kubevirt-console-plugin/kubevirt-console-plugin/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.001951232Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-console-plugin-78d96d6dd4-pt6xs/kubevirt-console-plugin/kubevirt-console-plugin/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.002049814Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-console-plugin-78d96d6dd4-pt6xs/kubevirt-console-plugin/kubevirt-console-plugin/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.002195947Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-console-plugin-78d96d6dd4-pt6xs/kubevirt-console-plugin/kubevirt-console-plugin/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.00231687Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-console-plugin-78d96d6dd4-pt6xs/kubevirt-console-plugin/kubevirt-console-plugin/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.002358141Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-console-plugin-78d96d6dd4-tjj5h/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.002424712Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-console-plugin-78d96d6dd4-tjj5h/kubevirt-console-plugin-78d96d6dd4-tjj5h.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:41.002519114Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-console-plugin-78d96d6dd4-tjj5h/kubevirt-console-plugin/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.002527665Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-console-plugin-78d96d6dd4-tjj5h/kubevirt-console-plugin/kubevirt-console-plugin/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.002531755Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-console-plugin-78d96d6dd4-tjj5h/kubevirt-console-plugin/kubevirt-console-plugin/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.002616016Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-console-plugin-78d96d6dd4-tjj5h/kubevirt-console-plugin/kubevirt-console-plugin/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.002750479Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-console-plugin-78d96d6dd4-tjj5h/kubevirt-console-plugin/kubevirt-console-plugin/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.002887073Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-console-plugin-78d96d6dd4-tjj5h/kubevirt-console-plugin/kubevirt-console-plugin/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.002944984Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-ipam-controller-manager-566c85498f-q4mhg/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.003025676Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-ipam-controller-manager-566c85498f-q4mhg/kubevirt-ipam-controller-manager-566c85498f-q4mhg.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:41.003122067Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-ipam-controller-manager-566c85498f-q4mhg/manager/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.003129578Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-ipam-controller-manager-566c85498f-q4mhg/manager/manager/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.003133608Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-ipam-controller-manager-566c85498f-q4mhg/manager/manager/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.003213769Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-ipam-controller-manager-566c85498f-q4mhg/manager/manager/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.003357403Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-ipam-controller-manager-566c85498f-q4mhg/manager/manager/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.003480175Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-ipam-controller-manager-566c85498f-q4mhg/manager/manager/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.003534716Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-ipam-controller-manager-566c85498f-zpgf8/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.003612008Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-ipam-controller-manager-566c85498f-zpgf8/kubevirt-ipam-controller-manager-566c85498f-zpgf8.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:41.00370033Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-ipam-controller-manager-566c85498f-zpgf8/manager/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.00370764Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-ipam-controller-manager-566c85498f-zpgf8/manager/manager/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.00371176Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-ipam-controller-manager-566c85498f-zpgf8/manager/manager/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.003819273Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-ipam-controller-manager-566c85498f-zpgf8/manager/manager/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.003981796Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-ipam-controller-manager-566c85498f-zpgf8/manager/manager/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.004094608Z clusters/0cd423ed/namespaces/openshift-cnv/pods/kubevirt-ipam-controller-manager-566c85498f-zpgf8/manager/manager/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.00414817Z clusters/0cd423ed/namespaces/openshift-cnv/pods/ssp-operator-59ddfd9959-2f89k/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.004230722Z clusters/0cd423ed/namespaces/openshift-cnv/pods/ssp-operator-59ddfd9959-2f89k/ssp-operator-59ddfd9959-2f89k.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:41.004339224Z clusters/0cd423ed/namespaces/openshift-cnv/pods/ssp-operator-59ddfd9959-2f89k/manager/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.004347024Z clusters/0cd423ed/namespaces/openshift-cnv/pods/ssp-operator-59ddfd9959-2f89k/manager/manager/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.004351244Z clusters/0cd423ed/namespaces/openshift-cnv/pods/ssp-operator-59ddfd9959-2f89k/manager/manager/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.004426406Z clusters/0cd423ed/namespaces/openshift-cnv/pods/ssp-operator-59ddfd9959-2f89k/manager/manager/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.006872959Z clusters/0cd423ed/namespaces/openshift-cnv/pods/ssp-operator-59ddfd9959-2f89k/manager/manager/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.007009622Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-api-5d697454c5-7qvf2/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.007091424Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-api-5d697454c5-7qvf2/virt-api-5d697454c5-7qvf2.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:41.007190096Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-api-5d697454c5-7qvf2/virt-api/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.007202116Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-api-5d697454c5-7qvf2/virt-api/virt-api/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.007209266Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-api-5d697454c5-7qvf2/virt-api/virt-api/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.007286038Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-api-5d697454c5-7qvf2/virt-api/virt-api/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.009072336Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-api-5d697454c5-7qvf2/virt-api/virt-api/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.00921886Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-api-5d697454c5-7qvf2/virt-api/virt-api/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.009272861Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-api-5d697454c5-xj7wc/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.009347133Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-api-5d697454c5-xj7wc/virt-api-5d697454c5-xj7wc.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:41.009453325Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-api-5d697454c5-xj7wc/virt-api/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.009466585Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-api-5d697454c5-xj7wc/virt-api/virt-api/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.009472785Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-api-5d697454c5-xj7wc/virt-api/virt-api/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.009546547Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-api-5d697454c5-xj7wc/virt-api/virt-api/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.011401157Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-api-5d697454c5-xj7wc/virt-api/virt-api/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.01152779Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-api-5d697454c5-xj7wc/virt-api/virt-api/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.011580681Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-controller-64dcd467b4-72bgj/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.011661673Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-controller-64dcd467b4-72bgj/virt-controller-64dcd467b4-72bgj.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:41.011780675Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-controller-64dcd467b4-72bgj/virt-controller/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.011793886Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-controller-64dcd467b4-72bgj/virt-controller/virt-controller/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.011807536Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-controller-64dcd467b4-72bgj/virt-controller/virt-controller/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.011880728Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-controller-64dcd467b4-72bgj/virt-controller/virt-controller/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.012072012Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-controller-64dcd467b4-72bgj/virt-controller/virt-controller/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.012195604Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-controller-64dcd467b4-72bgj/virt-controller/virt-controller/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.012247516Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-controller-64dcd467b4-98fx5/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.012316017Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-controller-64dcd467b4-98fx5/virt-controller-64dcd467b4-98fx5.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:41.012408509Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-controller-64dcd467b4-98fx5/virt-controller/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.012416039Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-controller-64dcd467b4-98fx5/virt-controller/virt-controller/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.012420259Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-controller-64dcd467b4-98fx5/virt-controller/virt-controller/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.012496251Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-controller-64dcd467b4-98fx5/virt-controller/virt-controller/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.012653174Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-controller-64dcd467b4-98fx5/virt-controller/virt-controller/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.012786647Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-controller-64dcd467b4-98fx5/virt-controller/virt-controller/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.012845778Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-exportproxy-6b85c76b89-l827n/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.01292141Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-exportproxy-6b85c76b89-l827n/virt-exportproxy-6b85c76b89-l827n.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:41.013016362Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-exportproxy-6b85c76b89-l827n/exportproxy/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.013023742Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-exportproxy-6b85c76b89-l827n/exportproxy/exportproxy/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.013027932Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-exportproxy-6b85c76b89-l827n/exportproxy/exportproxy/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.013113654Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-exportproxy-6b85c76b89-l827n/exportproxy/exportproxy/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.013263158Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-exportproxy-6b85c76b89-l827n/exportproxy/exportproxy/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.01338168Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-exportproxy-6b85c76b89-l827n/exportproxy/exportproxy/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.013430911Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-exportproxy-6b85c76b89-wbjdg/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.013507043Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-exportproxy-6b85c76b89-wbjdg/virt-exportproxy-6b85c76b89-wbjdg.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:41.013597455Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-exportproxy-6b85c76b89-wbjdg/exportproxy/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.013605015Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-exportproxy-6b85c76b89-wbjdg/exportproxy/exportproxy/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.013618005Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-exportproxy-6b85c76b89-wbjdg/exportproxy/exportproxy/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.013701687Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-exportproxy-6b85c76b89-wbjdg/exportproxy/exportproxy/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.01385941Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-exportproxy-6b85c76b89-wbjdg/exportproxy/exportproxy/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.013993763Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-exportproxy-6b85c76b89-wbjdg/exportproxy/exportproxy/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.014039114Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-4n22f/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.014121846Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-4n22f/virt-handler-4n22f.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:41.014235148Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-4n22f/virt-handler/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.014242639Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-4n22f/virt-handler/virt-handler/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.014246819Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-4n22f/virt-handler/virt-handler/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.0143197Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-4n22f/virt-handler/virt-handler/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.014551115Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-4n22f/virt-handler/virt-handler/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.014660718Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-4n22f/virt-handler/virt-handler/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.014708689Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-4n22f/virt-launcher/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.014716419Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-4n22f/virt-launcher/virt-launcher/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.014720509Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-4n22f/virt-launcher/virt-launcher/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.014823311Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-4n22f/virt-launcher/virt-launcher/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.014961784Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-4n22f/virt-launcher/virt-launcher/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.015082157Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-4n22f/virt-launcher/virt-launcher/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.015133978Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-kmbqz/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.0152222Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-kmbqz/virt-handler-kmbqz.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:41.015318842Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-kmbqz/virt-handler/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.015326682Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-kmbqz/virt-handler/virt-handler/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.015331102Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-kmbqz/virt-handler/virt-handler/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.015412034Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-kmbqz/virt-handler/virt-handler/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.015646899Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-kmbqz/virt-handler/virt-handler/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.015778912Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-kmbqz/virt-handler/virt-handler/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.015834703Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-kmbqz/virt-launcher/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.015843094Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-kmbqz/virt-launcher/virt-launcher/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.015847483Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-kmbqz/virt-launcher/virt-launcher/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.015927255Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-kmbqz/virt-launcher/virt-launcher/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.016068048Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-kmbqz/virt-launcher/virt-launcher/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.016185951Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-kmbqz/virt-launcher/virt-launcher/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.016236852Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-xjvqm/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.016312694Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-xjvqm/virt-handler-xjvqm.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:41.016416486Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-xjvqm/virt-handler/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.016423666Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-xjvqm/virt-handler/virt-handler/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.016427776Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-xjvqm/virt-handler/virt-handler/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.016507288Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-xjvqm/virt-handler/virt-handler/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.016714772Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-xjvqm/virt-handler/virt-handler/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.016857785Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-xjvqm/virt-handler/virt-handler/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.016919157Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-xjvqm/virt-launcher/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.016932957Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-xjvqm/virt-launcher/virt-launcher/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.016939517Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-xjvqm/virt-launcher/virt-launcher/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.017009059Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-xjvqm/virt-launcher/virt-launcher/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.017151262Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-xjvqm/virt-launcher/virt-launcher/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.017283825Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-handler-xjvqm/virt-launcher/virt-launcher/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.017334496Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-operator-6d67dd8845-h4bj4/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.017415028Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-operator-6d67dd8845-h4bj4/virt-operator-6d67dd8845-h4bj4.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:41.017560141Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-operator-6d67dd8845-h4bj4/virt-operator/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.017567581Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-operator-6d67dd8845-h4bj4/virt-operator/virt-operator/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.017573541Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-operator-6d67dd8845-h4bj4/virt-operator/virt-operator/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.017662413Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-operator-6d67dd8845-h4bj4/virt-operator/virt-operator/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.017881688Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-operator-6d67dd8845-h4bj4/virt-operator/virt-operator/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.01800034Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-operator-6d67dd8845-h4bj4/virt-operator/virt-operator/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.018048241Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-operator-6d67dd8845-vs5mv/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.018124263Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-operator-6d67dd8845-vs5mv/virt-operator-6d67dd8845-vs5mv.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:41.018240655Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-operator-6d67dd8845-vs5mv/virt-operator/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.018248816Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-operator-6d67dd8845-vs5mv/virt-operator/virt-operator/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.018252956Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-operator-6d67dd8845-vs5mv/virt-operator/virt-operator/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.018323287Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-operator-6d67dd8845-vs5mv/virt-operator/virt-operator/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.021633389Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-operator-6d67dd8845-vs5mv/virt-operator/virt-operator/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.021751371Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-operator-6d67dd8845-vs5mv/virt-operator/virt-operator/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.021814933Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-template-validator-5f84757967-dl766/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.021901595Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-template-validator-5f84757967-dl766/virt-template-validator-5f84757967-dl766.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:41.021997927Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-template-validator-5f84757967-dl766/webhook/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.022007527Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-template-validator-5f84757967-dl766/webhook/webhook/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.022012567Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-template-validator-5f84757967-dl766/webhook/webhook/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.022094849Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-template-validator-5f84757967-dl766/webhook/webhook/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.022227662Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-template-validator-5f84757967-dl766/webhook/webhook/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.022349925Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-template-validator-5f84757967-dl766/webhook/webhook/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.022403316Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-template-validator-5f84757967-kvs2d/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.022483067Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-template-validator-5f84757967-kvs2d/virt-template-validator-5f84757967-kvs2d.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:41.022568639Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-template-validator-5f84757967-kvs2d/webhook/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.02257598Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-template-validator-5f84757967-kvs2d/webhook/webhook/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.02257996Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-template-validator-5f84757967-kvs2d/webhook/webhook/logs/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.022661261Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-template-validator-5f84757967-kvs2d/webhook/webhook/logs/current.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.022843085Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-template-validator-5f84757967-kvs2d/webhook/webhook/logs/previous.insecure.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.022968688Z clusters/0cd423ed/namespaces/openshift-cnv/pods/virt-template-validator-5f84757967-kvs2d/webhook/webhook/logs/previous.log [must-gather-vqqmv] OUT 2026-01-14T16:44:41.023017839Z clusters/0cd423ed/namespaces/openshift-cnv/policy/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.023096761Z clusters/0cd423ed/namespaces/openshift-cnv/policy/poddisruptionbudgets.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:41.023183013Z clusters/0cd423ed/namespaces/openshift-cnv/pool.kubevirt.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.023253084Z clusters/0cd423ed/namespaces/openshift-cnv/pool.kubevirt.io/virtualmachinepools.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:41.023317256Z clusters/0cd423ed/namespaces/openshift-cnv/route.openshift.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.023391367Z clusters/0cd423ed/namespaces/openshift-cnv/route.openshift.io/routes.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:41.023485339Z clusters/0cd423ed/namespaces/openshift-cnv/snapshot.kubevirt.io/ [must-gather-vqqmv] OUT 2026-01-14T16:44:41.023554381Z clusters/0cd423ed/namespaces/openshift-cnv/snapshot.kubevirt.io/virtualmachinerestores.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:41.023675833Z clusters/0cd423ed/namespaces/openshift-cnv/snapshot.kubevirt.io/virtualmachinesnapshotcontents.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:41.023864888Z clusters/0cd423ed/namespaces/openshift-cnv/snapshot.kubevirt.io/virtualmachinesnapshots.yaml [must-gather-vqqmv] OUT 2026-01-14T16:44:41.029274375Z [must-gather-vqqmv] OUT 2026-01-14T16:44:41.029312986Z sent 6,511 bytes received 1,700,938 bytes 1,138,299.33 bytes/sec [must-gather-vqqmv] OUT 2026-01-14T16:44:41.029320396Z total size is 25,036,946 speedup is 14.66 [must-gather ] OUT 2026-01-14T16:44:41.247163352Z namespace/openshift-must-gather-q9fnk deleted Reprinting Cluster State: When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: 0cd423ed-2178-4c99-b8d7-354d4fc33d55 ClientVersion: 4.20.8 ClusterVersion: Stable at "4.21.0-0.nightly-2026-01-13-111112" ClusterOperators: clusteroperator/operator-lifecycle-manager is not upgradeable because ClusterServiceVersions blocking minor version upgrades to 4.22.0 or higher: - maximum supported OCP version for openshift-storage/odf-operator.v4.20.5-rhodf is 4.21 - maximum supported OCP version for openshift-storage/odf-dependencies.v4.20.5-rhodf is 4.21 Checking for additional logs in /alabama/cspi/e2e/logs Copying /alabama/cspi/e2e/logs to /logs/artifacts... It_Backup_restore_tests_Application_backup_tc-id_OADP-122_interop_skip-disconnected_Tier0_vf_csi-snapshots_Django_application_with_BSL_CSI_exclude_aro-4 It_Backup_restore_tests_Application_backup_tc-id_OADP-437_interop_smoke_Tier0_vf_fsb-kopia_MySQL_application_with_filesystem_Kopia_mr-check It_Incremental_backup_restore_tests_Incremental_restore_pod_count_tc-id_OADP-165_interop_Tier0_vf_restore-policies_Todolist_app_with_CSI_-_policy_update It_datamover_DataMover_Backup_Restore_stateful_application_with_CSI_tc-id_OADP-439_interop_Tier0_vf_data-mover_MySQL_application It_skip-disconnected_Restore_hooks_tests_Successful_Init_hook_tc-id_OADP-164_interop_smoke_Tier0_MySQL_app_with_Kopia artifacts Checking if yq exists Installing yq % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 12.9M 100 12.9M 0 0 92.9M 0 --:--:-- --:--:-- --:--:-- 92.9M Collecting original results in /logs/artifacts/original_results/ Patching Tests Result File: /logs/artifacts/junit_oadp_cnv_results.xml Mapping Test Suite Name To: OADP-lp-interop Patching Tests Result File: /logs/artifacts/junit_oadp_interop_results.xml Mapping Test Suite Name To: OADP-lp-interop Patching Tests Result File: /logs/artifacts/original_results/junit_oadp_cnv_results.xml Mapping Test Suite Name To: OADP-lp-interop Patching Tests Result File: /logs/artifacts/original_results/junit_oadp_interop_results.xml Mapping Test Suite Name To: OADP-lp-interop