Extract /home/jenkins/oadp-e2e-qe.tar.gz to /alabama/cspi Extract /home/jenkins/oadp-apps-deployer.tar.gz to /alabama/oadpApps Extract /home/jenkins/mtc-python-client.tar.gz to /alabama/pyclient Create and populate /tmp/test-settings... Login as Kubeadmin to the test cluster at https://api.ci-op-bkpv69pr-78a3d.cspilp.interop.ccitredhat.com:6443... WARNING: Using insecure TLS client config. Setting this option is not supported! Login successful. You have access to 78 projects, the list has been suppressed. You can list all projects with 'oc projects' Using project "default". Create virtual environment and install required packages... Collecting ansible_runner Downloading ansible_runner-2.4.0-py3-none-any.whl.metadata (3.5 kB) Collecting pexpect>=4.5 (from ansible_runner) Downloading pexpect-4.9.0-py2.py3-none-any.whl.metadata (2.5 kB) Collecting packaging (from ansible_runner) Downloading packaging-24.2-py3-none-any.whl.metadata (3.2 kB) Collecting python-daemon (from ansible_runner) Downloading python_daemon-3.1.2-py3-none-any.whl.metadata (4.8 kB) Collecting pyyaml (from ansible_runner) Downloading PyYAML-6.0.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB) Collecting ptyprocess>=0.5 (from pexpect>=4.5->ansible_runner) Downloading ptyprocess-0.7.0-py2.py3-none-any.whl.metadata (1.3 kB) Collecting lockfile>=0.10 (from python-daemon->ansible_runner) Downloading lockfile-0.12.2-py2.py3-none-any.whl.metadata (2.4 kB) Downloading ansible_runner-2.4.0-py3-none-any.whl (79 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 79.7/79.7 kB 901.5 kB/s eta 0:00:00 Downloading pexpect-4.9.0-py2.py3-none-any.whl (63 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 63.8/63.8 kB 925.3 kB/s eta 0:00:00 Downloading packaging-24.2-py3-none-any.whl (65 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65.5/65.5 kB 537.1 kB/s eta 0:00:00 Downloading python_daemon-3.1.2-py3-none-any.whl (30 kB) Downloading PyYAML-6.0.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (767 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 767.5/767.5 kB 6.8 MB/s eta 0:00:00 Downloading lockfile-0.12.2-py2.py3-none-any.whl (13 kB) Downloading ptyprocess-0.7.0-py2.py3-none-any.whl (13 kB) Installing collected packages: ptyprocess, lockfile, pyyaml, python-daemon, pexpect, packaging, ansible_runner Successfully installed ansible_runner-2.4.0 lockfile-0.12.2 packaging-24.2 pexpect-4.9.0 ptyprocess-0.7.0 python-daemon-3.1.2 pyyaml-6.0.2 [notice] A new release of pip is available: 23.3.2 -> 24.3.1 [notice] To update, run: pip install --upgrade pip Processing /alabama/oadpApps Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' Preparing metadata (pyproject.toml): started Preparing metadata (pyproject.toml): finished with status 'done' Building wheels for collected packages: ocpdeployer Building wheel for ocpdeployer (pyproject.toml): started Building wheel for ocpdeployer (pyproject.toml): finished with status 'done' Created wheel for ocpdeployer: filename=ocpdeployer-0.0.1-py2.py3-none-any.whl size=225442 sha256=45a1a3781e3fbe205dd39f29e931f502841cddf047f74e88c66a98def4776574 Stored in directory: /tmp/pip-ephem-wheel-cache-yl6nlnf_/wheels/55/c3/15/eb89266a7928fafe53678a24892891bbfb18405fbd475eb4c6 Successfully built ocpdeployer Installing collected packages: ocpdeployer Successfully installed ocpdeployer-0.0.1 [notice] A new release of pip is available: 23.3.2 -> 24.3.1 [notice] To update, run: pip install --upgrade pip Processing /alabama/pyclient Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' Preparing metadata (pyproject.toml): started Preparing metadata (pyproject.toml): finished with status 'done' Collecting suds-py3 (from mtc==0.0.1) Downloading suds_py3-1.4.5.0-py3-none-any.whl.metadata (778 bytes) Collecting requests (from mtc==0.0.1) Downloading requests-2.32.3-py3-none-any.whl.metadata (4.6 kB) Collecting jinja2 (from mtc==0.0.1) Downloading jinja2-3.1.5-py3-none-any.whl.metadata (2.6 kB) Collecting kubernetes==11.0.0 (from mtc==0.0.1) Downloading kubernetes-11.0.0-py3-none-any.whl.metadata (1.5 kB) Collecting openshift==0.11.2 (from mtc==0.0.1) Downloading openshift-0.11.2.tar.gz (19 kB) Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' Preparing metadata (pyproject.toml): started Preparing metadata (pyproject.toml): finished with status 'done' Collecting certifi>=14.05.14 (from kubernetes==11.0.0->mtc==0.0.1) Downloading certifi-2024.12.14-py3-none-any.whl.metadata (2.3 kB) Collecting six>=1.9.0 (from kubernetes==11.0.0->mtc==0.0.1) Downloading six-1.17.0-py2.py3-none-any.whl.metadata (1.7 kB) Collecting python-dateutil>=2.5.3 (from kubernetes==11.0.0->mtc==0.0.1) Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) Collecting setuptools>=21.0.0 (from kubernetes==11.0.0->mtc==0.0.1) Using cached setuptools-75.7.0-py3-none-any.whl.metadata (6.7 kB) Requirement already satisfied: pyyaml>=3.12 in /alabama/venv/lib64/python3.12/site-packages (from kubernetes==11.0.0->mtc==0.0.1) (6.0.2) Collecting google-auth>=1.0.1 (from kubernetes==11.0.0->mtc==0.0.1) Downloading google_auth-2.37.0-py2.py3-none-any.whl.metadata (4.8 kB) Collecting websocket-client!=0.40.0,!=0.41.*,!=0.42.*,>=0.32.0 (from kubernetes==11.0.0->mtc==0.0.1) Downloading websocket_client-1.8.0-py3-none-any.whl.metadata (8.0 kB) Collecting requests-oauthlib (from kubernetes==11.0.0->mtc==0.0.1) Downloading requests_oauthlib-2.0.0-py2.py3-none-any.whl.metadata (11 kB) Collecting urllib3>=1.24.2 (from kubernetes==11.0.0->mtc==0.0.1) Downloading urllib3-2.3.0-py3-none-any.whl.metadata (6.5 kB) Collecting python-string-utils (from openshift==0.11.2->mtc==0.0.1) Downloading python_string_utils-1.0.0-py3-none-any.whl.metadata (12 kB) Collecting ruamel.yaml>=0.15 (from openshift==0.11.2->mtc==0.0.1) Downloading ruamel.yaml-0.18.9-py3-none-any.whl.metadata (22 kB) Collecting MarkupSafe>=2.0 (from jinja2->mtc==0.0.1) Downloading MarkupSafe-3.0.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.0 kB) Collecting charset-normalizer<4,>=2 (from requests->mtc==0.0.1) Downloading charset_normalizer-3.4.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (35 kB) Collecting idna<4,>=2.5 (from requests->mtc==0.0.1) Downloading idna-3.10-py3-none-any.whl.metadata (10 kB) Collecting cachetools<6.0,>=2.0.0 (from google-auth>=1.0.1->kubernetes==11.0.0->mtc==0.0.1) Downloading cachetools-5.5.0-py3-none-any.whl.metadata (5.3 kB) Collecting pyasn1-modules>=0.2.1 (from google-auth>=1.0.1->kubernetes==11.0.0->mtc==0.0.1) Downloading pyasn1_modules-0.4.1-py3-none-any.whl.metadata (3.5 kB) Collecting rsa<5,>=3.1.4 (from google-auth>=1.0.1->kubernetes==11.0.0->mtc==0.0.1) Downloading rsa-4.9-py3-none-any.whl.metadata (4.2 kB) Collecting ruamel.yaml.clib>=0.2.7 (from ruamel.yaml>=0.15->openshift==0.11.2->mtc==0.0.1) Downloading ruamel.yaml.clib-0.2.12-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.7 kB) Collecting oauthlib>=3.0.0 (from requests-oauthlib->kubernetes==11.0.0->mtc==0.0.1) Downloading oauthlib-3.2.2-py3-none-any.whl.metadata (7.5 kB) Collecting pyasn1<0.7.0,>=0.4.6 (from pyasn1-modules>=0.2.1->google-auth>=1.0.1->kubernetes==11.0.0->mtc==0.0.1) Downloading pyasn1-0.6.1-py3-none-any.whl.metadata (8.4 kB) Downloading kubernetes-11.0.0-py3-none-any.whl (1.5 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.5/1.5 MB 34.8 MB/s eta 0:00:00 Downloading jinja2-3.1.5-py3-none-any.whl (134 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 134.6/134.6 kB 16.9 MB/s eta 0:00:00 Downloading requests-2.32.3-py3-none-any.whl (64 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.9/64.9 kB 7.5 MB/s eta 0:00:00 Downloading suds_py3-1.4.5.0-py3-none-any.whl (298 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 298.8/298.8 kB 30.9 MB/s eta 0:00:00 Downloading certifi-2024.12.14-py3-none-any.whl (164 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 164.9/164.9 kB 19.4 MB/s eta 0:00:00 Downloading charset_normalizer-3.4.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (145 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 145.3/145.3 kB 19.4 MB/s eta 0:00:00 Downloading google_auth-2.37.0-py2.py3-none-any.whl (209 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 209.8/209.8 kB 22.6 MB/s eta 0:00:00 Downloading idna-3.10-py3-none-any.whl (70 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 70.4/70.4 kB 7.9 MB/s eta 0:00:00 Downloading MarkupSafe-3.0.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (23 kB) Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 229.9/229.9 kB 25.1 MB/s eta 0:00:00 Downloading ruamel.yaml-0.18.9-py3-none-any.whl (117 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 117.6/117.6 kB 14.3 MB/s eta 0:00:00 Using cached setuptools-75.7.0-py3-none-any.whl (1.2 MB) Downloading six-1.17.0-py2.py3-none-any.whl (11 kB) Downloading urllib3-2.3.0-py3-none-any.whl (128 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 128.4/128.4 kB 15.7 MB/s eta 0:00:00 Downloading websocket_client-1.8.0-py3-none-any.whl (58 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.8/58.8 kB 6.9 MB/s eta 0:00:00 Downloading python_string_utils-1.0.0-py3-none-any.whl (26 kB) Downloading requests_oauthlib-2.0.0-py2.py3-none-any.whl (24 kB) Downloading cachetools-5.5.0-py3-none-any.whl (9.5 kB) Downloading oauthlib-3.2.2-py3-none-any.whl (151 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 151.7/151.7 kB 17.8 MB/s eta 0:00:00 Downloading pyasn1_modules-0.4.1-py3-none-any.whl (181 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 181.5/181.5 kB 20.9 MB/s eta 0:00:00 Downloading rsa-4.9-py3-none-any.whl (34 kB) Downloading ruamel.yaml.clib-0.2.12-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (754 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 754.1/754.1 kB 60.7 MB/s eta 0:00:00 Downloading pyasn1-0.6.1-py3-none-any.whl (83 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 83.1/83.1 kB 9.5 MB/s eta 0:00:00 Building wheels for collected packages: mtc, openshift Building wheel for mtc (pyproject.toml): started Building wheel for mtc (pyproject.toml): finished with status 'done' Created wheel for mtc: filename=mtc-0.0.1-py3-none-any.whl size=31106 sha256=9200e1d1787b559238e40ca3ef72875dd62b453d39475a0a2a478a37a2a7e07f Stored in directory: /tmp/pip-ephem-wheel-cache-dws5p09v/wheels/f1/2c/83/c09cb54cb0e821a8186cf5320758c27e7227ec862045210509 Building wheel for openshift (pyproject.toml): started Building wheel for openshift (pyproject.toml): finished with status 'done' Created wheel for openshift: filename=openshift-0.11.2-py3-none-any.whl size=19831 sha256=fe5289e389e7d949637eb8ac2dcc4d258319b094ffe0620ac538397d5753ce85 Stored in directory: /alabama/.cache/pip/wheels/34/b7/02/4eb142942314b119c5fb3d4e595ac59486c1f3d79ff665397d Successfully built mtc openshift Installing collected packages: suds-py3, websocket-client, urllib3, six, setuptools, ruamel.yaml.clib, python-string-utils, pyasn1, oauthlib, MarkupSafe, idna, charset-normalizer, certifi, cachetools, ruamel.yaml, rsa, requests, python-dateutil, pyasn1-modules, jinja2, requests-oauthlib, google-auth, kubernetes, openshift, mtc Successfully installed MarkupSafe-3.0.2 cachetools-5.5.0 certifi-2024.12.14 charset-normalizer-3.4.1 google-auth-2.37.0 idna-3.10 jinja2-3.1.5 kubernetes-11.0.0 mtc-0.0.1 oauthlib-3.2.2 openshift-0.11.2 pyasn1-0.6.1 pyasn1-modules-0.4.1 python-dateutil-2.9.0.post0 python-string-utils-1.0.0 requests-2.32.3 requests-oauthlib-2.0.0 rsa-4.9 ruamel.yaml-0.18.9 ruamel.yaml.clib-0.2.12 setuptools-75.7.0 six-1.17.0 suds-py3-1.4.5.0 urllib3-2.3.0 websocket-client-1.8.0 [notice] A new release of pip is available: 23.3.2 -> 24.3.1 [notice] To update, run: pip install --upgrade pip go: downloading github.com/openshift/oadp-operator v1.0.2-0.20241025125902-d7cb5732aa2f go: downloading github.com/onsi/gomega v1.34.2 go: downloading github.com/onsi/ginkgo/v2 v2.22.0 go: downloading github.com/vmware-tanzu/velero v1.14.0 go: downloading k8s.io/apimachinery v0.29.0 go: downloading k8s.io/api v0.29.0 go: downloading github.com/operator-framework/api v0.14.1-0.20220413143725-33310d6154f3 go: downloading k8s.io/utils v0.0.0-20230726121419-3b25d923346b go: downloading github.com/aws/aws-sdk-go v1.44.253 go: downloading github.com/kubernetes-csi/external-snapshotter/client/v4 v4.2.0 go: downloading gopkg.in/yaml.v2 v2.4.0 go: downloading sigs.k8s.io/controller-runtime v0.17.2 go: downloading k8s.io/client-go v0.29.0 go: downloading github.com/andygrunwald/go-jira v1.16.0 go: downloading github.com/apenella/go-ansible v1.1.5 go: downloading github.com/google/uuid v1.6.0 go: downloading github.com/openshift/api v0.0.0-20230414143018-3367bc7e6ac7 go: downloading github.com/openshift/client-go v0.0.0-20211209144617-7385dd6338e3 go: downloading k8s.io/kubectl v0.29.0 go: downloading sigs.k8s.io/yaml v1.4.0 go: downloading github.com/google/go-cmp v0.6.0 go: downloading github.com/go-logr/logr v1.4.2 go: downloading github.com/gogo/protobuf v1.3.2 go: downloading github.com/google/gofuzz v1.2.0 go: downloading k8s.io/klog/v2 v2.110.1 go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.4.1 go: downloading gopkg.in/inf.v0 v0.9.1 go: downloading github.com/spf13/pflag v1.0.5 go: downloading github.com/fatih/structs v1.1.0 go: downloading github.com/golang-jwt/jwt/v4 v4.5.0 go: downloading github.com/google/go-querystring v1.1.0 go: downloading github.com/pkg/errors v0.9.1 go: downloading github.com/trivago/tgo v1.0.7 go: downloading github.com/evanphx/json-patch/v5 v5.8.0 go: downloading github.com/sirupsen/logrus v1.9.3 go: downloading github.com/evanphx/json-patch v5.6.0+incompatible go: downloading github.com/apenella/go-common-utils/data v0.0.0-20210528133155-34ba915e28c8 go: downloading github.com/apenella/go-common-utils/error v0.0.0-20210528133155-34ba915e28c8 go: downloading github.com/imdario/mergo v0.3.13 go: downloading golang.org/x/term v0.25.0 go: downloading github.com/gorilla/websocket v1.5.0 go: downloading golang.org/x/net v0.30.0 go: downloading gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c go: downloading github.com/stretchr/testify v1.9.0 go: downloading gopkg.in/yaml.v3 v3.0.1 go: downloading golang.org/x/sys v0.26.0 go: downloading github.com/spf13/cobra v1.7.0 go: downloading k8s.io/cli-runtime v0.29.0 go: downloading k8s.io/component-base v0.29.0 go: downloading sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd go: downloading github.com/json-iterator/go v1.1.12 go: downloading k8s.io/kube-openapi v0.0.0-20231010175941-2dd684a91f00 go: downloading k8s.io/apiextensions-apiserver v0.29.0 go: downloading golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56 go: downloading go.uber.org/zap v1.27.0 go: downloading github.com/go-logr/zapr v1.3.0 go: downloading github.com/blang/semver/v4 v4.0.0 go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da go: downloading github.com/golang/protobuf v1.5.4 go: downloading github.com/google/gnostic-models v0.6.8 go: downloading google.golang.org/protobuf v1.34.1 go: downloading golang.org/x/time v0.5.0 go: downloading github.com/moby/spdystream v0.2.0 go: downloading github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 go: downloading golang.org/x/oauth2 v0.19.0 go: downloading github.com/kr/pretty v0.3.1 go: downloading github.com/davecgh/go-spew v1.1.1 go: downloading github.com/pmezard/go-difflib v1.0.0 go: downloading golang.org/x/text v0.19.0 go: downloading github.com/go-task/slim-sprig/v3 v3.0.0 go: downloading golang.org/x/tools v0.26.0 go: downloading github.com/inconshreveable/mousetrap v1.1.0 go: downloading github.com/kubernetes-csi/external-snapshotter/client/v7 v7.0.0 go: downloading github.com/jonboulle/clockwork v0.2.2 go: downloading k8s.io/component-helpers v0.29.0 go: downloading github.com/daviddengcn/go-colortext v1.0.0 go: downloading github.com/bombsimon/logrusr/v3 v3.0.0 go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de go: downloading github.com/prometheus/client_golang v1.19.0 go: downloading github.com/distribution/reference v0.5.0 go: downloading github.com/moby/term v0.0.0-20221205130635-1aeaba878587 go: downloading github.com/fvbommel/sortorder v1.1.0 go: downloading sigs.k8s.io/kustomize/kustomize/v5 v5.0.4-0.20230601165947-6ce0bf390ce3 go: downloading sigs.k8s.io/kustomize/kyaml v0.14.3-0.20230601165947-6ce0bf390ce3 go: downloading github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d go: downloading github.com/lithammer/dedent v1.1.0 go: downloading k8s.io/metrics v0.29.0 go: downloading github.com/MakeNowJust/heredoc v1.0.0 go: downloading github.com/mitchellh/go-wordwrap v1.0.1 go: downloading github.com/chai2010/gettext-go v1.0.2 go: downloading github.com/russross/blackfriday/v2 v2.1.0 go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd go: downloading github.com/modern-go/reflect2 v1.0.2 go: downloading go.uber.org/multierr v1.11.0 go: downloading github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f go: downloading github.com/kr/text v0.2.0 go: downloading github.com/rogpeppe/go-internal v1.12.0 go: downloading github.com/google/pprof v0.0.0-20241029153458-d1b30febd7db go: downloading github.com/go-openapi/jsonreference v0.20.2 go: downloading github.com/go-openapi/swag v0.22.3 go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 go: downloading golang.org/x/sync v0.8.0 go: downloading sigs.k8s.io/kustomize/api v0.13.5-0.20230601165947-6ce0bf390ce3 go: downloading github.com/fatih/camelcase v1.0.0 go: downloading github.com/golangplus/testing v1.0.0 go: downloading github.com/gobwas/glob v0.2.3 go: downloading github.com/hashicorp/go-plugin v1.6.0 go: downloading google.golang.org/grpc v1.63.2 go: downloading github.com/robfig/cron v1.1.0 go: downloading github.com/onsi/ginkgo v1.16.5 go: downloading github.com/opencontainers/go-digest v1.0.0 go: downloading github.com/hashicorp/go-hclog v1.2.0 go: downloading github.com/prometheus/client_model v0.6.1 go: downloading github.com/prometheus/common v0.52.3 go: downloading github.com/stretchr/objx v0.5.2 go: downloading github.com/creack/pty v1.1.18 go: downloading go.uber.org/goleak v1.3.0 go: downloading github.com/spf13/afero v1.9.3 go: downloading github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7 go: downloading github.com/peterbourgon/diskv v2.0.1+incompatible go: downloading github.com/go-openapi/jsonpointer v0.19.6 go: downloading github.com/emicklei/go-restful/v3 v3.11.0 go: downloading github.com/mailru/easyjson v0.7.7 go: downloading github.com/aws/aws-sdk-go-v2 v1.24.1 go: downloading github.com/aws/aws-sdk-go-v2/config v1.26.3 go: downloading github.com/aws/aws-sdk-go-v2/credentials v1.16.14 go: downloading github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.15.11 go: downloading github.com/aws/aws-sdk-go-v2/service/s3 v1.48.0 go: downloading github.com/aws/aws-sdk-go-v2/service/sts v1.26.7 go: downloading github.com/project-velero/kopia v0.0.0-20240417031915-e07d5b7de567 go: downloading github.com/beorn7/perks v1.0.1 go: downloading github.com/cespare/xxhash/v2 v2.2.0 go: downloading github.com/prometheus/procfs v0.12.0 go: downloading github.com/Azure/azure-sdk-for-go/sdk/azcore v1.11.1 go: downloading github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 go: downloading gomodules.xyz/jsonpatch/v2 v2.4.0 go: downloading github.com/hashicorp/yamux v0.1.1 go: downloading github.com/mitchellh/go-testing-interface v1.0.0 go: downloading github.com/oklog/run v1.0.0 go: downloading github.com/jhump/protoreflect v1.15.1 go: downloading github.com/fatih/color v1.16.0 go: downloading github.com/mattn/go-colorable v0.1.13 go: downloading github.com/mattn/go-isatty v0.0.20 go: downloading github.com/go-errors/errors v1.4.2 go: downloading github.com/google/btree v1.0.1 go: downloading github.com/sergi/go-diff v1.1.0 go: downloading github.com/josharian/intern v1.0.0 go: downloading github.com/aws/smithy-go v1.19.0 go: downloading github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.6.0 go: downloading github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.3.2 go: downloading github.com/joho/godotenv v1.3.0 go: downloading github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/storage/armstorage v1.5.0 go: downloading github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.14.11 go: downloading github.com/aws/aws-sdk-go-v2/internal/ini v1.7.2 go: downloading github.com/aws/aws-sdk-go-v2/service/sso v1.18.6 go: downloading github.com/aws/aws-sdk-go-v2/service/ssooidc v1.21.6 go: downloading github.com/aws/aws-sdk-go-v2/internal/configsources v1.2.10 go: downloading github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.10.4 go: downloading github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.10.10 go: downloading github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.5.4 go: downloading github.com/aws/aws-sdk-go-v2/internal/v4a v1.2.10 go: downloading github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.2.10 go: downloading github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.16.10 go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 go: downloading github.com/xlab/treeprint v1.2.0 go: downloading github.com/fsnotify/fsnotify v1.7.0 go: downloading google.golang.org/genproto/googleapis/rpc v0.0.0-20240401170217-c3f982113cda go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: downloading google.golang.org/genproto v0.0.0-20240227224415-6ceb2ff114de go: downloading github.com/nxadm/tail v1.4.8 go: downloading github.com/Azure/azure-sdk-for-go/sdk/internal v1.8.0 go: downloading github.com/AzureAD/microsoft-authentication-library-for-go v1.2.2 go: downloading golang.org/x/crypto v0.28.0 go: downloading github.com/golang-jwt/jwt/v5 v5.2.1 go: downloading github.com/jmespath/go-jmespath v0.4.0 go: downloading github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.5.10 go: downloading github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/internal/v2 v2.0.0 go: downloading github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/resources/armresources v1.1.1 go: downloading go.opentelemetry.io/otel v1.25.0 go: downloading go.opentelemetry.io/otel/trace v1.25.0 go: downloading github.com/mxk/go-vss v1.2.0 go: downloading github.com/kylelemons/godebug v1.1.0 go: downloading github.com/bufbuild/protocompile v0.4.0 go: downloading github.com/edsrzf/mmap-go v1.1.0 go: downloading github.com/hashicorp/cronexpr v1.1.2 go: downloading gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 go: downloading github.com/klauspost/compress v1.17.8 go: downloading github.com/gofrs/flock v0.8.1 go: downloading github.com/klauspost/pgzip v1.2.6 go: downloading github.com/pierrec/lz4 v2.6.1+incompatible go: downloading go.starlark.net v0.0.0-20230525235612-a134d8f9ddca go: downloading github.com/natefinch/atomic v1.0.1 go: downloading github.com/klauspost/reedsolomon v1.12.1 go: downloading github.com/zeebo/blake3 v0.2.3 go: downloading github.com/chmduquesne/rollinghash v4.0.0+incompatible go: downloading github.com/gorilla/mux v1.8.1 go: downloading github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c go: downloading cloud.google.com/go/storage v1.40.0 go: downloading google.golang.org/api v0.172.0 go: downloading github.com/minio/minio-go/v7 v7.0.69 go: downloading github.com/go-ole/go-ole v1.3.0 go: downloading github.com/jmespath/go-jmespath/internal/testify v1.5.1 go: downloading go.opentelemetry.io/otel/metric v1.25.0 go: downloading github.com/go-logr/stdr v1.2.2 go: downloading cloud.google.com/go v0.112.1 go: downloading github.com/tg123/go-htpasswd v1.2.2 go: downloading github.com/zalando/go-keyring v0.2.4 go: downloading github.com/kopia/htmluibuild v0.0.1-0.20240405040759-90bfbf000696 go: downloading cloud.google.com/go/compute/metadata v0.2.3 go: downloading cloud.google.com/go/compute v1.24.0 go: downloading github.com/zeebo/assert v1.1.0 go: downloading github.com/klauspost/cpuid/v2 v2.2.6 go: downloading gopkg.in/ini.v1 v1.67.0 go: downloading github.com/dustin/go-humanize v1.0.1 go: downloading github.com/minio/md5-simd v1.1.2 go: downloading github.com/minio/sha256-simd v1.0.1 go: downloading github.com/hanwen/go-fuse/v2 v2.5.1 go: downloading github.com/GehirnInc/crypt v0.0.0-20230320061759-8cc1b52080c5 go: downloading cloud.google.com/go/iam v1.1.7 go: downloading github.com/googleapis/gax-go/v2 v2.12.3 go: downloading golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028 go: downloading github.com/rs/xid v1.5.0 go: downloading github.com/alessio/shellescape v1.4.1 go: downloading github.com/danieljoos/wincred v1.2.0 go: downloading github.com/godbus/dbus/v5 v5.1.0 go: downloading go.opencensus.io v0.24.0 go: downloading go.opentelemetry.io/otel/sdk v1.25.0 go: downloading google.golang.org/genproto/googleapis/api v0.0.0-20240314234333-6e1732d8331c go: downloading github.com/zeebo/pcg v1.0.1 go: downloading github.com/google/martian/v3 v3.3.2 go: downloading github.com/google/martian v2.1.0+incompatible go: downloading github.com/frankban/quicktest v1.14.3 go: downloading github.com/googleapis/enterprise-certificate-proxy v0.3.2 go: downloading go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.49.0 go: downloading github.com/google/s2a-go v0.1.7 go: downloading go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.49.0 go: downloading github.com/felixge/httpsnoop v1.0.4 storageclass.storage.k8s.io/gp2-csi annotated storageclass.storage.k8s.io/gp3-csi annotated storageclass.storage.k8s.io/odf-operator-ceph-rbd annotated storageclass.storage.k8s.io/odf-operator-ceph-rbd-virtualization annotated storageclass.storage.k8s.io/odf-operator-cephfs annotated storageclass.storage.k8s.io/openshift-storage.noobaa.io annotated storageclass.storage.k8s.io/odf-operator-ceph-rbd annotated + readonly 'RED=\e[31m' + RED='\e[31m' + readonly 'BLUE=\033[34m' + BLUE='\033[34m' + readonly 'CLEAR=\e[39m' + CLEAR='\e[39m' ++ oc get infrastructures cluster -o 'jsonpath={.status.platform}' ++ awk '{print tolower($0)}' + CLOUD_PROVIDER=aws + [[ '' == \t\r\u\e ]] + echo /home/jenkins/.kube/config /home/jenkins/.kube/config + [[ aws == *-arm* ]] + [[ aws == *-fips* ]] + E2E_TIMEOUT_MULTIPLIER=2 + export NAMESPACE=openshift-adp + NAMESPACE=openshift-adp + export PROVIDER=aws + PROVIDER=aws ++ echo aws ++ awk '{print tolower($0)}' + BACKUP_LOCATION=aws + export BACKUP_LOCATION=aws + BACKUP_LOCATION=aws + export BUCKET=ci-op-bkpv69pr-interopoadp + BUCKET=ci-op-bkpv69pr-interopoadp + OADP_CREDS_FILE=/tmp/test-settings/credentials + OADP_VSL_CREDS_FILE=/tmp/test-settings/aws_vsl_creds +++ readlink -f /alabama/cspi/test_settings/scripts/test_runner.sh ++ dirname /alabama/cspi/test_settings/scripts/test_runner.sh + readonly SCRIPT_DIR=/alabama/cspi/test_settings/scripts + SCRIPT_DIR=/alabama/cspi/test_settings/scripts ++ cd /alabama/cspi/test_settings/scripts ++ git rev-parse --show-toplevel + readonly TOP_DIR=/alabama/cspi + TOP_DIR=/alabama/cspi + echo /alabama/cspi /alabama/cspi + TESTS_FOLDER=/alabama/cspi/e2e/kubevirt-plugin ++ oc get nodes -o 'jsonpath={.items[*].metadata.labels.kubernetes\.io/arch}' ++ tr ' ' '\n' ++ sort -u ++ xargs + export NODES_ARCHITECTURE=amd64 + NODES_ARCHITECTURE=amd64 + export OADP_REPOSITORY=redhat + OADP_REPOSITORY=redhat + SKIP_DPA_CREATION=false ++ oc get ns openshift-storage ++ echo true + OPENSHIFT_STORAGE=true + '[' redhat == upstream-velero ']' + '[' true == true ']' ++ oc get sc ++ awk '$1 ~ /^.+ceph-rbd$/ {print $1}' ++ tail -1 + CEPH_RBD_STORAGE_CLASS=odf-operator-ceph-rbd + '[' -n odf-operator-ceph-rbd ']' + export CEPH_RBD_STORAGE_CLASS + echo 'ceph-rbd StorageClass found: odf-operator-ceph-rbd' ceph-rbd StorageClass found: odf-operator-ceph-rbd ++ oc get storageclass -o 'jsonpath={range .items[*]}{@.metadata.name}{" "}{@.metadata.annotations.storageclass\.kubernetes\.io/is-default-class}{"\n"}{end}' ++ awk '$2=="true"{print $1}' ++ wc -l + NUM_DEFAULT_STORAGE_CLASS=1 + '[' 1 -ne 1 ']' ++ oc get storageclass -o 'jsonpath={.items[?(@.metadata.annotations.storageclass\.kubernetes\.io/is-default-class=='\''true'\'')].metadata.name}' + DEFAULT_SC=odf-operator-ceph-rbd + export STORAGE_CLASS=odf-operator-ceph-rbd + STORAGE_CLASS=odf-operator-ceph-rbd + '[' -n odf-operator-ceph-rbd ']' + '[' odf-operator-ceph-rbd '!=' odf-operator-ceph-rbd ']' + export STORAGE_CLASS_OPENSHIFT_STORAGE=odf-operator-ceph-rbd + STORAGE_CLASS_OPENSHIFT_STORAGE=odf-operator-ceph-rbd + echo 'Using the StorageClass for openshift-storage: odf-operator-ceph-rbd' Using the StorageClass for openshift-storage: odf-operator-ceph-rbd + [[ amd64 != \a\m\d\6\4 ]] + TEST_FILTER='!// || (// && !exclude_aws && (!/target/ || target_aws) ) ' + [[ aws =~ ^osp ]] + [[ aws =~ ^vsphere ]] + [[ aws =~ ^gcp-wif ]] + [[ aws =~ ^ibmcloud ]] ++ oc config current-context ++ awk -F / '{print $2}' + SETTINGS_TMP=/alabama/cspi/output_files/api-ci-op-bkpv69pr-78a3d-cspilp-interop-ccitredhat-com:6443 + mkdir -p /alabama/cspi/output_files/api-ci-op-bkpv69pr-78a3d-cspilp-interop-ccitredhat-com:6443 ++ oc get authentication cluster -o 'jsonpath={.spec.serviceAccountIssuer}' + IS_OIDC= + '[' '!' -z ']' + [[ aws == \a\w\s ]] + export PROVIDER=aws + PROVIDER=aws + export CREDS_SECRET_REF=cloud-credentials + CREDS_SECRET_REF=cloud-credentials ++ oc get infrastructures cluster -o 'jsonpath={.status.platformStatus.aws.region}' --allow-missing-template-keys=false + export REGION=us-east-2 + REGION=us-east-2 + settings_script=aws_settings.sh + '[' aws == aws-sts ']' + BUCKET=ci-op-bkpv69pr-interopoadp + TMP_DIR=/alabama/cspi/output_files/api-ci-op-bkpv69pr-78a3d-cspilp-interop-ccitredhat-com:6443 + source /alabama/cspi/test_settings/scripts/aws_settings.sh ++ cat ++ [[ aws == *aws* ]] ++ cat ++ echo -e '\n }\n}' +++ cat /alabama/cspi/output_files/api-ci-op-bkpv69pr-78a3d-cspilp-interop-ccitredhat-com:6443/settings.json ++ x='{ "metadata": { "namespace": "openshift-adp" }, "spec": { "configuration":{ "velero":{ "defaultPlugins": [ "openshift", "aws" ] } }, "backupLocations": [ { "velero": { "provider": "aws", "default": true, "config": { "region": "us-east-2" }, "credential":{ "name": "cloud-credentials", "key": "cloud" }, "objectStorage":{ "bucket": "ci-op-bkpv69pr-interopoadp" } } } ] , "snapshotLocations": [ { "velero": { "provider": "aws", "config": { "profile": "default", "region": "us-east-2" } } } ] } }' ++ echo '{ "metadata": { "namespace": "openshift-adp" }, "spec": { "configuration":{ "velero":{ "defaultPlugins": [ "openshift", "aws" ] } }, "backupLocations": [ { "velero": { "provider": "aws", "default": true, "config": { "region": "us-east-2" }, "credential":{ "name": "cloud-credentials", "key": "cloud" }, "objectStorage":{ "bucket": "ci-op-bkpv69pr-interopoadp" } } } ] , "snapshotLocations": [ { "velero": { "provider": "aws", "config": { "profile": "default", "region": "us-east-2" } } } ] } }' ++ grep -o '^[^#]*' + FILE_SETTINGS_NAME=settings.json + printf '\033[34mGenerated settings file under /alabama/cspi/output_files/api-ci-op-bkpv69pr-78a3d-cspilp-interop-ccitredhat-com:6443/settings.json\e[39m\n' Generated settings file under /alabama/cspi/output_files/api-ci-op-bkpv69pr-78a3d-cspilp-interop-ccitredhat-com:6443/settings.json + cat /alabama/cspi/output_files/api-ci-op-bkpv69pr-78a3d-cspilp-interop-ccitredhat-com:6443/settings.json ++ oc get volumesnapshotclass -o name + for i in $(oc get volumesnapshotclass -o name) + oc annotate volumesnapshotclass.snapshot.storage.k8s.io/csi-aws-vsc snapshot.storage.kubernetes.io/is-default-class- volumesnapshotclass.snapshot.storage.k8s.io/csi-aws-vsc annotated + for i in $(oc get volumesnapshotclass -o name) + oc annotate volumesnapshotclass.snapshot.storage.k8s.io/odf-operator-cephfsplugin-snapclass snapshot.storage.kubernetes.io/is-default-class- volumesnapshotclass.snapshot.storage.k8s.io/odf-operator-cephfsplugin-snapclass annotated + for i in $(oc get volumesnapshotclass -o name) + oc annotate volumesnapshotclass.snapshot.storage.k8s.io/odf-operator-rbdplugin-snapclass snapshot.storage.kubernetes.io/is-default-class- volumesnapshotclass.snapshot.storage.k8s.io/odf-operator-rbdplugin-snapclass annotated ++ ./e2e/must-gather/get-latest-build.sh + oc get configmaps -n default must-gather-image + UPSTREAM_VERSION=99.0.0 ++ oc get OperatorCondition -n openshift-adp -o 'jsonpath={.items[*].metadata.name}' ++ awk -F v '{print $2}' + OADP_VERSION=1.4.1 + '[' -z 1.4.1 ']' + '[' 1.4.1 == 99.0.0 ']' ++ oc get sub redhat-oadp-operator -n openshift-adp -o 'jsonpath={.spec.source}' + OADP_REPO=redhat-operators + '[' -z redhat-operators ']' + '[' redhat-operators == redhat-operators ']' + REGISTRY_PATH=registry.redhat.io/oadp/oadp-mustgather-rhel9: + TAG=1.4.1 + export MUST_GATHER_BUILD=registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 + MUST_GATHER_BUILD=registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 + echo registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 + export MUST_GATHER_BUILD=registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 + MUST_GATHER_BUILD=registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 + '[' -z registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 ']' + export NUM_OF_OADP_INSTANCES=1 + NUM_OF_OADP_INSTANCES=1 + ginkgo run -p --nodes=1 -mod=mod /alabama/cspi/e2e/kubevirt-plugin/ --show-node-events -- -credentials_file=/tmp/test-settings/credentials -vsl_credentials_file=/tmp/test-settings/aws_vsl_creds -oadp_namespace=openshift-adp -settings=/alabama/cspi/output_files/api-ci-op-bkpv69pr-78a3d-cspilp-interop-ccitredhat-com:6443/settings.json -must_gather_image=registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 -timeout_multiplier=2 -skip_dpa_creation=false --ginkgo.junit-report=/logs/artifacts/junit_oadp_cnv_results.xml '--ginkgo.label-filter=!// || (// && !exclude_aws && (!/target/ || target_aws) ) ' --ginkgo.skip=tc-id:OADP-555 Ginkgo detected a version mismatch between the Ginkgo CLI and the version of Ginkgo imported by your packages: Ginkgo CLI Version: 2.22.2 Mismatched package versions found: 2.22.0 used by kubevirt-plugin Ginkgo will continue to attempt to run but you may see errors (including flag parsing errors) and should either update your go.mod or your version of the Ginkgo CLI to match. To install the matching version of the CLI run go install github.com/onsi/ginkgo/v2/ginkgo from a path that contains a go.mod file. Alternatively you can use go run github.com/onsi/ginkgo/v2/ginkgo from a path that contains a go.mod file to invoke the matching version of the Ginkgo CLI. If you are attempting to test multiple packages that each have a different version of the Ginkgo library with a single Ginkgo CLI that is currently unsupported.  2025/01/06 07:40:08 Setting up clients 2025/01/06 07:40:08 Getting default StorageClass... 2025/01/06 07:40:08 Checking default storage class count Run the command: oc get sc 2025/01/06 07:40:08 Got default StorageClass odf-operator-ceph-rbd 2025/01/06 07:40:08 oc get sc NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE gp2-csi ebs.csi.aws.com Delete WaitForFirstConsumer true 65m gp3-csi ebs.csi.aws.com Delete WaitForFirstConsumer true 65m odf-operator-ceph-rbd (default) openshift-storage.rbd.csi.ceph.com Delete Immediate true 4m50s odf-operator-ceph-rbd-virtualization openshift-storage.rbd.csi.ceph.com Delete Immediate true 4m50s odf-operator-cephfs openshift-storage.cephfs.csi.ceph.com Delete Immediate true 4m39s openshift-storage.noobaa.io openshift-storage.noobaa.io/obc Delete Immediate false 3m22s 2025/01/06 07:40:08 Using velero prefix: velero-e2e-kubevirt-73cbbaa4-cc01-11ef-8010-0a580a83a847 Running Suite: OADP E2E Virtualization Workloads Suite - /alabama/cspi/e2e/kubevirt-plugin ========================================================================================== Random Seed: 1736149124 Will run 4 of 5 specs ------------------------------ [BeforeSuite]  /alabama/cspi/e2e/kubevirt-plugin/kubevirt_suite_test.go:62 [BeforeSuite] PASSED [0.041 seconds] ------------------------------ CSI: Backup/Restore Openshift Virtualization Workloads  [tc-id:OADP-185] [kubevirt] Backing up started VM should succeed /alabama/cspi/e2e/kubevirt-plugin/backup_restore_csi.go:34 2025/01/06 07:40:08 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 01/06/25 07:40:08.207 2025/01/06 07:40:08 csi 2025/01/06 07:40:08 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "c8f02dfe-44f0-4d73-933a-d86441c290ae", "resourceVersion": "64486", "generation": 1, "creationTimestamp": "2025-01-06T07:40:08Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2025-01-06T07:40:08Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {} } }, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-bkpv69pr-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "csi" ] } }, "features": null }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/06/25 07:40:08.229 2025/01/06 07:40:08 Waiting for velero pod to be running 2025/01/06 07:40:13 pod: velero-94ff5594b-m5plg is not yet running with status: {Pending [{PodReadyToStartContainers False 0001-01-01 00:00:00 +0000 UTC 2025-01-06 07:40:08 +0000 UTC } {Initialized False 0001-01-01 00:00:00 +0000 UTC 2025-01-06 07:40:08 +0000 UTC ContainersNotInitialized containers with incomplete status: [openshift-velero-plugin velero-plugin-for-aws kubevirt-velero-plugin]} {Ready False 0001-01-01 00:00:00 +0000 UTC 2025-01-06 07:40:08 +0000 UTC ContainersNotReady containers with unready status: [velero]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2025-01-06 07:40:08 +0000 UTC ContainersNotReady containers with unready status: [velero]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2025-01-06 07:40:08 +0000 UTC }] 10.0.3.10 [{10.0.3.10}] [] 2025-01-06 07:40:08 +0000 UTC [{openshift-velero-plugin {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:a8725b48079a61e393583a3e8f0c9c70da31b08886cc9a8d115884886d623ff0 0xc000f9dc19 map[] nil} {velero-plugin-for-aws {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:55aec191293ea79b76761161ddbb9db1a68d0f98c531ea97ce543dca2e9de967 0xc000f9dc1a map[] nil} {kubevirt-velero-plugin {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:342dc1fcb6a8b5f01cafa01f6c4849888a6b3f78c3e5fa94a55cdc7566e088b1 0xc000f9dc1b map[] nil}] [{velero {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-rhel9@sha256:cf9ebc90a72bf5043ef275babb19a4eebe830db9d7194a3e973ea64910f1ae29 0xc000f9dc66 map[] nil}] Burstable [] []} 2025/01/06 07:40:18 pod: velero-94ff5594b-m5plg is not yet running with status: {Pending [{PodReadyToStartContainers True 0001-01-01 00:00:00 +0000 UTC 2025-01-06 07:40:14 +0000 UTC } {Initialized False 0001-01-01 00:00:00 +0000 UTC 2025-01-06 07:40:08 +0000 UTC ContainersNotInitialized containers with incomplete status: [velero-plugin-for-aws kubevirt-velero-plugin]} {Ready False 0001-01-01 00:00:00 +0000 UTC 2025-01-06 07:40:08 +0000 UTC ContainersNotReady containers with unready status: [velero]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2025-01-06 07:40:08 +0000 UTC ContainersNotReady containers with unready status: [velero]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2025-01-06 07:40:08 +0000 UTC }] 10.0.3.10 [{10.0.3.10}] 10.131.0.51 [{10.131.0.51}] 2025-01-06 07:40:08 +0000 UTC [{openshift-velero-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2025-01-06 07:40:14 +0000 UTC,FinishedAt:2025-01-06 07:40:14 +0000 UTC,ContainerID:cri-o://184756e4cf302a00969f5e3e664f2d320cd4bf23c6845119db815ce54cb858bc,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:a8725b48079a61e393583a3e8f0c9c70da31b08886cc9a8d115884886d623ff0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:3b66ca3d09fc4578af372c19c675c284d0eebef3daf0ef1bf748e76fa1526a08 cri-o://184756e4cf302a00969f5e3e664f2d320cd4bf23c6845119db815ce54cb858bc 0xc00083e529 map[] nil} {velero-plugin-for-aws {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:55aec191293ea79b76761161ddbb9db1a68d0f98c531ea97ce543dca2e9de967 0xc00083e52a map[] nil} {kubevirt-velero-plugin {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:342dc1fcb6a8b5f01cafa01f6c4849888a6b3f78c3e5fa94a55cdc7566e088b1 0xc00083e52b map[] nil}] [{velero {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-rhel9@sha256:cf9ebc90a72bf5043ef275babb19a4eebe830db9d7194a3e973ea64910f1ae29 0xc00083e596 map[] nil}] Burstable [] []} 2025/01/06 07:40:23 pod: velero-94ff5594b-m5plg is not yet running with status: {Pending [{PodReadyToStartContainers True 0001-01-01 00:00:00 +0000 UTC 2025-01-06 07:40:14 +0000 UTC } {Initialized False 0001-01-01 00:00:00 +0000 UTC 2025-01-06 07:40:08 +0000 UTC ContainersNotInitialized containers with incomplete status: [kubevirt-velero-plugin]} {Ready False 0001-01-01 00:00:00 +0000 UTC 2025-01-06 07:40:08 +0000 UTC ContainersNotReady containers with unready status: [velero]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2025-01-06 07:40:08 +0000 UTC ContainersNotReady containers with unready status: [velero]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2025-01-06 07:40:08 +0000 UTC }] 10.0.3.10 [{10.0.3.10}] 10.131.0.51 [{10.131.0.51}] 2025-01-06 07:40:08 +0000 UTC [{openshift-velero-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2025-01-06 07:40:14 +0000 UTC,FinishedAt:2025-01-06 07:40:14 +0000 UTC,ContainerID:cri-o://184756e4cf302a00969f5e3e664f2d320cd4bf23c6845119db815ce54cb858bc,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:a8725b48079a61e393583a3e8f0c9c70da31b08886cc9a8d115884886d623ff0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:3b66ca3d09fc4578af372c19c675c284d0eebef3daf0ef1bf748e76fa1526a08 cri-o://184756e4cf302a00969f5e3e664f2d320cd4bf23c6845119db815ce54cb858bc 0xc00083efb9 map[] nil} {velero-plugin-for-aws {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2025-01-06 07:40:18 +0000 UTC,FinishedAt:2025-01-06 07:40:18 +0000 UTC,ContainerID:cri-o://e91093f9e5f69702eda3f74f40854764911155fe9dfb621ff93d7a375a80195a,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:55aec191293ea79b76761161ddbb9db1a68d0f98c531ea97ce543dca2e9de967 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:55aec191293ea79b76761161ddbb9db1a68d0f98c531ea97ce543dca2e9de967 cri-o://e91093f9e5f69702eda3f74f40854764911155fe9dfb621ff93d7a375a80195a 0xc00083f009 map[] nil} {kubevirt-velero-plugin {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:342dc1fcb6a8b5f01cafa01f6c4849888a6b3f78c3e5fa94a55cdc7566e088b1 0xc00083f00a map[] nil}] [{velero {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-rhel9@sha256:cf9ebc90a72bf5043ef275babb19a4eebe830db9d7194a3e973ea64910f1ae29 0xc00083f026 map[] nil}] Burstable [] []} 2025/01/06 07:40:28 pod: velero-94ff5594b-m5plg is not yet running with status: {Pending [{PodReadyToStartContainers True 0001-01-01 00:00:00 +0000 UTC 2025-01-06 07:40:14 +0000 UTC } {Initialized True 0001-01-01 00:00:00 +0000 UTC 2025-01-06 07:40:26 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2025-01-06 07:40:08 +0000 UTC ContainersNotReady containers with unready status: [velero]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2025-01-06 07:40:08 +0000 UTC ContainersNotReady containers with unready status: [velero]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2025-01-06 07:40:08 +0000 UTC }] 10.0.3.10 [{10.0.3.10}] 10.131.0.51 [{10.131.0.51}] 2025-01-06 07:40:08 +0000 UTC [{openshift-velero-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2025-01-06 07:40:14 +0000 UTC,FinishedAt:2025-01-06 07:40:14 +0000 UTC,ContainerID:cri-o://184756e4cf302a00969f5e3e664f2d320cd4bf23c6845119db815ce54cb858bc,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:a8725b48079a61e393583a3e8f0c9c70da31b08886cc9a8d115884886d623ff0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:3b66ca3d09fc4578af372c19c675c284d0eebef3daf0ef1bf748e76fa1526a08 cri-o://184756e4cf302a00969f5e3e664f2d320cd4bf23c6845119db815ce54cb858bc 0xc0005de3e9 map[] nil} {velero-plugin-for-aws {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2025-01-06 07:40:18 +0000 UTC,FinishedAt:2025-01-06 07:40:18 +0000 UTC,ContainerID:cri-o://e91093f9e5f69702eda3f74f40854764911155fe9dfb621ff93d7a375a80195a,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:55aec191293ea79b76761161ddbb9db1a68d0f98c531ea97ce543dca2e9de967 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:55aec191293ea79b76761161ddbb9db1a68d0f98c531ea97ce543dca2e9de967 cri-o://e91093f9e5f69702eda3f74f40854764911155fe9dfb621ff93d7a375a80195a 0xc0005de439 map[] nil} {kubevirt-velero-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2025-01-06 07:40:25 +0000 UTC,FinishedAt:2025-01-06 07:40:26 +0000 UTC,ContainerID:cri-o://fd8f113dd84cc000add19268de7571e50bdf276804f5fcbc5d16c370982fd72d,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:342dc1fcb6a8b5f01cafa01f6c4849888a6b3f78c3e5fa94a55cdc7566e088b1 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:342dc1fcb6a8b5f01cafa01f6c4849888a6b3f78c3e5fa94a55cdc7566e088b1 cri-o://fd8f113dd84cc000add19268de7571e50bdf276804f5fcbc5d16c370982fd72d 0xc0005de449 map[] nil}] [{velero {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-rhel9@sha256:cf9ebc90a72bf5043ef275babb19a4eebe830db9d7194a3e973ea64910f1ae29 0xc0005de45f map[] nil}] Burstable [] []} 2025/01/06 07:40:33 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' STEP: Prepare backup resources, depending on the volumes backup type @ 01/06/25 07:40:33.284 Run the command: oc get ns openshift-storage &> /dev/null && echo true || echo false 2025/01/06 07:40:33 The 'openshift-storage' namespace exists 2025/01/06 07:40:33 Checking default storage class count 2025/01/06 07:40:33 Using the CSI driver: openshift-storage.rbd.csi.ceph.com 2025/01/06 07:40:33 Snapclass 'example-snapclass' doesn't exist, creating 2025/01/06 07:40:33 Setting new default StorageClass 'odf-operator-ceph-rbd' 2025/01/06 07:40:33 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd STEP: Installing application for case ocp-kubevirt @ 01/06/25 07:40:33.553 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] /usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py:1013: InsecureRequestWarning: Unverified HTTPS request is being made to host 'api.ci-op-bkpv69pr-78a3d.cspilp.interop.ccitredhat.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings warnings.warn( TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Create namespace] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Deploy vm test-vm] *** changed: [localhost] FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (60 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (59 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (58 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (57 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (56 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (55 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (54 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (53 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (52 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (51 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (50 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (49 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (48 retries left). TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Wait for VM to be Running & Ready] *** ok: [localhost] PLAY RECAP ********************************************************************* localhost : ok=13  changed=5  unreachable=0 failed=0 skipped=6  rescued=0 ignored=0 2025/01/06 07:41:53 2025-01-06 07:40:35,541 p=24715 u=1003440000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2025-01-06 07:40:35,541 p=24715 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:40:35,915 p=24715 u=1003440000 n=ansible INFO| TASK [Get cluster endpoint] **************************************************** 2025-01-06 07:40:35,915 p=24715 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:40:36,295 p=24715 u=1003440000 n=ansible INFO| TASK [Get current admin token] ************************************************* 2025-01-06 07:40:36,295 p=24715 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:40:36,316 p=24715 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:40:36,316 p=24715 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:40:36,765 p=24715 u=1003440000 n=ansible INFO| TASK [Extract kubernetes minor version from cluster_info] ********************** 2025-01-06 07:40:36,765 p=24715 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:40:36,797 p=24715 u=1003440000 n=ansible INFO| TASK [Map kubernetes minor to ocp releases] ************************************ 2025-01-06 07:40:36,797 p=24715 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:40:36,816 p=24715 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:40:36,816 p=24715 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:40:36,818 p=24715 u=1003440000 n=ansible INFO| PLAY [Execute Task] ************************************************************ 2025-01-06 07:40:37,446 p=24715 u=1003440000 n=ansible INFO| TASK [Gathering Facts] ********************************************************* 2025-01-06 07:40:37,446 p=24715 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:40:38,295 p=24715 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Create namespace] *** 2025-01-06 07:40:38,296 p=24715 u=1003440000 n=ansible WARNING| [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2025-01-06 07:40:38,296 p=24715 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:40:39,010 p=24715 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Deploy vm test-vm] *** 2025-01-06 07:40:39,010 p=24715 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:40:39,841 p=24715 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (60 retries left). 2025-01-06 07:40:45,549 p=24715 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (59 retries left). 2025-01-06 07:40:51,584 p=24715 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (58 retries left). 2025-01-06 07:40:57,243 p=24715 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (57 retries left). 2025-01-06 07:41:02,877 p=24715 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (56 retries left). 2025-01-06 07:41:08,664 p=24715 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (55 retries left). 2025-01-06 07:41:14,266 p=24715 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (54 retries left). 2025-01-06 07:41:19,874 p=24715 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (53 retries left). 2025-01-06 07:41:25,612 p=24715 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (52 retries left). 2025-01-06 07:41:31,218 p=24715 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (51 retries left). 2025-01-06 07:41:36,803 p=24715 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (50 retries left). 2025-01-06 07:41:42,416 p=24715 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (49 retries left). 2025-01-06 07:41:48,008 p=24715 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (48 retries left). 2025-01-06 07:41:53,594 p=24715 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Wait for VM to be Running & Ready] *** 2025-01-06 07:41:53,594 p=24715 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:41:53,688 p=24715 u=1003440000 n=ansible INFO| PLAY RECAP ********************************************************************* 2025-01-06 07:41:53,688 p=24715 u=1003440000 n=ansible INFO| localhost : ok=13 changed=5 unreachable=0 failed=0 skipped=6 rescued=0 ignored=0 STEP: Verify Application deployment @ 01/06/25 07:41:53.728 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Wait for VM to be Running & Ready] *** ok: [localhost] FAILED - RETRYING: [localhost]: Wait for VM to have AgentConnected status True indicating the guest agent is running (60 retries left). FAILED - RETRYING: [localhost]: Wait for VM to have AgentConnected status True indicating the guest agent is running (59 retries left). TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Wait for VM to have AgentConnected status True indicating the guest agent is running] *** ok: [localhost] PLAY RECAP ********************************************************************* localhost : ok=12  changed=3  unreachable=0 failed=0 skipped=7  rescued=0 ignored=0 2025/01/06 07:42:09 2025-01-06 07:41:55,126 p=25077 u=1003440000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2025-01-06 07:41:55,127 p=25077 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:41:55,388 p=25077 u=1003440000 n=ansible INFO| TASK [Get cluster endpoint] **************************************************** 2025-01-06 07:41:55,388 p=25077 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:41:55,644 p=25077 u=1003440000 n=ansible INFO| TASK [Get current admin token] ************************************************* 2025-01-06 07:41:55,644 p=25077 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:41:55,657 p=25077 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:41:55,658 p=25077 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:41:55,951 p=25077 u=1003440000 n=ansible INFO| TASK [Extract kubernetes minor version from cluster_info] ********************** 2025-01-06 07:41:55,952 p=25077 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:41:55,979 p=25077 u=1003440000 n=ansible INFO| TASK [Map kubernetes minor to ocp releases] ************************************ 2025-01-06 07:41:55,979 p=25077 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:41:55,995 p=25077 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:41:55,995 p=25077 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:41:55,997 p=25077 u=1003440000 n=ansible INFO| PLAY [Execute Task] ************************************************************ 2025-01-06 07:41:56,553 p=25077 u=1003440000 n=ansible INFO| TASK [Gathering Facts] ********************************************************* 2025-01-06 07:41:56,553 p=25077 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:41:57,411 p=25077 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Wait for VM to be Running & Ready] *** 2025-01-06 07:41:57,412 p=25077 u=1003440000 n=ansible WARNING| [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2025-01-06 07:41:57,412 p=25077 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:41:58,022 p=25077 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to have AgentConnected status True indicating the guest agent is running (60 retries left). 2025-01-06 07:42:03,618 p=25077 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to have AgentConnected status True indicating the guest agent is running (59 retries left). 2025-01-06 07:42:09,203 p=25077 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Wait for VM to have AgentConnected status True indicating the guest agent is running] *** 2025-01-06 07:42:09,203 p=25077 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:42:09,208 p=25077 u=1003440000 n=ansible INFO| PLAY RECAP ********************************************************************* 2025-01-06 07:42:09,208 p=25077 u=1003440000 n=ansible INFO| localhost : ok=12 changed=3 unreachable=0 failed=0 skipped=7 rescued=0 ignored=0 2025/01/06 07:42:09 {{ } { } [{{ } {test-vm-dv test-oadp-185 03e422f3-c79f-41fb-8073-0ee02464c239 66004 0 2025-01-06 07:40:39 +0000 UTC map[app:containerized-data-importer app.kubernetes.io/component:storage app.kubernetes.io/managed-by:cdi-controller app.kubernetes.io/part-of:hyperconverged-cluster app.kubernetes.io/version:4.17.2 kubevirt.io/created-by:70aff22e-9aa8-4c39-8d32-71fa16a89d9c] map[cdi.kubevirt.io/allowClaimAdoption:true cdi.kubevirt.io/createdForDataVolume:1c75b0e8-92d1-4711-83c9-322eb5e7588b cdi.kubevirt.io/storage.condition.running:false cdi.kubevirt.io/storage.condition.running.message:Import Complete cdi.kubevirt.io/storage.condition.running.reason:Completed cdi.kubevirt.io/storage.contentType:kubevirt cdi.kubevirt.io/storage.deleteAfterCompletion:false cdi.kubevirt.io/storage.pod.phase:Succeeded cdi.kubevirt.io/storage.pod.restarts:0 cdi.kubevirt.io/storage.populator.progress:100.0% cdi.kubevirt.io/storage.preallocation.requested:false cdi.kubevirt.io/storage.usePopulator:true pv.kubernetes.io/bind-completed:yes pv.kubernetes.io/bound-by-controller:yes volume.beta.kubernetes.io/storage-provisioner:openshift-storage.rbd.csi.ceph.com volume.kubernetes.io/storage-provisioner:openshift-storage.rbd.csi.ceph.com] [{cdi.kubevirt.io/v1beta1 DataVolume test-vm-dv 1c75b0e8-92d1-4711-83c9-322eb5e7588b 0xc000a91dda 0xc000a91ddb}] [kubernetes.io/pvc-protection] [{kube-controller-manager Update v1 2025-01-06 07:41:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:pv.kubernetes.io/bind-completed":{},"f:pv.kubernetes.io/bound-by-controller":{},"f:volume.beta.kubernetes.io/storage-provisioner":{},"f:volume.kubernetes.io/storage-provisioner":{}}},"f:spec":{"f:volumeName":{}}} } {kube-controller-manager Update v1 2025-01-06 07:41:39 +0000 UTC FieldsV1 {"f:status":{"f:accessModes":{},"f:capacity":{".":{},"f:storage":{}},"f:phase":{}}} status} {virt-cdi-controller Update v1 2025-01-06 07:41:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:cdi.kubevirt.io/allowClaimAdoption":{},"f:cdi.kubevirt.io/createdForDataVolume":{},"f:cdi.kubevirt.io/storage.condition.running":{},"f:cdi.kubevirt.io/storage.condition.running.message":{},"f:cdi.kubevirt.io/storage.condition.running.reason":{},"f:cdi.kubevirt.io/storage.contentType":{},"f:cdi.kubevirt.io/storage.deleteAfterCompletion":{},"f:cdi.kubevirt.io/storage.pod.phase":{},"f:cdi.kubevirt.io/storage.pod.restarts":{},"f:cdi.kubevirt.io/storage.populator.progress":{},"f:cdi.kubevirt.io/storage.preallocation.requested":{},"f:cdi.kubevirt.io/storage.usePopulator":{}},"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/component":{},"f:app.kubernetes.io/managed-by":{},"f:app.kubernetes.io/part-of":{},"f:app.kubernetes.io/version":{},"f:kubevirt.io/created-by":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"1c75b0e8-92d1-4711-83c9-322eb5e7588b\"}":{}}},"f:spec":{"f:accessModes":{},"f:dataSourceRef":{".":{},"f:apiGroup":{},"f:kind":{},"f:name":{}},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}}} }]} {[ReadWriteOnce] nil {map[] map[storage:{{5368709120 0} {} 5Gi BinarySI}]} pvc-34526c93-c545-4d87-b7af-28a347cad468 0xc000628740 0xc000628750 &TypedLocalObjectReference{APIGroup:*cdi.kubevirt.io,Kind:VolumeImportSource,Name:volume-import-source-1c75b0e8-92d1-4711-83c9-322eb5e7588b,} &TypedObjectReference{APIGroup:*cdi.kubevirt.io,Kind:VolumeImportSource,Name:volume-import-source-1c75b0e8-92d1-4711-83c9-322eb5e7588b,Namespace:nil,} } {Bound [ReadWriteOnce] map[storage:{{5368709120 0} {} 5Gi BinarySI}] [] map[] map[] nil}}]} STEP: Creating backup ocp-kubevirt-73d3076c-cc01-11ef-8010-0a580a83a847 @ 01/06/25 07:42:09.264 2025/01/06 07:42:09 Wait until backup ocp-kubevirt-73d3076c-cc01-11ef-8010-0a580a83a847 is completed backup phase: Completed 2025/01/06 07:42:29 Verify the PVC and VSC count is matching 2025/01/06 07:42:29 Verify the Backup has CSIVolumeSnapshotsAttempted and CSIVolumeSnapshotsCompleted field on status 2025/01/06 07:42:29 Run velero describe on the backup 2025/01/06 07:42:29 [./velero describe backup ocp-kubevirt-73d3076c-cc01-11ef-8010-0a580a83a847 -n openshift-adp --details --insecure-skip-tls-verify] 2025/01/06 07:42:29 Exec stderr: "" 2025/01/06 07:42:29 Name: ocp-kubevirt-73d3076c-cc01-11ef-8010-0a580a83a847 Namespace: openshift-adp Labels: velero.io/storage-location=ts-dpa-1 Annotations: velero.io/resource-timeout=10m0s velero.io/source-cluster-k8s-gitversion=v1.31.3 velero.io/source-cluster-k8s-major-version=1 velero.io/source-cluster-k8s-minor-version=31 Phase: Completed Namespaces: Included: test-oadp-185 Excluded: Resources: Included: * Excluded: Cluster-scoped: auto Label selector: Or label selector: Storage Location: ts-dpa-1 Velero-Native Snapshot PVs: auto Snapshot Move Data: false Data Mover: velero TTL: 720h0m0s CSISnapshotTimeout: 10m0s ItemOperationTimeout: 4h0m0s Hooks: Backup Format Version: 1.1.0 Started: 2025-01-06 07:42:09 +0000 UTC Completed: 2025-01-06 07:42:17 +0000 UTC Expiration: 2025-02-05 07:42:09 +0000 UTC Total items to be backed up: 81 Items backed up: 81 Backup Item Operations: Operation for volumesnapshots.snapshot.storage.k8s.io test-oadp-185/velero-test-vm-dv-94h6f: Backup Item Action Plugin: velero.io/csi-volumesnapshot-backupper Operation ID: test-oadp-185/velero-test-vm-dv-94h6f/2025-01-06T07:42:16Z Items to Update: volumesnapshots.snapshot.storage.k8s.io test-oadp-185/velero-test-vm-dv-94h6f volumesnapshotcontents.snapshot.storage.k8s.io /snapcontent-4c4393d1-7971-469f-ae95-889e90b1fe43 Phase: Completed Created: 2025-01-06 07:42:16 +0000 UTC Started: 2025-01-06 07:42:16 +0000 UTC Updated: 2025-01-06 07:42:16 +0000 UTC Resource List: apiextensions.k8s.io/v1/CustomResourceDefinition: - datavolumes.cdi.kubevirt.io - virtualmachineinstances.kubevirt.io - virtualmachines.kubevirt.io apps/v1/ControllerRevision: - test-oadp-185/revision-start-vm-70aff22e-9aa8-4c39-8d32-71fa16a89d9c-1 authorization.openshift.io/v1/RoleBinding: - test-oadp-185/system:deployers - test-oadp-185/system:image-builders - test-oadp-185/system:image-pullers cdi.kubevirt.io/v1beta1/DataVolume: - test-oadp-185/test-vm-dv kubevirt.io/v1/VirtualMachine: - test-oadp-185/test-vm kubevirt.io/v1/VirtualMachineInstance: - test-oadp-185/test-vm policy/v1/PodDisruptionBudget: - test-oadp-185/kubevirt-disruption-budget-2k4rn rbac.authorization.k8s.io/v1/RoleBinding: - test-oadp-185/system:deployers - test-oadp-185/system:image-builders - test-oadp-185/system:image-pullers snapshot.storage.k8s.io/v1/VolumeSnapshot: - test-oadp-185/velero-test-vm-dv-94h6f snapshot.storage.k8s.io/v1/VolumeSnapshotClass: - example-snapclass snapshot.storage.k8s.io/v1/VolumeSnapshotContent: - snapcontent-4c4393d1-7971-469f-ae95-889e90b1fe43 v1/ConfigMap: - test-oadp-185/kube-root-ca.crt - test-oadp-185/openshift-service-ca.crt v1/Event: - test-oadp-185/importer-prime-03e422f3-c79f-41fb-8073-0ee02464c239.18180a88d7dd5f64 - test-oadp-185/importer-prime-03e422f3-c79f-41fb-8073-0ee02464c239.18180a893e76c435 - test-oadp-185/importer-prime-03e422f3-c79f-41fb-8073-0ee02464c239.18180a8961aef6cb - test-oadp-185/importer-prime-03e422f3-c79f-41fb-8073-0ee02464c239.18180a8b481df8de - test-oadp-185/importer-prime-03e422f3-c79f-41fb-8073-0ee02464c239.18180a8b481e29ca - test-oadp-185/importer-prime-03e422f3-c79f-41fb-8073-0ee02464c239.18180a8b6ee9e26d - test-oadp-185/importer-prime-03e422f3-c79f-41fb-8073-0ee02464c239.18180a8b70627896 - test-oadp-185/importer-prime-03e422f3-c79f-41fb-8073-0ee02464c239.18180a8b7a0f6521 - test-oadp-185/importer-prime-03e422f3-c79f-41fb-8073-0ee02464c239.18180a8b8133dfc8 - test-oadp-185/importer-prime-03e422f3-c79f-41fb-8073-0ee02464c239.18180a8bb3fa9c9d - test-oadp-185/importer-prime-03e422f3-c79f-41fb-8073-0ee02464c239.18180a8bbdd7fc48 - test-oadp-185/importer-prime-03e422f3-c79f-41fb-8073-0ee02464c239.18180a8bc379dc54 - test-oadp-185/importer-prime-03e422f3-c79f-41fb-8073-0ee02464c239.18180a8bc385f502 - test-oadp-185/importer-prime-03e422f3-c79f-41fb-8073-0ee02464c239.18180a924ac5f12f - test-oadp-185/importer-prime-03e422f3-c79f-41fb-8073-0ee02464c239.18180a924aebfb93 - test-oadp-185/importer-prime-03e422f3-c79f-41fb-8073-0ee02464c239.18180a927f9435bc - test-oadp-185/importer-prime-03e422f3-c79f-41fb-8073-0ee02464c239.18180a9289f6d48b - test-oadp-185/importer-prime-03e422f3-c79f-41fb-8073-0ee02464c239.18180a928ebe0a5c - test-oadp-185/kubevirt-disruption-budget-2k4rn.18180a96cba2e244 - test-oadp-185/prime-03e422f3-c79f-41fb-8073-0ee02464c239.18180a88d2e8b419 - test-oadp-185/prime-03e422f3-c79f-41fb-8073-0ee02464c239.18180a88d2f139fc - test-oadp-185/prime-03e422f3-c79f-41fb-8073-0ee02464c239.18180a88d99dd574 - test-oadp-185/prime-03e422f3-c79f-41fb-8073-0ee02464c239.18180a96c5fbef7f - test-oadp-185/prime-03e422f3-c79f-41fb-8073-0ee02464c239.18180a9a1ea55da6 - test-oadp-185/test-vm-dv.18180a88d133fa1a - test-oadp-185/test-vm-dv.18180a88d18930ea - test-oadp-185/test-vm-dv.18180a88d1958054 - test-oadp-185/test-vm-dv.18180a88d195eae5 - test-oadp-185/test-vm-dv.18180a88d276f680 - test-oadp-185/test-vm-dv.18180a88d3a9b305 - test-oadp-185/test-vm-dv.18180a894273d930 - test-oadp-185/test-vm-dv.18180a92bf4e888e - test-oadp-185/test-vm-dv.18180a96415c5ebc - test-oadp-185/test-vm-dv.18180a96c7bd0335 - test-oadp-185/test-vm-dv.18180a96c84cf619 - test-oadp-185/test-vm-dv.18180a96cabb3eee - test-oadp-185/test-vm.18180a88ce28c3dc - test-oadp-185/test-vm.18180a96cb36f098 - test-oadp-185/test-vm.18180a96cbade306 - test-oadp-185/test-vm.18180a96d0252597 - test-oadp-185/test-vm.18180a988c4047f9 - test-oadp-185/test-vm.18180a994164bd39 - test-oadp-185/test-vm.18180a9946ba2e0a - test-oadp-185/virt-launcher-test-vm-mzpsl.18180a96d0a5a0dd - test-oadp-185/virt-launcher-test-vm-mzpsl.18180a96d611abe7 - test-oadp-185/virt-launcher-test-vm-mzpsl.18180a974801a08f - test-oadp-185/virt-launcher-test-vm-mzpsl.18180a98399658dc - test-oadp-185/virt-launcher-test-vm-mzpsl.18180a9839968b10 - test-oadp-185/virt-launcher-test-vm-mzpsl.18180a986018f283 - test-oadp-185/virt-launcher-test-vm-mzpsl.18180a98618205b9 - test-oadp-185/virt-launcher-test-vm-mzpsl.18180a98702fdb76 - test-oadp-185/virt-launcher-test-vm-mzpsl.18180a9874e9e4a3 v1/Namespace: - test-oadp-185 v1/PersistentVolume: - pvc-34526c93-c545-4d87-b7af-28a347cad468 v1/PersistentVolumeClaim: - test-oadp-185/test-vm-dv v1/Pod: - test-oadp-185/virt-launcher-test-vm-mzpsl v1/Secret: - test-oadp-185/builder-dockercfg-ndqhp - test-oadp-185/default-dockercfg-nkg8q - test-oadp-185/deployer-dockercfg-vnsdn v1/ServiceAccount: - test-oadp-185/builder - test-oadp-185/default - test-oadp-185/deployer Backup Volumes: Velero-Native Snapshots: CSI Snapshots: test-oadp-185/test-vm-dv: Snapshot: Operation ID: test-oadp-185/velero-test-vm-dv-94h6f/2025-01-06T07:42:16Z Snapshot Content Name: snapcontent-4c4393d1-7971-469f-ae95-889e90b1fe43 Storage Snapshot ID: 0001-0011-openshift-storage-0000000000000001-3772f678-6753-45f0-9fd7-ee538396051c Snapshot Size (bytes): 5368709120 CSI Driver: openshift-storage.rbd.csi.ceph.com Pod Volume Backups: HooksAttempted: 2 HooksFailed: 0 STEP: Verify backup ocp-kubevirt-73d3076c-cc01-11ef-8010-0a580a83a847 has completed successfully @ 01/06/25 07:42:29.979 2025/01/06 07:42:30 Backup for case ocp-kubevirt succeeded STEP: Delete the appplication resources ocp-kubevirt @ 01/06/25 07:42:30.025 STEP: Cleanup Application for case ocp-kubevirt @ 01/06/25 07:42:30.025 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Remove namespace test-oadp-185] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=11  changed=4  unreachable=0 failed=0 skipped=8  rescued=0 ignored=0 2025/01/06 07:42:48 2025-01-06 07:42:31,433 p=25298 u=1003440000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2025-01-06 07:42:31,434 p=25298 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:42:31,692 p=25298 u=1003440000 n=ansible INFO| TASK [Get cluster endpoint] **************************************************** 2025-01-06 07:42:31,692 p=25298 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:42:31,945 p=25298 u=1003440000 n=ansible INFO| TASK [Get current admin token] ************************************************* 2025-01-06 07:42:31,945 p=25298 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:42:31,959 p=25298 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:42:31,959 p=25298 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:42:32,263 p=25298 u=1003440000 n=ansible INFO| TASK [Extract kubernetes minor version from cluster_info] ********************** 2025-01-06 07:42:32,263 p=25298 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:42:32,290 p=25298 u=1003440000 n=ansible INFO| TASK [Map kubernetes minor to ocp releases] ************************************ 2025-01-06 07:42:32,291 p=25298 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:42:32,308 p=25298 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:42:32,308 p=25298 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:42:32,309 p=25298 u=1003440000 n=ansible INFO| PLAY [Execute Task] ************************************************************ 2025-01-06 07:42:32,861 p=25298 u=1003440000 n=ansible INFO| TASK [Gathering Facts] ********************************************************* 2025-01-06 07:42:32,861 p=25298 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:42:48,642 p=25298 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Remove namespace test-oadp-185] *** 2025-01-06 07:42:48,642 p=25298 u=1003440000 n=ansible WARNING| [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2025-01-06 07:42:48,642 p=25298 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:42:48,779 p=25298 u=1003440000 n=ansible INFO| PLAY RECAP ********************************************************************* 2025-01-06 07:42:48,779 p=25298 u=1003440000 n=ansible INFO| localhost : ok=11 changed=4 unreachable=0 failed=0 skipped=8 rescued=0 ignored=0 2025/01/06 07:42:48 Creating restore ocp-kubevirt-73d3076c-cc01-11ef-8010-0a580a83a847 for case ocp-kubevirt-73d3076c-cc01-11ef-8010-0a580a83a847 STEP: Create restore ocp-kubevirt-73d3076c-cc01-11ef-8010-0a580a83a847 from backup ocp-kubevirt-73d3076c-cc01-11ef-8010-0a580a83a847 @ 01/06/25 07:42:48.819 2025/01/06 07:42:48 Wait until restore ocp-kubevirt-73d3076c-cc01-11ef-8010-0a580a83a847 is complete restore phase: Finalizing restore phase: Completed STEP: Verify restore ocp-kubevirt-73d3076c-cc01-11ef-8010-0a580a83a847has completed successfully @ 01/06/25 07:43:08.865 STEP: Verify Application restore @ 01/06/25 07:43:08.868 STEP: Verify Application deployment for case ocp-kubevirt @ 01/06/25 07:43:08.868 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Wait for VM to be Running & Ready] *** ok: [localhost] FAILED - RETRYING: [localhost]: Wait for VM to have AgentConnected status True indicating the guest agent is running (60 retries left). TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Wait for VM to have AgentConnected status True indicating the guest agent is running] *** ok: [localhost] PLAY RECAP ********************************************************************* localhost : ok=12  changed=3  unreachable=0 failed=0 skipped=7  rescued=0 ignored=0 2025/01/06 07:43:19 2025-01-06 07:43:10,341 p=25477 u=1003440000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2025-01-06 07:43:10,341 p=25477 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:43:10,618 p=25477 u=1003440000 n=ansible INFO| TASK [Get cluster endpoint] **************************************************** 2025-01-06 07:43:10,618 p=25477 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:43:10,898 p=25477 u=1003440000 n=ansible INFO| TASK [Get current admin token] ************************************************* 2025-01-06 07:43:10,898 p=25477 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:43:10,917 p=25477 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:43:10,917 p=25477 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:43:11,246 p=25477 u=1003440000 n=ansible INFO| TASK [Extract kubernetes minor version from cluster_info] ********************** 2025-01-06 07:43:11,247 p=25477 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:43:11,280 p=25477 u=1003440000 n=ansible INFO| TASK [Map kubernetes minor to ocp releases] ************************************ 2025-01-06 07:43:11,280 p=25477 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:43:11,300 p=25477 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:43:11,300 p=25477 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:43:11,302 p=25477 u=1003440000 n=ansible INFO| PLAY [Execute Task] ************************************************************ 2025-01-06 07:43:11,929 p=25477 u=1003440000 n=ansible INFO| TASK [Gathering Facts] ********************************************************* 2025-01-06 07:43:11,930 p=25477 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:43:12,953 p=25477 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Wait for VM to be Running & Ready] *** 2025-01-06 07:43:12,953 p=25477 u=1003440000 n=ansible WARNING| [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2025-01-06 07:43:12,953 p=25477 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:43:13,609 p=25477 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to have AgentConnected status True indicating the guest agent is running (60 retries left). 2025-01-06 07:43:19,215 p=25477 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Wait for VM to have AgentConnected status True indicating the guest agent is running] *** 2025-01-06 07:43:19,215 p=25477 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:43:19,220 p=25477 u=1003440000 n=ansible INFO| PLAY RECAP ********************************************************************* 2025-01-06 07:43:19,220 p=25477 u=1003440000 n=ansible INFO| localhost : ok=12 changed=3 unreachable=0 failed=0 skipped=7 rescued=0 ignored=0 2025/01/06 07:43:19 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 2025/01/06 07:43:19 Cleaning app [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Remove namespace test-oadp-185] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=11  changed=4  unreachable=0 failed=0 skipped=8  rescued=0 ignored=0 2025/01/06 07:43:38 2025-01-06 07:43:20,766 p=25680 u=1003440000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2025-01-06 07:43:20,766 p=25680 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:43:21,040 p=25680 u=1003440000 n=ansible INFO| TASK [Get cluster endpoint] **************************************************** 2025-01-06 07:43:21,040 p=25680 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:43:21,326 p=25680 u=1003440000 n=ansible INFO| TASK [Get current admin token] ************************************************* 2025-01-06 07:43:21,326 p=25680 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:43:21,341 p=25680 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:43:21,342 p=25680 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:43:21,658 p=25680 u=1003440000 n=ansible INFO| TASK [Extract kubernetes minor version from cluster_info] ********************** 2025-01-06 07:43:21,658 p=25680 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:43:21,688 p=25680 u=1003440000 n=ansible INFO| TASK [Map kubernetes minor to ocp releases] ************************************ 2025-01-06 07:43:21,688 p=25680 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:43:21,706 p=25680 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:43:21,706 p=25680 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:43:21,708 p=25680 u=1003440000 n=ansible INFO| PLAY [Execute Task] ************************************************************ 2025-01-06 07:43:22,274 p=25680 u=1003440000 n=ansible INFO| TASK [Gathering Facts] ********************************************************* 2025-01-06 07:43:22,275 p=25680 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:43:38,348 p=25680 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Remove namespace test-oadp-185] *** 2025-01-06 07:43:38,349 p=25680 u=1003440000 n=ansible WARNING| [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2025-01-06 07:43:38,349 p=25680 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:43:38,592 p=25680 u=1003440000 n=ansible INFO| PLAY RECAP ********************************************************************* 2025-01-06 07:43:38,592 p=25680 u=1003440000 n=ansible INFO| localhost : ok=11 changed=4 unreachable=0 failed=0 skipped=8 rescued=0 ignored=0 2025/01/06 07:43:38 Cleaning setup resources for the backup 2025/01/06 07:43:38 Setting new default StorageClass 'odf-operator-ceph-rbd' 2025/01/06 07:43:38 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2025/01/06 07:43:38 Deleting VolumeSnapshotClass 'example-snapclass' • [210.545 seconds] ------------------------------ CSI: Backup/Restore Openshift Virtualization Workloads  [tc-id:OADP-186] [kubevirt] Stopped VM should be restored /alabama/cspi/e2e/kubevirt-plugin/backup_restore_csi.go:51 2025/01/06 07:43:38 Delete all downloadrequest ocp-kubevirt-73d3076c-cc01-11ef-8010-0a580a83a847-311a1dae-944e-4ea8-a934-cababab27c11 ocp-kubevirt-73d3076c-cc01-11ef-8010-0a580a83a847-70f82d4d-7f6a-4d5c-b4fa-db3d3bebf7b4 ocp-kubevirt-73d3076c-cc01-11ef-8010-0a580a83a847-fb40586b-d919-4016-97cc-8c02889dcf78 STEP: Create DPA CR @ 01/06/25 07:43:38.902 2025/01/06 07:43:38 csi 2025/01/06 07:43:38 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "4ee32b7e-4554-4304-ae45-78277ef08993", "resourceVersion": "68080", "generation": 1, "creationTimestamp": "2025-01-06T07:43:38Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2025-01-06T07:43:38Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {} } }, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-bkpv69pr-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "csi" ] } }, "features": null }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/06/25 07:43:38.956 2025/01/06 07:43:38 Waiting for velero pod to be running 2025/01/06 07:43:38 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2025/01/06 07:43:38 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "4ee32b7e-4554-4304-ae45-78277ef08993", "resourceVersion": "68080", "generation": 1, "creationTimestamp": "2025-01-06T07:43:38Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2025-01-06T07:43:38Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {} } }, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-bkpv69pr-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "csi" ] } }, "features": null }, "status": {} } STEP: Prepare backup resources, depending on the volumes backup type @ 01/06/25 07:43:43.977 Run the command: oc get ns openshift-storage &> /dev/null && echo true || echo false 2025/01/06 07:43:44 The 'openshift-storage' namespace exists 2025/01/06 07:43:44 Checking default storage class count 2025/01/06 07:43:44 Using the CSI driver: openshift-storage.rbd.csi.ceph.com 2025/01/06 07:43:44 Snapclass 'example-snapclass' doesn't exist, creating 2025/01/06 07:43:44 Setting new default StorageClass 'odf-operator-ceph-rbd' 2025/01/06 07:43:44 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd STEP: Installing application for case ocp-kubevirt @ 01/06/25 07:43:44.251 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Create namespace] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Deploy vm test-vm] *** changed: [localhost] FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (60 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (59 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (58 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (57 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (56 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (55 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (54 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (53 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (52 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (51 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (50 retries left). TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Wait for VM to be Running & Ready] *** ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Shutdown the VM if required] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Wait for VM status to become 'Stopped'] *** ok: [localhost] PLAY RECAP ********************************************************************* localhost : ok=15  changed=6  unreachable=0 failed=0 skipped=4  rescued=0 ignored=0 2025/01/06 07:44:58 2025-01-06 07:43:46,273 p=25870 u=1003440000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2025-01-06 07:43:46,273 p=25870 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:43:46,642 p=25870 u=1003440000 n=ansible INFO| TASK [Get cluster endpoint] **************************************************** 2025-01-06 07:43:46,642 p=25870 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:43:47,076 p=25870 u=1003440000 n=ansible INFO| TASK [Get current admin token] ************************************************* 2025-01-06 07:43:47,076 p=25870 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:43:47,095 p=25870 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:43:47,095 p=25870 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:43:47,533 p=25870 u=1003440000 n=ansible INFO| TASK [Extract kubernetes minor version from cluster_info] ********************** 2025-01-06 07:43:47,533 p=25870 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:43:47,574 p=25870 u=1003440000 n=ansible INFO| TASK [Map kubernetes minor to ocp releases] ************************************ 2025-01-06 07:43:47,575 p=25870 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:43:47,601 p=25870 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:43:47,602 p=25870 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:43:47,604 p=25870 u=1003440000 n=ansible INFO| PLAY [Execute Task] ************************************************************ 2025-01-06 07:43:48,408 p=25870 u=1003440000 n=ansible INFO| TASK [Gathering Facts] ********************************************************* 2025-01-06 07:43:48,409 p=25870 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:43:49,638 p=25870 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Create namespace] *** 2025-01-06 07:43:49,639 p=25870 u=1003440000 n=ansible WARNING| [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2025-01-06 07:43:49,639 p=25870 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:43:50,697 p=25870 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Deploy vm test-vm] *** 2025-01-06 07:43:50,697 p=25870 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:43:51,937 p=25870 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (60 retries left). 2025-01-06 07:43:57,946 p=25870 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (59 retries left). 2025-01-06 07:44:03,818 p=25870 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (58 retries left). 2025-01-06 07:44:09,627 p=25870 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (57 retries left). 2025-01-06 07:44:15,598 p=25870 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (56 retries left). 2025-01-06 07:44:21,213 p=25870 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (55 retries left). 2025-01-06 07:44:26,940 p=25870 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (54 retries left). 2025-01-06 07:44:32,554 p=25870 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (53 retries left). 2025-01-06 07:44:38,646 p=25870 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (52 retries left). 2025-01-06 07:44:44,624 p=25870 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (51 retries left). 2025-01-06 07:44:50,601 p=25870 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (50 retries left). 2025-01-06 07:44:56,585 p=25870 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Wait for VM to be Running & Ready] *** 2025-01-06 07:44:56,585 p=25870 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:44:57,648 p=25870 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Shutdown the VM if required] *** 2025-01-06 07:44:57,648 p=25870 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:44:58,649 p=25870 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Wait for VM status to become 'Stopped'] *** 2025-01-06 07:44:58,649 p=25870 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:44:58,748 p=25870 u=1003440000 n=ansible INFO| PLAY RECAP ********************************************************************* 2025-01-06 07:44:58,748 p=25870 u=1003440000 n=ansible INFO| localhost : ok=15 changed=6 unreachable=0 failed=0 skipped=4 rescued=0 ignored=0 STEP: Verify Application deployment @ 01/06/25 07:44:58.845 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Verify VM is not in running state] *** ok: [localhost] PLAY RECAP ********************************************************************* localhost : ok=11  changed=3  unreachable=0 failed=0 skipped=8  rescued=0 ignored=0 2025/01/06 07:45:04 2025-01-06 07:45:01,049 p=26231 u=1003440000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2025-01-06 07:45:01,049 p=26231 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:45:01,507 p=26231 u=1003440000 n=ansible INFO| TASK [Get cluster endpoint] **************************************************** 2025-01-06 07:45:01,507 p=26231 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:45:02,000 p=26231 u=1003440000 n=ansible INFO| TASK [Get current admin token] ************************************************* 2025-01-06 07:45:02,001 p=26231 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:45:02,024 p=26231 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:45:02,024 p=26231 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:45:02,473 p=26231 u=1003440000 n=ansible INFO| TASK [Extract kubernetes minor version from cluster_info] ********************** 2025-01-06 07:45:02,473 p=26231 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:45:02,517 p=26231 u=1003440000 n=ansible INFO| TASK [Map kubernetes minor to ocp releases] ************************************ 2025-01-06 07:45:02,517 p=26231 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:45:02,544 p=26231 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:45:02,544 p=26231 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:45:02,547 p=26231 u=1003440000 n=ansible INFO| PLAY [Execute Task] ************************************************************ 2025-01-06 07:45:03,335 p=26231 u=1003440000 n=ansible INFO| TASK [Gathering Facts] ********************************************************* 2025-01-06 07:45:03,335 p=26231 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:45:04,713 p=26231 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Verify VM is not in running state] *** 2025-01-06 07:45:04,714 p=26231 u=1003440000 n=ansible WARNING| [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2025-01-06 07:45:04,714 p=26231 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:45:04,787 p=26231 u=1003440000 n=ansible INFO| PLAY RECAP ********************************************************************* 2025-01-06 07:45:04,787 p=26231 u=1003440000 n=ansible INFO| localhost : ok=11 changed=3 unreachable=0 failed=0 skipped=8 rescued=0 ignored=0 2025/01/06 07:45:04 {{ } { } [{{ } {test-vm-dv test-oadp-186 20b2ac57-b26d-498f-854a-7c5e8355c375 69156 0 2025-01-06 07:43:50 +0000 UTC map[app:containerized-data-importer app.kubernetes.io/component:storage app.kubernetes.io/managed-by:cdi-controller app.kubernetes.io/part-of:hyperconverged-cluster app.kubernetes.io/version:4.17.2 kubevirt.io/created-by:a5f4c0e7-1dd6-40c4-ae60-efe45bc39a93] map[cdi.kubevirt.io/allowClaimAdoption:true cdi.kubevirt.io/createdForDataVolume:04112486-eadd-4048-bf39-f37e3d5ee969 cdi.kubevirt.io/storage.condition.running:false cdi.kubevirt.io/storage.condition.running.message:Import Complete cdi.kubevirt.io/storage.condition.running.reason:Completed cdi.kubevirt.io/storage.contentType:kubevirt cdi.kubevirt.io/storage.deleteAfterCompletion:false cdi.kubevirt.io/storage.pod.phase:Succeeded cdi.kubevirt.io/storage.pod.restarts:0 cdi.kubevirt.io/storage.populator.progress:100.0% cdi.kubevirt.io/storage.preallocation.requested:false cdi.kubevirt.io/storage.usePopulator:true pv.kubernetes.io/bind-completed:yes pv.kubernetes.io/bound-by-controller:yes volume.beta.kubernetes.io/storage-provisioner:openshift-storage.rbd.csi.ceph.com volume.kubernetes.io/storage-provisioner:openshift-storage.rbd.csi.ceph.com] [{cdi.kubevirt.io/v1beta1 DataVolume test-vm-dv 04112486-eadd-4048-bf39-f37e3d5ee969 0xc0007193ca 0xc0007193cb}] [kubernetes.io/pvc-protection] [{kube-controller-manager Update v1 2025-01-06 07:44:44 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:pv.kubernetes.io/bind-completed":{},"f:pv.kubernetes.io/bound-by-controller":{},"f:volume.beta.kubernetes.io/storage-provisioner":{},"f:volume.kubernetes.io/storage-provisioner":{}}},"f:spec":{"f:volumeName":{}}} } {kube-controller-manager Update v1 2025-01-06 07:44:44 +0000 UTC FieldsV1 {"f:status":{"f:accessModes":{},"f:capacity":{".":{},"f:storage":{}},"f:phase":{}}} status} {virt-cdi-controller Update v1 2025-01-06 07:44:44 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:cdi.kubevirt.io/allowClaimAdoption":{},"f:cdi.kubevirt.io/createdForDataVolume":{},"f:cdi.kubevirt.io/storage.condition.running":{},"f:cdi.kubevirt.io/storage.condition.running.message":{},"f:cdi.kubevirt.io/storage.condition.running.reason":{},"f:cdi.kubevirt.io/storage.contentType":{},"f:cdi.kubevirt.io/storage.deleteAfterCompletion":{},"f:cdi.kubevirt.io/storage.pod.phase":{},"f:cdi.kubevirt.io/storage.pod.restarts":{},"f:cdi.kubevirt.io/storage.populator.progress":{},"f:cdi.kubevirt.io/storage.preallocation.requested":{},"f:cdi.kubevirt.io/storage.usePopulator":{}},"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/component":{},"f:app.kubernetes.io/managed-by":{},"f:app.kubernetes.io/part-of":{},"f:app.kubernetes.io/version":{},"f:kubevirt.io/created-by":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"04112486-eadd-4048-bf39-f37e3d5ee969\"}":{}}},"f:spec":{"f:accessModes":{},"f:dataSourceRef":{".":{},"f:apiGroup":{},"f:kind":{},"f:name":{}},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}}} }]} {[ReadWriteOnce] nil {map[] map[storage:{{5368709120 0} {} 5Gi BinarySI}]} pvc-d2e89338-ccc0-4964-9b4c-3e7029d8fa7e 0xc000073060 0xc000073070 &TypedLocalObjectReference{APIGroup:*cdi.kubevirt.io,Kind:VolumeImportSource,Name:volume-import-source-04112486-eadd-4048-bf39-f37e3d5ee969,} &TypedObjectReference{APIGroup:*cdi.kubevirt.io,Kind:VolumeImportSource,Name:volume-import-source-04112486-eadd-4048-bf39-f37e3d5ee969,Namespace:nil,} } {Bound [ReadWriteOnce] map[storage:{{5368709120 0} {} 5Gi BinarySI}] [] map[] map[] nil}}]} STEP: Creating backup ocp-kubevirt-f157e83c-cc01-11ef-8010-0a580a83a847 @ 01/06/25 07:45:04.904 2025/01/06 07:45:04 Wait until backup ocp-kubevirt-f157e83c-cc01-11ef-8010-0a580a83a847 is completed backup phase: Completed 2025/01/06 07:45:24 Verify the PVC and VSC count is matching 2025/01/06 07:45:24 Verify the Backup has CSIVolumeSnapshotsAttempted and CSIVolumeSnapshotsCompleted field on status 2025/01/06 07:45:25 Run velero describe on the backup 2025/01/06 07:45:25 [./velero describe backup ocp-kubevirt-f157e83c-cc01-11ef-8010-0a580a83a847 -n openshift-adp --details --insecure-skip-tls-verify] 2025/01/06 07:45:25 Exec stderr: "" 2025/01/06 07:45:25 Name: ocp-kubevirt-f157e83c-cc01-11ef-8010-0a580a83a847 Namespace: openshift-adp Labels: velero.io/storage-location=ts-dpa-1 Annotations: velero.io/resource-timeout=10m0s velero.io/source-cluster-k8s-gitversion=v1.31.3 velero.io/source-cluster-k8s-major-version=1 velero.io/source-cluster-k8s-minor-version=31 Phase: Completed Namespaces: Included: test-oadp-186 Excluded: Resources: Included: * Excluded: Cluster-scoped: auto Label selector: Or label selector: Storage Location: ts-dpa-1 Velero-Native Snapshot PVs: auto Snapshot Move Data: false Data Mover: velero TTL: 720h0m0s CSISnapshotTimeout: 10m0s ItemOperationTimeout: 4h0m0s Hooks: Backup Format Version: 1.1.0 Started: 2025-01-06 07:45:05 +0000 UTC Completed: 2025-01-06 07:45:12 +0000 UTC Expiration: 2025-02-05 07:45:04 +0000 UTC Total items to be backed up: 81 Items backed up: 81 Backup Item Operations: Operation for volumesnapshots.snapshot.storage.k8s.io test-oadp-186/velero-test-vm-dv-54qjl: Backup Item Action Plugin: velero.io/csi-volumesnapshot-backupper Operation ID: test-oadp-186/velero-test-vm-dv-54qjl/2025-01-06T07:45:11Z Items to Update: volumesnapshots.snapshot.storage.k8s.io test-oadp-186/velero-test-vm-dv-54qjl volumesnapshotcontents.snapshot.storage.k8s.io /snapcontent-44fe67a7-fd12-4388-979e-5bf5f47b871f Phase: Completed Created: 2025-01-06 07:45:11 +0000 UTC Started: 2025-01-06 07:45:11 +0000 UTC Updated: 2025-01-06 07:45:11 +0000 UTC Resource List: apiextensions.k8s.io/v1/CustomResourceDefinition: - datavolumes.cdi.kubevirt.io - virtualmachines.kubevirt.io authorization.openshift.io/v1/RoleBinding: - test-oadp-186/system:deployers - test-oadp-186/system:image-builders - test-oadp-186/system:image-pullers cdi.kubevirt.io/v1beta1/DataVolume: - test-oadp-186/test-vm-dv kubevirt.io/v1/VirtualMachine: - test-oadp-186/test-vm rbac.authorization.k8s.io/v1/RoleBinding: - test-oadp-186/system:deployers - test-oadp-186/system:image-builders - test-oadp-186/system:image-pullers snapshot.storage.k8s.io/v1/VolumeSnapshot: - test-oadp-186/velero-test-vm-dv-54qjl snapshot.storage.k8s.io/v1/VolumeSnapshotClass: - example-snapclass snapshot.storage.k8s.io/v1/VolumeSnapshotContent: - snapcontent-44fe67a7-fd12-4388-979e-5bf5f47b871f v1/ConfigMap: - test-oadp-186/kube-root-ca.crt - test-oadp-186/openshift-service-ca.crt v1/Event: - test-oadp-186/importer-prime-20b2ac57-b26d-498f-854a-7c5e8355c375.18180ab571434f56 - test-oadp-186/importer-prime-20b2ac57-b26d-498f-854a-7c5e8355c375.18180ab5be21cb59 - test-oadp-186/importer-prime-20b2ac57-b26d-498f-854a-7c5e8355c375.18180ab5e3b7b14d - test-oadp-186/importer-prime-20b2ac57-b26d-498f-854a-7c5e8355c375.18180ab61bd2a30f - test-oadp-186/importer-prime-20b2ac57-b26d-498f-854a-7c5e8355c375.18180ab61bd2e682 - test-oadp-186/importer-prime-20b2ac57-b26d-498f-854a-7c5e8355c375.18180ab6419d533c - test-oadp-186/importer-prime-20b2ac57-b26d-498f-854a-7c5e8355c375.18180ab6436289e6 - test-oadp-186/importer-prime-20b2ac57-b26d-498f-854a-7c5e8355c375.18180ab64ee195d4 - test-oadp-186/importer-prime-20b2ac57-b26d-498f-854a-7c5e8355c375.18180ab656ba618b - test-oadp-186/importer-prime-20b2ac57-b26d-498f-854a-7c5e8355c375.18180ab68c7c87ae - test-oadp-186/importer-prime-20b2ac57-b26d-498f-854a-7c5e8355c375.18180ab696df5b5d - test-oadp-186/importer-prime-20b2ac57-b26d-498f-854a-7c5e8355c375.18180ab69c2f6a86 - test-oadp-186/importer-prime-20b2ac57-b26d-498f-854a-7c5e8355c375.18180ab69c3bff5f - test-oadp-186/importer-prime-20b2ac57-b26d-498f-854a-7c5e8355c375.18180ab6a608604c - test-oadp-186/importer-prime-20b2ac57-b26d-498f-854a-7c5e8355c375.18180ab6aab808ad - test-oadp-186/kubevirt-disruption-budget-g6rlh.18180ac20d365984 - test-oadp-186/prime-20b2ac57-b26d-498f-854a-7c5e8355c375.18180ab56eb1d77b - test-oadp-186/prime-20b2ac57-b26d-498f-854a-7c5e8355c375.18180ab56ecf129d - test-oadp-186/prime-20b2ac57-b26d-498f-854a-7c5e8355c375.18180ab57233e0ba - test-oadp-186/prime-20b2ac57-b26d-498f-854a-7c5e8355c375.18180ab57479cf1a - test-oadp-186/prime-20b2ac57-b26d-498f-854a-7c5e8355c375.18180abb0d226fbd - test-oadp-186/prime-20b2ac57-b26d-498f-854a-7c5e8355c375.18180ac40802cb57 - test-oadp-186/test-vm-dv.18180ab56e44045b - test-oadp-186/test-vm-dv.18180ab56e88a781 - test-oadp-186/test-vm-dv.18180ab56e93e319 - test-oadp-186/test-vm-dv.18180ab56e945559 - test-oadp-186/test-vm-dv.18180ab56e947c3e - test-oadp-186/test-vm-dv.18180ab572a1d999 - test-oadp-186/test-vm-dv.18180ab574648c8f - test-oadp-186/test-vm-dv.18180ab5c18f1986 - test-oadp-186/test-vm-dv.18180ab6cb65f825 - test-oadp-186/test-vm-dv.18180aba8a09a104 - test-oadp-186/test-vm-dv.18180ac208df3068 - test-oadp-186/test-vm-dv.18180ac209f09106 - test-oadp-186/test-vm-dv.18180ac20ad03f89 - test-oadp-186/test-vm.18180ab56bce4d16 - test-oadp-186/test-vm.18180ac20cd2ce5b - test-oadp-186/test-vm.18180ac20d411836 - test-oadp-186/test-vm.18180ac20fe59fce - test-oadp-186/test-vm.18180ac38fb1e682 - test-oadp-186/test-vm.18180ac44237cdc7 - test-oadp-186/test-vm.18180ac4476d125f - test-oadp-186/test-vm.18180ac4fda5b50e - test-oadp-186/test-vm.18180ac4fda9173c - test-oadp-186/test-vm.18180ac4fdde3fb8 - test-oadp-186/test-vm.18180ac50b656174 - test-oadp-186/test-vm.18180ac50d1f0e37 - test-oadp-186/test-vm.18180ac50fce777d - test-oadp-186/virt-launcher-test-vm-gtb7c.18180ac2105a1988 - test-oadp-186/virt-launcher-test-vm-gtb7c.18180ac2331e3f42 - test-oadp-186/virt-launcher-test-vm-gtb7c.18180ac343ca6df9 - test-oadp-186/virt-launcher-test-vm-gtb7c.18180ac343ca9b14 - test-oadp-186/virt-launcher-test-vm-gtb7c.18180ac36aafb44b - test-oadp-186/virt-launcher-test-vm-gtb7c.18180ac36ca4a8de - test-oadp-186/virt-launcher-test-vm-gtb7c.18180ac37ab86060 - test-oadp-186/virt-launcher-test-vm-gtb7c.18180ac37feed218 - test-oadp-186/virt-launcher-test-vm-gtb7c.18180ac4fdf907e4 v1/Namespace: - test-oadp-186 v1/PersistentVolume: - pvc-d2e89338-ccc0-4964-9b4c-3e7029d8fa7e v1/PersistentVolumeClaim: - test-oadp-186/test-vm-dv v1/Secret: - test-oadp-186/builder-dockercfg-pqm2t - test-oadp-186/default-dockercfg-t5vmv - test-oadp-186/deployer-dockercfg-bmqjt v1/ServiceAccount: - test-oadp-186/builder - test-oadp-186/default - test-oadp-186/deployer Backup Volumes: Velero-Native Snapshots: CSI Snapshots: test-oadp-186/test-vm-dv: Snapshot: Operation ID: test-oadp-186/velero-test-vm-dv-54qjl/2025-01-06T07:45:11Z Snapshot Content Name: snapcontent-44fe67a7-fd12-4388-979e-5bf5f47b871f Storage Snapshot ID: 0001-0011-openshift-storage-0000000000000001-55f7b668-85a4-4c57-bb1a-660d951b8f98 Snapshot Size (bytes): 5368709120 CSI Driver: openshift-storage.rbd.csi.ceph.com Pod Volume Backups: HooksAttempted: 0 HooksFailed: 0 STEP: Verify backup ocp-kubevirt-f157e83c-cc01-11ef-8010-0a580a83a847 has completed successfully @ 01/06/25 07:45:25.574 2025/01/06 07:45:25 Backup for case ocp-kubevirt succeeded STEP: Delete the appplication resources ocp-kubevirt @ 01/06/25 07:45:25.632 STEP: Cleanup Application for case ocp-kubevirt @ 01/06/25 07:45:25.632 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Remove namespace test-oadp-186] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=11  changed=4  unreachable=0 failed=0 skipped=8  rescued=0 ignored=0 2025/01/06 07:45:45 2025-01-06 07:45:27,442 p=26395 u=1003440000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2025-01-06 07:45:27,443 p=26395 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:45:27,826 p=26395 u=1003440000 n=ansible INFO| TASK [Get cluster endpoint] **************************************************** 2025-01-06 07:45:27,827 p=26395 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:45:28,162 p=26395 u=1003440000 n=ansible INFO| TASK [Get current admin token] ************************************************* 2025-01-06 07:45:28,162 p=26395 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:45:28,178 p=26395 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:45:28,178 p=26395 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:45:28,561 p=26395 u=1003440000 n=ansible INFO| TASK [Extract kubernetes minor version from cluster_info] ********************** 2025-01-06 07:45:28,562 p=26395 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:45:28,600 p=26395 u=1003440000 n=ansible INFO| TASK [Map kubernetes minor to ocp releases] ************************************ 2025-01-06 07:45:28,600 p=26395 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:45:28,627 p=26395 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:45:28,627 p=26395 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:45:28,630 p=26395 u=1003440000 n=ansible INFO| PLAY [Execute Task] ************************************************************ 2025-01-06 07:45:29,412 p=26395 u=1003440000 n=ansible INFO| TASK [Gathering Facts] ********************************************************* 2025-01-06 07:45:29,413 p=26395 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:45:45,644 p=26395 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Remove namespace test-oadp-186] *** 2025-01-06 07:45:45,644 p=26395 u=1003440000 n=ansible WARNING| [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2025-01-06 07:45:45,645 p=26395 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:45:45,805 p=26395 u=1003440000 n=ansible INFO| PLAY RECAP ********************************************************************* 2025-01-06 07:45:45,805 p=26395 u=1003440000 n=ansible INFO| localhost : ok=11 changed=4 unreachable=0 failed=0 skipped=8 rescued=0 ignored=0 2025/01/06 07:45:45 Creating restore ocp-kubevirt-f157e83c-cc01-11ef-8010-0a580a83a847 for case ocp-kubevirt-f157e83c-cc01-11ef-8010-0a580a83a847 STEP: Create restore ocp-kubevirt-f157e83c-cc01-11ef-8010-0a580a83a847 from backup ocp-kubevirt-f157e83c-cc01-11ef-8010-0a580a83a847 @ 01/06/25 07:45:45.856 2025/01/06 07:45:45 Wait until restore ocp-kubevirt-f157e83c-cc01-11ef-8010-0a580a83a847 is complete restore phase: Finalizing restore phase: Completed STEP: Verify restore ocp-kubevirt-f157e83c-cc01-11ef-8010-0a580a83a847has completed successfully @ 01/06/25 07:46:05.886 STEP: Verify Application restore @ 01/06/25 07:46:05.889 STEP: Verify Application deployment for case ocp-kubevirt @ 01/06/25 07:46:05.889 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Verify VM is not in running state] *** ok: [localhost] PLAY RECAP ********************************************************************* localhost : ok=11  changed=3  unreachable=0 failed=0 skipped=8  rescued=0 ignored=0 2025/01/06 07:46:12 2025-01-06 07:46:08,179 p=26566 u=1003440000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2025-01-06 07:46:08,179 p=26566 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:46:08,661 p=26566 u=1003440000 n=ansible INFO| TASK [Get cluster endpoint] **************************************************** 2025-01-06 07:46:08,661 p=26566 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:46:09,100 p=26566 u=1003440000 n=ansible INFO| TASK [Get current admin token] ************************************************* 2025-01-06 07:46:09,100 p=26566 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:46:09,123 p=26566 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:46:09,123 p=26566 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:46:09,579 p=26566 u=1003440000 n=ansible INFO| TASK [Extract kubernetes minor version from cluster_info] ********************** 2025-01-06 07:46:09,579 p=26566 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:46:09,618 p=26566 u=1003440000 n=ansible INFO| TASK [Map kubernetes minor to ocp releases] ************************************ 2025-01-06 07:46:09,618 p=26566 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:46:09,650 p=26566 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:46:09,650 p=26566 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:46:09,653 p=26566 u=1003440000 n=ansible INFO| PLAY [Execute Task] ************************************************************ 2025-01-06 07:46:10,499 p=26566 u=1003440000 n=ansible INFO| TASK [Gathering Facts] ********************************************************* 2025-01-06 07:46:10,499 p=26566 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:46:11,982 p=26566 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Verify VM is not in running state] *** 2025-01-06 07:46:11,983 p=26566 u=1003440000 n=ansible WARNING| [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2025-01-06 07:46:11,983 p=26566 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:46:12,051 p=26566 u=1003440000 n=ansible INFO| PLAY RECAP ********************************************************************* 2025-01-06 07:46:12,052 p=26566 u=1003440000 n=ansible INFO| localhost : ok=11 changed=3 unreachable=0 failed=0 skipped=8 rescued=0 ignored=0 2025/01/06 07:46:12 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 2025/01/06 07:46:12 Cleaning app [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Remove namespace test-oadp-186] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=11  changed=4  unreachable=0 failed=0 skipped=8  rescued=0 ignored=0 2025/01/06 07:46:33 2025-01-06 07:46:14,427 p=26731 u=1003440000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2025-01-06 07:46:14,427 p=26731 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:46:14,931 p=26731 u=1003440000 n=ansible INFO| TASK [Get cluster endpoint] **************************************************** 2025-01-06 07:46:14,932 p=26731 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:46:15,378 p=26731 u=1003440000 n=ansible INFO| TASK [Get current admin token] ************************************************* 2025-01-06 07:46:15,379 p=26731 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:46:15,402 p=26731 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:46:15,403 p=26731 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:46:15,855 p=26731 u=1003440000 n=ansible INFO| TASK [Extract kubernetes minor version from cluster_info] ********************** 2025-01-06 07:46:15,855 p=26731 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:46:15,898 p=26731 u=1003440000 n=ansible INFO| TASK [Map kubernetes minor to ocp releases] ************************************ 2025-01-06 07:46:15,898 p=26731 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:46:15,930 p=26731 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:46:15,930 p=26731 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:46:15,933 p=26731 u=1003440000 n=ansible INFO| PLAY [Execute Task] ************************************************************ 2025-01-06 07:46:16,747 p=26731 u=1003440000 n=ansible INFO| TASK [Gathering Facts] ********************************************************* 2025-01-06 07:46:16,747 p=26731 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:46:33,007 p=26731 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Remove namespace test-oadp-186] *** 2025-01-06 07:46:33,007 p=26731 u=1003440000 n=ansible WARNING| [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2025-01-06 07:46:33,008 p=26731 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:46:33,223 p=26731 u=1003440000 n=ansible INFO| PLAY RECAP ********************************************************************* 2025-01-06 07:46:33,223 p=26731 u=1003440000 n=ansible INFO| localhost : ok=11 changed=4 unreachable=0 failed=0 skipped=8 rescued=0 ignored=0 2025/01/06 07:46:33 Cleaning setup resources for the backup 2025/01/06 07:46:33 Setting new default StorageClass 'odf-operator-ceph-rbd' 2025/01/06 07:46:33 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2025/01/06 07:46:33 Deleting VolumeSnapshotClass 'example-snapclass' • [174.693 seconds] ------------------------------ CSI: Backup/Restore Openshift Virtualization Workloads  [tc-id:OADP-187] [kubevirt] Backup-restore data volume /alabama/cspi/e2e/kubevirt-plugin/backup_restore_csi.go:68 2025/01/06 07:46:33 Delete all downloadrequest ocp-kubevirt-f157e83c-cc01-11ef-8010-0a580a83a847-64e55c77-087d-4c90-918c-11c2e5fd17b0 ocp-kubevirt-f157e83c-cc01-11ef-8010-0a580a83a847-dc45d880-ee64-40c3-8a4b-ba75efee8a8a ocp-kubevirt-f157e83c-cc01-11ef-8010-0a580a83a847-e1767fbc-84ce-4031-bba4-d5f5bc66947a STEP: Create DPA CR @ 01/06/25 07:46:33.554 2025/01/06 07:46:33 csi 2025/01/06 07:46:33 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "22e47b87-e77e-4990-9a14-995c15d49c95", "resourceVersion": "71127", "generation": 1, "creationTimestamp": "2025-01-06T07:46:33Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2025-01-06T07:46:33Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {} } }, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-bkpv69pr-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "csi" ] } }, "features": null }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/06/25 07:46:33.58 2025/01/06 07:46:33 Waiting for velero pod to be running 2025/01/06 07:46:33 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2025/01/06 07:46:33 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "22e47b87-e77e-4990-9a14-995c15d49c95", "resourceVersion": "71127", "generation": 1, "creationTimestamp": "2025-01-06T07:46:33Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2025-01-06T07:46:33Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {} } }, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-bkpv69pr-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "csi" ] } }, "features": null }, "status": {} } STEP: Prepare backup resources, depending on the volumes backup type @ 01/06/25 07:46:38.596 Run the command: oc get ns openshift-storage &> /dev/null && echo true || echo false 2025/01/06 07:46:38 The 'openshift-storage' namespace exists 2025/01/06 07:46:38 Checking default storage class count 2025/01/06 07:46:38 Using the CSI driver: openshift-storage.rbd.csi.ceph.com 2025/01/06 07:46:38 Snapclass 'example-snapclass' doesn't exist, creating 2025/01/06 07:46:38 Setting new default StorageClass 'odf-operator-ceph-rbd' 2025/01/06 07:46:38 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd STEP: Installing application for case ocp-datavolume @ 01/06/25 07:46:38.971 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-datavolume : Create namespace] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-datavolume : Deploy DataVolume test-dv] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=12  changed=5  unreachable=0 failed=0 skipped=2  rescued=0 ignored=0 2025/01/06 07:46:44 2025-01-06 07:46:40,795 p=26913 u=1003440000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2025-01-06 07:46:40,795 p=26913 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:46:41,079 p=26913 u=1003440000 n=ansible INFO| TASK [Get cluster endpoint] **************************************************** 2025-01-06 07:46:41,079 p=26913 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:46:41,394 p=26913 u=1003440000 n=ansible INFO| TASK [Get current admin token] ************************************************* 2025-01-06 07:46:41,394 p=26913 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:46:41,410 p=26913 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:46:41,411 p=26913 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:46:41,765 p=26913 u=1003440000 n=ansible INFO| TASK [Extract kubernetes minor version from cluster_info] ********************** 2025-01-06 07:46:41,765 p=26913 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:46:41,809 p=26913 u=1003440000 n=ansible INFO| TASK [Map kubernetes minor to ocp releases] ************************************ 2025-01-06 07:46:41,809 p=26913 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:46:41,847 p=26913 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:46:41,847 p=26913 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:46:41,849 p=26913 u=1003440000 n=ansible INFO| PLAY [Execute Task] ************************************************************ 2025-01-06 07:46:42,450 p=26913 u=1003440000 n=ansible INFO| TASK [Gathering Facts] ********************************************************* 2025-01-06 07:46:42,450 p=26913 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:46:43,306 p=26913 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-datavolume : Create namespace] *** 2025-01-06 07:46:43,306 p=26913 u=1003440000 n=ansible WARNING| [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2025-01-06 07:46:43,306 p=26913 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:46:43,990 p=26913 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-datavolume : Deploy DataVolume test-dv] *** 2025-01-06 07:46:43,990 p=26913 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:46:44,012 p=26913 u=1003440000 n=ansible INFO| PLAY RECAP ********************************************************************* 2025-01-06 07:46:44,012 p=26913 u=1003440000 n=ansible INFO| localhost : ok=12 changed=5 unreachable=0 failed=0 skipped=2 rescued=0 ignored=0 STEP: Verify Application deployment @ 01/06/25 07:46:44.054 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. FAILED - RETRYING: [localhost]: Wait for DataVolume to be in Succeeded phase (30 retries left). TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-datavolume : Wait for DataVolume to be in Succeeded phase] *** ok: [localhost] PLAY RECAP ********************************************************************* localhost : ok=11  changed=3  unreachable=0 failed=0 skipped=3  rescued=0 ignored=0 2025/01/06 07:46:59 2025-01-06 07:46:45,585 p=27094 u=1003440000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2025-01-06 07:46:45,586 p=27094 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:46:45,852 p=27094 u=1003440000 n=ansible INFO| TASK [Get cluster endpoint] **************************************************** 2025-01-06 07:46:45,852 p=27094 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:46:46,122 p=27094 u=1003440000 n=ansible INFO| TASK [Get current admin token] ************************************************* 2025-01-06 07:46:46,123 p=27094 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:46:46,140 p=27094 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:46:46,141 p=27094 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:46:46,503 p=27094 u=1003440000 n=ansible INFO| TASK [Extract kubernetes minor version from cluster_info] ********************** 2025-01-06 07:46:46,503 p=27094 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:46:46,581 p=27094 u=1003440000 n=ansible INFO| TASK [Map kubernetes minor to ocp releases] ************************************ 2025-01-06 07:46:46,581 p=27094 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:46:46,620 p=27094 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:46:46,621 p=27094 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:46:46,623 p=27094 u=1003440000 n=ansible INFO| PLAY [Execute Task] ************************************************************ 2025-01-06 07:46:47,319 p=27094 u=1003440000 n=ansible INFO| TASK [Gathering Facts] ********************************************************* 2025-01-06 07:46:47,320 p=27094 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:46:48,321 p=27094 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for DataVolume to be in Succeeded phase (30 retries left). 2025-01-06 07:46:59,301 p=27094 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-datavolume : Wait for DataVolume to be in Succeeded phase] *** 2025-01-06 07:46:59,302 p=27094 u=1003440000 n=ansible WARNING| [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2025-01-06 07:46:59,302 p=27094 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:46:59,308 p=27094 u=1003440000 n=ansible INFO| PLAY RECAP ********************************************************************* 2025-01-06 07:46:59,308 p=27094 u=1003440000 n=ansible INFO| localhost : ok=11 changed=3 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0 2025/01/06 07:46:59 {{ } { } [{{ } {prime-2301fef4-bbb4-4daa-bb59-76b162a89c54 test-oadp-187 471b031b-d99b-45bd-820b-2145163d7d6b 71627 0 2025-01-06 07:46:43 +0000 UTC map[app:containerized-data-importer app.kubernetes.io/component:storage app.kubernetes.io/managed-by:cdi-controller app.kubernetes.io/part-of:hyperconverged-cluster app.kubernetes.io/version:4.17.2 velero.io/exclude-from-backup:true] map[cdi.kubevirt.io/storage.bind.immediate.requested: cdi.kubevirt.io/storage.condition.running:false cdi.kubevirt.io/storage.condition.running.message:Import Complete cdi.kubevirt.io/storage.condition.running.reason:Completed cdi.kubevirt.io/storage.contentType:kubevirt cdi.kubevirt.io/storage.import.importPodName:importer-prime-2301fef4-bbb4-4daa-bb59-76b162a89c54 cdi.kubevirt.io/storage.import.source:none cdi.kubevirt.io/storage.pod.phase:Succeeded cdi.kubevirt.io/storage.pod.restarts:0 cdi.kubevirt.io/storage.populator.kind:VolumeImportSource cdi.kubevirt.io/storage.preallocation.requested:false linkerd.io/inject:disabled pv.kubernetes.io/bind-completed:yes pv.kubernetes.io/bound-by-controller:yes sidecar.istio.io/inject:false volume.beta.kubernetes.io/storage-provisioner:openshift-storage.rbd.csi.ceph.com volume.kubernetes.io/storage-provisioner:openshift-storage.rbd.csi.ceph.com] [{v1 PersistentVolumeClaim test-dv 2301fef4-bbb4-4daa-bb59-76b162a89c54 0xc00083f58f 0xc00083f590}] [kubernetes.io/pvc-protection] [{kube-controller-manager Update v1 2025-01-06 07:46:44 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:pv.kubernetes.io/bind-completed":{},"f:pv.kubernetes.io/bound-by-controller":{},"f:volume.beta.kubernetes.io/storage-provisioner":{},"f:volume.kubernetes.io/storage-provisioner":{}}},"f:spec":{"f:volumeName":{}}} } {kube-controller-manager Update v1 2025-01-06 07:46:44 +0000 UTC FieldsV1 {"f:status":{"f:accessModes":{},"f:capacity":{".":{},"f:storage":{}},"f:phase":{}}} status} {virt-cdi-controller Update v1 2025-01-06 07:46:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:cdi.kubevirt.io/storage.bind.immediate.requested":{},"f:cdi.kubevirt.io/storage.condition.running":{},"f:cdi.kubevirt.io/storage.condition.running.message":{},"f:cdi.kubevirt.io/storage.condition.running.reason":{},"f:cdi.kubevirt.io/storage.contentType":{},"f:cdi.kubevirt.io/storage.import.importPodName":{},"f:cdi.kubevirt.io/storage.import.source":{},"f:cdi.kubevirt.io/storage.pod.phase":{},"f:cdi.kubevirt.io/storage.pod.restarts":{},"f:cdi.kubevirt.io/storage.populator.kind":{},"f:cdi.kubevirt.io/storage.preallocation.requested":{},"f:linkerd.io/inject":{},"f:sidecar.istio.io/inject":{}},"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/component":{},"f:app.kubernetes.io/managed-by":{},"f:app.kubernetes.io/part-of":{},"f:app.kubernetes.io/version":{},"f:velero.io/exclude-from-backup":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"2301fef4-bbb4-4daa-bb59-76b162a89c54\"}":{}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:storageClassName":{},"f:volumeMode":{}}} }]} {[ReadWriteOnce] nil {map[] map[storage:{{104857600 0} {} 100Mi BinarySI}]} pvc-471b031b-d99b-45bd-820b-2145163d7d6b 0xc000628ab0 0xc000628ac0 nil nil } {Bound [ReadWriteOnce] map[storage:{{104857600 0} {} 100Mi BinarySI}] [] map[] map[] nil}} {{ } {test-dv test-oadp-187 2301fef4-bbb4-4daa-bb59-76b162a89c54 71638 0 2025-01-06 07:46:43 +0000 UTC map[alerts.k8s.io/KubePersistentVolumeFillingUp:disabled app:containerized-data-importer app.kubernetes.io/component:storage app.kubernetes.io/managed-by:cdi-controller app.kubernetes.io/part-of:hyperconverged-cluster app.kubernetes.io/version:4.17.2] map[cdi.kubevirt.io/createdForDataVolume:bb5f98b3-47f9-448a-9f2d-fde8aa624b93 cdi.kubevirt.io/storage.bind.immediate.requested:true cdi.kubevirt.io/storage.condition.running:false cdi.kubevirt.io/storage.condition.running.message:Import Complete cdi.kubevirt.io/storage.condition.running.reason:Completed cdi.kubevirt.io/storage.contentType:kubevirt cdi.kubevirt.io/storage.deleteAfterCompletion:false cdi.kubevirt.io/storage.pod.phase:Succeeded cdi.kubevirt.io/storage.pod.restarts:0 cdi.kubevirt.io/storage.populator.progress:100.0% cdi.kubevirt.io/storage.preallocation.requested:false cdi.kubevirt.io/storage.usePopulator:true pv.kubernetes.io/bind-completed:yes pv.kubernetes.io/bound-by-controller:yes volume.beta.kubernetes.io/storage-provisioner:openshift-storage.rbd.csi.ceph.com volume.kubernetes.io/storage-provisioner:openshift-storage.rbd.csi.ceph.com] [{cdi.kubevirt.io/v1beta1 DataVolume test-dv bb5f98b3-47f9-448a-9f2d-fde8aa624b93 0xc00083f8a7 0xc00083f8a8}] [kubernetes.io/pvc-protection] [{kube-controller-manager Update v1 2025-01-06 07:46:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:pv.kubernetes.io/bind-completed":{},"f:pv.kubernetes.io/bound-by-controller":{},"f:volume.beta.kubernetes.io/storage-provisioner":{},"f:volume.kubernetes.io/storage-provisioner":{}}},"f:spec":{"f:volumeName":{}}} } {kube-controller-manager Update v1 2025-01-06 07:46:56 +0000 UTC FieldsV1 {"f:status":{"f:accessModes":{},"f:capacity":{".":{},"f:storage":{}},"f:phase":{}}} status} {virt-cdi-controller Update v1 2025-01-06 07:46:56 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:cdi.kubevirt.io/createdForDataVolume":{},"f:cdi.kubevirt.io/storage.bind.immediate.requested":{},"f:cdi.kubevirt.io/storage.condition.running":{},"f:cdi.kubevirt.io/storage.condition.running.message":{},"f:cdi.kubevirt.io/storage.condition.running.reason":{},"f:cdi.kubevirt.io/storage.contentType":{},"f:cdi.kubevirt.io/storage.deleteAfterCompletion":{},"f:cdi.kubevirt.io/storage.pod.phase":{},"f:cdi.kubevirt.io/storage.pod.restarts":{},"f:cdi.kubevirt.io/storage.populator.progress":{},"f:cdi.kubevirt.io/storage.preallocation.requested":{},"f:cdi.kubevirt.io/storage.usePopulator":{}},"f:labels":{".":{},"f:alerts.k8s.io/KubePersistentVolumeFillingUp":{},"f:app":{},"f:app.kubernetes.io/component":{},"f:app.kubernetes.io/managed-by":{},"f:app.kubernetes.io/part-of":{},"f:app.kubernetes.io/version":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"bb5f98b3-47f9-448a-9f2d-fde8aa624b93\"}":{}}},"f:spec":{"f:accessModes":{},"f:dataSourceRef":{".":{},"f:apiGroup":{},"f:kind":{},"f:name":{}},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}}} }]} {[ReadWriteOnce] nil {map[] map[storage:{{104857600 0} {} 100Mi BinarySI}]} pvc-471b031b-d99b-45bd-820b-2145163d7d6b 0xc000628d20 0xc000628d30 &TypedLocalObjectReference{APIGroup:*cdi.kubevirt.io,Kind:VolumeImportSource,Name:volume-import-source-bb5f98b3-47f9-448a-9f2d-fde8aa624b93,} &TypedObjectReference{APIGroup:*cdi.kubevirt.io,Kind:VolumeImportSource,Name:volume-import-source-bb5f98b3-47f9-448a-9f2d-fde8aa624b93,Namespace:nil,} } {Bound [ReadWriteOnce] map[storage:{{104857600 0} {} 100Mi BinarySI}] [] map[] map[] nil}}]} STEP: Creating backup ocp-datavolume-5973bc77-cc02-11ef-8010-0a580a83a847 @ 01/06/25 07:46:59.435 2025/01/06 07:46:59 Wait until backup ocp-datavolume-5973bc77-cc02-11ef-8010-0a580a83a847 is completed backup phase: Completed 2025/01/06 07:47:19 Verify the PVC and VSC count is matching [FAILED] in [It] - /alabama/cspi/lib/backup.go:331 @ 01/06/25 07:47:19.458 2025/01/06 07:47:19 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 STEP: Get the failed spec name @ 01/06/25 07:47:19.458 2025/01/06 07:47:19 The failed spec name is: CSI: Backup/Restore Openshift Virtualization Workloads [tc-id:OADP-187] [kubevirt] Backup-restore data volume STEP: Create a folder for all must-gather files if it doesn't exists already @ 01/06/25 07:47:19.458 2025/01/06 07:47:19 The folder logs does not exists, creating new folder with the name: logs STEP: Create a folder for the failed spec if it doesn't exists already @ 01/06/25 07:47:19.458 2025/01/06 07:47:19 The folder logs/It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-187_kubevirt_Backup-restore_data_volume does not exists, creating new folder with the name: logs/It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-187_kubevirt_Backup-restore_data_volume STEP: Run must-gather because the spec failed @ 01/06/25 07:47:19.458 2025/01/06 07:47:19 Log the present working directory path:- /alabama/cspi/e2e/kubevirt-plugin 2025/01/06 07:47:19 [adm must-gather --dest-dir /alabama/cspi/e2e/kubevirt-plugin/logs/It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-187_kubevirt_Backup-restore_data_volume --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1] 2025/01/06 07:47:58 Log all the files present in /alabama/cspi/e2e/kubevirt-plugin/logs directory 2025/01/06 07:47:58 It_CSI_Backup_Restore_Openshift_Virtualization_Workloads_tc-id_OADP-187_kubevirt_Backup-restore_data_volume STEP: Find must-gather folder and rename it to a shorter more readable name @ 01/06/25 07:47:58.214 2025/01/06 07:47:58 Cleaning app [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-datavolume : Remove namespace test-oadp-187] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=11  changed=4  unreachable=0 failed=0 skipped=3  rescued=0 ignored=0 2025/01/06 07:48:18 2025-01-06 07:48:00,296 p=27315 u=1003440000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2025-01-06 07:48:00,296 p=27315 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:48:00,641 p=27315 u=1003440000 n=ansible INFO| TASK [Get cluster endpoint] **************************************************** 2025-01-06 07:48:00,641 p=27315 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:48:01,010 p=27315 u=1003440000 n=ansible INFO| TASK [Get current admin token] ************************************************* 2025-01-06 07:48:01,011 p=27315 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:48:01,037 p=27315 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:48:01,037 p=27315 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:48:01,496 p=27315 u=1003440000 n=ansible INFO| TASK [Extract kubernetes minor version from cluster_info] ********************** 2025-01-06 07:48:01,496 p=27315 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:48:01,577 p=27315 u=1003440000 n=ansible INFO| TASK [Map kubernetes minor to ocp releases] ************************************ 2025-01-06 07:48:01,577 p=27315 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:48:01,644 p=27315 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:48:01,645 p=27315 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:48:01,647 p=27315 u=1003440000 n=ansible INFO| PLAY [Execute Task] ************************************************************ 2025-01-06 07:48:02,342 p=27315 u=1003440000 n=ansible INFO| TASK [Gathering Facts] ********************************************************* 2025-01-06 07:48:02,342 p=27315 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:48:18,508 p=27315 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-datavolume : Remove namespace test-oadp-187] *** 2025-01-06 07:48:18,508 p=27315 u=1003440000 n=ansible WARNING| [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2025-01-06 07:48:18,508 p=27315 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:48:18,611 p=27315 u=1003440000 n=ansible INFO| PLAY RECAP ********************************************************************* 2025-01-06 07:48:18,612 p=27315 u=1003440000 n=ansible INFO| localhost : ok=11 changed=4 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0 2025/01/06 07:48:18 Cleaning setup resources for the backup 2025/01/06 07:48:18 Setting new default StorageClass 'odf-operator-ceph-rbd' 2025/01/06 07:48:18 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2025/01/06 07:48:18 Deleting VolumeSnapshotClass 'example-snapclass' • [FAILED] [105.365 seconds] CSI: Backup/Restore Openshift Virtualization Workloads  [It] [tc-id:OADP-187] [kubevirt] Backup-restore data volume /alabama/cspi/e2e/kubevirt-plugin/backup_restore_csi.go:68 [FAILED] PVC and VSC count is not matching Expected : 1 to equal : 2 In [It] at: /alabama/cspi/lib/backup.go:331 @ 01/06/25 07:47:19.458 ------------------------------ S ------------------------------ Native CSI Data Mover: Backup/Restore Openshift Virtualization Workloads  [tc-id:OADP-401] [kubevirt] Started VM should over ceph filesytem mode /alabama/cspi/e2e/kubevirt-plugin/backup_restore_datamover.go:129 2025/01/06 07:48:18 Delete all downloadrequest ocp-datavolume-5973bc77-cc02-11ef-8010-0a580a83a847-2d7058e4-c4c5-4cff-a6b6-78b47316bee1 ocp-datavolume-5973bc77-cc02-11ef-8010-0a580a83a847-9e735aaa-ec26-4b37-a5d3-a04ccfb82d8e ocp-datavolume-5973bc77-cc02-11ef-8010-0a580a83a847-e1430e04-30ed-4a89-a2ca-1554b5b7d413 ocp-datavolume-5973bc77-cc02-11ef-8010-0a580a83a847-f8a6e1b2-12dc-49e0-a490-483733efb1fb ocp-kubevirt-73d3076c-cc01-11ef-8010-0a580a83a847-1dbdf35c-bf14-4ee2-bccb-ced5b99497cc ocp-kubevirt-73d3076c-cc01-11ef-8010-0a580a83a847-85ffa07b-891f-4c79-8ce1-ecc6f4ebaee1 ocp-kubevirt-73d3076c-cc01-11ef-8010-0a580a83a847-96b577b3-a02b-4794-994b-acc885a4dc3b ocp-kubevirt-73d3076c-cc01-11ef-8010-0a580a83a847-af92d2d6-646b-4feb-8228-c36f89881a45 ocp-kubevirt-73d3076c-cc01-11ef-8010-0a580a83a847-b53ebf39-4f7b-437d-86c3-2c15992fde76 ocp-kubevirt-73d3076c-cc01-11ef-8010-0a580a83a847-b639e711-2b53-4499-a3b2-9fa8cf7075a8 ocp-kubevirt-73d3076c-cc01-11ef-8010-0a580a83a847-e5bcbb45-edd0-41d4-98f7-4c664bfa150f ocp-kubevirt-73d3076c-cc01-11ef-8010-0a580a83a847-f46a5e89-0170-4050-8c2d-caa5b9e6b41b ocp-kubevirt-f157e83c-cc01-11ef-8010-0a580a83a847-036b28f0-ec05-434a-a866-81b91c02a11c ocp-kubevirt-f157e83c-cc01-11ef-8010-0a580a83a847-03915733-7018-4e83-b0bd-89f8911c1083 ocp-kubevirt-f157e83c-cc01-11ef-8010-0a580a83a847-1c7c1898-521c-4957-9029-85e7ecb3aa2f ocp-kubevirt-f157e83c-cc01-11ef-8010-0a580a83a847-29b3b112-2a22-4ce5-bce4-002134dd7a59 ocp-kubevirt-f157e83c-cc01-11ef-8010-0a580a83a847-2be3a484-1d7a-4782-bedd-165e97dd552f ocp-kubevirt-f157e83c-cc01-11ef-8010-0a580a83a847-405ff7cf-5071-4707-b24a-a9f61200c290 ocp-kubevirt-f157e83c-cc01-11ef-8010-0a580a83a847-823b7211-f4d2-4d41-bb45-fa02b12f1a3d ocp-kubevirt-f157e83c-cc01-11ef-8010-0a580a83a847-cd0ae943-1238-4b4b-88cd-f04d33b5004d STEP: Create DPA CR @ 01/06/25 07:48:20.817 2025/01/06 07:48:20 native-datamover 2025/01/06 07:48:20 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "1d91fdaa-6990-471a-9992-edd11fddab06", "resourceVersion": "73032", "generation": 1, "creationTimestamp": "2025-01-06T07:48:20Z", "managedFields": [ { "manager": "kubevirt-plugin.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2025-01-06T07:48:20Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:nodeAgent": { ".": {}, "f:enable": {}, "f:podConfig": { ".": {}, "f:resourceAllocations": {} }, "f:uploaderType": {} }, "f:velero": { ".": {}, "f:defaultPlugins": {} } }, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "velero": { "provider": "aws", "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "cloud" }, "objectStorage": { "bucket": "ci-op-bkpv69pr-interopoadp", "prefix": "kubevirt" }, "default": true } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "csi" ] }, "nodeAgent": { "enable": true, "podConfig": { "resourceAllocations": {} }, "uploaderType": "kopia" } }, "features": null }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 01/06/25 07:48:20.84 2025/01/06 07:48:20 Waiting for velero pod to be running 2025/01/06 07:48:25 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' STEP: Prepare backup resources, depending on the volumes backup type @ 01/06/25 07:48:25.861 Run the command: oc get ns openshift-storage &> /dev/null && echo true || echo false 2025/01/06 07:48:25 The 'openshift-storage' namespace exists 2025/01/06 07:48:25 Checking default storage class count 2025/01/06 07:48:26 Using the CSI driver: openshift-storage.rbd.csi.ceph.com 2025/01/06 07:48:26 Snapclass 'example-snapclass' doesn't exist, creating 2025/01/06 07:48:26 Setting new default StorageClass 'odf-operator-ceph-rbd' 2025/01/06 07:48:26 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2025/01/06 07:48:26 Checking for correct number of running NodeAgent pods... 2025/01/06 07:48:26 pod: node-agent-gdjlh is not yet running with status: {Pending [{PodReadyToStartContainers False 0001-01-01 00:00:00 +0000 UTC 2025-01-06 07:48:21 +0000 UTC } {Initialized True 0001-01-01 00:00:00 +0000 UTC 2025-01-06 07:48:21 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2025-01-06 07:48:21 +0000 UTC ContainersNotReady containers with unready status: [node-agent]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2025-01-06 07:48:21 +0000 UTC ContainersNotReady containers with unready status: [node-agent]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2025-01-06 07:48:21 +0000 UTC }] 10.0.106.2 [{10.0.106.2}] [] 2025-01-06 07:48:21 +0000 UTC [] [{node-agent {&ContainerStateWaiting{Reason:ContainerCreating,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-rhel9@sha256:cf9ebc90a72bf5043ef275babb19a4eebe830db9d7194a3e973ea64910f1ae29 0xc0009a59ba map[] nil}] Burstable [] []} STEP: Installing application for case ocp-kubevirt @ 01/06/25 07:48:31.236 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Create namespace] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Deploy vm test-vm] *** changed: [localhost] FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (60 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (59 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (58 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (57 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (56 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (55 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (54 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (53 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (52 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (51 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (50 retries left). TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Wait for VM to be Running & Ready] *** ok: [localhost] PLAY RECAP ********************************************************************* localhost : ok=13  changed=5  unreachable=0 failed=0 skipped=6  rescued=0 ignored=0 2025/01/06 07:49:40 2025-01-06 07:48:33,051 p=27505 u=1003440000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2025-01-06 07:48:33,051 p=27505 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:48:33,382 p=27505 u=1003440000 n=ansible INFO| TASK [Get cluster endpoint] **************************************************** 2025-01-06 07:48:33,382 p=27505 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:48:33,687 p=27505 u=1003440000 n=ansible INFO| TASK [Get current admin token] ************************************************* 2025-01-06 07:48:33,687 p=27505 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:48:33,705 p=27505 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:48:33,705 p=27505 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:48:34,053 p=27505 u=1003440000 n=ansible INFO| TASK [Extract kubernetes minor version from cluster_info] ********************** 2025-01-06 07:48:34,054 p=27505 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:48:34,092 p=27505 u=1003440000 n=ansible INFO| TASK [Map kubernetes minor to ocp releases] ************************************ 2025-01-06 07:48:34,092 p=27505 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:48:34,118 p=27505 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:48:34,118 p=27505 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:48:34,120 p=27505 u=1003440000 n=ansible INFO| PLAY [Execute Task] ************************************************************ 2025-01-06 07:48:34,746 p=27505 u=1003440000 n=ansible INFO| TASK [Gathering Facts] ********************************************************* 2025-01-06 07:48:34,746 p=27505 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:48:35,850 p=27505 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Create namespace] *** 2025-01-06 07:48:35,851 p=27505 u=1003440000 n=ansible WARNING| [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2025-01-06 07:48:35,851 p=27505 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:48:36,765 p=27505 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Deploy vm test-vm] *** 2025-01-06 07:48:36,765 p=27505 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:48:37,687 p=27505 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (60 retries left). 2025-01-06 07:48:43,670 p=27505 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (59 retries left). 2025-01-06 07:48:49,407 p=27505 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (58 retries left). 2025-01-06 07:48:55,511 p=27505 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (57 retries left). 2025-01-06 07:49:01,509 p=27505 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (56 retries left). 2025-01-06 07:49:07,149 p=27505 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (55 retries left). 2025-01-06 07:49:12,789 p=27505 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (54 retries left). 2025-01-06 07:49:18,392 p=27505 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (53 retries left). 2025-01-06 07:49:23,981 p=27505 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (52 retries left). 2025-01-06 07:49:29,582 p=27505 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (51 retries left). 2025-01-06 07:49:35,178 p=27505 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (50 retries left). 2025-01-06 07:49:40,793 p=27505 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Wait for VM to be Running & Ready] *** 2025-01-06 07:49:40,793 p=27505 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:49:40,893 p=27505 u=1003440000 n=ansible INFO| PLAY RECAP ********************************************************************* 2025-01-06 07:49:40,893 p=27505 u=1003440000 n=ansible INFO| localhost : ok=13 changed=5 unreachable=0 failed=0 skipped=6 rescued=0 ignored=0 STEP: Verify Application deployment @ 01/06/25 07:49:40.935 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Wait for VM to be Running & Ready] *** ok: [localhost] FAILED - RETRYING: [localhost]: Wait for VM to have AgentConnected status True indicating the guest agent is running (60 retries left). FAILED - RETRYING: [localhost]: Wait for VM to have AgentConnected status True indicating the guest agent is running (59 retries left). FAILED - RETRYING: [localhost]: Wait for VM to have AgentConnected status True indicating the guest agent is running (58 retries left). TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Wait for VM to have AgentConnected status True indicating the guest agent is running] *** ok: [localhost] PLAY RECAP ********************************************************************* localhost : ok=12  changed=3  unreachable=0 failed=0 skipped=7  rescued=0 ignored=0 2025/01/06 07:50:02 2025-01-06 07:49:42,334 p=27849 u=1003440000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2025-01-06 07:49:42,334 p=27849 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:49:42,595 p=27849 u=1003440000 n=ansible INFO| TASK [Get cluster endpoint] **************************************************** 2025-01-06 07:49:42,595 p=27849 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:49:42,848 p=27849 u=1003440000 n=ansible INFO| TASK [Get current admin token] ************************************************* 2025-01-06 07:49:42,848 p=27849 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:49:42,864 p=27849 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:49:42,865 p=27849 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:49:43,164 p=27849 u=1003440000 n=ansible INFO| TASK [Extract kubernetes minor version from cluster_info] ********************** 2025-01-06 07:49:43,164 p=27849 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:49:43,191 p=27849 u=1003440000 n=ansible INFO| TASK [Map kubernetes minor to ocp releases] ************************************ 2025-01-06 07:49:43,191 p=27849 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:49:43,207 p=27849 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:49:43,208 p=27849 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:49:43,209 p=27849 u=1003440000 n=ansible INFO| PLAY [Execute Task] ************************************************************ 2025-01-06 07:49:43,760 p=27849 u=1003440000 n=ansible INFO| TASK [Gathering Facts] ********************************************************* 2025-01-06 07:49:43,760 p=27849 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:49:44,605 p=27849 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Wait for VM to be Running & Ready] *** 2025-01-06 07:49:44,605 p=27849 u=1003440000 n=ansible WARNING| [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2025-01-06 07:49:44,605 p=27849 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:49:45,232 p=27849 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to have AgentConnected status True indicating the guest agent is running (60 retries left). 2025-01-06 07:49:50,843 p=27849 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to have AgentConnected status True indicating the guest agent is running (59 retries left). 2025-01-06 07:49:56,430 p=27849 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to have AgentConnected status True indicating the guest agent is running (58 retries left). 2025-01-06 07:50:02,016 p=27849 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Wait for VM to have AgentConnected status True indicating the guest agent is running] *** 2025-01-06 07:50:02,016 p=27849 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:50:02,021 p=27849 u=1003440000 n=ansible INFO| PLAY RECAP ********************************************************************* 2025-01-06 07:50:02,021 p=27849 u=1003440000 n=ansible INFO| localhost : ok=12 changed=3 unreachable=0 failed=0 skipped=7 rescued=0 ignored=0 STEP: Creating backup ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847 @ 01/06/25 07:50:02.079 2025/01/06 07:50:02 Wait until backup ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847 is completed backup phase: WaitingForPluginOperations DataUpload ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-tbmzb phase: Accepted DataUpload Name: ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-tbmzb and status: Accepted 2025/01/06 07:50:22 { "kind": "DataUpload", "apiVersion": "velero.io/v2alpha1", "metadata": { "name": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-tbmzb", "generateName": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-", "namespace": "openshift-adp", "uid": "5dcd1ade-5612-4a5d-b1a5-1e284b61f64d", "resourceVersion": "74851", "generation": 2, "creationTimestamp": "2025-01-06T07:50:10Z", "labels": { "velero.io/accepted-by": "ip-10-0-21-92.us-east-2.compute.internal", "velero.io/async-operation-id": "du-d173214e-4527-430f-9886-c2b4f5c869d0.ce023406-c2eb-43f4a49b2", "velero.io/backup-name": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847", "velero.io/backup-uid": "d173214e-4527-430f-9886-c2b4f5c869d0", "velero.io/pvc-uid": "ce023406-c2eb-43f7-9b8d-bede1f7bd7bb" }, "ownerReferences": [ { "apiVersion": "velero.io/v1", "kind": "Backup", "name": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847", "uid": "d173214e-4527-430f-9886-c2b4f5c869d0", "controller": true } ], "finalizers": [ "velero.io/data-upload-download-finalizer" ], "managedFields": [ { "manager": "node-agent-server", "operation": "Update", "apiVersion": "velero.io/v2alpha1", "time": "2025-01-06T07:50:10Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:metadata": { "f:finalizers": { ".": {}, "v:\"velero.io/data-upload-download-finalizer\"": {} }, "f:labels": { "f:velero.io/accepted-by": {} } }, "f:status": { "f:phase": {} } } }, { "manager": "velero", "operation": "Update", "apiVersion": "velero.io/v2alpha1", "time": "2025-01-06T07:50:10Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:metadata": { "f:generateName": {}, "f:labels": { ".": {}, "f:velero.io/async-operation-id": {}, "f:velero.io/backup-name": {}, "f:velero.io/backup-uid": {}, "f:velero.io/pvc-uid": {} }, "f:ownerReferences": { ".": {}, "k:{\"uid\":\"d173214e-4527-430f-9886-c2b4f5c869d0\"}": {} } }, "f:spec": { ".": {}, "f:backupStorageLocation": {}, "f:csiSnapshot": { ".": {}, "f:snapshotClass": {}, "f:storageClass": {}, "f:volumeSnapshot": {} }, "f:operationTimeout": {}, "f:snapshotType": {}, "f:sourceNamespace": {}, "f:sourcePVC": {} }, "f:status": { ".": {}, "f:progress": {} } } } ] }, "spec": { "snapshotType": "CSI", "csiSnapshot": { "volumeSnapshot": "velero-test-vm-dv-2qphs", "storageClass": "odf-operator-cephfs", "snapshotClass": "odf-operator-cephfsplugin-snapclass" }, "sourcePVC": "test-vm-dv", "backupStorageLocation": "ts-dpa-1", "sourceNamespace": "test-oadp-401", "operationTimeout": "10m0s" }, "status": { "phase": "Accepted", "progress": {} } } backup phase: WaitingForPluginOperations DataUpload ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-tbmzb phase: Accepted DataUpload Name: ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-tbmzb and status: Accepted 2025/01/06 07:50:42 { "kind": "DataUpload", "apiVersion": "velero.io/v2alpha1", "metadata": { "name": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-tbmzb", "generateName": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-", "namespace": "openshift-adp", "uid": "5dcd1ade-5612-4a5d-b1a5-1e284b61f64d", "resourceVersion": "74851", "generation": 2, "creationTimestamp": "2025-01-06T07:50:10Z", "labels": { "velero.io/accepted-by": "ip-10-0-21-92.us-east-2.compute.internal", "velero.io/async-operation-id": "du-d173214e-4527-430f-9886-c2b4f5c869d0.ce023406-c2eb-43f4a49b2", "velero.io/backup-name": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847", "velero.io/backup-uid": "d173214e-4527-430f-9886-c2b4f5c869d0", "velero.io/pvc-uid": "ce023406-c2eb-43f7-9b8d-bede1f7bd7bb" }, "ownerReferences": [ { "apiVersion": "velero.io/v1", "kind": "Backup", "name": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847", "uid": "d173214e-4527-430f-9886-c2b4f5c869d0", "controller": true } ], "finalizers": [ "velero.io/data-upload-download-finalizer" ], "managedFields": [ { "manager": "node-agent-server", "operation": "Update", "apiVersion": "velero.io/v2alpha1", "time": "2025-01-06T07:50:10Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:metadata": { "f:finalizers": { ".": {}, "v:\"velero.io/data-upload-download-finalizer\"": {} }, "f:labels": { "f:velero.io/accepted-by": {} } }, "f:status": { "f:phase": {} } } }, { "manager": "velero", "operation": "Update", "apiVersion": "velero.io/v2alpha1", "time": "2025-01-06T07:50:10Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:metadata": { "f:generateName": {}, "f:labels": { ".": {}, "f:velero.io/async-operation-id": {}, "f:velero.io/backup-name": {}, "f:velero.io/backup-uid": {}, "f:velero.io/pvc-uid": {} }, "f:ownerReferences": { ".": {}, "k:{\"uid\":\"d173214e-4527-430f-9886-c2b4f5c869d0\"}": {} } }, "f:spec": { ".": {}, "f:backupStorageLocation": {}, "f:csiSnapshot": { ".": {}, "f:snapshotClass": {}, "f:storageClass": {}, "f:volumeSnapshot": {} }, "f:operationTimeout": {}, "f:snapshotType": {}, "f:sourceNamespace": {}, "f:sourcePVC": {} }, "f:status": { ".": {}, "f:progress": {} } } } ] }, "spec": { "snapshotType": "CSI", "csiSnapshot": { "volumeSnapshot": "velero-test-vm-dv-2qphs", "storageClass": "odf-operator-cephfs", "snapshotClass": "odf-operator-cephfsplugin-snapclass" }, "sourcePVC": "test-vm-dv", "backupStorageLocation": "ts-dpa-1", "sourceNamespace": "test-oadp-401", "operationTimeout": "10m0s" }, "status": { "phase": "Accepted", "progress": {} } } backup phase: WaitingForPluginOperations DataUpload ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-tbmzb phase: Accepted DataUpload Name: ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-tbmzb and status: Accepted 2025/01/06 07:51:02 { "kind": "DataUpload", "apiVersion": "velero.io/v2alpha1", "metadata": { "name": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-tbmzb", "generateName": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-", "namespace": "openshift-adp", "uid": "5dcd1ade-5612-4a5d-b1a5-1e284b61f64d", "resourceVersion": "74851", "generation": 2, "creationTimestamp": "2025-01-06T07:50:10Z", "labels": { "velero.io/accepted-by": "ip-10-0-21-92.us-east-2.compute.internal", "velero.io/async-operation-id": "du-d173214e-4527-430f-9886-c2b4f5c869d0.ce023406-c2eb-43f4a49b2", "velero.io/backup-name": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847", "velero.io/backup-uid": "d173214e-4527-430f-9886-c2b4f5c869d0", "velero.io/pvc-uid": "ce023406-c2eb-43f7-9b8d-bede1f7bd7bb" }, "ownerReferences": [ { "apiVersion": "velero.io/v1", "kind": "Backup", "name": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847", "uid": "d173214e-4527-430f-9886-c2b4f5c869d0", "controller": true } ], "finalizers": [ "velero.io/data-upload-download-finalizer" ], "managedFields": [ { "manager": "node-agent-server", "operation": "Update", "apiVersion": "velero.io/v2alpha1", "time": "2025-01-06T07:50:10Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:metadata": { "f:finalizers": { ".": {}, "v:\"velero.io/data-upload-download-finalizer\"": {} }, "f:labels": { "f:velero.io/accepted-by": {} } }, "f:status": { "f:phase": {} } } }, { "manager": "velero", "operation": "Update", "apiVersion": "velero.io/v2alpha1", "time": "2025-01-06T07:50:10Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:metadata": { "f:generateName": {}, "f:labels": { ".": {}, "f:velero.io/async-operation-id": {}, "f:velero.io/backup-name": {}, "f:velero.io/backup-uid": {}, "f:velero.io/pvc-uid": {} }, "f:ownerReferences": { ".": {}, "k:{\"uid\":\"d173214e-4527-430f-9886-c2b4f5c869d0\"}": {} } }, "f:spec": { ".": {}, "f:backupStorageLocation": {}, "f:csiSnapshot": { ".": {}, "f:snapshotClass": {}, "f:storageClass": {}, "f:volumeSnapshot": {} }, "f:operationTimeout": {}, "f:snapshotType": {}, "f:sourceNamespace": {}, "f:sourcePVC": {} }, "f:status": { ".": {}, "f:progress": {} } } } ] }, "spec": { "snapshotType": "CSI", "csiSnapshot": { "volumeSnapshot": "velero-test-vm-dv-2qphs", "storageClass": "odf-operator-cephfs", "snapshotClass": "odf-operator-cephfsplugin-snapclass" }, "sourcePVC": "test-vm-dv", "backupStorageLocation": "ts-dpa-1", "sourceNamespace": "test-oadp-401", "operationTimeout": "10m0s" }, "status": { "phase": "Accepted", "progress": {} } } backup phase: WaitingForPluginOperations DataUpload ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-tbmzb phase: Accepted DataUpload Name: ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-tbmzb and status: Accepted 2025/01/06 07:51:22 { "kind": "DataUpload", "apiVersion": "velero.io/v2alpha1", "metadata": { "name": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-tbmzb", "generateName": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-", "namespace": "openshift-adp", "uid": "5dcd1ade-5612-4a5d-b1a5-1e284b61f64d", "resourceVersion": "74851", "generation": 2, "creationTimestamp": "2025-01-06T07:50:10Z", "labels": { "velero.io/accepted-by": "ip-10-0-21-92.us-east-2.compute.internal", "velero.io/async-operation-id": "du-d173214e-4527-430f-9886-c2b4f5c869d0.ce023406-c2eb-43f4a49b2", "velero.io/backup-name": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847", "velero.io/backup-uid": "d173214e-4527-430f-9886-c2b4f5c869d0", "velero.io/pvc-uid": "ce023406-c2eb-43f7-9b8d-bede1f7bd7bb" }, "ownerReferences": [ { "apiVersion": "velero.io/v1", "kind": "Backup", "name": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847", "uid": "d173214e-4527-430f-9886-c2b4f5c869d0", "controller": true } ], "finalizers": [ "velero.io/data-upload-download-finalizer" ], "managedFields": [ { "manager": "node-agent-server", "operation": "Update", "apiVersion": "velero.io/v2alpha1", "time": "2025-01-06T07:50:10Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:metadata": { "f:finalizers": { ".": {}, "v:\"velero.io/data-upload-download-finalizer\"": {} }, "f:labels": { "f:velero.io/accepted-by": {} } }, "f:status": { "f:phase": {} } } }, { "manager": "velero", "operation": "Update", "apiVersion": "velero.io/v2alpha1", "time": "2025-01-06T07:50:10Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:metadata": { "f:generateName": {}, "f:labels": { ".": {}, "f:velero.io/async-operation-id": {}, "f:velero.io/backup-name": {}, "f:velero.io/backup-uid": {}, "f:velero.io/pvc-uid": {} }, "f:ownerReferences": { ".": {}, "k:{\"uid\":\"d173214e-4527-430f-9886-c2b4f5c869d0\"}": {} } }, "f:spec": { ".": {}, "f:backupStorageLocation": {}, "f:csiSnapshot": { ".": {}, "f:snapshotClass": {}, "f:storageClass": {}, "f:volumeSnapshot": {} }, "f:operationTimeout": {}, "f:snapshotType": {}, "f:sourceNamespace": {}, "f:sourcePVC": {} }, "f:status": { ".": {}, "f:progress": {} } } } ] }, "spec": { "snapshotType": "CSI", "csiSnapshot": { "volumeSnapshot": "velero-test-vm-dv-2qphs", "storageClass": "odf-operator-cephfs", "snapshotClass": "odf-operator-cephfsplugin-snapclass" }, "sourcePVC": "test-vm-dv", "backupStorageLocation": "ts-dpa-1", "sourceNamespace": "test-oadp-401", "operationTimeout": "10m0s" }, "status": { "phase": "Accepted", "progress": {} } } backup phase: WaitingForPluginOperations DataUpload ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-tbmzb phase: InProgress DataUpload Name: ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-tbmzb and status: InProgress 2025/01/06 07:51:42 { "kind": "DataUpload", "apiVersion": "velero.io/v2alpha1", "metadata": { "name": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-tbmzb", "generateName": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-", "namespace": "openshift-adp", "uid": "5dcd1ade-5612-4a5d-b1a5-1e284b61f64d", "resourceVersion": "76262", "generation": 5, "creationTimestamp": "2025-01-06T07:50:10Z", "labels": { "velero.io/accepted-by": "ip-10-0-21-92.us-east-2.compute.internal", "velero.io/async-operation-id": "du-d173214e-4527-430f-9886-c2b4f5c869d0.ce023406-c2eb-43f4a49b2", "velero.io/backup-name": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847", "velero.io/backup-uid": "d173214e-4527-430f-9886-c2b4f5c869d0", "velero.io/pvc-uid": "ce023406-c2eb-43f7-9b8d-bede1f7bd7bb" }, "ownerReferences": [ { "apiVersion": "velero.io/v1", "kind": "Backup", "name": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847", "uid": "d173214e-4527-430f-9886-c2b4f5c869d0", "controller": true } ], "finalizers": [ "velero.io/data-upload-download-finalizer" ], "managedFields": [ { "manager": "velero", "operation": "Update", "apiVersion": "velero.io/v2alpha1", "time": "2025-01-06T07:50:10Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:metadata": { "f:generateName": {}, "f:labels": { ".": {}, "f:velero.io/async-operation-id": {}, "f:velero.io/backup-name": {}, "f:velero.io/backup-uid": {}, "f:velero.io/pvc-uid": {} }, "f:ownerReferences": { ".": {}, "k:{\"uid\":\"d173214e-4527-430f-9886-c2b4f5c869d0\"}": {} } }, "f:spec": { ".": {}, "f:backupStorageLocation": {}, "f:csiSnapshot": { ".": {}, "f:snapshotClass": {}, "f:storageClass": {}, "f:volumeSnapshot": {} }, "f:operationTimeout": {}, "f:snapshotType": {}, "f:sourceNamespace": {}, "f:sourcePVC": {} }, "f:status": { ".": {}, "f:progress": {} } } }, { "manager": "node-agent-server", "operation": "Update", "apiVersion": "velero.io/v2alpha1", "time": "2025-01-06T07:51:33Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:metadata": { "f:finalizers": { ".": {}, "v:\"velero.io/data-upload-download-finalizer\"": {} }, "f:labels": { "f:velero.io/accepted-by": {} } }, "f:status": { "f:node": {}, "f:phase": {}, "f:progress": { "f:totalBytes": {} }, "f:startTimestamp": {} } } } ] }, "spec": { "snapshotType": "CSI", "csiSnapshot": { "volumeSnapshot": "velero-test-vm-dv-2qphs", "storageClass": "odf-operator-cephfs", "snapshotClass": "odf-operator-cephfsplugin-snapclass" }, "sourcePVC": "test-vm-dv", "backupStorageLocation": "ts-dpa-1", "sourceNamespace": "test-oadp-401", "operationTimeout": "10m0s" }, "status": { "phase": "InProgress", "startTimestamp": "2025-01-06T07:51:30Z", "progress": { "totalBytes": 5073010688 }, "node": "ip-10-0-21-92.us-east-2.compute.internal" } } backup phase: WaitingForPluginOperations DataUpload ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-tbmzb phase: InProgress DataUpload Name: ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-tbmzb and status: InProgress 2025/01/06 07:52:02 { "kind": "DataUpload", "apiVersion": "velero.io/v2alpha1", "metadata": { "name": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-tbmzb", "generateName": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-", "namespace": "openshift-adp", "uid": "5dcd1ade-5612-4a5d-b1a5-1e284b61f64d", "resourceVersion": "76856", "generation": 7, "creationTimestamp": "2025-01-06T07:50:10Z", "labels": { "velero.io/accepted-by": "ip-10-0-21-92.us-east-2.compute.internal", "velero.io/async-operation-id": "du-d173214e-4527-430f-9886-c2b4f5c869d0.ce023406-c2eb-43f4a49b2", "velero.io/backup-name": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847", "velero.io/backup-uid": "d173214e-4527-430f-9886-c2b4f5c869d0", "velero.io/pvc-uid": "ce023406-c2eb-43f7-9b8d-bede1f7bd7bb" }, "ownerReferences": [ { "apiVersion": "velero.io/v1", "kind": "Backup", "name": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847", "uid": "d173214e-4527-430f-9886-c2b4f5c869d0", "controller": true } ], "finalizers": [ "velero.io/data-upload-download-finalizer" ], "managedFields": [ { "manager": "velero", "operation": "Update", "apiVersion": "velero.io/v2alpha1", "time": "2025-01-06T07:50:10Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:metadata": { "f:generateName": {}, "f:labels": { ".": {}, "f:velero.io/async-operation-id": {}, "f:velero.io/backup-name": {}, "f:velero.io/backup-uid": {}, "f:velero.io/pvc-uid": {} }, "f:ownerReferences": { ".": {}, "k:{\"uid\":\"d173214e-4527-430f-9886-c2b4f5c869d0\"}": {} } }, "f:spec": { ".": {}, "f:backupStorageLocation": {}, "f:csiSnapshot": { ".": {}, "f:snapshotClass": {}, "f:storageClass": {}, "f:volumeSnapshot": {} }, "f:operationTimeout": {}, "f:snapshotType": {}, "f:sourceNamespace": {}, "f:sourcePVC": {} }, "f:status": { ".": {}, "f:progress": {} } } }, { "manager": "node-agent-server", "operation": "Update", "apiVersion": "velero.io/v2alpha1", "time": "2025-01-06T07:51:53Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:metadata": { "f:finalizers": { ".": {}, "v:\"velero.io/data-upload-download-finalizer\"": {} }, "f:labels": { "f:velero.io/accepted-by": {} } }, "f:status": { "f:node": {}, "f:phase": {}, "f:progress": { "f:bytesDone": {}, "f:totalBytes": {} }, "f:startTimestamp": {} } } } ] }, "spec": { "snapshotType": "CSI", "csiSnapshot": { "volumeSnapshot": "velero-test-vm-dv-2qphs", "storageClass": "odf-operator-cephfs", "snapshotClass": "odf-operator-cephfsplugin-snapclass" }, "sourcePVC": "test-vm-dv", "backupStorageLocation": "ts-dpa-1", "sourceNamespace": "test-oadp-401", "operationTimeout": "10m0s" }, "status": { "phase": "InProgress", "startTimestamp": "2025-01-06T07:51:30Z", "progress": { "totalBytes": 5073010688, "bytesDone": 3005284352 }, "node": "ip-10-0-21-92.us-east-2.compute.internal" } } backup phase: Completed STEP: Verify backup ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847 has completed successfully @ 01/06/25 07:52:22.233 2025/01/06 07:52:22 Backup for case ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847 succeeded STEP: Delete the appplication resources ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847 @ 01/06/25 07:52:22.236 STEP: Cleanup Application for case ocp-kubevirt @ 01/06/25 07:52:22.236 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Remove namespace test-oadp-401] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=11  changed=4  unreachable=0 failed=0 skipped=8  rescued=0 ignored=0 2025/01/06 07:52:41 2025-01-06 07:52:23,645 p=28080 u=1003440000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2025-01-06 07:52:23,645 p=28080 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:52:23,899 p=28080 u=1003440000 n=ansible INFO| TASK [Get cluster endpoint] **************************************************** 2025-01-06 07:52:23,900 p=28080 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:52:24,153 p=28080 u=1003440000 n=ansible INFO| TASK [Get current admin token] ************************************************* 2025-01-06 07:52:24,154 p=28080 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:52:24,168 p=28080 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:52:24,168 p=28080 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:52:24,463 p=28080 u=1003440000 n=ansible INFO| TASK [Extract kubernetes minor version from cluster_info] ********************** 2025-01-06 07:52:24,463 p=28080 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:52:24,491 p=28080 u=1003440000 n=ansible INFO| TASK [Map kubernetes minor to ocp releases] ************************************ 2025-01-06 07:52:24,491 p=28080 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:52:24,508 p=28080 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:52:24,508 p=28080 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:52:24,509 p=28080 u=1003440000 n=ansible INFO| PLAY [Execute Task] ************************************************************ 2025-01-06 07:52:25,074 p=28080 u=1003440000 n=ansible INFO| TASK [Gathering Facts] ********************************************************* 2025-01-06 07:52:25,075 p=28080 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:52:40,877 p=28080 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Remove namespace test-oadp-401] *** 2025-01-06 07:52:40,878 p=28080 u=1003440000 n=ansible WARNING| [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2025-01-06 07:52:40,878 p=28080 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:52:41,018 p=28080 u=1003440000 n=ansible INFO| PLAY RECAP ********************************************************************* 2025-01-06 07:52:41,018 p=28080 u=1003440000 n=ansible INFO| localhost : ok=11 changed=4 unreachable=0 failed=0 skipped=8 rescued=0 ignored=0 STEP: Create restore ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847 from backup ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847 @ 01/06/25 07:52:41.058 2025/01/06 07:52:41 Wait until restore ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847 completes restore phase: WaitingForPluginOperations DataDownload ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-f57sd phase: InProgress DataDownload Name: ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-f57sd and status: InProgress 2025/01/06 07:53:01 { "kind": "DataDownload", "apiVersion": "velero.io/v2alpha1", "metadata": { "name": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-f57sd", "generateName": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-", "namespace": "openshift-adp", "uid": "4659299b-8aac-4ec0-9d5d-418f6d7e76a2", "resourceVersion": "77892", "generation": 4, "creationTimestamp": "2025-01-06T07:52:43Z", "labels": { "velero.io/accepted-by": "ip-10-0-106-2.us-east-2.compute.internal", "velero.io/async-operation-id": "dd-d558d344-8147-4b37-b9c6-26bac80a89b9.ce023406-c2eb-43fec6e4f", "velero.io/restore-name": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847", "velero.io/restore-uid": "d558d344-8147-4b37-b9c6-26bac80a89b9" }, "ownerReferences": [ { "apiVersion": "velero.io/v1", "kind": "Restore", "name": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847", "uid": "d558d344-8147-4b37-b9c6-26bac80a89b9", "controller": true } ], "finalizers": [ "velero.io/data-upload-download-finalizer" ], "managedFields": [ { "manager": "velero", "operation": "Update", "apiVersion": "velero.io/v2alpha1", "time": "2025-01-06T07:52:43Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:metadata": { "f:generateName": {}, "f:labels": { ".": {}, "f:velero.io/async-operation-id": {}, "f:velero.io/restore-name": {}, "f:velero.io/restore-uid": {} }, "f:ownerReferences": { ".": {}, "k:{\"uid\":\"d558d344-8147-4b37-b9c6-26bac80a89b9\"}": {} } }, "f:spec": { ".": {}, "f:backupStorageLocation": {}, "f:operationTimeout": {}, "f:snapshotID": {}, "f:sourceNamespace": {}, "f:targetVolume": { ".": {}, "f:namespace": {}, "f:pv": {}, "f:pvc": {} } }, "f:status": { ".": {}, "f:progress": {} } } }, { "manager": "node-agent-server", "operation": "Update", "apiVersion": "velero.io/v2alpha1", "time": "2025-01-06T07:52:51Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:metadata": { "f:finalizers": { ".": {}, "v:\"velero.io/data-upload-download-finalizer\"": {} }, "f:labels": { "f:velero.io/accepted-by": {} } }, "f:status": { "f:node": {}, "f:phase": {}, "f:startTimestamp": {} } } } ] }, "spec": { "targetVolume": { "pvc": "test-vm-dv", "pv": "", "namespace": "test-oadp-401" }, "backupStorageLocation": "ts-dpa-1", "snapshotID": "02fee8d15c8dfe74fbcf9b44906b0f0c", "sourceNamespace": "test-oadp-401", "operationTimeout": "10m0s" }, "status": { "phase": "InProgress", "startTimestamp": "2025-01-06T07:52:51Z", "progress": {}, "node": "ip-10-0-21-92.us-east-2.compute.internal" } } restore phase: WaitingForPluginOperations DataDownload ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-f57sd phase: InProgress DataDownload Name: ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-f57sd and status: InProgress 2025/01/06 07:53:21 { "kind": "DataDownload", "apiVersion": "velero.io/v2alpha1", "metadata": { "name": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-f57sd", "generateName": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-", "namespace": "openshift-adp", "uid": "4659299b-8aac-4ec0-9d5d-418f6d7e76a2", "resourceVersion": "77892", "generation": 4, "creationTimestamp": "2025-01-06T07:52:43Z", "labels": { "velero.io/accepted-by": "ip-10-0-106-2.us-east-2.compute.internal", "velero.io/async-operation-id": "dd-d558d344-8147-4b37-b9c6-26bac80a89b9.ce023406-c2eb-43fec6e4f", "velero.io/restore-name": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847", "velero.io/restore-uid": "d558d344-8147-4b37-b9c6-26bac80a89b9" }, "ownerReferences": [ { "apiVersion": "velero.io/v1", "kind": "Restore", "name": "ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847", "uid": "d558d344-8147-4b37-b9c6-26bac80a89b9", "controller": true } ], "finalizers": [ "velero.io/data-upload-download-finalizer" ], "managedFields": [ { "manager": "velero", "operation": "Update", "apiVersion": "velero.io/v2alpha1", "time": "2025-01-06T07:52:43Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:metadata": { "f:generateName": {}, "f:labels": { ".": {}, "f:velero.io/async-operation-id": {}, "f:velero.io/restore-name": {}, "f:velero.io/restore-uid": {} }, "f:ownerReferences": { ".": {}, "k:{\"uid\":\"d558d344-8147-4b37-b9c6-26bac80a89b9\"}": {} } }, "f:spec": { ".": {}, "f:backupStorageLocation": {}, "f:operationTimeout": {}, "f:snapshotID": {}, "f:sourceNamespace": {}, "f:targetVolume": { ".": {}, "f:namespace": {}, "f:pv": {}, "f:pvc": {} } }, "f:status": { ".": {}, "f:progress": {} } } }, { "manager": "node-agent-server", "operation": "Update", "apiVersion": "velero.io/v2alpha1", "time": "2025-01-06T07:52:51Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:metadata": { "f:finalizers": { ".": {}, "v:\"velero.io/data-upload-download-finalizer\"": {} }, "f:labels": { "f:velero.io/accepted-by": {} } }, "f:status": { "f:node": {}, "f:phase": {}, "f:startTimestamp": {} } } } ] }, "spec": { "targetVolume": { "pvc": "test-vm-dv", "pv": "", "namespace": "test-oadp-401" }, "backupStorageLocation": "ts-dpa-1", "snapshotID": "02fee8d15c8dfe74fbcf9b44906b0f0c", "sourceNamespace": "test-oadp-401", "operationTimeout": "10m0s" }, "status": { "phase": "InProgress", "startTimestamp": "2025-01-06T07:52:51Z", "progress": {}, "node": "ip-10-0-21-92.us-east-2.compute.internal" } } restore phase: WaitingForPluginOperations DataDownload ocp-kubevirt-983e5412-cc02-11ef-8010-0a580a83a847-f57sd phase: Completed restore phase: Completed STEP: Validate the application after restore @ 01/06/25 07:54:01.131 STEP: Verify Application deployment for case ocp-kubevirt @ 01/06/25 07:54:01.131 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (60 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (59 retries left). FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (58 retries left). TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Wait for VM to be Running & Ready] *** ok: [localhost] FAILED - RETRYING: [localhost]: Wait for VM to have AgentConnected status True indicating the guest agent is running (60 retries left). FAILED - RETRYING: [localhost]: Wait for VM to have AgentConnected status True indicating the guest agent is running (59 retries left). TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Wait for VM to have AgentConnected status True indicating the guest agent is running] *** ok: [localhost] PLAY RECAP ********************************************************************* localhost : ok=12  changed=3  unreachable=0 failed=0 skipped=7  rescued=0 ignored=0 2025/01/06 07:54:33 2025-01-06 07:54:02,525 p=28259 u=1003440000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2025-01-06 07:54:02,525 p=28259 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:54:02,781 p=28259 u=1003440000 n=ansible INFO| TASK [Get cluster endpoint] **************************************************** 2025-01-06 07:54:02,781 p=28259 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:54:03,034 p=28259 u=1003440000 n=ansible INFO| TASK [Get current admin token] ************************************************* 2025-01-06 07:54:03,034 p=28259 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:54:03,048 p=28259 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:54:03,048 p=28259 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:54:03,345 p=28259 u=1003440000 n=ansible INFO| TASK [Extract kubernetes minor version from cluster_info] ********************** 2025-01-06 07:54:03,346 p=28259 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:54:03,373 p=28259 u=1003440000 n=ansible INFO| TASK [Map kubernetes minor to ocp releases] ************************************ 2025-01-06 07:54:03,373 p=28259 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:54:03,389 p=28259 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:54:03,389 p=28259 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:54:03,391 p=28259 u=1003440000 n=ansible INFO| PLAY [Execute Task] ************************************************************ 2025-01-06 07:54:03,948 p=28259 u=1003440000 n=ansible INFO| TASK [Gathering Facts] ********************************************************* 2025-01-06 07:54:03,948 p=28259 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:54:04,868 p=28259 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (60 retries left). 2025-01-06 07:54:10,474 p=28259 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (59 retries left). 2025-01-06 07:54:16,065 p=28259 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to be Running & Ready (58 retries left). 2025-01-06 07:54:21,668 p=28259 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Wait for VM to be Running & Ready] *** 2025-01-06 07:54:21,668 p=28259 u=1003440000 n=ansible WARNING| [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2025-01-06 07:54:21,669 p=28259 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:54:22,296 p=28259 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to have AgentConnected status True indicating the guest agent is running (60 retries left). 2025-01-06 07:54:27,886 p=28259 u=1003440000 n=ansible INFO| FAILED - RETRYING: [localhost]: Wait for VM to have AgentConnected status True indicating the guest agent is running (59 retries left). 2025-01-06 07:54:33,510 p=28259 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Wait for VM to have AgentConnected status True indicating the guest agent is running] *** 2025-01-06 07:54:33,510 p=28259 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:54:33,513 p=28259 u=1003440000 n=ansible INFO| PLAY RECAP ********************************************************************* 2025-01-06 07:54:33,514 p=28259 u=1003440000 n=ansible INFO| localhost : ok=12 changed=3 unreachable=0 failed=0 skipped=7 rescued=0 ignored=0 2025/01/06 07:54:33 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 2025/01/06 07:54:33 Cleaning app [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Remove namespace test-oadp-401] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=11  changed=4  unreachable=0 failed=0 skipped=8  rescued=0 ignored=0 2025/01/06 07:54:52 2025-01-06 07:54:34,950 p=28516 u=1003440000 n=ansible INFO| TASK [Remove all the contents from the file] *********************************** 2025-01-06 07:54:34,950 p=28516 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:54:35,207 p=28516 u=1003440000 n=ansible INFO| TASK [Get cluster endpoint] **************************************************** 2025-01-06 07:54:35,207 p=28516 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:54:35,462 p=28516 u=1003440000 n=ansible INFO| TASK [Get current admin token] ************************************************* 2025-01-06 07:54:35,463 p=28516 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:54:35,476 p=28516 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:54:35,476 p=28516 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:54:35,774 p=28516 u=1003440000 n=ansible INFO| TASK [Extract kubernetes minor version from cluster_info] ********************** 2025-01-06 07:54:35,774 p=28516 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:54:35,801 p=28516 u=1003440000 n=ansible INFO| TASK [Map kubernetes minor to ocp releases] ************************************ 2025-01-06 07:54:35,801 p=28516 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:54:35,818 p=28516 u=1003440000 n=ansible INFO| TASK [set_fact] **************************************************************** 2025-01-06 07:54:35,818 p=28516 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:54:35,819 p=28516 u=1003440000 n=ansible INFO| PLAY [Execute Task] ************************************************************ 2025-01-06 07:54:36,371 p=28516 u=1003440000 n=ansible INFO| TASK [Gathering Facts] ********************************************************* 2025-01-06 07:54:36,371 p=28516 u=1003440000 n=ansible INFO| ok: [localhost] 2025-01-06 07:54:52,170 p=28516 u=1003440000 n=ansible INFO| TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-kubevirt : Remove namespace test-oadp-401] *** 2025-01-06 07:54:52,170 p=28516 u=1003440000 n=ansible WARNING| [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2025-01-06 07:54:52,170 p=28516 u=1003440000 n=ansible INFO| changed: [localhost] 2025-01-06 07:54:52,308 p=28516 u=1003440000 n=ansible INFO| PLAY RECAP ********************************************************************* 2025-01-06 07:54:52,308 p=28516 u=1003440000 n=ansible INFO| localhost : ok=11 changed=4 unreachable=0 failed=0 skipped=8 rescued=0 ignored=0 2025/01/06 07:54:52 Cleaning setup resources for the backup 2025/01/06 07:54:52 Setting new default StorageClass 'odf-operator-ceph-rbd' 2025/01/06 07:54:52 Checking default storage class count Skipping creation of StorageClass The current StorageClass: odf-operator-ceph-rbd matches the new StorageClass: odf-operator-ceph-rbd 2025/01/06 07:54:52 Deleting VolumeSnapshotClass 'example-snapclass' • [393.582 seconds] ------------------------------ [AfterSuite]  /alabama/cspi/e2e/kubevirt-plugin/kubevirt_suite_test.go:105 [AfterSuite] PASSED [0.006 seconds] ------------------------------ [ReportAfterSuite] Autogenerated ReportAfterSuite for --junit-report autogenerated by Ginkgo [ReportAfterSuite] PASSED [0.002 seconds] ------------------------------ Summarizing 1 Failure: [FAIL] CSI: Backup/Restore Openshift Virtualization Workloads  [It] [tc-id:OADP-187] [kubevirt] Backup-restore data volume /alabama/cspi/lib/backup.go:331 Ran 4 of 5 Specs in 884.232 seconds FAIL! -- 3 Passed | 1 Failed | 0 Pending | 1 Skipped --- FAIL: TestOADPE2E (884.24s) FAIL Ginkgo ran 1 suite in 16m8.159353886s Test Suite Failed + readonly 'RED=\e[31m' + RED='\e[31m' + readonly 'BLUE=\033[34m' + BLUE='\033[34m' + readonly 'CLEAR=\e[39m' + CLEAR='\e[39m' ++ oc get infrastructures cluster -o 'jsonpath={.status.platform}' ++ awk '{print tolower($0)}' + CLOUD_PROVIDER=aws + [[ '' == \t\r\u\e ]] + echo /home/jenkins/.kube/config /home/jenkins/.kube/config + [[ aws == *-arm* ]] + [[ aws == *-fips* ]] + E2E_TIMEOUT_MULTIPLIER=2 + export NAMESPACE=openshift-adp + NAMESPACE=openshift-adp + export PROVIDER=aws + PROVIDER=aws ++ echo aws ++ awk '{print tolower($0)}' + BACKUP_LOCATION=aws + export BACKUP_LOCATION=aws + BACKUP_LOCATION=aws + export BUCKET=ci-op-bkpv69pr-interopoadp + BUCKET=ci-op-bkpv69pr-interopoadp + OADP_CREDS_FILE=/tmp/test-settings/credentials + OADP_VSL_CREDS_FILE=/tmp/test-settings/aws_vsl_creds +++ readlink -f /alabama/cspi/test_settings/scripts/test_runner.sh ++ dirname /alabama/cspi/test_settings/scripts/test_runner.sh + readonly SCRIPT_DIR=/alabama/cspi/test_settings/scripts + SCRIPT_DIR=/alabama/cspi/test_settings/scripts ++ cd /alabama/cspi/test_settings/scripts ++ git rev-parse --show-toplevel + readonly TOP_DIR=/alabama/cspi + TOP_DIR=/alabama/cspi + echo /alabama/cspi /alabama/cspi + TESTS_FOLDER=/alabama/cspi/e2e/kubevirt-plugin ++ oc get nodes -o 'jsonpath={.items[*].metadata.labels.kubernetes\.io/arch}' ++ tr ' ' '\n' ++ sort -u ++ xargs + export NODES_ARCHITECTURE=amd64 + NODES_ARCHITECTURE=amd64 + export OADP_REPOSITORY=redhat + OADP_REPOSITORY=redhat + SKIP_DPA_CREATION=false ++ oc get ns openshift-storage ++ echo true + OPENSHIFT_STORAGE=true + '[' redhat == upstream-velero ']' + '[' true == true ']' ++ oc get sc ++ awk '$1 ~ /^.+ceph-rbd$/ {print $1}' ++ tail -1 + CEPH_RBD_STORAGE_CLASS=odf-operator-ceph-rbd + '[' -n odf-operator-ceph-rbd ']' + export CEPH_RBD_STORAGE_CLASS + echo 'ceph-rbd StorageClass found: odf-operator-ceph-rbd' ceph-rbd StorageClass found: odf-operator-ceph-rbd ++ oc get storageclass -o 'jsonpath={range .items[*]}{@.metadata.name}{" "}{@.metadata.annotations.storageclass\.kubernetes\.io/is-default-class}{"\n"}{end}' ++ awk '$2=="true"{print $1}' ++ wc -l + NUM_DEFAULT_STORAGE_CLASS=1 + '[' 1 -ne 1 ']' ++ oc get storageclass -o 'jsonpath={.items[?(@.metadata.annotations.storageclass\.kubernetes\.io/is-default-class=='\''true'\'')].metadata.name}' + DEFAULT_SC=odf-operator-ceph-rbd + export STORAGE_CLASS=odf-operator-ceph-rbd + STORAGE_CLASS=odf-operator-ceph-rbd + '[' -n odf-operator-ceph-rbd ']' + '[' odf-operator-ceph-rbd '!=' odf-operator-ceph-rbd ']' + export STORAGE_CLASS_OPENSHIFT_STORAGE=odf-operator-ceph-rbd + STORAGE_CLASS_OPENSHIFT_STORAGE=odf-operator-ceph-rbd + echo 'Using the StorageClass for openshift-storage: odf-operator-ceph-rbd' Using the StorageClass for openshift-storage: odf-operator-ceph-rbd + [[ amd64 != \a\m\d\6\4 ]] + TEST_FILTER='!// || (// && !exclude_aws && (!/target/ || target_aws) ) ' + [[ aws =~ ^osp ]] + [[ aws =~ ^vsphere ]] + [[ aws =~ ^gcp-wif ]] + [[ aws =~ ^ibmcloud ]] ++ oc config current-context ++ awk -F / '{print $2}' + SETTINGS_TMP=/alabama/cspi/output_files/api-ci-op-bkpv69pr-78a3d-cspilp-interop-ccitredhat-com:6443 + mkdir -p /alabama/cspi/output_files/api-ci-op-bkpv69pr-78a3d-cspilp-interop-ccitredhat-com:6443 ++ oc get authentication cluster -o 'jsonpath={.spec.serviceAccountIssuer}' + IS_OIDC= + '[' '!' -z ']' + [[ aws == \a\w\s ]] + export PROVIDER=aws + PROVIDER=aws + export CREDS_SECRET_REF=cloud-credentials + CREDS_SECRET_REF=cloud-credentials ++ oc get infrastructures cluster -o 'jsonpath={.status.platformStatus.aws.region}' --allow-missing-template-keys=false + export REGION=us-east-2 + REGION=us-east-2 + settings_script=aws_settings.sh + '[' aws == aws-sts ']' + BUCKET=ci-op-bkpv69pr-interopoadp + TMP_DIR=/alabama/cspi/output_files/api-ci-op-bkpv69pr-78a3d-cspilp-interop-ccitredhat-com:6443 + source /alabama/cspi/test_settings/scripts/aws_settings.sh ++ cat ++ [[ aws == *aws* ]] ++ cat ++ echo -e '\n }\n}' +++ cat /alabama/cspi/output_files/api-ci-op-bkpv69pr-78a3d-cspilp-interop-ccitredhat-com:6443/settings.json ++ x='{ "metadata": { "namespace": "openshift-adp" }, "spec": { "configuration":{ "velero":{ "defaultPlugins": [ "openshift", "aws" ] } }, "backupLocations": [ { "velero": { "provider": "aws", "default": true, "config": { "region": "us-east-2" }, "credential":{ "name": "cloud-credentials", "key": "cloud" }, "objectStorage":{ "bucket": "ci-op-bkpv69pr-interopoadp" } } } ] , "snapshotLocations": [ { "velero": { "provider": "aws", "config": { "profile": "default", "region": "us-east-2" } } } ] } }' ++ echo '{ "metadata": { "namespace": "openshift-adp" }, "spec": { "configuration":{ "velero":{ "defaultPlugins": [ "openshift", "aws" ] } }, "backupLocations": [ { "velero": { "provider": "aws", "default": true, "config": { "region": "us-east-2" }, "credential":{ "name": "cloud-credentials", "key": "cloud" }, "objectStorage":{ "bucket": "ci-op-bkpv69pr-interopoadp" } } } ] , "snapshotLocations": [ { "velero": { "provider": "aws", "config": { "profile": "default", "region": "us-east-2" } } } ] } }' ++ grep -o '^[^#]*' + FILE_SETTINGS_NAME=settings.json + printf '\033[34mGenerated settings file under /alabama/cspi/output_files/api-ci-op-bkpv69pr-78a3d-cspilp-interop-ccitredhat-com:6443/settings.json\e[39m\n' Generated settings file under /alabama/cspi/output_files/api-ci-op-bkpv69pr-78a3d-cspilp-interop-ccitredhat-com:6443/settings.json + cat /alabama/cspi/output_files/api-ci-op-bkpv69pr-78a3d-cspilp-interop-ccitredhat-com:6443/settings.json ++ oc get volumesnapshotclass -o name + for i in $(oc get volumesnapshotclass -o name) + oc annotate volumesnapshotclass.snapshot.storage.k8s.io/csi-aws-vsc snapshot.storage.kubernetes.io/is-default-class- volumesnapshotclass.snapshot.storage.k8s.io/csi-aws-vsc annotated + for i in $(oc get volumesnapshotclass -o name) + oc annotate volumesnapshotclass.snapshot.storage.k8s.io/odf-operator-cephfsplugin-snapclass snapshot.storage.kubernetes.io/is-default-class- volumesnapshotclass.snapshot.storage.k8s.io/odf-operator-cephfsplugin-snapclass annotated + for i in $(oc get volumesnapshotclass -o name) + oc annotate volumesnapshotclass.snapshot.storage.k8s.io/odf-operator-rbdplugin-snapclass snapshot.storage.kubernetes.io/is-default-class- volumesnapshotclass.snapshot.storage.k8s.io/odf-operator-rbdplugin-snapclass annotated ++ ./e2e/must-gather/get-latest-build.sh + oc get configmaps -n default must-gather-image + UPSTREAM_VERSION=99.0.0 ++ oc get OperatorCondition -n openshift-adp -o 'jsonpath={.items[*].metadata.name}' ++ awk -F v '{print $2}' + OADP_VERSION=1.4.1 + '[' -z 1.4.1 ']' + '[' 1.4.1 == 99.0.0 ']' ++ oc get sub redhat-oadp-operator -n openshift-adp -o 'jsonpath={.spec.source}' + OADP_REPO=redhat-operators + '[' -z redhat-operators ']' + '[' redhat-operators == redhat-operators ']' + REGISTRY_PATH=registry.redhat.io/oadp/oadp-mustgather-rhel9: + TAG=1.4.1 + export MUST_GATHER_BUILD=registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 + MUST_GATHER_BUILD=registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 + echo registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 + export MUST_GATHER_BUILD=registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 + MUST_GATHER_BUILD=registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 + '[' -z registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 ']' + export NUM_OF_OADP_INSTANCES=1 + NUM_OF_OADP_INSTANCES=1 + ginkgo run -p --nodes=1 -mod=mod /alabama/cspi/e2e/kubevirt-plugin/ --show-node-events -- -credentials_file=/tmp/test-settings/credentials -vsl_credentials_file=/tmp/test-settings/aws_vsl_creds -oadp_namespace=openshift-adp -settings=/alabama/cspi/output_files/api-ci-op-bkpv69pr-78a3d-cspilp-interop-ccitredhat-com:6443/settings.json -must_gather_image=registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 -timeout_multiplier=2 -skip_dpa_creation=false --ginkgo.junit-report=/logs/artifacts/junit_oadp_interop_results.xml '--ginkgo.label-filter=!// || (// && !exclude_aws && (!/target/ || target_aws) ) ' --ginkgo.focus=interop Ginkgo detected a version mismatch between the Ginkgo CLI and the version of Ginkgo imported by your packages: Ginkgo CLI Version: 2.22.2 Mismatched package versions found: 2.22.0 used by kubevirt-plugin Ginkgo will continue to attempt to run but you may see errors (including flag parsing errors) and should either update your go.mod or your version of the Ginkgo CLI to match. To install the matching version of the CLI run go install github.com/onsi/ginkgo/v2/ginkgo from a path that contains a go.mod file. Alternatively you can use go run github.com/onsi/ginkgo/v2/ginkgo from a path that contains a go.mod file to invoke the matching version of the Ginkgo CLI. If you are attempting to test multiple packages that each have a different version of the Ginkgo library with a single Ginkgo CLI that is currently unsupported.  2025/01/06 07:55:05 Setting up clients 2025/01/06 07:55:05 Getting default StorageClass... 2025/01/06 07:55:05 Checking default storage class count Run the command: oc get sc 2025/01/06 07:55:05 Got default StorageClass odf-operator-ceph-rbd 2025/01/06 07:55:05 oc get sc NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE gp2-csi ebs.csi.aws.com Delete WaitForFirstConsumer true 80m gp3-csi ebs.csi.aws.com Delete WaitForFirstConsumer true 80m odf-operator-ceph-rbd (default) openshift-storage.rbd.csi.ceph.com Delete Immediate true 19m odf-operator-ceph-rbd-virtualization openshift-storage.rbd.csi.ceph.com Delete Immediate true 19m odf-operator-cephfs openshift-storage.cephfs.csi.ceph.com Delete Immediate true 19m openshift-storage.noobaa.io openshift-storage.noobaa.io/obc Delete Immediate false 18m 2025/01/06 07:55:05 Using velero prefix: velero-e2e-kubevirt-8acbe3ac-cc03-11ef-8977-0a580a83a847 Running Suite: OADP E2E Virtualization Workloads Suite - /alabama/cspi/e2e/kubevirt-plugin ========================================================================================== Random Seed: 1736150094 Will run 0 of 5 specs SSSSS ------------------------------ [ReportAfterSuite] Autogenerated ReportAfterSuite for --junit-report autogenerated by Ginkgo [ReportAfterSuite] PASSED [0.000 seconds] ------------------------------ Ran 0 of 5 Specs in 0.000 seconds SUCCESS! -- 0 Passed | 0 Failed | 0 Pending | 5 Skipped PASS Ginkgo ran 1 suite in 11.589218085s Test Suite Passed [must-gather ] OUT 2025-01-06T07:55:35.825497721Z Using must-gather plug-in image: registry.redhat.io/oadp/oadp-mustgather-rhel9:v1.4 When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: d9768d32-db3b-4eff-9be9-12cb926a6b21 ClientVersion: 4.17.9 ClusterVersion: Stable at "4.18.0-0.nightly-2025-01-04-101226" ClusterOperators: clusteroperator/operator-lifecycle-manager is not upgradeable because ClusterServiceVersions blocking minor version upgrades to 4.17.0 or higher: - maximum supported OCP version for openshift-storage/odf-operator.v4.15.9-rhodf is 4.16 [must-gather ] OUT 2025-01-06T07:55:35.863488459Z namespace/openshift-must-gather-76tjf created [must-gather ] OUT 2025-01-06T07:55:35.879570179Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-nrs7w created Warning: spec.nodeSelector[node-role.kubernetes.io/master]: use "node-role.kubernetes.io/control-plane" instead [must-gather ] OUT 2025-01-06T07:55:35.945254294Z pod for plug-in image registry.redhat.io/oadp/oadp-mustgather-rhel9:v1.4 created [must-gather-88b56] POD 2025-01-06T07:55:38.874230285Z volume percentage checker started..... [must-gather-88b56] POD 2025-01-06T07:55:38.883076782Z volume usage percentage 0 [must-gather-88b56] POD 2025-01-06T07:55:39.292079473Z No resources found [must-gather-88b56] POD 2025-01-06T07:55:39.296973983Z Gathering Velero and OADP CRs [must-gather-88b56] POD 2025-01-06T07:55:39.297150693Z Gathering Velero and OADP CRDs [must-gather-88b56] POD 2025-01-06T07:55:39.297323122Z [cluster=d9768d32] Gathering logs for namespaces [] [must-gather-88b56] POD 2025-01-06T07:55:39.297581476Z [cluster=d9768d32] Gathering prometheus metrics [must-gather-88b56] POD 2025-01-06T07:55:39.297825560Z Gathering version information for top-level debug file [must-gather-88b56] POD 2025-01-06T07:55:39.298054122Z Waiting on subprocesses to finish execution. [must-gather-88b56] POD 2025-01-06T07:55:40.541856259Z Starting collection of: cloudstorages.oadp.openshift.io dataprotectionapplications.oadp.openshift.io backuprepositories.velero.io backups.velero.io backupstoragelocations.velero.io datadownloads.velero.io datauploads.velero.io deletebackuprequests.velero.io downloadrequests.velero.io podvolumebackups.velero.io podvolumerestores.velero.io restores.velero.io schedules.velero.io serverstatusrequests.velero.io volumesnapshotlocations.velero.io [must-gather-88b56] POD 2025-01-06T07:55:40.642488025Z Starting collection of CRDs: cloudstorages.oadp.openshift.io dataprotectionapplications.oadp.openshift.io backuprepositories.velero.io backups.velero.io backupstoragelocations.velero.io datadownloads.velero.io datauploads.velero.io deletebackuprequests.velero.io downloadrequests.velero.io podvolumebackups.velero.io podvolumerestores.velero.io restores.velero.io schedules.velero.io serverstatusrequests.velero.io volumesnapshotlocations.velero.io [must-gather-88b56] POD 2025-01-06T07:55:40.642682226Z Collecting cloudstorages.oadp.openshift.io CRD [must-gather-88b56] POD 2025-01-06T07:55:40.648154678Z Collecting dataprotectionapplications.oadp.openshift.io CRD [must-gather-88b56] POD 2025-01-06T07:55:40.649949467Z Collecting backuprepositories.velero.io CRD [must-gather-88b56] POD 2025-01-06T07:55:40.658603634Z Collecting backups.velero.io CRD [must-gather-88b56] POD 2025-01-06T07:55:40.687145246Z Collecting backupstoragelocations.velero.io CRD [must-gather-88b56] POD 2025-01-06T07:55:40.713623506Z Collecting datadownloads.velero.io CRD [must-gather-88b56] POD 2025-01-06T07:55:40.730328157Z Collecting datauploads.velero.io CRD [must-gather-88b56] POD 2025-01-06T07:55:40.781436273Z Collecting deletebackuprequests.velero.io CRD [must-gather-88b56] POD 2025-01-06T07:55:40.815929244Z Collecting downloadrequests.velero.io CRD [must-gather-88b56] POD 2025-01-06T07:55:40.875957052Z Collecting podvolumebackups.velero.io CRD [must-gather-88b56] POD 2025-01-06T07:55:41.010159028Z Collecting podvolumerestores.velero.io CRD [must-gather-88b56] POD 2025-01-06T07:55:41.083826118Z Collecting restores.velero.io CRD [must-gather-88b56] POD 2025-01-06T07:55:41.100382331Z Collecting schedules.velero.io CRD [must-gather-88b56] POD 2025-01-06T07:55:41.110852437Z Collecting serverstatusrequests.velero.io CRD [must-gather-88b56] POD 2025-01-06T07:55:41.120759043Z Collecting volumesnapshotlocations.velero.io CRD [must-gather-88b56] POD 2025-01-06T07:55:41.280172629Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather-88b56] POD 2025-01-06T07:55:41.368858797Z Warning: kubevirt.io/v1 VirtualMachineInstancePresets is now deprecated and will be removed in v2. [must-gather-88b56] POD 2025-01-06T07:55:41.493425072Z [cluster=host] Dumping Prometheus metadata ... [must-gather-88b56] POD 2025-01-06T07:55:42.957824086Z [cluster=host] Capturing prometheus metric [velero_backup_total] [must-gather-88b56] POD 2025-01-06T07:55:42.973090878Z Collecting backuprepositories in openshift-adp namespace [must-gather-88b56] POD 2025-01-06T07:55:43.304236234Z Collecting backups in openshift-adp namespace [must-gather-88b56] POD 2025-01-06T07:55:43.408761966Z [cluster=host] Building chart for prometheus metric [velero_backup_total] [must-gather-88b56] POD 2025-01-06T07:55:43.419206891Z [cluster=host] Capturing prometheus metric [velero_restore_total] [must-gather-88b56] POD 2025-01-06T07:55:43.860901318Z [cluster=host] Building chart for prometheus metric [velero_restore_total] [must-gather-88b56] POD 2025-01-06T07:55:43.867629548Z Collecting datadownloads in openshift-adp namespace [must-gather-88b56] POD 2025-01-06T07:55:43.897000244Z volume usage percentage 0 [must-gather-88b56] POD 2025-01-06T07:55:44.120772908Z Collecting datauploads in openshift-adp namespace [must-gather-88b56] POD 2025-01-06T07:55:45.087852087Z Collecting restores in openshift-adp namespace [must-gather-88b56] POD 2025-01-06T07:55:45.653591640Z Full must-gather was requested. Keeping full log payload from oc adm inspect [must-gather-88b56] POD 2025-01-06T07:55:45.653591640Z Tarring must-gather artifacts... [must-gather-88b56] POD 2025-01-06T07:55:45.656342491Z tar: Removing leading `/' from member names [must-gather-88b56] POD 2025-01-06T07:55:45.788869174Z Created /must-gather/must-gather.tar.gz [must-gather-88b56] POD 2025-01-06T07:55:45.788869174Z Waiting for copy phase... [must-gather-88b56] POD 2025-01-06T07:55:44.120772908Z Collecting datauploads in openshift-adp namespace [must-gather-88b56] POD 2025-01-06T07:55:45.087852087Z Collecting restores in openshift-adp namespace [must-gather-88b56] POD 2025-01-06T07:55:45.653591640Z Full must-gather was requested. Keeping full log payload from oc adm inspect [must-gather-88b56] POD 2025-01-06T07:55:45.653591640Z Tarring must-gather artifacts... [must-gather-88b56] POD 2025-01-06T07:55:45.656342491Z tar: Removing leading `/' from member names [must-gather-88b56] POD 2025-01-06T07:55:45.788869174Z Created /must-gather/must-gather.tar.gz [must-gather-88b56] POD 2025-01-06T07:55:45.788869174Z Waiting for copy phase... [must-gather-88b56] POD 2025-01-06T07:55:44.120772908Z Collecting datauploads in openshift-adp namespace [must-gather-88b56] POD 2025-01-06T07:55:45.087852087Z Collecting restores in openshift-adp namespace [must-gather-88b56] POD 2025-01-06T07:55:45.653591640Z Full must-gather was requested. Keeping full log payload from oc adm inspect [must-gather-88b56] POD 2025-01-06T07:55:45.653591640Z Tarring must-gather artifacts... [must-gather-88b56] POD 2025-01-06T07:55:45.656342491Z tar: Removing leading `/' from member names [must-gather-88b56] POD 2025-01-06T07:55:45.788869174Z Created /must-gather/must-gather.tar.gz [must-gather-88b56] POD 2025-01-06T07:55:45.788869174Z Waiting for copy phase... [must-gather-88b56] POD 2025-01-06T07:55:44.120772908Z Collecting datauploads in openshift-adp namespace [must-gather-88b56] POD 2025-01-06T07:55:45.087852087Z Collecting restores in openshift-adp namespace [must-gather-88b56] POD 2025-01-06T07:55:45.653591640Z Full must-gather was requested. Keeping full log payload from oc adm inspect [must-gather-88b56] POD 2025-01-06T07:55:45.653591640Z Tarring must-gather artifacts... [must-gather-88b56] POD 2025-01-06T07:55:45.656342491Z tar: Removing leading `/' from member names [must-gather-88b56] POD 2025-01-06T07:55:45.788869174Z Created /must-gather/must-gather.tar.gz [must-gather-88b56] POD 2025-01-06T07:55:45.788869174Z Waiting for copy phase... [must-gather-88b56] POD 2025-01-06T07:55:44.120772908Z Collecting datauploads in openshift-adp namespace [must-gather-88b56] POD 2025-01-06T07:55:45.087852087Z Collecting restores in openshift-adp namespace [must-gather-88b56] POD 2025-01-06T07:55:45.653591640Z Full must-gather was requested. Keeping full log payload from oc adm inspect [must-gather-88b56] POD 2025-01-06T07:55:45.653591640Z Tarring must-gather artifacts... [must-gather-88b56] POD 2025-01-06T07:55:45.656342491Z tar: Removing leading `/' from member names [must-gather-88b56] POD 2025-01-06T07:55:45.788869174Z Created /must-gather/must-gather.tar.gz [must-gather-88b56] POD 2025-01-06T07:55:45.788869174Z Waiting for copy phase... [must-gather-88b56] POD 2025-01-06T07:55:44.120772908Z Collecting datauploads in openshift-adp namespace [must-gather-88b56] POD 2025-01-06T07:55:45.087852087Z Collecting restores in openshift-adp namespace [must-gather-88b56] POD 2025-01-06T07:55:45.653591640Z Full must-gather was requested. Keeping full log payload from oc adm inspect [must-gather-88b56] POD 2025-01-06T07:55:45.653591640Z Tarring must-gather artifacts... [must-gather-88b56] POD 2025-01-06T07:55:45.656342491Z tar: Removing leading `/' from member names [must-gather-88b56] POD 2025-01-06T07:55:45.788869174Z Created /must-gather/must-gather.tar.gz [must-gather-88b56] POD 2025-01-06T07:55:45.788869174Z Waiting for copy phase... [must-gather-88b56] POD 2025-01-06T07:55:44.120772908Z Collecting datauploads in openshift-adp namespace [must-gather-88b56] POD 2025-01-06T07:55:45.087852087Z Collecting restores in openshift-adp namespace [must-gather-88b56] POD 2025-01-06T07:55:45.653591640Z Full must-gather was requested. Keeping full log payload from oc adm inspect [must-gather-88b56] POD 2025-01-06T07:55:45.653591640Z Tarring must-gather artifacts... [must-gather-88b56] POD 2025-01-06T07:55:45.656342491Z tar: Removing leading `/' from member names [must-gather-88b56] POD 2025-01-06T07:55:45.788869174Z Created /must-gather/must-gather.tar.gz [must-gather-88b56] POD 2025-01-06T07:55:45.788869174Z Waiting for copy phase... [must-gather-88b56] POD 2025-01-06T07:55:44.120772908Z Collecting datauploads in openshift-adp namespace [must-gather-88b56] POD 2025-01-06T07:55:45.087852087Z Collecting restores in openshift-adp namespace [must-gather-88b56] POD 2025-01-06T07:55:45.653591640Z Full must-gather was requested. Keeping full log payload from oc adm inspect [must-gather-88b56] POD 2025-01-06T07:55:45.653591640Z Tarring must-gather artifacts... [must-gather-88b56] POD 2025-01-06T07:55:45.656342491Z tar: Removing leading `/' from member names [must-gather-88b56] POD 2025-01-06T07:55:45.788869174Z Created /must-gather/must-gather.tar.gz [must-gather-88b56] POD 2025-01-06T07:55:45.788869174Z Waiting for copy phase... [must-gather-88b56] POD 2025-01-06T07:55:44.120772908Z Collecting datauploads in openshift-adp namespace [must-gather-88b56] POD 2025-01-06T07:55:45.087852087Z Collecting restores in openshift-adp namespace [must-gather-88b56] POD 2025-01-06T07:55:45.653591640Z Full must-gather was requested. Keeping full log payload from oc adm inspect [must-gather-88b56] POD 2025-01-06T07:55:45.653591640Z Tarring must-gather artifacts... [must-gather-88b56] POD 2025-01-06T07:55:45.656342491Z tar: Removing leading `/' from member names [must-gather-88b56] POD 2025-01-06T07:55:45.788869174Z Created /must-gather/must-gather.tar.gz [must-gather-88b56] POD 2025-01-06T07:55:45.788869174Z Waiting for copy phase... [must-gather-88b56] POD 2025-01-06T07:55:44.120772908Z Collecting datauploads in openshift-adp namespace [must-gather-88b56] POD 2025-01-06T07:55:45.087852087Z Collecting restores in openshift-adp namespace [must-gather-88b56] POD 2025-01-06T07:55:45.653591640Z Full must-gather was requested. Keeping full log payload from oc adm inspect [must-gather-88b56] POD 2025-01-06T07:55:45.653591640Z Tarring must-gather artifacts... [must-gather-88b56] POD 2025-01-06T07:55:45.656342491Z tar: Removing leading `/' from member names [must-gather-88b56] POD 2025-01-06T07:55:45.788869174Z Created /must-gather/must-gather.tar.gz [must-gather-88b56] POD 2025-01-06T07:55:45.788869174Z Waiting for copy phase... [must-gather-88b56] POD 2025-01-06T07:55:45.087852087Z Collecting restores in openshift-adp namespace [must-gather-88b56] POD 2025-01-06T07:55:45.653591640Z Full must-gather was requested. Keeping full log payload from oc adm inspect [must-gather-88b56] POD 2025-01-06T07:55:45.653591640Z Tarring must-gather artifacts... [must-gather-88b56] POD 2025-01-06T07:55:45.656342491Z tar: Removing leading `/' from member names [must-gather-88b56] POD 2025-01-06T07:55:45.788869174Z Created /must-gather/must-gather.tar.gz [must-gather-88b56] POD 2025-01-06T07:55:45.788869174Z Waiting for copy phase... [must-gather-88b56] OUT 2025-01-06T07:55:46.365922891Z waiting for gather to complete [must-gather-88b56] OUT 2025-01-06T07:55:46.565945621Z downloading gather output [must-gather-88b56] OUT 2025-01-06T07:55:47.188362127Z receiving incremental file list [must-gather-88b56] OUT 2025-01-06T07:55:47.192486145Z ./ [must-gather-88b56] OUT 2025-01-06T07:55:47.192726858Z must-gather.tar.gz [must-gather-88b56] OUT 2025-01-06T07:55:47.197673666Z [must-gather-88b56] OUT 2025-01-06T07:55:47.197705106Z sent 46 bytes received 608,536 bytes 405,721.33 bytes/sec [must-gather-88b56] OUT 2025-01-06T07:55:47.197715547Z total size is 608,276 speedup is 1.00 [must-gather ] OUT 2025-01-06T07:55:47.226254494Z namespace/openshift-must-gather-76tjf deleted Reprinting Cluster State: When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: d9768d32-db3b-4eff-9be9-12cb926a6b21 ClientVersion: 4.17.9 ClusterVersion: Stable at "4.18.0-0.nightly-2025-01-04-101226" ClusterOperators: clusteroperator/operator-lifecycle-manager is not upgradeable because ClusterServiceVersions blocking minor version upgrades to 4.17.0 or higher: - maximum supported OCP version for openshift-storage/odf-operator.v4.15.9-rhodf is 4.16 Checking for additional logs in /alabama/cspi/e2e/logs