Extract /home/jenkins/oadp-e2e-qe.tar.gz to /alabama/cspi Extract /home/jenkins/oadp-apps-deployer.tar.gz to /alabama/oadpApps Extract /home/jenkins/mtc-python-client.tar.gz to /alabama/pyclient Create and populate /tmp/test-settings... Login as Kubeadmin to the test cluster at https://api.ci-rosa-s-916u.wmvn.s1.devshift.org:6443... WARNING: Using insecure TLS client config. Setting this option is not supported! Login successful. You have access to 103 projects, the list has been suppressed. You can list all projects with 'oc projects' Using project "default". Create virtual environment and install required packages... Collecting ansible_runner Downloading ansible_runner-2.4.0-py3-none-any.whl (79 kB) Collecting python-daemon Downloading python_daemon-3.1.0-py3-none-any.whl (30 kB) Collecting packaging Downloading packaging-24.1-py3-none-any.whl (53 kB) Collecting pexpect>=4.5 Downloading pexpect-4.9.0-py2.py3-none-any.whl (63 kB) Collecting pyyaml Downloading PyYAML-6.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (751 kB) Collecting ptyprocess>=0.5 Downloading ptyprocess-0.7.0-py2.py3-none-any.whl (13 kB) Collecting lockfile>=0.10 Downloading lockfile-0.12.2-py2.py3-none-any.whl (13 kB) Collecting setuptools>=62.4.0 Downloading setuptools-75.2.0-py3-none-any.whl (1.2 MB) Installing collected packages: setuptools, ptyprocess, lockfile, pyyaml, python-daemon, pexpect, packaging, ansible-runner Attempting uninstall: setuptools Found existing installation: setuptools 57.4.0 Uninstalling setuptools-57.4.0: Successfully uninstalled setuptools-57.4.0 Successfully installed ansible-runner-2.4.0 lockfile-0.12.2 packaging-24.1 pexpect-4.9.0 ptyprocess-0.7.0 python-daemon-3.1.0 pyyaml-6.0.2 setuptools-75.2.0 WARNING: You are using pip version 21.2.3; however, version 24.2 is available. You should consider upgrading via the '/alabama/venv/bin/python3 -m pip install --upgrade pip' command. Processing /alabama/oadpApps DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default. pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555. Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' Preparing wheel metadata: started Preparing wheel metadata: finished with status 'done' Building wheels for collected packages: ocpdeployer Building wheel for ocpdeployer (PEP 517): started Building wheel for ocpdeployer (PEP 517): finished with status 'done' Created wheel for ocpdeployer: filename=ocpdeployer-0.0.1-py2.py3-none-any.whl size=224961 sha256=f019f4b0f0971a4682a39955a87ede11002270d02bc6611741c408c042eed94e Stored in directory: /tmp/pip-ephem-wheel-cache-ywnarr7k/wheels/ea/a6/87/9d98fc51cc395d30bd2147a7e53e3f5a2e80044d9dd9e64977 Successfully built ocpdeployer Installing collected packages: ocpdeployer WARNING: Value for scheme.platlib does not match. Please report this to distutils: /tmp/pip-target-naw9_hyr/lib64/python sysconfig: /tmp/pip-target-naw9_hyr/lib/python WARNING: Additional context: user = False home = '/tmp/pip-target-naw9_hyr' root = None prefix = None Successfully installed ocpdeployer-0.0.1 WARNING: You are using pip version 21.2.3; however, version 24.2 is available. You should consider upgrading via the '/alabama/venv/bin/python3 -m pip install --upgrade pip' command. Processing /alabama/pyclient DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default. pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555. Collecting suds-py3 Downloading suds_py3-1.4.5.0-py3-none-any.whl (298 kB) Collecting requests Downloading requests-2.32.3-py3-none-any.whl (64 kB) Collecting jinja2 Downloading jinja2-3.1.4-py3-none-any.whl (133 kB) Collecting kubernetes==11.0.0 Downloading kubernetes-11.0.0-py3-none-any.whl (1.5 MB) Collecting openshift==0.11.2 Downloading openshift-0.11.2.tar.gz (19 kB) Collecting python-dateutil>=2.5.3 Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) Collecting certifi>=14.05.14 Downloading certifi-2024.8.30-py3-none-any.whl (167 kB) Requirement already satisfied: setuptools>=21.0.0 in /alabama/venv/lib/python3.10/site-packages (from kubernetes==11.0.0->mtc==0.0.1) (75.2.0) Collecting urllib3>=1.24.2 Downloading urllib3-2.2.3-py3-none-any.whl (126 kB) Collecting google-auth>=1.0.1 Downloading google_auth-2.35.0-py2.py3-none-any.whl (208 kB) Requirement already satisfied: pyyaml>=3.12 in /alabama/venv/lib/python3.10/site-packages (from kubernetes==11.0.0->mtc==0.0.1) (6.0.2) Collecting six>=1.9.0 Downloading six-1.16.0-py2.py3-none-any.whl (11 kB) Collecting requests-oauthlib Downloading requests_oauthlib-2.0.0-py2.py3-none-any.whl (24 kB) Collecting websocket-client!=0.40.0,!=0.41.*,!=0.42.*,>=0.32.0 Downloading websocket_client-1.8.0-py3-none-any.whl (58 kB) Collecting python-string-utils Downloading python_string_utils-1.0.0-py3-none-any.whl (26 kB) Collecting ruamel.yaml>=0.15 Downloading ruamel.yaml-0.18.6-py3-none-any.whl (117 kB) Collecting cachetools<6.0,>=2.0.0 Downloading cachetools-5.5.0-py3-none-any.whl (9.5 kB) Collecting rsa<5,>=3.1.4 Downloading rsa-4.9-py3-none-any.whl (34 kB) Collecting pyasn1-modules>=0.2.1 Downloading pyasn1_modules-0.4.1-py3-none-any.whl (181 kB) Collecting pyasn1<0.7.0,>=0.4.6 Downloading pyasn1-0.6.1-py3-none-any.whl (83 kB) Collecting ruamel.yaml.clib>=0.2.7 Downloading ruamel.yaml.clib-0.2.12-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (722 kB) Collecting MarkupSafe>=2.0 Downloading MarkupSafe-3.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (20 kB) Collecting idna<4,>=2.5 Downloading idna-3.10-py3-none-any.whl (70 kB) Collecting charset-normalizer<4,>=2 Downloading charset_normalizer-3.4.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (144 kB) Collecting oauthlib>=3.0.0 Downloading oauthlib-3.2.2-py3-none-any.whl (151 kB) Using legacy 'setup.py install' for mtc, since package 'wheel' is not installed. Using legacy 'setup.py install' for openshift, since package 'wheel' is not installed. Installing collected packages: urllib3, pyasn1, idna, charset-normalizer, certifi, six, rsa, requests, pyasn1-modules, oauthlib, cachetools, websocket-client, ruamel.yaml.clib, requests-oauthlib, python-dateutil, MarkupSafe, google-auth, ruamel.yaml, python-string-utils, kubernetes, jinja2, suds-py3, openshift, mtc Running setup.py install for openshift: started Running setup.py install for openshift: finished with status 'done' Running setup.py install for mtc: started Running setup.py install for mtc: finished with status 'done' Successfully installed MarkupSafe-3.0.2 cachetools-5.5.0 certifi-2024.8.30 charset-normalizer-3.4.0 google-auth-2.35.0 idna-3.10 jinja2-3.1.4 kubernetes-11.0.0 mtc-0.0.1 oauthlib-3.2.2 openshift-0.11.2 pyasn1-0.6.1 pyasn1-modules-0.4.1 python-dateutil-2.9.0.post0 python-string-utils-1.0.0 requests-2.32.3 requests-oauthlib-2.0.0 rsa-4.9 ruamel.yaml-0.18.6 ruamel.yaml.clib-0.2.12 six-1.16.0 suds-py3-1.4.5.0 urllib3-2.2.3 websocket-client-1.8.0 WARNING: You are using pip version 21.2.3; however, version 24.2 is available. You should consider upgrading via the '/alabama/venv/bin/python3 -m pip install --upgrade pip' command. go: downloading k8s.io/apimachinery v0.25.6 go: downloading k8s.io/client-go v0.25.6 go: downloading k8s.io/api v0.25.6 go: downloading github.com/onsi/gomega v1.27.8 go: downloading github.com/onsi/ginkgo/v2 v2.11.0 go: downloading github.com/vmware-tanzu/velero v1.12.0 go: downloading github.com/openshift/oadp-operator v1.0.2-0.20231003152846-925e5cc16370 go: downloading github.com/operator-framework/api v0.14.1-0.20220413143725-33310d6154f3 go: downloading k8s.io/utils v0.0.0-20230115233650-391b47cb4029 go: downloading sigs.k8s.io/controller-runtime v0.12.2 go: downloading github.com/kubernetes-csi/external-snapshotter/client/v4 v4.2.0 go: downloading gopkg.in/yaml.v2 v2.4.0 go: downloading github.com/andygrunwald/go-jira v1.16.0 go: downloading k8s.io/kubectl v0.25.6 go: downloading github.com/backube/volsync v0.7.0 go: downloading github.com/apenella/go-ansible v1.1.5 go: downloading github.com/google/uuid v1.3.0 go: downloading github.com/konveyor/volume-snapshot-mover v0.0.0-20230510202920-14b0e79dbaaa go: downloading github.com/openshift/api v0.0.0-20230414143018-3367bc7e6ac7 go: downloading github.com/openshift/client-go v0.0.0-20211209144617-7385dd6338e3 go: downloading sigs.k8s.io/yaml v1.3.0 go: downloading github.com/google/go-cmp v0.5.9 go: downloading github.com/evanphx/json-patch v5.6.0+incompatible go: downloading github.com/go-logr/logr v1.2.4 go: downloading github.com/gogo/protobuf v1.3.2 go: downloading github.com/google/gofuzz v1.2.0 go: downloading gopkg.in/inf.v0 v0.9.1 go: downloading github.com/spf13/pflag v1.0.5 go: downloading k8s.io/klog/v2 v2.90.0 go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.2.3 go: downloading github.com/sirupsen/logrus v1.9.0 go: downloading github.com/fatih/structs v1.1.0 go: downloading github.com/golang-jwt/jwt/v4 v4.5.0 go: downloading github.com/google/go-querystring v1.1.0 go: downloading github.com/pkg/errors v0.9.1 go: downloading github.com/trivago/tgo v1.0.7 go: downloading github.com/imdario/mergo v0.3.13 go: downloading golang.org/x/net v0.12.0 go: downloading golang.org/x/term v0.10.0 go: downloading github.com/davecgh/go-spew v1.1.1 go: downloading github.com/spf13/cobra v1.6.1 go: downloading k8s.io/cli-runtime v0.25.6 go: downloading k8s.io/component-base v0.26.1 go: downloading github.com/apenella/go-common-utils/data v0.0.0-20210528133155-34ba915e28c8 go: downloading github.com/apenella/go-common-utils/error v0.0.0-20210528133155-34ba915e28c8 go: downloading gopkg.in/yaml.v3 v3.0.1 go: downloading sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd go: downloading k8s.io/kube-openapi v0.0.0-20230118215034-64b6bb138190 go: downloading golang.org/x/time v0.3.0 go: downloading k8s.io/apiextensions-apiserver v0.26.1 go: downloading github.com/go-logr/zapr v1.2.3 go: downloading go.uber.org/zap v1.24.0 go: downloading golang.org/x/sys v0.10.0 go: downloading github.com/json-iterator/go v1.1.12 go: downloading github.com/blang/semver/v4 v4.0.0 go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da go: downloading github.com/golang/protobuf v1.5.3 go: downloading github.com/google/gnostic v0.6.9 go: downloading google.golang.org/protobuf v1.30.0 go: downloading golang.org/x/oauth2 v0.7.0 go: downloading github.com/moby/spdystream v0.2.0 go: downloading github.com/inconshreveable/mousetrap v1.1.0 go: downloading github.com/jonboulle/clockwork v0.2.2 go: downloading k8s.io/component-helpers v0.26.1 go: downloading github.com/daviddengcn/go-colortext v1.0.0 go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de go: downloading github.com/docker/distribution v2.8.1+incompatible go: downloading github.com/moby/term v0.0.0-20220808134915-39b0c02b01ae go: downloading github.com/fvbommel/sortorder v1.0.1 go: downloading sigs.k8s.io/kustomize/kustomize/v4 v4.5.7 go: downloading sigs.k8s.io/kustomize/kyaml v0.13.9 go: downloading github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d go: downloading github.com/lithammer/dedent v1.1.0 go: downloading k8s.io/metrics v0.26.1 go: downloading github.com/chai2010/gettext-go v1.0.2 go: downloading github.com/MakeNowJust/heredoc v1.0.0 go: downloading github.com/mitchellh/go-wordwrap v1.0.0 go: downloading github.com/russross/blackfriday v1.5.2 go: downloading golang.org/x/text v0.11.0 go: downloading gomodules.xyz/jsonpatch/v2 v2.2.0 go: downloading go.uber.org/atomic v1.10.0 go: downloading go.uber.org/multierr v1.11.0 go: downloading github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 go: downloading golang.org/x/tools v0.11.0 go: downloading github.com/bombsimon/logrusr/v3 v3.0.0 go: downloading github.com/prometheus/client_golang v1.15.0 go: downloading k8s.io/kube-aggregator v0.19.12 go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd go: downloading github.com/modern-go/reflect2 v1.0.2 go: downloading github.com/onsi/ginkgo v1.16.5 go: downloading github.com/stretchr/testify v1.8.2 go: downloading gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c go: downloading github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 go: downloading github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153 go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 go: downloading github.com/go-openapi/swag v0.22.3 go: downloading sigs.k8s.io/kustomize/api v0.12.1 go: downloading github.com/fatih/camelcase v1.0.0 go: downloading github.com/golangplus/testing v1.0.0 go: downloading github.com/creack/pty v1.1.11 go: downloading gotest.tools/v3 v3.0.3 go: downloading gotest.tools v2.2.0+incompatible go: downloading github.com/opencontainers/go-digest v1.0.0 go: downloading github.com/fsnotify/fsnotify v1.6.0 go: downloading github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7 go: downloading github.com/peterbourgon/diskv v2.0.1+incompatible go: downloading github.com/google/pprof v0.0.0-20230705174524-200ffdc848b8 go: downloading github.com/robfig/cron v1.1.0 go: downloading github.com/hashicorp/go-hclog v1.2.0 go: downloading github.com/hashicorp/go-plugin v1.4.3 go: downloading go.uber.org/goleak v1.1.12 go: downloading github.com/prometheus/client_model v0.3.0 go: downloading github.com/prometheus/common v0.42.0 go: downloading github.com/spf13/afero v1.9.3 go: downloading google.golang.org/grpc v1.54.0 go: downloading github.com/gobwas/glob v0.2.3 go: downloading github.com/emicklei/go-restful/v3 v3.10.1 go: downloading github.com/go-openapi/jsonreference v0.20.2 go: downloading github.com/kr/pretty v0.3.1 go: downloading google.golang.org/appengine v1.6.7 go: downloading github.com/pmezard/go-difflib v1.0.0 go: downloading github.com/stretchr/objx v0.5.0 go: downloading github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 go: downloading github.com/mailru/easyjson v0.7.7 go: downloading github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f go: downloading github.com/beorn7/perks v1.0.1 go: downloading github.com/cespare/xxhash/v2 v2.2.0 go: downloading github.com/prometheus/procfs v0.9.0 go: downloading github.com/go-errors/errors v1.0.1 go: downloading github.com/Azure/azure-sdk-for-go v67.2.0+incompatible go: downloading github.com/Azure/go-autorest/autorest v0.11.27 go: downloading github.com/Azure/go-autorest v14.2.0+incompatible go: downloading github.com/Azure/go-autorest/autorest/azure/auth v0.5.8 go: downloading github.com/aws/aws-sdk-go v1.44.253 go: downloading github.com/joho/godotenv v1.3.0 go: downloading github.com/kopia/kopia v0.13.0 go: downloading github.com/Azure/go-autorest/autorest/to v0.3.0 go: downloading github.com/google/btree v1.0.1 go: downloading github.com/fatih/color v1.15.0 go: downloading github.com/mattn/go-colorable v0.1.13 go: downloading github.com/mattn/go-isatty v0.0.17 go: downloading github.com/hashicorp/yamux v0.0.0-20180604194846-3520598351bb go: downloading github.com/mitchellh/go-testing-interface v1.0.0 go: downloading github.com/oklog/run v1.0.0 go: downloading github.com/jhump/protoreflect v1.6.0 go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.4 go: downloading github.com/go-openapi/jsonpointer v0.19.6 go: downloading github.com/rogpeppe/go-internal v1.10.0 go: downloading github.com/kr/text v0.2.0 go: downloading github.com/josharian/intern v1.0.0 go: downloading github.com/nxadm/tail v1.4.8 go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 go: downloading github.com/xlab/treeprint v1.1.0 go: downloading github.com/Azure/go-autorest/logger v0.2.1 go: downloading github.com/Azure/go-autorest/tracing v0.6.0 go: downloading github.com/Azure/go-autorest/autorest/mocks v0.4.2 go: downloading github.com/Azure/go-autorest/autorest/adal v0.9.20 go: downloading github.com/Azure/go-autorest/autorest/azure/cli v0.4.2 go: downloading github.com/dimchansky/utfbom v1.1.1 go: downloading google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: downloading gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 go: downloading github.com/mitchellh/go-homedir v1.1.0 go: downloading github.com/Azure/go-autorest/autorest/date v0.3.0 go: downloading golang.org/x/crypto v0.11.0 go: downloading golang.org/x/mod v0.12.0 go: downloading go.opentelemetry.io/otel v1.14.0 go: downloading go.opentelemetry.io/otel/trace v1.14.0 go: downloading golang.org/x/sync v0.3.0 go: downloading github.com/kylelemons/godebug v1.1.0 go: downloading github.com/edsrzf/mmap-go v1.1.0 go: downloading github.com/klauspost/compress v1.16.5 go: downloading github.com/klauspost/pgzip v1.2.5 go: downloading github.com/pierrec/lz4 v2.6.1+incompatible go: downloading github.com/gofrs/flock v0.8.1 go: downloading github.com/sergi/go-diff v1.1.0 go: downloading go.starlark.net v0.0.0-20201006213952-227f4aabceb5 go: downloading github.com/natefinch/atomic v1.0.1 go: downloading golang.org/x/exp v0.0.0-20230213192124-5e25df0256eb go: downloading github.com/klauspost/reedsolomon v1.11.7 go: downloading github.com/zeebo/blake3 v0.2.3 go: downloading github.com/chmduquesne/rollinghash v4.0.0+incompatible go: downloading github.com/gorilla/mux v1.8.0 go: downloading github.com/benbjohnson/clock v1.1.0 go: downloading github.com/go-logr/stdr v1.2.2 go: downloading github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v0.3.0 go: downloading github.com/Azure/azure-storage-blob-go v0.15.0 go: downloading cloud.google.com/go/storage v1.30.1 go: downloading google.golang.org/api v0.120.0 go: downloading cloud.google.com/go v0.110.0 go: downloading github.com/minio/minio-go/v7 v7.0.52 go: downloading github.com/zeebo/assert v1.1.0 go: downloading github.com/klauspost/cpuid/v2 v2.2.4 go: downloading github.com/tg123/go-htpasswd v1.2.1 go: downloading github.com/zalando/go-keyring v0.2.2 go: downloading github.com/kopia/htmluibuild v0.0.0-20230326183719-f482ef17e2c9 go: downloading cloud.google.com/go/compute/metadata v0.2.3 go: downloading github.com/Azure/azure-pipeline-go v0.2.3 go: downloading cloud.google.com/go/compute v1.19.0 go: downloading cloud.google.com/go/iam v0.13.0 go: downloading github.com/googleapis/gax-go/v2 v2.8.0 go: downloading golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 go: downloading github.com/dustin/go-humanize v1.0.1 go: downloading gopkg.in/ini.v1 v1.67.0 go: downloading github.com/minio/md5-simd v1.1.2 go: downloading github.com/minio/sha256-simd v1.0.0 go: downloading github.com/alessio/shellescape v1.4.1 go: downloading github.com/danieljoos/wincred v1.1.2 go: downloading github.com/godbus/dbus/v5 v5.1.0 go: downloading github.com/hanwen/go-fuse/v2 v2.3.0 go: downloading github.com/mattn/go-ieproxy v0.0.1 go: downloading github.com/GehirnInc/crypt v0.0.0-20200316065508-bb7000b8a962 go: downloading go.opencensus.io v0.24.0 go: downloading github.com/rs/xid v1.4.0 go: downloading cloud.google.com/go/longrunning v0.4.1 go: downloading github.com/frankban/quicktest v1.14.3 go: downloading github.com/google/martian/v3 v3.3.2 go: downloading github.com/google/martian v2.1.0+incompatible go: downloading github.com/jmespath/go-jmespath v0.4.0 go: downloading github.com/zeebo/pcg v1.0.1 go: downloading github.com/jmespath/go-jmespath/internal/testify v1.5.1 go: downloading github.com/googleapis/enterprise-certificate-proxy v0.2.3 go: downloading github.com/google/s2a-go v0.1.2 go: downloading github.com/Azure/go-autorest/autorest/validation v0.2.0 go: downloading github.com/Azure/azure-sdk-for-go/sdk/internal v0.8.3 go: downloading github.com/Azure/azure-sdk-for-go/sdk/azcore v0.21.1 go: downloading github.com/dnaeon/go-vcr v1.2.0 + readonly 'RED=\e[31m' + RED='\e[31m' + readonly 'BLUE=\033[34m' + BLUE='\033[34m' + readonly 'CLEAR=\e[39m' + CLEAR='\e[39m' ++ oc get infrastructures cluster -o 'jsonpath={.status.platform}' ++ awk '{print tolower($0)}' + CLOUD_PROVIDER=aws + [[ aws == *-arm* ]] + [[ aws == *-fips* ]] + E2E_TIMEOUT_MULTIPLIER=2 + export NAMESPACE=openshift-adp + NAMESPACE=openshift-adp + export PROVIDER=aws + PROVIDER=aws ++ echo aws ++ awk '{print tolower($0)}' + BACKUP_LOCATION=aws + export BACKUP_LOCATION=aws + BACKUP_LOCATION=aws + export BUCKET=ci-op-jdjixbyi-interopoadp + BUCKET=ci-op-jdjixbyi-interopoadp + OADP_CREDS_FILE=/tmp/test-settings/credentials + OADP_VSL_CREDS_FILE=/tmp/test-settings/aws_vsl_creds +++ readlink -f /alabama/cspi/test_settings/scripts/test_runner.sh ++ dirname /alabama/cspi/test_settings/scripts/test_runner.sh + readonly SCRIPT_DIR=/alabama/cspi/test_settings/scripts + SCRIPT_DIR=/alabama/cspi/test_settings/scripts ++ cd /alabama/cspi/test_settings/scripts ++ git rev-parse --show-toplevel + readonly TOP_DIR=/alabama/cspi + TOP_DIR=/alabama/cspi + echo /alabama/cspi /alabama/cspi + TESTS_FOLDER=e2e ++ oc get nodes -o 'jsonpath={.items[*].metadata.labels.kubernetes\.io/arch}' ++ tr ' ' '\n' ++ sort -u ++ xargs + export NODES_ARCHITECTURE=amd64 + NODES_ARCHITECTURE=amd64 + export OADP_REPOSITORY=redhat + OADP_REPOSITORY=redhat + SKIP_DPA_CREATION=false ++ oc get ns openshift-storage ++ echo false + OPENSHIFT_STORAGE=false + '[' redhat == upstream-velero ']' + '[' false == true ']' ++ oc get storageclass -o 'jsonpath={range .items[*]}{@.metadata.name}{" "}{@.metadata.annotations.storageclass\.kubernetes\.io/is-default-class}{"\n"}{end}' ++ awk '$2=="true"{print $1}' ++ wc -l + NUM_DEFAULT_STORAGE_CLASS=1 + '[' 1 -ne 1 ']' ++ oc get storageclass -o 'jsonpath={.items[?(@.metadata.annotations.storageclass\.kubernetes\.io/is-default-class=='\''true'\'')].metadata.name}' + DEFAULT_SC=gp3 + export STORAGE_CLASS=gp3 + STORAGE_CLASS=gp3 + '[' -n gp3 ']' + '[' gp3 '!=' gp3 ']' + export STORAGE_CLASS_OPENSHIFT_STORAGE=gp3 + STORAGE_CLASS_OPENSHIFT_STORAGE=gp3 + echo 'Using the StorageClass for openshift-storage: gp3' Using the StorageClass for openshift-storage: gp3 + [[ amd64 != \a\m\d\6\4 ]] + TEST_FILTER='!// || (// && !exclude_aws && (!/target/ || target_aws) ) ' + [[ aws =~ ^osp ]] + [[ aws =~ ^vsphere ]] + [[ aws =~ ^gcp-wif ]] + [[ aws =~ ^ibmcloud ]] ++ oc config current-context ++ awk -F / '{print $2}' + SETTINGS_TMP=/alabama/cspi/output_files/api-ci-rosa-s-916u-wmvn-s1-devshift-org:6443 + mkdir -p /alabama/cspi/output_files/api-ci-rosa-s-916u-wmvn-s1-devshift-org:6443 ++ oc get authentication cluster -o 'jsonpath={.spec.serviceAccountIssuer}' + IS_OIDC=https://oidc.os1.devshift.org/2ekemulnfc21b5g95ie9l30q2mqe193p + '[' '!' -z https://oidc.os1.devshift.org/2ekemulnfc21b5g95ie9l30q2mqe193p ']' + '[' aws '!=' gcp-wif ']' + CLOUD_PROVIDER=aws-sts + TESTS_FOLDER=e2e/cloudstorage + [[ aws == \a\w\s ]] + export PROVIDER=aws + PROVIDER=aws + export CREDS_SECRET_REF=cloud-credentials + CREDS_SECRET_REF=cloud-credentials ++ oc get infrastructures cluster -o 'jsonpath={.status.platformStatus.aws.region}' --allow-missing-template-keys=false + export REGION=us-east-2 + REGION=us-east-2 + settings_script=aws_settings.sh + '[' aws-sts == aws-sts ']' + settings_script=aws-sts_settings.sh + BUCKET=ci-op-jdjixbyi-interopoadp + TMP_DIR=/alabama/cspi/output_files/api-ci-rosa-s-916u-wmvn-s1-devshift-org:6443 + source /alabama/cspi/test_settings/scripts/aws-sts_settings.sh ++ cat ++ [[ aws-sts == \a\w\s\-\s\t\s ]] ++ cat ++ echo -e '\n }\n}' +++ cat /alabama/cspi/output_files/api-ci-rosa-s-916u-wmvn-s1-devshift-org:6443/settings.json ++ x='{ "metadata": { "namespace": "openshift-adp" }, "spec": { "backupImages": false, "configuration":{ "velero":{ "defaultPlugins": [ "openshift", "aws" ] } }, "backupLocations": [ { "bucket": { "cloudStorageRef": { "name": "ci-op-jdjixbyi-interopoadp" }, "default": true, "credential":{ "name": "cloud-credentials", "key": "credentials" }, "config": { "region": "us-east-2" } } } ] , "snapshotLocations": [ { "velero": { "provider": "aws", "config": { "profile": "default", "region": "us-east-2", "enableSharedConfig": "true", "credentialsFile": "/tmp/credentials/openshift-adp/cloud-credentials-credentials" } } } ] } }' ++ echo '{ "metadata": { "namespace": "openshift-adp" }, "spec": { "backupImages": false, "configuration":{ "velero":{ "defaultPlugins": [ "openshift", "aws" ] } }, "backupLocations": [ { "bucket": { "cloudStorageRef": { "name": "ci-op-jdjixbyi-interopoadp" }, "default": true, "credential":{ "name": "cloud-credentials", "key": "credentials" }, "config": { "region": "us-east-2" } } } ] , "snapshotLocations": [ { "velero": { "provider": "aws", "config": { "profile": "default", "region": "us-east-2", "enableSharedConfig": "true", "credentialsFile": "/tmp/credentials/openshift-adp/cloud-credentials-credentials" } } } ] } }' ++ grep -o '^[^#]*' + FILE_SETTINGS_NAME=settings.json + printf '\033[34mGenerated settings file under /alabama/cspi/output_files/api-ci-rosa-s-916u-wmvn-s1-devshift-org:6443/settings.json\e[39m\n' Generated settings file under /alabama/cspi/output_files/api-ci-rosa-s-916u-wmvn-s1-devshift-org:6443/settings.json + cat /alabama/cspi/output_files/api-ci-rosa-s-916u-wmvn-s1-devshift-org:6443/settings.json ++ oc get volumesnapshotclass -o name + for i in $(oc get volumesnapshotclass -o name) + oc annotate volumesnapshotclass.snapshot.storage.k8s.io/csi-aws-vsc snapshot.storage.kubernetes.io/is-default-class- volumesnapshotclass.snapshot.storage.k8s.io/csi-aws-vsc annotated ++ ./e2e/must-gather/get-latest-build.sh + oc get configmaps -n default must-gather-image + UPSTREAM_VERSION=99.0.0 ++ oc get OperatorCondition -n openshift-adp -o 'jsonpath={.items[*].metadata.name}' ++ awk -F v '{print $2}' + OADP_VERSION=1.4.1 + '[' -z 1.4.1 ']' + '[' 1.4.1 == 99.0.0 ']' ++ oc get sub redhat-oadp-operator -n openshift-adp -o 'jsonpath={.spec.source}' + OADP_REPO=redhat-operators + '[' -z redhat-operators ']' + '[' redhat-operators == redhat-operators ']' + REGISTRY_PATH=registry.redhat.io/oadp/oadp-mustgather-rhel9: + TAG=1.4.1 + export MUST_GATHER_BUILD=registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 + MUST_GATHER_BUILD=registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 + echo registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 + export MUST_GATHER_BUILD=registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 + MUST_GATHER_BUILD=registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 + '[' -z registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 ']' + export NUM_OF_OADP_INSTANCES=1 + NUM_OF_OADP_INSTANCES=1 + ginkgo run -p --nodes=1 -mod=mod e2e/cloudstorage/ --show-node-events -- -credentials_file=/tmp/test-settings/credentials -vsl_credentials_file=/tmp/test-settings/aws_vsl_creds -oadp_namespace=openshift-adp -settings=/alabama/cspi/output_files/api-ci-rosa-s-916u-wmvn-s1-devshift-org:6443/settings.json -must_gather_image=registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 -timeout_multiplier=2 -skip_dpa_creation=false --ginkgo.junit-report=/logs/artifacts/junit_oadp_interop_results.xml '--ginkgo.label-filter=!// || (// && !exclude_aws && (!/target/ || target_aws) ) ' --ginkgo.focus=interop Ginkgo detected a version mismatch between the Ginkgo CLI and the version of Ginkgo imported by your packages: Ginkgo CLI Version: 2.20.2 Mismatched package versions found: 2.11.0 used by cloudstorage Ginkgo will continue to attempt to run but you may see errors (including flag parsing errors) and should either update your go.mod or your version of the Ginkgo CLI to match. To install the matching version of the CLI run go install github.com/onsi/ginkgo/v2/ginkgo from a path that contains a go.mod file. Alternatively you can use go run github.com/onsi/ginkgo/v2/ginkgo from a path that contains a go.mod file to invoke the matching version of the Ginkgo CLI. If you are attempting to test multiple packages that each have a different version of the Ginkgo library with a single Ginkgo CLI that is currently unsupported.  2024/10/25 08:42:42 Setting up clients 2024/10/25 08:42:42 Getting default StorageClass... 2024/10/25 08:42:42 Checking default storage class count Run the command: oc get sc 2024/10/25 08:42:42 Got default StorageClass gp3 2024/10/25 08:42:42 oc get sc NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE gp2 kubernetes.io/aws-ebs Delete WaitForFirstConsumer true 84m gp2-csi ebs.csi.aws.com Delete WaitForFirstConsumer true 81m gp3 (default) ebs.csi.aws.com Delete WaitForFirstConsumer true 84m gp3-csi ebs.csi.aws.com Delete WaitForFirstConsumer true 81m 2024/10/25 08:42:42 Using velero prefix: velero-e2e-197f22f9-92ad-11ef-8d75-0a580a81aa41 Running Suite: OADP CloudStorage Suite - /alabama/cspi/e2e/cloudstorage ======================================================================= Random Seed: 1729845718 Will run 8 of 14 specs ------------------------------ [BeforeSuite]  /alabama/cspi/e2e/cloudstorage/cloudstorage_suite_test.go:61 STEP: Setup CloudStorage in case using CloudStorageLocation @ 10/25/24 08:42:42.686 2024/10/25 08:42:44 pod "openshift-adp-controller-manager-57b8d6b4f6-x2z25" deleted 2024/10/25 08:42:44 pod: openshift-adp-controller-manager-57b8d6b4f6-k87r5 is not yet running with status: {Pending [{PodReadyToStartContainers False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:42:42 +0000 UTC } {Initialized True 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:42:42 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:42:42 +0000 UTC ContainersNotReady containers with unready status: [manager]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:42:42 +0000 UTC ContainersNotReady containers with unready status: [manager]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:42:42 +0000 UTC }] 10.0.37.172 [] 2024-10-25 08:42:42 +0000 UTC [] [{manager {&ContainerStateWaiting{Reason:ContainerCreating,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-rhel9-operator@sha256:6855c0e12c1730c757daafb7f28cfccf9f468c7a8d28e30cd61a1837c138b9eb 0xc00069d8f7}] Burstable []} 2024/10/25 08:42:49 pod: openshift-adp-controller-manager-57b8d6b4f6-k87r5 is not yet running with status: {Pending [{PodReadyToStartContainers False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:42:42 +0000 UTC } {Initialized True 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:42:42 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:42:42 +0000 UTC ContainersNotReady containers with unready status: [manager]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:42:42 +0000 UTC ContainersNotReady containers with unready status: [manager]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:42:42 +0000 UTC }] 10.0.37.172 [] 2024-10-25 08:42:42 +0000 UTC [] [{manager {&ContainerStateWaiting{Reason:ContainerCreating,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-rhel9-operator@sha256:6855c0e12c1730c757daafb7f28cfccf9f468c7a8d28e30cd61a1837c138b9eb 0xc000e8a087}] Burstable []} 2024/10/25 08:42:54 I1025 08:42:48.938773 1 leaderelection.go:78] The leader election gives 4 retries and allows for 30s of clock skew. The kube-apiserver downtime tolerance is 78s. Worst non-graceful lease acquisition is 2m43s. Worst graceful lease acquisition is {26s}. I1025 08:42:48.954922 1 leaderelection.go:35] infrastructure object status: &{ci-rosa-s-916u-r5g9t AWS 0xc000b1c200 https://api.ci-rosa-s-916u.wmvn.s1.devshift.org:6443 https://api-int.ci-rosa-s-916u.wmvn.s1.devshift.org:6443 HighlyAvailable HighlyAvailable} 2024-10-25T08:42:48Z INFO setup patching operator namespace with Pod security admission (PSA) labels to privileged 2024-10-25T08:42:48Z INFO setup starting manager 2024-10-25T08:42:48Z INFO controller-runtime.metrics Starting metrics server 2024-10-25T08:42:48Z INFO starting server {"kind": "health probe", "addr": "[::]:8081"} 2024-10-25T08:42:48Z INFO controller-runtime.metrics Serving metrics server {"bindAddress": ":8080", "secure": false} I1025 08:42:48.970019 1 leaderelection.go:250] attempting to acquire leader lease openshift-adp/oadp.openshift.io... 2024/10/25 08:43:24 I1025 08:42:48.938773 1 leaderelection.go:78] The leader election gives 4 retries and allows for 30s of clock skew. The kube-apiserver downtime tolerance is 78s. Worst non-graceful lease acquisition is 2m43s. Worst graceful lease acquisition is {26s}. I1025 08:42:48.954922 1 leaderelection.go:35] infrastructure object status: &{ci-rosa-s-916u-r5g9t AWS 0xc000b1c200 https://api.ci-rosa-s-916u.wmvn.s1.devshift.org:6443 https://api-int.ci-rosa-s-916u.wmvn.s1.devshift.org:6443 HighlyAvailable HighlyAvailable} 2024-10-25T08:42:48Z INFO setup patching operator namespace with Pod security admission (PSA) labels to privileged 2024-10-25T08:42:48Z INFO setup starting manager 2024-10-25T08:42:48Z INFO controller-runtime.metrics Starting metrics server 2024-10-25T08:42:48Z INFO starting server {"kind": "health probe", "addr": "[::]:8081"} 2024-10-25T08:42:48Z INFO controller-runtime.metrics Serving metrics server {"bindAddress": ":8080", "secure": false} I1025 08:42:48.970019 1 leaderelection.go:250] attempting to acquire leader lease openshift-adp/oadp.openshift.io... 2024/10/25 08:43:54 I1025 08:42:48.938773 1 leaderelection.go:78] The leader election gives 4 retries and allows for 30s of clock skew. The kube-apiserver downtime tolerance is 78s. Worst non-graceful lease acquisition is 2m43s. Worst graceful lease acquisition is {26s}. I1025 08:42:48.954922 1 leaderelection.go:35] infrastructure object status: &{ci-rosa-s-916u-r5g9t AWS 0xc000b1c200 https://api.ci-rosa-s-916u.wmvn.s1.devshift.org:6443 https://api-int.ci-rosa-s-916u.wmvn.s1.devshift.org:6443 HighlyAvailable HighlyAvailable} 2024-10-25T08:42:48Z INFO setup patching operator namespace with Pod security admission (PSA) labels to privileged 2024-10-25T08:42:48Z INFO setup starting manager 2024-10-25T08:42:48Z INFO controller-runtime.metrics Starting metrics server 2024-10-25T08:42:48Z INFO starting server {"kind": "health probe", "addr": "[::]:8081"} 2024-10-25T08:42:48Z INFO controller-runtime.metrics Serving metrics server {"bindAddress": ":8080", "secure": false} I1025 08:42:48.970019 1 leaderelection.go:250] attempting to acquire leader lease openshift-adp/oadp.openshift.io... 2024/10/25 08:44:24 I1025 08:42:48.938773 1 leaderelection.go:78] The leader election gives 4 retries and allows for 30s of clock skew. The kube-apiserver downtime tolerance is 78s. Worst non-graceful lease acquisition is 2m43s. Worst graceful lease acquisition is {26s}. I1025 08:42:48.954922 1 leaderelection.go:35] infrastructure object status: &{ci-rosa-s-916u-r5g9t AWS 0xc000b1c200 https://api.ci-rosa-s-916u.wmvn.s1.devshift.org:6443 https://api-int.ci-rosa-s-916u.wmvn.s1.devshift.org:6443 HighlyAvailable HighlyAvailable} 2024-10-25T08:42:48Z INFO setup patching operator namespace with Pod security admission (PSA) labels to privileged 2024-10-25T08:42:48Z INFO setup starting manager 2024-10-25T08:42:48Z INFO controller-runtime.metrics Starting metrics server 2024-10-25T08:42:48Z INFO starting server {"kind": "health probe", "addr": "[::]:8081"} 2024-10-25T08:42:48Z INFO controller-runtime.metrics Serving metrics server {"bindAddress": ":8080", "secure": false} I1025 08:42:48.970019 1 leaderelection.go:250] attempting to acquire leader lease openshift-adp/oadp.openshift.io... 2024/10/25 08:44:54 I1025 08:42:48.938773 1 leaderelection.go:78] The leader election gives 4 retries and allows for 30s of clock skew. The kube-apiserver downtime tolerance is 78s. Worst non-graceful lease acquisition is 2m43s. Worst graceful lease acquisition is {26s}. I1025 08:42:48.954922 1 leaderelection.go:35] infrastructure object status: &{ci-rosa-s-916u-r5g9t AWS 0xc000b1c200 https://api.ci-rosa-s-916u.wmvn.s1.devshift.org:6443 https://api-int.ci-rosa-s-916u.wmvn.s1.devshift.org:6443 HighlyAvailable HighlyAvailable} 2024-10-25T08:42:48Z INFO setup patching operator namespace with Pod security admission (PSA) labels to privileged 2024-10-25T08:42:48Z INFO setup starting manager 2024-10-25T08:42:48Z INFO controller-runtime.metrics Starting metrics server 2024-10-25T08:42:48Z INFO starting server {"kind": "health probe", "addr": "[::]:8081"} 2024-10-25T08:42:48Z INFO controller-runtime.metrics Serving metrics server {"bindAddress": ":8080", "secure": false} I1025 08:42:48.970019 1 leaderelection.go:250] attempting to acquire leader lease openshift-adp/oadp.openshift.io... 2024/10/25 08:45:24 I1025 08:42:48.938773 1 leaderelection.go:78] The leader election gives 4 retries and allows for 30s of clock skew. The kube-apiserver downtime tolerance is 78s. Worst non-graceful lease acquisition is 2m43s. Worst graceful lease acquisition is {26s}. I1025 08:42:48.954922 1 leaderelection.go:35] infrastructure object status: &{ci-rosa-s-916u-r5g9t AWS 0xc000b1c200 https://api.ci-rosa-s-916u.wmvn.s1.devshift.org:6443 https://api-int.ci-rosa-s-916u.wmvn.s1.devshift.org:6443 HighlyAvailable HighlyAvailable} 2024-10-25T08:42:48Z INFO setup patching operator namespace with Pod security admission (PSA) labels to privileged 2024-10-25T08:42:48Z INFO setup starting manager 2024-10-25T08:42:48Z INFO controller-runtime.metrics Starting metrics server 2024-10-25T08:42:48Z INFO starting server {"kind": "health probe", "addr": "[::]:8081"} 2024-10-25T08:42:48Z INFO controller-runtime.metrics Serving metrics server {"bindAddress": ":8080", "secure": false} I1025 08:42:48.970019 1 leaderelection.go:250] attempting to acquire leader lease openshift-adp/oadp.openshift.io... 2024/10/25 08:45:54 I1025 08:42:48.938773 1 leaderelection.go:78] The leader election gives 4 retries and allows for 30s of clock skew. The kube-apiserver downtime tolerance is 78s. Worst non-graceful lease acquisition is 2m43s. Worst graceful lease acquisition is {26s}. I1025 08:42:48.954922 1 leaderelection.go:35] infrastructure object status: &{ci-rosa-s-916u-r5g9t AWS 0xc000b1c200 https://api.ci-rosa-s-916u.wmvn.s1.devshift.org:6443 https://api-int.ci-rosa-s-916u.wmvn.s1.devshift.org:6443 HighlyAvailable HighlyAvailable} 2024-10-25T08:42:48Z INFO setup patching operator namespace with Pod security admission (PSA) labels to privileged 2024-10-25T08:42:48Z INFO setup starting manager 2024-10-25T08:42:48Z INFO controller-runtime.metrics Starting metrics server 2024-10-25T08:42:48Z INFO starting server {"kind": "health probe", "addr": "[::]:8081"} 2024-10-25T08:42:48Z INFO controller-runtime.metrics Serving metrics server {"bindAddress": ":8080", "secure": false} I1025 08:42:48.970019 1 leaderelection.go:250] attempting to acquire leader lease openshift-adp/oadp.openshift.io... I1025 08:45:33.506939 1 leaderelection.go:260] successfully acquired lease openshift-adp/oadp.openshift.io 2024-10-25T08:45:33Z INFO Starting EventSource {"controller": "dataprotectionapplication", "controllerGroup": "oadp.openshift.io", "controllerKind": "DataProtectionApplication", "source": "kind source: *v1alpha1.DataProtectionApplication"} 2024-10-25T08:45:33Z INFO Starting EventSource {"controller": "dataprotectionapplication", "controllerGroup": "oadp.openshift.io", "controllerKind": "DataProtectionApplication", "source": "kind source: *v1.Deployment"} 2024-10-25T08:45:33Z INFO Starting EventSource {"controller": "dataprotectionapplication", "controllerGroup": "oadp.openshift.io", "controllerKind": "DataProtectionApplication", "source": "kind source: *v1.BackupStorageLocation"} 2024-10-25T08:45:33Z INFO Starting EventSource {"controller": "dataprotectionapplication", "controllerGroup": "oadp.openshift.io", "controllerKind": "DataProtectionApplication", "source": "kind source: *v1.VolumeSnapshotLocation"} 2024-10-25T08:45:33Z INFO Starting EventSource {"controller": "dataprotectionapplication", "controllerGroup": "oadp.openshift.io", "controllerKind": "DataProtectionApplication", "source": "kind source: *v1.DaemonSet"} 2024-10-25T08:45:33Z INFO Starting EventSource {"controller": "dataprotectionapplication", "controllerGroup": "oadp.openshift.io", "controllerKind": "DataProtectionApplication", "source": "kind source: *v1.SecurityContextConstraints"} 2024-10-25T08:45:33Z INFO Starting EventSource {"controller": "dataprotectionapplication", "controllerGroup": "oadp.openshift.io", "controllerKind": "DataProtectionApplication", "source": "kind source: *v1.Service"} 2024-10-25T08:45:33Z INFO Starting EventSource {"controller": "cloudstorage", "controllerGroup": "oadp.openshift.io", "controllerKind": "CloudStorage", "source": "kind source: *v1alpha1.CloudStorage"} 2024-10-25T08:45:33Z INFO Starting EventSource {"controller": "dataprotectionapplication", "controllerGroup": "oadp.openshift.io", "controllerKind": "DataProtectionApplication", "source": "kind source: *v1.Route"} 2024-10-25T08:45:33Z INFO Starting EventSource {"controller": "dataprotectionapplication", "controllerGroup": "oadp.openshift.io", "controllerKind": "DataProtectionApplication", "source": "kind source: *v1.ConfigMap"} 2024-10-25T08:45:33Z INFO Starting EventSource {"controller": "dataprotectionapplication", "controllerGroup": "oadp.openshift.io", "controllerKind": "DataProtectionApplication", "source": "kind source: *v1.Secret"} 2024-10-25T08:45:33Z INFO Starting Controller {"controller": "dataprotectionapplication", "controllerGroup": "oadp.openshift.io", "controllerKind": "DataProtectionApplication"} 2024-10-25T08:45:33Z DEBUG events openshift-adp-controller-manager-57b8d6b4f6-k87r5_d3da2644-1c84-4bae-9bd5-22e7c279b24f became leader {"type": "Normal", "object": {"kind":"Lease","namespace":"openshift-adp","name":"oadp.openshift.io","uid":"9abb4a37-a7cc-42af-8976-a5737dfa5215","apiVersion":"coordination.k8s.io/v1","resourceVersion":"102519"}, "reason": "LeaderElection"} 2024-10-25T08:45:33Z INFO Starting Controller {"controller": "cloudstorage", "controllerGroup": "oadp.openshift.io", "controllerKind": "CloudStorage"} 2024-10-25T08:45:33Z INFO Starting workers {"controller": "cloudstorage", "controllerGroup": "oadp.openshift.io", "controllerKind": "CloudStorage", "worker count": 1} 2024-10-25T08:45:33Z INFO Starting workers {"controller": "dataprotectionapplication", "controllerGroup": "oadp.openshift.io", "controllerKind": "DataProtectionApplication", "worker count": 1} STEP: Create CloudStorage ci-op-jdjixbyi-interopoadp @ 10/25/24 08:45:54.483 2024/10/25 08:45:54 OADP namespace: openshift-adp [BeforeSuite] PASSED [191.882 seconds] ------------------------------ Backup restore tests [CSI] Application backup [tc-id:OADP-370][interop] [amd64] MSSQL application with CSI /alabama/cspi/e2e/cloudstorage/app_backup_csi.go:39 2024/10/25 08:45:54 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 10/25/24 08:45:54.569 2024/10/25 08:45:54 csi 2024/10/25 08:45:54 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "e9de968e-62a7-4def-9b78-9cec101f1089", "resourceVersion": "102726", "generation": 1, "creationTimestamp": "2024-10-25T08:45:54Z", "managedFields": [ { "manager": "cloudstorage.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2024-10-25T08:45:54Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {} } }, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "bucket": { "cloudStorageRef": { "name": "ci-op-jdjixbyi-interopoadp" }, "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "credentials" }, "default": true, "prefix": "velero-e2e-1981d203-92ad-11ef-8d75-0a580a81aa41" } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "csi" ] } }, "features": null }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 10/25/24 08:45:54.651 2024/10/25 08:45:54 Waiting for velero pod to be running 2024/10/25 08:45:59 pod: velero-684bf5d89-t4hm8 is not yet running with status: {Pending [{PodReadyToStartContainers False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:45:54 +0000 UTC } {Initialized False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:45:54 +0000 UTC ContainersNotInitialized containers with incomplete status: [openshift-velero-plugin velero-plugin-for-aws kubevirt-velero-plugin]} {Ready False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:45:54 +0000 UTC ContainersNotReady containers with unready status: [velero]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:45:54 +0000 UTC ContainersNotReady containers with unready status: [velero]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:45:54 +0000 UTC }] 10.0.65.50 [] 2024-10-25 08:45:54 +0000 UTC [{openshift-velero-plugin {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:a8725b48079a61e393583a3e8f0c9c70da31b08886cc9a8d115884886d623ff0 0xc000e8a7ca} {velero-plugin-for-aws {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:55aec191293ea79b76761161ddbb9db1a68d0f98c531ea97ce543dca2e9de967 0xc000e8a7cb} {kubevirt-velero-plugin {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:342dc1fcb6a8b5f01cafa01f6c4849888a6b3f78c3e5fa94a55cdc7566e088b1 0xc000e8a7cc}] [{velero {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-rhel9@sha256:cf9ebc90a72bf5043ef275babb19a4eebe830db9d7194a3e973ea64910f1ae29 0xc000e8a806}] Burstable []} 2024/10/25 08:46:04 pod: velero-684bf5d89-t4hm8 is not yet running with status: {Pending [{PodReadyToStartContainers True 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:46:00 +0000 UTC } {Initialized False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:45:54 +0000 UTC ContainersNotInitialized containers with incomplete status: [velero-plugin-for-aws kubevirt-velero-plugin]} {Ready False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:45:54 +0000 UTC ContainersNotReady containers with unready status: [velero]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:45:54 +0000 UTC ContainersNotReady containers with unready status: [velero]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:45:54 +0000 UTC }] 10.0.65.50 10.128.2.10 [{10.128.2.10}] 2024-10-25 08:45:54 +0000 UTC [{openshift-velero-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2024-10-25 08:46:00 +0000 UTC,FinishedAt:2024-10-25 08:46:00 +0000 UTC,ContainerID:cri-o://c9a1b22a450d140b884bd8463f4c5be30954e92acfdefa423008c5edcddd7d9c,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:a8725b48079a61e393583a3e8f0c9c70da31b08886cc9a8d115884886d623ff0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:3b66ca3d09fc4578af372c19c675c284d0eebef3daf0ef1bf748e76fa1526a08 cri-o://c9a1b22a450d140b884bd8463f4c5be30954e92acfdefa423008c5edcddd7d9c 0xc000e8b039} {velero-plugin-for-aws {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:55aec191293ea79b76761161ddbb9db1a68d0f98c531ea97ce543dca2e9de967 0xc000e8b03a} {kubevirt-velero-plugin {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:342dc1fcb6a8b5f01cafa01f6c4849888a6b3f78c3e5fa94a55cdc7566e088b1 0xc000e8b03b}] [{velero {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-rhel9@sha256:cf9ebc90a72bf5043ef275babb19a4eebe830db9d7194a3e973ea64910f1ae29 0xc000e8b066}] Burstable []} 2024/10/25 08:46:09 pod: velero-684bf5d89-t4hm8 is not yet running with status: {Pending [{PodReadyToStartContainers True 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:46:00 +0000 UTC } {Initialized False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:45:54 +0000 UTC ContainersNotInitialized containers with incomplete status: [velero-plugin-for-aws kubevirt-velero-plugin]} {Ready False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:45:54 +0000 UTC ContainersNotReady containers with unready status: [velero]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:45:54 +0000 UTC ContainersNotReady containers with unready status: [velero]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:45:54 +0000 UTC }] 10.0.65.50 10.128.2.10 [{10.128.2.10}] 2024-10-25 08:45:54 +0000 UTC [{openshift-velero-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2024-10-25 08:46:00 +0000 UTC,FinishedAt:2024-10-25 08:46:00 +0000 UTC,ContainerID:cri-o://c9a1b22a450d140b884bd8463f4c5be30954e92acfdefa423008c5edcddd7d9c,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:a8725b48079a61e393583a3e8f0c9c70da31b08886cc9a8d115884886d623ff0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:3b66ca3d09fc4578af372c19c675c284d0eebef3daf0ef1bf748e76fa1526a08 cri-o://c9a1b22a450d140b884bd8463f4c5be30954e92acfdefa423008c5edcddd7d9c 0xc0004799a9} {velero-plugin-for-aws {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:55aec191293ea79b76761161ddbb9db1a68d0f98c531ea97ce543dca2e9de967 0xc0004799aa} {kubevirt-velero-plugin {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:342dc1fcb6a8b5f01cafa01f6c4849888a6b3f78c3e5fa94a55cdc7566e088b1 0xc0004799ab}] [{velero {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-rhel9@sha256:cf9ebc90a72bf5043ef275babb19a4eebe830db9d7194a3e973ea64910f1ae29 0xc0004799d6}] Burstable []} 2024/10/25 08:46:14 pod: velero-684bf5d89-t4hm8 is not yet running with status: {Pending [{PodReadyToStartContainers True 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:46:00 +0000 UTC } {Initialized False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:45:54 +0000 UTC ContainersNotInitialized containers with incomplete status: [kubevirt-velero-plugin]} {Ready False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:45:54 +0000 UTC ContainersNotReady containers with unready status: [velero]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:45:54 +0000 UTC ContainersNotReady containers with unready status: [velero]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:45:54 +0000 UTC }] 10.0.65.50 10.128.2.10 [{10.128.2.10}] 2024-10-25 08:45:54 +0000 UTC [{openshift-velero-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2024-10-25 08:46:00 +0000 UTC,FinishedAt:2024-10-25 08:46:00 +0000 UTC,ContainerID:cri-o://c9a1b22a450d140b884bd8463f4c5be30954e92acfdefa423008c5edcddd7d9c,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:a8725b48079a61e393583a3e8f0c9c70da31b08886cc9a8d115884886d623ff0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:3b66ca3d09fc4578af372c19c675c284d0eebef3daf0ef1bf748e76fa1526a08 cri-o://c9a1b22a450d140b884bd8463f4c5be30954e92acfdefa423008c5edcddd7d9c 0xc000e0f149} {velero-plugin-for-aws {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2024-10-25 08:46:12 +0000 UTC,FinishedAt:2024-10-25 08:46:12 +0000 UTC,ContainerID:cri-o://4f5775c4973dc64232ab0dff6b8ae522321251f632e09fd03df6f2c51fc0c5d4,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:55aec191293ea79b76761161ddbb9db1a68d0f98c531ea97ce543dca2e9de967 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:55aec191293ea79b76761161ddbb9db1a68d0f98c531ea97ce543dca2e9de967 cri-o://4f5775c4973dc64232ab0dff6b8ae522321251f632e09fd03df6f2c51fc0c5d4 0xc000e0f159} {kubevirt-velero-plugin {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:342dc1fcb6a8b5f01cafa01f6c4849888a6b3f78c3e5fa94a55cdc7566e088b1 0xc000e0f15a}] [{velero {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-rhel9@sha256:cf9ebc90a72bf5043ef275babb19a4eebe830db9d7194a3e973ea64910f1ae29 0xc000e0f176}] Burstable []} 2024/10/25 08:46:19 pod: velero-684bf5d89-t4hm8 is not yet running with status: {Pending [{PodReadyToStartContainers True 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:46:00 +0000 UTC } {Initialized False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:45:54 +0000 UTC ContainersNotInitialized containers with incomplete status: [kubevirt-velero-plugin]} {Ready False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:45:54 +0000 UTC ContainersNotReady containers with unready status: [velero]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:45:54 +0000 UTC ContainersNotReady containers with unready status: [velero]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:45:54 +0000 UTC }] 10.0.65.50 10.128.2.10 [{10.128.2.10}] 2024-10-25 08:45:54 +0000 UTC [{openshift-velero-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2024-10-25 08:46:00 +0000 UTC,FinishedAt:2024-10-25 08:46:00 +0000 UTC,ContainerID:cri-o://c9a1b22a450d140b884bd8463f4c5be30954e92acfdefa423008c5edcddd7d9c,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:a8725b48079a61e393583a3e8f0c9c70da31b08886cc9a8d115884886d623ff0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:3b66ca3d09fc4578af372c19c675c284d0eebef3daf0ef1bf748e76fa1526a08 cri-o://c9a1b22a450d140b884bd8463f4c5be30954e92acfdefa423008c5edcddd7d9c 0xc000c13fd9} {velero-plugin-for-aws {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2024-10-25 08:46:12 +0000 UTC,FinishedAt:2024-10-25 08:46:12 +0000 UTC,ContainerID:cri-o://4f5775c4973dc64232ab0dff6b8ae522321251f632e09fd03df6f2c51fc0c5d4,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:55aec191293ea79b76761161ddbb9db1a68d0f98c531ea97ce543dca2e9de967 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:55aec191293ea79b76761161ddbb9db1a68d0f98c531ea97ce543dca2e9de967 cri-o://4f5775c4973dc64232ab0dff6b8ae522321251f632e09fd03df6f2c51fc0c5d4 0xc000c13fe9} {kubevirt-velero-plugin {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:342dc1fcb6a8b5f01cafa01f6c4849888a6b3f78c3e5fa94a55cdc7566e088b1 0xc000c13fea}] [{velero {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-rhel9@sha256:cf9ebc90a72bf5043ef275babb19a4eebe830db9d7194a3e973ea64910f1ae29 0xc0012b03c6}] Burstable []} 2024/10/25 08:46:24 pod: velero-684bf5d89-t4hm8 is not yet running with status: {Pending [{PodReadyToStartContainers True 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:46:00 +0000 UTC } {Initialized True 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:46:23 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:45:54 +0000 UTC ContainersNotReady containers with unready status: [velero]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:45:54 +0000 UTC ContainersNotReady containers with unready status: [velero]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:45:54 +0000 UTC }] 10.0.65.50 10.128.2.10 [{10.128.2.10}] 2024-10-25 08:45:54 +0000 UTC [{openshift-velero-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2024-10-25 08:46:00 +0000 UTC,FinishedAt:2024-10-25 08:46:00 +0000 UTC,ContainerID:cri-o://c9a1b22a450d140b884bd8463f4c5be30954e92acfdefa423008c5edcddd7d9c,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:a8725b48079a61e393583a3e8f0c9c70da31b08886cc9a8d115884886d623ff0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:3b66ca3d09fc4578af372c19c675c284d0eebef3daf0ef1bf748e76fa1526a08 cri-o://c9a1b22a450d140b884bd8463f4c5be30954e92acfdefa423008c5edcddd7d9c 0xc000e8bc19} {velero-plugin-for-aws {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2024-10-25 08:46:12 +0000 UTC,FinishedAt:2024-10-25 08:46:12 +0000 UTC,ContainerID:cri-o://4f5775c4973dc64232ab0dff6b8ae522321251f632e09fd03df6f2c51fc0c5d4,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:55aec191293ea79b76761161ddbb9db1a68d0f98c531ea97ce543dca2e9de967 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:55aec191293ea79b76761161ddbb9db1a68d0f98c531ea97ce543dca2e9de967 cri-o://4f5775c4973dc64232ab0dff6b8ae522321251f632e09fd03df6f2c51fc0c5d4 0xc000e8bc29} {kubevirt-velero-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2024-10-25 08:46:22 +0000 UTC,FinishedAt:2024-10-25 08:46:22 +0000 UTC,ContainerID:cri-o://4c7b630103ae6d6dabbc701c263b002094290aca484b79488ebbd7cd37b38127,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:342dc1fcb6a8b5f01cafa01f6c4849888a6b3f78c3e5fa94a55cdc7566e088b1 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:342dc1fcb6a8b5f01cafa01f6c4849888a6b3f78c3e5fa94a55cdc7566e088b1 cri-o://4c7b630103ae6d6dabbc701c263b002094290aca484b79488ebbd7cd37b38127 0xc000e8bc39}] [{velero {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-rhel9@sha256:cf9ebc90a72bf5043ef275babb19a4eebe830db9d7194a3e973ea64910f1ae29 0xc000e8bc4f}] Burstable []} 2024/10/25 08:46:29 pod: velero-684bf5d89-t4hm8 is not yet running with status: {Pending [{PodReadyToStartContainers True 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:46:00 +0000 UTC } {Initialized True 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:46:23 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:45:54 +0000 UTC ContainersNotReady containers with unready status: [velero]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:45:54 +0000 UTC ContainersNotReady containers with unready status: [velero]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:45:54 +0000 UTC }] 10.0.65.50 10.128.2.10 [{10.128.2.10}] 2024-10-25 08:45:54 +0000 UTC [{openshift-velero-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2024-10-25 08:46:00 +0000 UTC,FinishedAt:2024-10-25 08:46:00 +0000 UTC,ContainerID:cri-o://c9a1b22a450d140b884bd8463f4c5be30954e92acfdefa423008c5edcddd7d9c,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:a8725b48079a61e393583a3e8f0c9c70da31b08886cc9a8d115884886d623ff0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:3b66ca3d09fc4578af372c19c675c284d0eebef3daf0ef1bf748e76fa1526a08 cri-o://c9a1b22a450d140b884bd8463f4c5be30954e92acfdefa423008c5edcddd7d9c 0xc000e28559} {velero-plugin-for-aws {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2024-10-25 08:46:12 +0000 UTC,FinishedAt:2024-10-25 08:46:12 +0000 UTC,ContainerID:cri-o://4f5775c4973dc64232ab0dff6b8ae522321251f632e09fd03df6f2c51fc0c5d4,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:55aec191293ea79b76761161ddbb9db1a68d0f98c531ea97ce543dca2e9de967 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:55aec191293ea79b76761161ddbb9db1a68d0f98c531ea97ce543dca2e9de967 cri-o://4f5775c4973dc64232ab0dff6b8ae522321251f632e09fd03df6f2c51fc0c5d4 0xc000e28569} {kubevirt-velero-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2024-10-25 08:46:22 +0000 UTC,FinishedAt:2024-10-25 08:46:22 +0000 UTC,ContainerID:cri-o://4c7b630103ae6d6dabbc701c263b002094290aca484b79488ebbd7cd37b38127,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:342dc1fcb6a8b5f01cafa01f6c4849888a6b3f78c3e5fa94a55cdc7566e088b1 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:342dc1fcb6a8b5f01cafa01f6c4849888a6b3f78c3e5fa94a55cdc7566e088b1 cri-o://4c7b630103ae6d6dabbc701c263b002094290aca484b79488ebbd7cd37b38127 0xc000e28579}] [{velero {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-rhel9@sha256:cf9ebc90a72bf5043ef275babb19a4eebe830db9d7194a3e973ea64910f1ae29 0xc000e2858f}] Burstable []} 2024/10/25 08:46:34 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' STEP: Prepare backup resources, depending on the volumes backup type @ 10/25/24 08:46:34.964 Run the command: oc get ns openshift-storage &> /dev/null && echo true || echo false 2024/10/25 08:46:35 The 'openshift-storage' namespace does not exist 2024/10/25 08:46:35 Using default CSI driver based on infrastructure: ebs.csi.aws.com 2024/10/25 08:46:35 Snapclass 'example-snapclass' doesn't exist, creating 2024/10/25 08:46:35 Setting new default StorageClass 'gp2-csi' 2024/10/25 08:46:35 Checking default storage class count Run the command: oc get sc 2024/10/25 08:46:35 oc get sc NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE gp2 kubernetes.io/aws-ebs Delete WaitForFirstConsumer true 88m gp2-csi (default) ebs.csi.aws.com Delete WaitForFirstConsumer true 85m gp3 ebs.csi.aws.com Delete WaitForFirstConsumer true 88m gp3-csi ebs.csi.aws.com Delete WaitForFirstConsumer true 85m STEP: Installing application for case mssql @ 10/25/24 08:46:35.432 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] /usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py:1013: InsecureRequestWarning: Unverified HTTPS request is being made to host 'api.ci-rosa-s-916u.wmvn.s1.devshift.org'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings warnings.warn( TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : Create namespace for mssql] *** changed: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : Deploy mssql application] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=12  changed=5  unreachable=0 failed=0 skipped=10  rescued=0 ignored=0 2024/10/25 08:46:44 2024-10-25 08:46:36,987 p=21417 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 08:46:36,987 p=21417 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:46:37,239 p=21417 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 08:46:37,239 p=21417 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:46:37,497 p=21417 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 08:46:37,497 p=21417 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:46:37,512 p=21417 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:46:37,512 p=21417 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:46:37,951 p=21417 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 08:46:37,951 p=21417 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:46:37,988 p=21417 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 08:46:37,988 p=21417 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:46:38,009 p=21417 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:46:38,009 p=21417 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:46:38,021 p=21417 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 08:46:38,572 p=21417 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 08:46:38,572 p=21417 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:46:39,079 p=21417 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : Create namespace for mssql] *** 2024-10-25 08:46:39,079 p=21417 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:46:43,918 p=21417 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : Deploy mssql application] *** 2024-10-25 08:46:43,918 p=21417 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 08:46:43,918 p=21417 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:46:44,084 p=21417 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 08:46:44,085 p=21417 u=1002480000 n=ansible | localhost : ok=12 changed=5 unreachable=0 failed=0 skipped=10 rescued=0 ignored=0 STEP: Verify Application deployment @ 10/25/24 08:46:44.13 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. FAILED - RETRYING: [localhost]: Check mssql resources status (30 retries left). FAILED - RETRYING: [localhost]: Check mssql resources status (29 retries left). FAILED - RETRYING: [localhost]: Check mssql resources status (28 retries left). TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : Check mssql resources status] *** ok: [localhost] => (item=mssql) ok: [localhost] => (item=mssql-app) TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : Obtain mssql app route] *** ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : debug] *** ok: [localhost] => {  "msg": "Will check mssql at http://mssql-app-route-test-oadp-370.apps.ci-rosa-s-916u.wmvn.s1.devshift.org" } FAILED - RETRYING: [localhost]: Verify mssql app is available (30 retries left). TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : Verify mssql app is available] *** ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : Check mssql-scc SecurityContextConstraints resource exist] *** ok: [localhost] PLAY RECAP ********************************************************************* localhost : ok=15  changed=3  unreachable=0 failed=0 skipped=7  rescued=0 ignored=0 2024/10/25 08:47:36 2024-10-25 08:46:45,531 p=21656 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 08:46:45,531 p=21656 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:46:45,764 p=21656 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 08:46:45,764 p=21656 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:46:46,000 p=21656 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 08:46:46,001 p=21656 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:46:46,019 p=21656 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:46:46,019 p=21656 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:46:46,346 p=21656 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 08:46:46,346 p=21656 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:46:46,375 p=21656 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 08:46:46,375 p=21656 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:46:46,397 p=21656 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:46:46,397 p=21656 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:46:46,409 p=21656 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 08:46:46,997 p=21656 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 08:46:46,997 p=21656 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:47:20,219 p=21656 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : Check mssql resources status] *** 2024-10-25 08:47:20,220 p=21656 u=1002480000 n=ansible | ok: [localhost] => (item=mssql) 2024-10-25 08:47:20,923 p=21656 u=1002480000 n=ansible | ok: [localhost] => (item=mssql-app) 2024-10-25 08:47:20,926 p=21656 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 08:47:23,061 p=21656 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : Obtain mssql app route] *** 2024-10-25 08:47:23,062 p=21656 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:47:23,091 p=21656 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : debug] *** 2024-10-25 08:47:23,091 p=21656 u=1002480000 n=ansible | ok: [localhost] => { "msg": "Will check mssql at http://mssql-app-route-test-oadp-370.apps.ci-rosa-s-916u.wmvn.s1.devshift.org" } 2024-10-25 08:47:34,892 p=21656 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : Verify mssql app is available] *** 2024-10-25 08:47:34,893 p=21656 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:47:36,873 p=21656 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : Check mssql-scc SecurityContextConstraints resource exist] *** 2024-10-25 08:47:36,873 p=21656 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:47:36,918 p=21656 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 08:47:36,918 p=21656 u=1002480000 n=ansible | localhost : ok=15 changed=3 unreachable=0 failed=0 skipped=7 rescued=0 ignored=0 2024/10/25 08:47:37 {{ } { } [{{ } {mssql-pvc test-oadp-370 4581dbed-20c0-48ca-b6c7-df69f0e98ce3 103455 0 2024-10-25 08:46:39 +0000 UTC map[app:mssql] map[pv.kubernetes.io/bind-completed:yes pv.kubernetes.io/bound-by-controller:yes volume.beta.kubernetes.io/storage-provisioner:ebs.csi.aws.com volume.kubernetes.io/selected-node:ip-10-0-65-50.us-east-2.compute.internal volume.kubernetes.io/storage-provisioner:ebs.csi.aws.com] [] [kubernetes.io/pvc-protection] [{OpenAPI-Generator Update v1 2024-10-25 08:46:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{".":{},"f:app":{}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}}} } {kube-scheduler Update v1 2024-10-25 08:46:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volume.kubernetes.io/selected-node":{}}}} } {kube-controller-manager Update v1 2024-10-25 08:46:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:pv.kubernetes.io/bind-completed":{},"f:pv.kubernetes.io/bound-by-controller":{},"f:volume.beta.kubernetes.io/storage-provisioner":{},"f:volume.kubernetes.io/storage-provisioner":{}}},"f:spec":{"f:volumeName":{}}} } {kube-controller-manager Update v1 2024-10-25 08:46:49 +0000 UTC FieldsV1 {"f:status":{"f:accessModes":{},"f:capacity":{".":{},"f:storage":{}},"f:phase":{}}} status}]} {[ReadWriteOnce] nil {map[] map[storage:{{10737418240 0} {} 10Gi BinarySI}]} pvc-4581dbed-20c0-48ca-b6c7-df69f0e98ce3 0xc0003cf590 0xc0003cf5a0 nil nil} {Bound [ReadWriteOnce] map[storage:{{10737418240 0} {} 10Gi BinarySI}] [] map[] }}]} STEP: Creating backup mssql-8be0d094-92ad-11ef-8d75-0a580a81aa41 @ 10/25/24 08:47:37.016 2024/10/25 08:47:37 Wait until backup mssql-8be0d094-92ad-11ef-8d75-0a580a81aa41 is completed backup phase: WaitingForPluginOperations backup phase: WaitingForPluginOperations backup phase: Completed 2024/10/25 08:48:37 Verify the PVC and VSC count is matching 2024/10/25 08:48:37 Verify the Backup has CSIVolumeSnapshotsAttempted and CSIVolumeSnapshotsCompleted field on status 2024/10/25 08:48:37 Run velero describe on the backup 2024/10/25 08:48:37 [./velero describe backup mssql-8be0d094-92ad-11ef-8d75-0a580a81aa41 -n openshift-adp --details --insecure-skip-tls-verify] 2024/10/25 08:48:38 Exec stderr: "" 2024/10/25 08:48:38 Name: mssql-8be0d094-92ad-11ef-8d75-0a580a81aa41 Namespace: openshift-adp Labels: velero.io/storage-location=ts-dpa-1 Annotations: velero.io/resource-timeout=10m0s velero.io/source-cluster-k8s-gitversion=v1.29.8+632b078 velero.io/source-cluster-k8s-major-version=1 velero.io/source-cluster-k8s-minor-version=29 Phase: Completed Namespaces: Included: test-oadp-370 Excluded: Resources: Included: * Excluded: Cluster-scoped: auto Label selector: Or label selector: Storage Location: ts-dpa-1 Velero-Native Snapshot PVs: auto Snapshot Move Data: false Data Mover: velero TTL: 720h0m0s CSISnapshotTimeout: 10m0s ItemOperationTimeout: 4h0m0s Hooks: Backup Format Version: 1.1.0 Started: 2024-10-25 08:47:37 +0000 UTC Completed: 2024-10-25 08:48:31 +0000 UTC Expiration: 2024-11-24 08:47:37 +0000 UTC Total items to be backed up: 80 Items backed up: 80 Backup Item Operations: Operation for volumesnapshots.snapshot.storage.k8s.io test-oadp-370/velero-mssql-pvc-lrnc4: Backup Item Action Plugin: velero.io/csi-volumesnapshot-backupper Operation ID: test-oadp-370/velero-mssql-pvc-lrnc4/2024-10-25T08:47:44Z Items to Update: volumesnapshots.snapshot.storage.k8s.io test-oadp-370/velero-mssql-pvc-lrnc4 volumesnapshotcontents.snapshot.storage.k8s.io /snapcontent-133f3a5a-dbdc-4010-ba85-c93d0d58dae1 Phase: Completed Created: 2024-10-25 08:47:44 +0000 UTC Started: 2024-10-25 08:47:44 +0000 UTC Updated: 2024-10-25 08:48:30 +0000 UTC Resource List: apiextensions.k8s.io/v1/CustomResourceDefinition: - clusterserviceversions.operators.coreos.com apps.openshift.io/v1/DeploymentConfig: - test-oadp-370/mssql-deployment apps/v1/Deployment: - test-oadp-370/mssql-app-deployment apps/v1/ReplicaSet: - test-oadp-370/mssql-app-deployment-57f46ff5b authorization.openshift.io/v1/RoleBinding: - test-oadp-370/admin - test-oadp-370/admin-dedicated-admins - test-oadp-370/admin-system:serviceaccounts:dedicated-admin - test-oadp-370/alert-routing-edit-dedicated-admins - test-oadp-370/dedicated-admins-project-dedicated-admins - test-oadp-370/dedicated-admins-project-system:serviceaccounts:dedicated-admin - test-oadp-370/system:deployers - test-oadp-370/system:image-builders - test-oadp-370/system:image-pullers discovery.k8s.io/v1/EndpointSlice: - test-oadp-370/mssql-app-service-628d4 - test-oadp-370/mssql-service-9xcb4 operators.coreos.com/v1alpha1/ClusterServiceVersion: - test-oadp-370/observability-operator.v0.4.2 - test-oadp-370/route-monitor-operator.v0.1.696-gcea6d59 rbac.authorization.k8s.io/v1/RoleBinding: - test-oadp-370/admin - test-oadp-370/admin-dedicated-admins - test-oadp-370/admin-system:serviceaccounts:dedicated-admin - test-oadp-370/alert-routing-edit-dedicated-admins - test-oadp-370/dedicated-admins-project-dedicated-admins - test-oadp-370/dedicated-admins-project-system:serviceaccounts:dedicated-admin - test-oadp-370/system:deployers - test-oadp-370/system:image-builders - test-oadp-370/system:image-pullers route.openshift.io/v1/Route: - test-oadp-370/mssql-app-route security.openshift.io/v1/SecurityContextConstraints: - test-oadp-370-scc snapshot.storage.k8s.io/v1/VolumeSnapshot: - test-oadp-370/velero-mssql-pvc-lrnc4 snapshot.storage.k8s.io/v1/VolumeSnapshotClass: - example-snapclass snapshot.storage.k8s.io/v1/VolumeSnapshotContent: - snapcontent-133f3a5a-dbdc-4010-ba85-c93d0d58dae1 v1/ConfigMap: - test-oadp-370/kube-root-ca.crt - test-oadp-370/openshift-service-ca.crt v1/Endpoints: - test-oadp-370/mssql-app-service - test-oadp-370/mssql-service v1/Event: - test-oadp-370/mssql-app-deployment-57f46ff5b-t7ss7.1801a5c633c0b7d5 - test-oadp-370/mssql-app-deployment-57f46ff5b-t7ss7.1801a5c64b300f4b - test-oadp-370/mssql-app-deployment-57f46ff5b-t7ss7.1801a5c64c5e5b2b - test-oadp-370/mssql-app-deployment-57f46ff5b-t7ss7.1801a5c78d883a9e - test-oadp-370/mssql-app-deployment-57f46ff5b-t7ss7.1801a5c7927ea118 - test-oadp-370/mssql-app-deployment-57f46ff5b-t7ss7.1801a5c79350d457 - test-oadp-370/mssql-app-deployment-57f46ff5b.1801a5c632d63420 - test-oadp-370/mssql-app-deployment.1801a5c63205b41c - test-oadp-370/mssql-deployment-1-deploy.1801a5c6349de125 - test-oadp-370/mssql-deployment-1-deploy.1801a5c64cb8ef27 - test-oadp-370/mssql-deployment-1-deploy.1801a5c64de1fa37 - test-oadp-370/mssql-deployment-1-deploy.1801a5c6be5edff0 - test-oadp-370/mssql-deployment-1-deploy.1801a5c6c5e02762 - test-oadp-370/mssql-deployment-1-deploy.1801a5c6c6f3b6ea - test-oadp-370/mssql-deployment-1-s45xv.1801a5c80179e876 - test-oadp-370/mssql-deployment-1-s45xv.1801a5c88d75f2ce - test-oadp-370/mssql-deployment-1-s45xv.1801a5c97ee10704 - test-oadp-370/mssql-deployment-1-s45xv.1801a5c98037724f - test-oadp-370/mssql-deployment-1-s45xv.1801a5ce9050bda8 - test-oadp-370/mssql-deployment-1-s45xv.1801a5ce99285483 - test-oadp-370/mssql-deployment-1-s45xv.1801a5ce9a0b5b13 - test-oadp-370/mssql-deployment-1.1801a5c6d672f3f7 - test-oadp-370/mssql-deployment.1801a5c63001da58 - test-oadp-370/mssql-pvc.1801a5c599c1a6cc - test-oadp-370/mssql-pvc.1801a5c6d77cd263 - test-oadp-370/mssql-pvc.1801a5c6d78f58aa - test-oadp-370/mssql-pvc.1801a5c7e270aca7 v1/Namespace: - test-oadp-370 v1/PersistentVolume: - pvc-4581dbed-20c0-48ca-b6c7-df69f0e98ce3 v1/PersistentVolumeClaim: - test-oadp-370/mssql-pvc v1/Pod: - test-oadp-370/mssql-app-deployment-57f46ff5b-t7ss7 - test-oadp-370/mssql-deployment-1-deploy - test-oadp-370/mssql-deployment-1-s45xv v1/ReplicationController: - test-oadp-370/mssql-deployment-1 v1/Secret: - test-oadp-370/builder-dockercfg-hwh8r - test-oadp-370/default-dockercfg-6n8lg - test-oadp-370/deployer-dockercfg-72qv7 - test-oadp-370/mssql-secret - test-oadp-370/test-oadp-370-sa-dockercfg-f596n v1/Service: - test-oadp-370/mssql-app-service - test-oadp-370/mssql-service v1/ServiceAccount: - test-oadp-370/builder - test-oadp-370/default - test-oadp-370/deployer - test-oadp-370/test-oadp-370-sa Backup Volumes: Velero-Native Snapshots: CSI Snapshots: test-oadp-370/mssql-pvc: Snapshot: Operation ID: test-oadp-370/velero-mssql-pvc-lrnc4/2024-10-25T08:47:44Z Snapshot Content Name: snapcontent-133f3a5a-dbdc-4010-ba85-c93d0d58dae1 Storage Snapshot ID: snap-0d5d26e4fc3a91800 Snapshot Size (bytes): 10737418240 CSI Driver: ebs.csi.aws.com Pod Volume Backups: HooksAttempted: 0 HooksFailed: 0 STEP: Verify backup mssql-8be0d094-92ad-11ef-8d75-0a580a81aa41 has completed successfully @ 10/25/24 08:48:38.173 2024/10/25 08:48:38 Backup for case mssql succeeded STEP: Delete the appplication resources mssql @ 10/25/24 08:48:38.265 STEP: Cleanup Application for case mssql @ 10/25/24 08:48:38.265 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : Remove namespace test-oadp-370] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : Remove SCC resource test-oadp-370-scc] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=12  changed=5  unreachable=0 failed=0 skipped=10  rescued=0 ignored=0 2024/10/25 08:49:29 2024-10-25 08:48:40,063 p=21967 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 08:48:40,064 p=21967 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:48:40,370 p=21967 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 08:48:40,371 p=21967 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:48:40,620 p=21967 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 08:48:40,620 p=21967 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:48:40,636 p=21967 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:48:40,636 p=21967 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:48:40,921 p=21967 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 08:48:40,921 p=21967 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:48:40,949 p=21967 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 08:48:40,949 p=21967 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:48:40,967 p=21967 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:48:40,967 p=21967 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:48:40,977 p=21967 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 08:48:41,556 p=21967 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 08:48:41,556 p=21967 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:49:27,640 p=21967 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : Remove namespace test-oadp-370] *** 2024-10-25 08:49:27,640 p=21967 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 08:49:27,640 p=21967 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:49:29,328 p=21967 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : Remove SCC resource test-oadp-370-scc] *** 2024-10-25 08:49:29,328 p=21967 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:49:29,517 p=21967 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 08:49:29,517 p=21967 u=1002480000 n=ansible | localhost : ok=12 changed=5 unreachable=0 failed=0 skipped=10 rescued=0 ignored=0 2024/10/25 08:49:29 Creating restore mssql-8be0d094-92ad-11ef-8d75-0a580a81aa41 for case mssql-8be0d094-92ad-11ef-8d75-0a580a81aa41 STEP: Create restore mssql-8be0d094-92ad-11ef-8d75-0a580a81aa41 from backup mssql-8be0d094-92ad-11ef-8d75-0a580a81aa41 @ 10/25/24 08:49:29.557 2024/10/25 08:49:29 Wait until restore mssql-8be0d094-92ad-11ef-8d75-0a580a81aa41 is complete restore phase: InProgress restore phase: Completed STEP: Verify restore mssql-8be0d094-92ad-11ef-8d75-0a580a81aa41has completed successfully @ 10/25/24 08:49:39.619 STEP: Verify Application restore @ 10/25/24 08:49:39.635 STEP: Verify Application deployment for case mssql @ 10/25/24 08:49:39.635 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : Check mssql resources status] *** ok: [localhost] => (item=mssql) ok: [localhost] => (item=mssql-app) TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : Obtain mssql app route] *** ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : debug] *** ok: [localhost] => {  "msg": "Will check mssql at http://mssql-app-route-test-oadp-370.apps.ci-rosa-s-916u.wmvn.s1.devshift.org" } TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : Verify mssql app is available] *** ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : Check mssql-scc SecurityContextConstraints resource exist] *** ok: [localhost] PLAY RECAP ********************************************************************* localhost : ok=15  changed=3  unreachable=0 failed=0 skipped=7  rescued=0 ignored=0 2024/10/25 08:49:48 2024-10-25 08:49:41,015 p=22187 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 08:49:41,015 p=22187 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:49:41,257 p=22187 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 08:49:41,257 p=22187 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:49:41,481 p=22187 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 08:49:41,481 p=22187 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:49:41,496 p=22187 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:49:41,497 p=22187 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:49:41,788 p=22187 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 08:49:41,789 p=22187 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:49:41,816 p=22187 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 08:49:41,817 p=22187 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:49:41,837 p=22187 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:49:41,837 p=22187 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:49:41,847 p=22187 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 08:49:42,374 p=22187 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 08:49:42,374 p=22187 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:49:43,260 p=22187 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : Check mssql resources status] *** 2024-10-25 08:49:43,260 p=22187 u=1002480000 n=ansible | ok: [localhost] => (item=mssql) 2024-10-25 08:49:43,844 p=22187 u=1002480000 n=ansible | ok: [localhost] => (item=mssql-app) 2024-10-25 08:49:43,846 p=22187 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 08:49:45,490 p=22187 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : Obtain mssql app route] *** 2024-10-25 08:49:45,491 p=22187 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:49:45,511 p=22187 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : debug] *** 2024-10-25 08:49:45,512 p=22187 u=1002480000 n=ansible | ok: [localhost] => { "msg": "Will check mssql at http://mssql-app-route-test-oadp-370.apps.ci-rosa-s-916u.wmvn.s1.devshift.org" } 2024-10-25 08:49:46,821 p=22187 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : Verify mssql app is available] *** 2024-10-25 08:49:46,822 p=22187 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:49:48,553 p=22187 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : Check mssql-scc SecurityContextConstraints resource exist] *** 2024-10-25 08:49:48,554 p=22187 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:49:48,583 p=22187 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 08:49:48,583 p=22187 u=1002480000 n=ansible | localhost : ok=15 changed=3 unreachable=0 failed=0 skipped=7 rescued=0 ignored=0 2024/10/25 08:49:48 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 2024/10/25 08:49:48 Cleaning app [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : Remove namespace test-oadp-370] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : Remove SCC resource test-oadp-370-scc] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=12  changed=5  unreachable=0 failed=0 skipped=10  rescued=0 ignored=0 2024/10/25 08:50:34 2024-10-25 08:49:50,003 p=22447 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 08:49:50,003 p=22447 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:49:50,240 p=22447 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 08:49:50,240 p=22447 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:49:50,476 p=22447 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 08:49:50,477 p=22447 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:49:50,491 p=22447 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:49:50,491 p=22447 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:49:50,776 p=22447 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 08:49:50,777 p=22447 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:49:50,807 p=22447 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 08:49:50,808 p=22447 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:49:50,827 p=22447 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:49:50,827 p=22447 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:49:50,837 p=22447 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 08:49:51,353 p=22447 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 08:49:51,354 p=22447 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:50:32,337 p=22447 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : Remove namespace test-oadp-370] *** 2024-10-25 08:50:32,337 p=22447 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 08:50:32,337 p=22447 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:50:34,155 p=22447 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mssql : Remove SCC resource test-oadp-370-scc] *** 2024-10-25 08:50:34,155 p=22447 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:50:34,360 p=22447 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 08:50:34,360 p=22447 u=1002480000 n=ansible | localhost : ok=12 changed=5 unreachable=0 failed=0 skipped=10 rescued=0 ignored=0 2024/10/25 08:50:34 Cleaning setup resources for the backup 2024/10/25 08:50:34 Setting new default StorageClass 'gp3' 2024/10/25 08:50:34 Checking default storage class count Run the command: oc get sc 2024/10/25 08:50:34 oc get sc NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE gp2 kubernetes.io/aws-ebs Delete WaitForFirstConsumer true 92m gp2-csi ebs.csi.aws.com Delete WaitForFirstConsumer true 88m gp3 (default) ebs.csi.aws.com Delete WaitForFirstConsumer true 92m gp3-csi ebs.csi.aws.com Delete WaitForFirstConsumer true 88m 2024/10/25 08:50:34 Deleting VolumeSnapshotClass 'example-snapclass' • [280.138 seconds] ------------------------------ Backup restore tests [CSI] Application backup [tc-id:OADP-122] [bug-id:OADP-1057] [interop] Django application with CSI /alabama/cspi/e2e/cloudstorage/app_backup_csi.go:54 [FAILED] in [JustBeforeEach] - /alabama/cspi/e2e/cloudstorage/cloudstorage_suite_test.go:120 @ 10/25/24 08:50:34.919 2024/10/25 08:50:34 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 STEP: Get the failed spec name @ 10/25/24 08:50:34.919 2024/10/25 08:50:34 The failed spec name is: Backup restore tests [CSI] Application backup [tc-id:OADP-122] [bug-id:OADP-1057] [interop] Django application with CSI STEP: Create a folder for all must-gather files if it doesn't exists already @ 10/25/24 08:50:34.919 2024/10/25 08:50:34 The folder logs does not exists, creating new folder with the name: logs STEP: Create a folder for the failed spec if it doesn't exists already @ 10/25/24 08:50:34.919 2024/10/25 08:50:34 The folder logs/It_Backup_restore_tests_CSI_Application_backup_tc-id_OADP-122_bug-id_OADP-1057_interop_Django_application_with_CSI does not exists, creating new folder with the name: logs/It_Backup_restore_tests_CSI_Application_backup_tc-id_OADP-122_bug-id_OADP-1057_interop_Django_application_with_CSI STEP: Run must-gather because the spec failed @ 10/25/24 08:50:34.92 2024/10/25 08:50:34 Log the present working directory path:- /alabama/cspi/e2e/cloudstorage 2024/10/25 08:50:34 [adm must-gather --dest-dir /alabama/cspi/e2e/cloudstorage/logs/It_Backup_restore_tests_CSI_Application_backup_tc-id_OADP-122_bug-id_OADP-1057_interop_Django_application_with_CSI --image registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1] 2024/10/25 08:51:04 Log all the files present in /alabama/cspi/e2e/cloudstorage/logs directory 2024/10/25 08:51:04 It_Backup_restore_tests_CSI_Application_backup_tc-id_OADP-122_bug-id_OADP-1057_interop_Django_application_with_CSI STEP: Find must-gather folder and rename it to a shorter more readable name @ 10/25/24 08:51:04.91 • [FAILED] [30.236 seconds] TOP-LEVEL [JustBeforeEach] Backup restore tests [CSI] Application backup [tc-id:OADP-122] [bug-id:OADP-1057] [interop] Django application with CSI [JustBeforeEach] /alabama/cspi/e2e/cloudstorage/cloudstorage_suite_test.go:118 [It] /alabama/cspi/e2e/cloudstorage/app_backup_csi.go:54 [FAILED] Unexpected error: <*errors.withStack | 0xc0016bcf18>: 429 Too Many Requests: HTTP Status 429 – Too Many Requests

HTTP Status 429 – Too Many Requests

Type Status Report

Message Rate limit exceeded.

Description The user has sent too many requests in a given amount of time ("rate limiting").

Apache Tomcat/9.0.91

: request failed. Please analyze the request body for more details. Status code: 429 { error: <*errors.withMessage | 0xc00059ea00>{ cause: <*errors.errorString | 0xc001709ec0>{ s: "request failed. Please analyze the request body for more details. Status code: 429", }, msg: "429 Too Many Requests: HTTP Status 429 – Too Many Requests

HTTP Status 429 – Too Many Requests

Type Status Report

Message Rate limit exceeded.

Description The user has sent too many requests in a given amount of time (\"rate limiting\").

Apache Tomcat/9.0.91

", }, stack: [0x33be527, 0x33c0da7, 0x33c0e98, 0x35ea7b5, 0x35ea70f, 0x35f116b, 0x188841b, 0x189de38, 0x13ffee1], } occurred In [JustBeforeEach] at: /alabama/cspi/e2e/cloudstorage/cloudstorage_suite_test.go:120 @ 10/25/24 08:50:34.919 ------------------------------ Backup restore tests [CSI] Application backup [tc-id:OADP-79][interop] MySQL application with CSI /alabama/cspi/e2e/cloudstorage/app_backup_csi.go:71 2024/10/25 08:51:04 Delete all downloadrequest mssql-8be0d094-92ad-11ef-8d75-0a580a81aa41-b955fb73-e540-400e-ae52-446d7a52ca5b mssql-8be0d094-92ad-11ef-8d75-0a580a81aa41-bccd8add-1a68-414e-8b5b-a6929b4ee62c mssql-8be0d094-92ad-11ef-8d75-0a580a81aa41-fa1f2ef6-87ed-472f-9a8d-100f706be202 STEP: Create DPA CR @ 10/25/24 08:51:05.007 2024/10/25 08:51:05 csi 2024/10/25 08:51:05 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "73a078e1-fc6c-4484-a9d2-e57f78e17b13", "resourceVersion": "107260", "generation": 1, "creationTimestamp": "2024-10-25T08:51:05Z", "managedFields": [ { "manager": "cloudstorage.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2024-10-25T08:51:05Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {} } }, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "bucket": { "cloudStorageRef": { "name": "ci-op-jdjixbyi-interopoadp" }, "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "credentials" }, "default": true, "prefix": "velero-e2e-1981d203-92ad-11ef-8d75-0a580a81aa41" } } ], "snapshotLocations": [], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt", "csi" ] } }, "features": null }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 10/25/24 08:51:05.088 2024/10/25 08:51:05 Waiting for velero pod to be running 2024/10/25 08:51:10 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' STEP: Prepare backup resources, depending on the volumes backup type @ 10/25/24 08:51:10.153 Run the command: oc get ns openshift-storage &> /dev/null && echo true || echo false 2024/10/25 08:51:10 The 'openshift-storage' namespace does not exist 2024/10/25 08:51:10 Using default CSI driver based on infrastructure: ebs.csi.aws.com 2024/10/25 08:51:10 Snapclass 'example-snapclass' doesn't exist, creating 2024/10/25 08:51:10 Setting new default StorageClass 'gp2-csi' 2024/10/25 08:51:10 Checking default storage class count Run the command: oc get sc 2024/10/25 08:51:10 oc get sc NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE gp2 kubernetes.io/aws-ebs Delete WaitForFirstConsumer true 93m gp2-csi (default) ebs.csi.aws.com Delete WaitForFirstConsumer true 89m gp3 ebs.csi.aws.com Delete WaitForFirstConsumer true 93m gp3-csi ebs.csi.aws.com Delete WaitForFirstConsumer true 89m STEP: Installing application for case mysql @ 10/25/24 08:51:10.572 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Check namespace test-oadp-79] *** ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Create namespace] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Deploy a mysql pod] *** changed: [localhost] FAILED - RETRYING: [localhost]: Check pod status (30 retries left). FAILED - RETRYING: [localhost]: Check pod status (29 retries left). FAILED - RETRYING: [localhost]: Check pod status (28 retries left). TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Check pod status] *** ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Copy mysql provision script to pod] *** changed: [localhost] FAILED - RETRYING: [localhost]: Wait until service ready for connections (30 retries left). TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Wait until service ready for connections] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Provision the mysql database] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Add dummy data into mysql-data1 pvc] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Create md5 hashes for the files] *** changed: [localhost] Pausing for 30 seconds TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Pause After Create md5 hashes for the files] *** ok: [localhost] PLAY RECAP ********************************************************************* localhost : ok=20  changed=10  unreachable=0 failed=0 skipped=6  rescued=0 ignored=0 2024/10/25 08:52:12 2024-10-25 08:51:11,971 p=22767 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 08:51:11,971 p=22767 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:51:12,195 p=22767 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 08:51:12,195 p=22767 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:51:12,441 p=22767 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 08:51:12,441 p=22767 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:51:12,456 p=22767 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:51:12,456 p=22767 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:51:12,737 p=22767 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 08:51:12,737 p=22767 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:51:12,764 p=22767 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 08:51:12,764 p=22767 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:51:12,782 p=22767 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:51:12,783 p=22767 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:51:12,793 p=22767 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 08:51:13,334 p=22767 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 08:51:13,334 p=22767 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:51:14,197 p=22767 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Check namespace test-oadp-79] *** 2024-10-25 08:51:14,198 p=22767 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 08:51:14,198 p=22767 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:51:14,685 p=22767 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Create namespace] *** 2024-10-25 08:51:14,686 p=22767 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:51:15,950 p=22767 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Deploy a mysql pod] *** 2024-10-25 08:51:15,950 p=22767 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:51:33,611 p=22767 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Check pod status] *** 2024-10-25 08:51:33,611 p=22767 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:51:34,310 p=22767 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Copy mysql provision script to pod] *** 2024-10-25 08:51:34,310 p=22767 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:51:39,968 p=22767 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Wait until service ready for connections] *** 2024-10-25 08:51:39,968 p=22767 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:51:40,611 p=22767 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Provision the mysql database] *** 2024-10-25 08:51:40,612 p=22767 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:51:41,496 p=22767 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Add dummy data into mysql-data1 pvc] *** 2024-10-25 08:51:41,496 p=22767 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:51:42,191 p=22767 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Create md5 hashes for the files] *** 2024-10-25 08:51:42,191 p=22767 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:51:42,209 p=22767 u=1002480000 n=ansible | Pausing for 30 seconds 2024-10-25 08:52:12,243 p=22767 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Pause After Create md5 hashes for the files] *** 2024-10-25 08:52:12,243 p=22767 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:52:12,323 p=22767 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 08:52:12,323 p=22767 u=1002480000 n=ansible | localhost : ok=20 changed=10 unreachable=0 failed=0 skipped=6 rescued=0 ignored=0 STEP: Verify Application deployment @ 10/25/24 08:52:12.367 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Check mysql pod status] *** ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Wait until service ready for connections] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Query the inserted data] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Validate test1 file has correct md5 hash] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=14  changed=6  unreachable=0 failed=0 skipped=12  rescued=0 ignored=0 2024/10/25 08:52:17 2024-10-25 08:52:13,789 p=23264 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 08:52:13,789 p=23264 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:52:14,015 p=23264 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 08:52:14,015 p=23264 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:52:14,242 p=23264 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 08:52:14,243 p=23264 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:52:14,257 p=23264 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:52:14,257 p=23264 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:52:14,603 p=23264 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 08:52:14,603 p=23264 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:52:14,631 p=23264 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 08:52:14,632 p=23264 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:52:14,649 p=23264 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:52:14,649 p=23264 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:52:14,659 p=23264 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 08:52:15,193 p=23264 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 08:52:15,194 p=23264 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:52:16,142 p=23264 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Check mysql pod status] *** 2024-10-25 08:52:16,142 p=23264 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 08:52:16,143 p=23264 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:52:16,477 p=23264 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Wait until service ready for connections] *** 2024-10-25 08:52:16,478 p=23264 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:52:16,970 p=23264 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Query the inserted data] *** 2024-10-25 08:52:16,971 p=23264 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:52:17,666 p=23264 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Validate test1 file has correct md5 hash] *** 2024-10-25 08:52:17,667 p=23264 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:52:17,679 p=23264 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 08:52:17,679 p=23264 u=1002480000 n=ansible | localhost : ok=14 changed=6 unreachable=0 failed=0 skipped=12 rescued=0 ignored=0 2024/10/25 08:52:17 {{ } { } [{{ } {mysql-data test-oadp-79 1b74d18a-de69-445d-aa7e-33c734ce98cd 107592 0 2024-10-25 08:51:15 +0000 UTC map[app:mysql testlabel:selectors testlabel2:foo] map[pv.kubernetes.io/bind-completed:yes pv.kubernetes.io/bound-by-controller:yes volume.beta.kubernetes.io/storage-provisioner:ebs.csi.aws.com volume.kubernetes.io/selected-node:ip-10-0-65-50.us-east-2.compute.internal volume.kubernetes.io/storage-provisioner:ebs.csi.aws.com] [] [kubernetes.io/pvc-protection] [{OpenAPI-Generator Update v1 2024-10-25 08:51:15 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{".":{},"f:app":{},"f:testlabel":{},"f:testlabel2":{}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}}} } {kube-scheduler Update v1 2024-10-25 08:51:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volume.kubernetes.io/selected-node":{}}}} } {kube-controller-manager Update v1 2024-10-25 08:51:20 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:pv.kubernetes.io/bind-completed":{},"f:pv.kubernetes.io/bound-by-controller":{},"f:volume.beta.kubernetes.io/storage-provisioner":{},"f:volume.kubernetes.io/storage-provisioner":{}}},"f:spec":{"f:volumeName":{}}} } {kube-controller-manager Update v1 2024-10-25 08:51:20 +0000 UTC FieldsV1 {"f:status":{"f:accessModes":{},"f:capacity":{".":{},"f:storage":{}},"f:phase":{}}} status}]} {[ReadWriteOnce] nil {map[] map[storage:{{2147483648 0} {} 2Gi BinarySI}]} pvc-1b74d18a-de69-445d-aa7e-33c734ce98cd 0xc000c0f6d0 0xc000c0f6e0 nil nil} {Bound [ReadWriteOnce] map[storage:{{2147483648 0} {} 2Gi BinarySI}] [] map[] }} {{ } {mysql-data1 test-oadp-79 4ec62a87-d1e7-46a0-a76b-abe3c96540e1 107595 0 2024-10-25 08:51:15 +0000 UTC map[app:mysql testlabel:selectors testlabel2:foo] map[pv.kubernetes.io/bind-completed:yes pv.kubernetes.io/bound-by-controller:yes volume.beta.kubernetes.io/storage-provisioner:ebs.csi.aws.com volume.kubernetes.io/selected-node:ip-10-0-65-50.us-east-2.compute.internal volume.kubernetes.io/storage-provisioner:ebs.csi.aws.com] [] [kubernetes.io/pvc-protection] [{OpenAPI-Generator Update v1 2024-10-25 08:51:15 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{".":{},"f:app":{},"f:testlabel":{},"f:testlabel2":{}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}}} } {kube-scheduler Update v1 2024-10-25 08:51:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volume.kubernetes.io/selected-node":{}}}} } {kube-controller-manager Update v1 2024-10-25 08:51:20 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:pv.kubernetes.io/bind-completed":{},"f:pv.kubernetes.io/bound-by-controller":{},"f:volume.beta.kubernetes.io/storage-provisioner":{},"f:volume.kubernetes.io/storage-provisioner":{}}},"f:spec":{"f:volumeName":{}}} } {kube-controller-manager Update v1 2024-10-25 08:51:20 +0000 UTC FieldsV1 {"f:status":{"f:accessModes":{},"f:capacity":{".":{},"f:storage":{}},"f:phase":{}}} status}]} {[ReadWriteOnce] nil {map[] map[storage:{{2147483648 0} {} 2Gi BinarySI}]} pvc-4ec62a87-d1e7-46a0-a76b-abe3c96540e1 0xc000c0f800 0xc000c0f810 nil nil} {Bound [ReadWriteOnce] map[storage:{{2147483648 0} {} 2Gi BinarySI}] [] map[] }}]} STEP: Creating backup mysql-44e1284f-92ae-11ef-8d75-0a580a81aa41 @ 10/25/24 08:52:17.736 2024/10/25 08:52:17 Wait until backup mysql-44e1284f-92ae-11ef-8d75-0a580a81aa41 is completed backup phase: WaitingForPluginOperations backup phase: WaitingForPluginOperations backup phase: WaitingForPluginOperations backup phase: Completed 2024/10/25 08:53:37 Verify the PVC and VSC count is matching 2024/10/25 08:53:37 Verify the Backup has CSIVolumeSnapshotsAttempted and CSIVolumeSnapshotsCompleted field on status 2024/10/25 08:53:37 Run velero describe on the backup 2024/10/25 08:53:37 [./velero describe backup mysql-44e1284f-92ae-11ef-8d75-0a580a81aa41 -n openshift-adp --details --insecure-skip-tls-verify] 2024/10/25 08:53:38 Exec stderr: "" 2024/10/25 08:53:38 Name: mysql-44e1284f-92ae-11ef-8d75-0a580a81aa41 Namespace: openshift-adp Labels: velero.io/storage-location=ts-dpa-1 Annotations: velero.io/resource-timeout=10m0s velero.io/source-cluster-k8s-gitversion=v1.29.8+632b078 velero.io/source-cluster-k8s-major-version=1 velero.io/source-cluster-k8s-minor-version=29 Phase: Completed Warnings: Velero: message: /VolumeSnapshot has a temporary error Failed to create snapshot: error updating status for volume snapshot content snapcontent-6445627a-9086-4240-88eb-f6f3a06541a4: snapshot controller failed to update snapcontent-6445627a-9086-4240-88eb-f6f3a06541a4 on API server: Operation cannot be fulfilled on volumesnapshotcontents.snapshot.storage.k8s.io "snapcontent-6445627a-9086-4240-88eb-f6f3a06541a4": the object has been modified; please apply your changes to the latest version and try again. Snapshot controller will retry later. Cluster: Namespaces: Namespaces: Included: test-oadp-79 Excluded: Resources: Included: * Excluded: Cluster-scoped: auto Label selector: Or label selector: Storage Location: ts-dpa-1 Velero-Native Snapshot PVs: auto Snapshot Move Data: false Data Mover: velero TTL: 720h0m0s CSISnapshotTimeout: 10m0s ItemOperationTimeout: 4h0m0s Hooks: Backup Format Version: 1.1.0 Started: 2024-10-25 08:52:17 +0000 UTC Completed: 2024-10-25 08:53:29 +0000 UTC Expiration: 2024-11-24 08:52:17 +0000 UTC Total items to be backed up: 64 Items backed up: 64 Backup Item Operations: Operation for volumesnapshots.snapshot.storage.k8s.io test-oadp-79/velero-mysql-data-nh7r6: Backup Item Action Plugin: velero.io/csi-volumesnapshot-backupper Operation ID: test-oadp-79/velero-mysql-data-nh7r6/2024-10-25T08:52:23Z Items to Update: volumesnapshots.snapshot.storage.k8s.io test-oadp-79/velero-mysql-data-nh7r6 volumesnapshotcontents.snapshot.storage.k8s.io /snapcontent-458954b8-2301-4c2b-a74b-beb0422c2c7e Phase: Completed Created: 2024-10-25 08:52:23 +0000 UTC Started: 2024-10-25 08:52:23 +0000 UTC Updated: 2024-10-25 08:52:58 +0000 UTC Operation for volumesnapshots.snapshot.storage.k8s.io test-oadp-79/velero-mysql-data1-dscxv: Backup Item Action Plugin: velero.io/csi-volumesnapshot-backupper Operation ID: test-oadp-79/velero-mysql-data1-dscxv/2024-10-25T08:52:28Z Items to Update: volumesnapshots.snapshot.storage.k8s.io test-oadp-79/velero-mysql-data1-dscxv volumesnapshotcontents.snapshot.storage.k8s.io /snapcontent-6445627a-9086-4240-88eb-f6f3a06541a4 Phase: Completed Created: 2024-10-25 08:52:28 +0000 UTC Started: 2024-10-25 08:52:28 +0000 UTC Updated: 2024-10-25 08:53:28 +0000 UTC Resource List: apiextensions.k8s.io/v1/CustomResourceDefinition: - clusterserviceversions.operators.coreos.com apps/v1/Deployment: - test-oadp-79/mysql apps/v1/ReplicaSet: - test-oadp-79/mysql-845bdd7d8d authorization.openshift.io/v1/RoleBinding: - test-oadp-79/admin - test-oadp-79/admin-dedicated-admins - test-oadp-79/admin-system:serviceaccounts:dedicated-admin - test-oadp-79/alert-routing-edit-dedicated-admins - test-oadp-79/dedicated-admins-project-dedicated-admins - test-oadp-79/dedicated-admins-project-system:serviceaccounts:dedicated-admin - test-oadp-79/system:deployers - test-oadp-79/system:image-builders - test-oadp-79/system:image-pullers discovery.k8s.io/v1/EndpointSlice: - test-oadp-79/mysql-nw77b operators.coreos.com/v1alpha1/ClusterServiceVersion: - test-oadp-79/observability-operator.v0.4.2 - test-oadp-79/route-monitor-operator.v0.1.696-gcea6d59 rbac.authorization.k8s.io/v1/RoleBinding: - test-oadp-79/admin - test-oadp-79/admin-dedicated-admins - test-oadp-79/admin-system:serviceaccounts:dedicated-admin - test-oadp-79/alert-routing-edit-dedicated-admins - test-oadp-79/dedicated-admins-project-dedicated-admins - test-oadp-79/dedicated-admins-project-system:serviceaccounts:dedicated-admin - test-oadp-79/system:deployers - test-oadp-79/system:image-builders - test-oadp-79/system:image-pullers snapshot.storage.k8s.io/v1/VolumeSnapshot: - test-oadp-79/velero-mysql-data-nh7r6 - test-oadp-79/velero-mysql-data1-dscxv snapshot.storage.k8s.io/v1/VolumeSnapshotClass: - example-snapclass snapshot.storage.k8s.io/v1/VolumeSnapshotContent: - snapcontent-458954b8-2301-4c2b-a74b-beb0422c2c7e - snapcontent-6445627a-9086-4240-88eb-f6f3a06541a4 v1/ConfigMap: - test-oadp-79/kube-root-ca.crt - test-oadp-79/openshift-service-ca.crt v1/Endpoints: - test-oadp-79/mysql v1/Event: - test-oadp-79/mysql-845bdd7d8d-nrdx9.1801a6070411f716 - test-oadp-79/mysql-845bdd7d8d-nrdx9.1801a6076fea6119 - test-oadp-79/mysql-845bdd7d8d-nrdx9.1801a6078e426553 - test-oadp-79/mysql-845bdd7d8d-nrdx9.1801a607aac58350 - test-oadp-79/mysql-845bdd7d8d-nrdx9.1801a607ac08017e - test-oadp-79/mysql-845bdd7d8d-nrdx9.1801a6097dfb5d18 - test-oadp-79/mysql-845bdd7d8d-nrdx9.1801a60984b4e9a4 - test-oadp-79/mysql-845bdd7d8d-nrdx9.1801a6098590972a - test-oadp-79/mysql-845bdd7d8d.1801a605d5c9f4e5 - test-oadp-79/mysql-data.1801a605ce3757e6 - test-oadp-79/mysql-data.1801a605d95db832 - test-oadp-79/mysql-data.1801a605d95f7525 - test-oadp-79/mysql-data.1801a606dcfe9635 - test-oadp-79/mysql-data1.1801a605d089ee9f - test-oadp-79/mysql-data1.1801a605da6a8b5a - test-oadp-79/mysql-data1.1801a605da8ff72e - test-oadp-79/mysql-data1.1801a606dd0139d2 - test-oadp-79/mysql.1801a605d41915bc v1/Namespace: - test-oadp-79 v1/PersistentVolume: - pvc-1b74d18a-de69-445d-aa7e-33c734ce98cd - pvc-4ec62a87-d1e7-46a0-a76b-abe3c96540e1 v1/PersistentVolumeClaim: - test-oadp-79/mysql-data - test-oadp-79/mysql-data1 v1/Pod: - test-oadp-79/mysql-845bdd7d8d-nrdx9 v1/Secret: - test-oadp-79/builder-dockercfg-jvvm8 - test-oadp-79/default-dockercfg-b7kzg - test-oadp-79/deployer-dockercfg-psdv6 - test-oadp-79/mysql v1/Service: - test-oadp-79/mysql v1/ServiceAccount: - test-oadp-79/builder - test-oadp-79/default - test-oadp-79/deployer Backup Volumes: Velero-Native Snapshots: CSI Snapshots: test-oadp-79/mysql-data: Snapshot: Operation ID: test-oadp-79/velero-mysql-data-nh7r6/2024-10-25T08:52:23Z Snapshot Content Name: snapcontent-458954b8-2301-4c2b-a74b-beb0422c2c7e Storage Snapshot ID: snap-0ffb979e2330721c0 Snapshot Size (bytes): 2147483648 CSI Driver: ebs.csi.aws.com test-oadp-79/mysql-data1: Snapshot: Operation ID: test-oadp-79/velero-mysql-data1-dscxv/2024-10-25T08:52:28Z Snapshot Content Name: snapcontent-6445627a-9086-4240-88eb-f6f3a06541a4 Storage Snapshot ID: snap-07dcacdab2b028500 Snapshot Size (bytes): 2147483648 CSI Driver: ebs.csi.aws.com Pod Volume Backups: HooksAttempted: 0 HooksFailed: 0 STEP: Verify backup mysql-44e1284f-92ae-11ef-8d75-0a580a81aa41 has completed successfully @ 10/25/24 08:53:38.899 2024/10/25 08:53:38 Backup for case mysql succeeded STEP: Delete the appplication resources mysql @ 10/25/24 08:53:38.987 STEP: Cleanup Application for case mysql @ 10/25/24 08:53:38.987 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Remove namespace test-oadp-79] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=11  changed=4  unreachable=0 failed=0 skipped=15  rescued=0 ignored=0 2024/10/25 08:53:57 2024-10-25 08:53:40,384 p=23571 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 08:53:40,385 p=23571 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:53:40,613 p=23571 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 08:53:40,613 p=23571 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:53:40,837 p=23571 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 08:53:40,837 p=23571 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:53:40,851 p=23571 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:53:40,852 p=23571 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:53:41,138 p=23571 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 08:53:41,138 p=23571 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:53:41,164 p=23571 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 08:53:41,165 p=23571 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:53:41,182 p=23571 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:53:41,182 p=23571 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:53:41,192 p=23571 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 08:53:41,731 p=23571 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 08:53:41,731 p=23571 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:53:57,631 p=23571 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Remove namespace test-oadp-79] *** 2024-10-25 08:53:57,632 p=23571 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 08:53:57,632 p=23571 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:53:57,870 p=23571 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 08:53:57,870 p=23571 u=1002480000 n=ansible | localhost : ok=11 changed=4 unreachable=0 failed=0 skipped=15 rescued=0 ignored=0 2024/10/25 08:53:57 Creating restore mysql-44e1284f-92ae-11ef-8d75-0a580a81aa41 for case mysql-44e1284f-92ae-11ef-8d75-0a580a81aa41 STEP: Create restore mysql-44e1284f-92ae-11ef-8d75-0a580a81aa41 from backup mysql-44e1284f-92ae-11ef-8d75-0a580a81aa41 @ 10/25/24 08:53:57.918 2024/10/25 08:53:57 Wait until restore mysql-44e1284f-92ae-11ef-8d75-0a580a81aa41 is complete restore phase: InProgress restore phase: Completed STEP: Verify restore mysql-44e1284f-92ae-11ef-8d75-0a580a81aa41has completed successfully @ 10/25/24 08:54:07.972 STEP: Verify Application restore @ 10/25/24 08:54:07.988 STEP: Verify Application deployment for case mysql @ 10/25/24 08:54:07.989 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. FAILED - RETRYING: [localhost]: Check mysql pod status (30 retries left). FAILED - RETRYING: [localhost]: Check mysql pod status (29 retries left). TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Check mysql pod status] *** ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Wait until service ready for connections] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Query the inserted data] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Validate test1 file has correct md5 hash] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=14  changed=6  unreachable=0 failed=0 skipped=12  rescued=0 ignored=0 2024/10/25 08:54:24 2024-10-25 08:54:09,390 p=23783 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 08:54:09,390 p=23783 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:54:09,626 p=23783 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 08:54:09,626 p=23783 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:54:09,864 p=23783 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 08:54:09,864 p=23783 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:54:09,878 p=23783 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:54:09,878 p=23783 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:54:10,169 p=23783 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 08:54:10,169 p=23783 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:54:10,199 p=23783 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 08:54:10,199 p=23783 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:54:10,218 p=23783 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:54:10,218 p=23783 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:54:10,229 p=23783 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 08:54:10,769 p=23783 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 08:54:10,770 p=23783 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:54:22,972 p=23783 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Check mysql pod status] *** 2024-10-25 08:54:22,972 p=23783 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 08:54:22,972 p=23783 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:54:23,340 p=23783 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Wait until service ready for connections] *** 2024-10-25 08:54:23,340 p=23783 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:54:23,885 p=23783 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Query the inserted data] *** 2024-10-25 08:54:23,885 p=23783 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:54:24,607 p=23783 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Validate test1 file has correct md5 hash] *** 2024-10-25 08:54:24,607 p=23783 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:54:24,619 p=23783 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 08:54:24,619 p=23783 u=1002480000 n=ansible | localhost : ok=14 changed=6 unreachable=0 failed=0 skipped=12 rescued=0 ignored=0 2024/10/25 08:54:24 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 2024/10/25 08:54:24 Cleaning app [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Remove namespace test-oadp-79] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=11  changed=4  unreachable=0 failed=0 skipped=15  rescued=0 ignored=0 2024/10/25 08:54:43 2024-10-25 08:54:26,143 p=24119 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 08:54:26,143 p=24119 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:54:26,372 p=24119 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 08:54:26,372 p=24119 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:54:26,606 p=24119 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 08:54:26,606 p=24119 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:54:26,620 p=24119 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:54:26,621 p=24119 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:54:26,899 p=24119 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 08:54:26,899 p=24119 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:54:26,926 p=24119 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 08:54:26,926 p=24119 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:54:26,945 p=24119 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:54:26,945 p=24119 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:54:26,955 p=24119 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 08:54:27,486 p=24119 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 08:54:27,486 p=24119 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:54:43,350 p=24119 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Remove namespace test-oadp-79] *** 2024-10-25 08:54:43,351 p=24119 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 08:54:43,351 p=24119 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:54:43,611 p=24119 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 08:54:43,611 p=24119 u=1002480000 n=ansible | localhost : ok=11 changed=4 unreachable=0 failed=0 skipped=15 rescued=0 ignored=0 2024/10/25 08:54:43 Cleaning setup resources for the backup 2024/10/25 08:54:43 Setting new default StorageClass 'gp3' 2024/10/25 08:54:43 Checking default storage class count Run the command: oc get sc 2024/10/25 08:54:43 oc get sc NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE gp2 kubernetes.io/aws-ebs Delete WaitForFirstConsumer true 96m gp2-csi ebs.csi.aws.com Delete WaitForFirstConsumer true 93m gp3 (default) ebs.csi.aws.com Delete WaitForFirstConsumer true 96m gp3-csi ebs.csi.aws.com Delete WaitForFirstConsumer true 93m 2024/10/25 08:54:43 Deleting VolumeSnapshotClass 'example-snapclass' • [219.017 seconds] ------------------------------ S ------------------------------ Backup restore tests Application backup [tc-id:OADP-76][interop] MySQL application with VSL [vsl] /alabama/cspi/e2e/cloudstorage/app_backup_vsl.go:40 2024/10/25 08:54:43 Check if VSL custom credentials exist 2024/10/25 08:54:43 Check if the cloud provider is AWS 2024/10/25 08:54:43 Delete all downloadrequest mysql-44e1284f-92ae-11ef-8d75-0a580a81aa41-09b154ed-db59-4d0a-8d6f-7630e48c0732 mysql-44e1284f-92ae-11ef-8d75-0a580a81aa41-6da2ff08-5878-4565-a3eb-eaebb333ea2e mysql-44e1284f-92ae-11ef-8d75-0a580a81aa41-ce0627d1-011e-42c9-adbd-062d44c218cf mysql-44e1284f-92ae-11ef-8d75-0a580a81aa41-d0b29e3c-981e-4ad3-ac87-1be50b9728c7 STEP: Create DPA CR @ 10/25/24 08:54:44.068 2024/10/25 08:54:44 vsl 2024/10/25 08:54:44 Check if VSL custom credentials exist 2024/10/25 08:54:44 Check if the cloud provider is AWS 2024/10/25 08:54:44 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "33630b0a-13c6-4b0f-9b39-9a0b0692a1f2", "resourceVersion": "110792", "generation": 1, "creationTimestamp": "2024-10-25T08:54:44Z", "managedFields": [ { "manager": "cloudstorage.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2024-10-25T08:54:44Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {} } }, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "bucket": { "cloudStorageRef": { "name": "ci-op-jdjixbyi-interopoadp" }, "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "credentials" }, "default": true, "prefix": "velero-e2e-1981d203-92ad-11ef-8d75-0a580a81aa41" } } ], "snapshotLocations": [ { "velero": { "provider": "aws", "config": { "credentialsFile": "/tmp/credentials/openshift-adp/cloud-credentials-credentials", "enableSharedConfig": "true", "profile": "default", "region": "us-east-2" } } } ], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt" ] } }, "features": null }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 10/25/24 08:54:44.17 2024/10/25 08:54:44 Waiting for velero pod to be running 2024/10/25 08:54:44 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2024/10/25 08:54:44 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "33630b0a-13c6-4b0f-9b39-9a0b0692a1f2", "resourceVersion": "110792", "generation": 1, "creationTimestamp": "2024-10-25T08:54:44Z", "managedFields": [ { "manager": "cloudstorage.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2024-10-25T08:54:44Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {} } }, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "bucket": { "cloudStorageRef": { "name": "ci-op-jdjixbyi-interopoadp" }, "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "credentials" }, "default": true, "prefix": "velero-e2e-1981d203-92ad-11ef-8d75-0a580a81aa41" } } ], "snapshotLocations": [ { "velero": { "provider": "aws", "config": { "credentialsFile": "/tmp/credentials/openshift-adp/cloud-credentials-credentials", "enableSharedConfig": "true", "profile": "default", "region": "us-east-2" } } } ], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt" ] } }, "features": null }, "status": {} } STEP: Prepare backup resources, depending on the volumes backup type @ 10/25/24 08:54:49.229 2024/10/25 08:54:49 Checking default storage class count STEP: Installing application for case mysql @ 10/25/24 08:54:49.261 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Check namespace test-oadp-76] *** ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Create namespace] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Deploy a mysql pod] *** changed: [localhost] FAILED - RETRYING: [localhost]: Check pod status (30 retries left). FAILED - RETRYING: [localhost]: Check pod status (29 retries left). FAILED - RETRYING: [localhost]: Check pod status (28 retries left). TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Check pod status] *** ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Copy mysql provision script to pod] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Wait until service ready for connections] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Provision the mysql database] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Add dummy data into mysql-data1 pvc] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Create md5 hashes for the files] *** changed: [localhost] Pausing for 30 seconds TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Pause After Create md5 hashes for the files] *** ok: [localhost] PLAY RECAP ********************************************************************* localhost : ok=20  changed=10  unreachable=0 failed=0 skipped=6  rescued=0 ignored=0 2024/10/25 08:55:45 2024-10-25 08:54:50,662 p=24350 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 08:54:50,662 p=24350 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:54:50,908 p=24350 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 08:54:50,909 p=24350 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:54:51,150 p=24350 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 08:54:51,150 p=24350 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:54:51,167 p=24350 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:54:51,167 p=24350 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:54:51,495 p=24350 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 08:54:51,495 p=24350 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:54:51,526 p=24350 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 08:54:51,526 p=24350 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:54:51,546 p=24350 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:54:51,546 p=24350 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:54:51,556 p=24350 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 08:54:52,090 p=24350 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 08:54:52,090 p=24350 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:54:52,904 p=24350 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Check namespace test-oadp-76] *** 2024-10-25 08:54:52,904 p=24350 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 08:54:52,905 p=24350 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:54:53,355 p=24350 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Create namespace] *** 2024-10-25 08:54:53,355 p=24350 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:54:54,341 p=24350 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Deploy a mysql pod] *** 2024-10-25 08:54:54,341 p=24350 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:55:11,812 p=24350 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Check pod status] *** 2024-10-25 08:55:11,813 p=24350 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:55:12,508 p=24350 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Copy mysql provision script to pod] *** 2024-10-25 08:55:12,508 p=24350 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:55:12,853 p=24350 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Wait until service ready for connections] *** 2024-10-25 08:55:12,853 p=24350 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:55:13,530 p=24350 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Provision the mysql database] *** 2024-10-25 08:55:13,530 p=24350 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:55:14,402 p=24350 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Add dummy data into mysql-data1 pvc] *** 2024-10-25 08:55:14,402 p=24350 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:55:15,094 p=24350 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Create md5 hashes for the files] *** 2024-10-25 08:55:15,094 p=24350 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:55:15,112 p=24350 u=1002480000 n=ansible | Pausing for 30 seconds 2024-10-25 08:55:45,123 p=24350 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Pause After Create md5 hashes for the files] *** 2024-10-25 08:55:45,124 p=24350 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:55:45,199 p=24350 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 08:55:45,199 p=24350 u=1002480000 n=ansible | localhost : ok=20 changed=10 unreachable=0 failed=0 skipped=6 rescued=0 ignored=0 STEP: Verify Application deployment @ 10/25/24 08:55:45.243 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Check mysql pod status] *** ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Wait until service ready for connections] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Query the inserted data] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Validate test1 file has correct md5 hash] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=14  changed=6  unreachable=0 failed=0 skipped=12  rescued=0 ignored=0 2024/10/25 08:55:50 2024-10-25 08:55:46,724 p=24822 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 08:55:46,724 p=24822 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:55:46,979 p=24822 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 08:55:46,979 p=24822 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:55:47,236 p=24822 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 08:55:47,236 p=24822 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:55:47,252 p=24822 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:55:47,252 p=24822 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:55:47,554 p=24822 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 08:55:47,555 p=24822 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:55:47,586 p=24822 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 08:55:47,586 p=24822 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:55:47,607 p=24822 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:55:47,607 p=24822 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:55:47,618 p=24822 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 08:55:48,176 p=24822 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 08:55:48,176 p=24822 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:55:49,248 p=24822 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Check mysql pod status] *** 2024-10-25 08:55:49,248 p=24822 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 08:55:49,248 p=24822 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:55:49,609 p=24822 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Wait until service ready for connections] *** 2024-10-25 08:55:49,609 p=24822 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:55:50,148 p=24822 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Query the inserted data] *** 2024-10-25 08:55:50,148 p=24822 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:55:50,868 p=24822 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Validate test1 file has correct md5 hash] *** 2024-10-25 08:55:50,868 p=24822 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:55:50,881 p=24822 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 08:55:50,881 p=24822 u=1002480000 n=ansible | localhost : ok=14 changed=6 unreachable=0 failed=0 skipped=12 rescued=0 ignored=0 2024/10/25 08:55:50 {{ } { } [{{ } {mysql-data test-oadp-76 55c5fd2d-ea30-4bdc-bdf6-d6dbd13921c1 111070 0 2024-10-25 08:54:54 +0000 UTC map[app:mysql testlabel:selectors testlabel2:foo] map[pv.kubernetes.io/bind-completed:yes pv.kubernetes.io/bound-by-controller:yes volume.beta.kubernetes.io/storage-provisioner:ebs.csi.aws.com volume.kubernetes.io/selected-node:ip-10-0-65-50.us-east-2.compute.internal volume.kubernetes.io/storage-provisioner:ebs.csi.aws.com] [] [kubernetes.io/pvc-protection] [{OpenAPI-Generator Update v1 2024-10-25 08:54:54 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{".":{},"f:app":{},"f:testlabel":{},"f:testlabel2":{}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}}} } {kube-scheduler Update v1 2024-10-25 08:54:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volume.kubernetes.io/selected-node":{}}}} } {kube-controller-manager Update v1 2024-10-25 08:54:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:pv.kubernetes.io/bind-completed":{},"f:pv.kubernetes.io/bound-by-controller":{},"f:volume.beta.kubernetes.io/storage-provisioner":{},"f:volume.kubernetes.io/storage-provisioner":{}}},"f:spec":{"f:volumeName":{}}} } {kube-controller-manager Update v1 2024-10-25 08:54:58 +0000 UTC FieldsV1 {"f:status":{"f:accessModes":{},"f:capacity":{".":{},"f:storage":{}},"f:phase":{}}} status}]} {[ReadWriteOnce] nil {map[] map[storage:{{2147483648 0} {} 2Gi BinarySI}]} pvc-55c5fd2d-ea30-4bdc-bdf6-d6dbd13921c1 0xc0005c0bb0 0xc0005c0bc0 nil nil} {Bound [ReadWriteOnce] map[storage:{{2147483648 0} {} 2Gi BinarySI}] [] map[] }} {{ } {mysql-data1 test-oadp-76 41c81c1c-3898-4a67-b1eb-dceac2265f25 111066 0 2024-10-25 08:54:54 +0000 UTC map[app:mysql testlabel:selectors testlabel2:foo] map[pv.kubernetes.io/bind-completed:yes pv.kubernetes.io/bound-by-controller:yes volume.beta.kubernetes.io/storage-provisioner:ebs.csi.aws.com volume.kubernetes.io/selected-node:ip-10-0-65-50.us-east-2.compute.internal volume.kubernetes.io/storage-provisioner:ebs.csi.aws.com] [] [kubernetes.io/pvc-protection] [{OpenAPI-Generator Update v1 2024-10-25 08:54:54 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{".":{},"f:app":{},"f:testlabel":{},"f:testlabel2":{}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}}} } {kube-scheduler Update v1 2024-10-25 08:54:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volume.kubernetes.io/selected-node":{}}}} } {kube-controller-manager Update v1 2024-10-25 08:54:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:pv.kubernetes.io/bind-completed":{},"f:pv.kubernetes.io/bound-by-controller":{},"f:volume.beta.kubernetes.io/storage-provisioner":{},"f:volume.kubernetes.io/storage-provisioner":{}}},"f:spec":{"f:volumeName":{}}} } {kube-controller-manager Update v1 2024-10-25 08:54:58 +0000 UTC FieldsV1 {"f:status":{"f:accessModes":{},"f:capacity":{".":{},"f:storage":{}},"f:phase":{}}} status}]} {[ReadWriteOnce] nil {map[] map[storage:{{2147483648 0} {} 2Gi BinarySI}]} pvc-41c81c1c-3898-4a67-b1eb-dceac2265f25 0xc0005c0d10 0xc0005c0d20 nil nil} {Bound [ReadWriteOnce] map[storage:{{2147483648 0} {} 2Gi BinarySI}] [] map[] }}]} STEP: Creating backup mysql-c76cab85-92ae-11ef-8d75-0a580a81aa41 @ 10/25/24 08:55:50.948 2024/10/25 08:55:50 Wait until backup mysql-c76cab85-92ae-11ef-8d75-0a580a81aa41 is completed backup phase: Completed STEP: Verify backup mysql-c76cab85-92ae-11ef-8d75-0a580a81aa41 has completed successfully @ 10/25/24 08:56:11.001 2024/10/25 08:56:11 Backup for case mysql succeeded STEP: Delete the appplication resources mysql @ 10/25/24 08:56:11.097 STEP: Cleanup Application for case mysql @ 10/25/24 08:56:11.097 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Remove namespace test-oadp-76] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=11  changed=4  unreachable=0 failed=0 skipped=15  rescued=0 ignored=0 2024/10/25 08:56:25 2024-10-25 08:56:12,506 p=25126 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 08:56:12,506 p=25126 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:56:12,747 p=25126 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 08:56:12,747 p=25126 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:56:13,008 p=25126 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 08:56:13,009 p=25126 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:56:13,024 p=25126 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:56:13,024 p=25126 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:56:13,339 p=25126 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 08:56:13,339 p=25126 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:56:13,366 p=25126 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 08:56:13,367 p=25126 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:56:13,384 p=25126 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:56:13,384 p=25126 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:56:13,394 p=25126 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 08:56:13,909 p=25126 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 08:56:13,910 p=25126 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:56:24,748 p=25126 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Remove namespace test-oadp-76] *** 2024-10-25 08:56:24,748 p=25126 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 08:56:24,748 p=25126 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:56:24,991 p=25126 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 08:56:24,992 p=25126 u=1002480000 n=ansible | localhost : ok=11 changed=4 unreachable=0 failed=0 skipped=15 rescued=0 ignored=0 2024/10/25 08:56:25 Creating restore mysql-c76cab85-92ae-11ef-8d75-0a580a81aa41 for case mysql-c76cab85-92ae-11ef-8d75-0a580a81aa41 STEP: Create restore mysql-c76cab85-92ae-11ef-8d75-0a580a81aa41 from backup mysql-c76cab85-92ae-11ef-8d75-0a580a81aa41 @ 10/25/24 08:56:25.039 2024/10/25 08:56:25 Wait until restore mysql-c76cab85-92ae-11ef-8d75-0a580a81aa41 is complete restore phase: InProgress restore phase: InProgress restore phase: InProgress restore phase: InProgress restore phase: Completed STEP: Verify restore mysql-c76cab85-92ae-11ef-8d75-0a580a81aa41has completed successfully @ 10/25/24 08:57:05.16 STEP: Verify Application restore @ 10/25/24 08:57:05.176 STEP: Verify Application deployment for case mysql @ 10/25/24 08:57:05.176 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Check mysql pod status] *** ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Wait until service ready for connections] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Query the inserted data] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Validate test1 file has correct md5 hash] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=14  changed=6  unreachable=0 failed=0 skipped=12  rescued=0 ignored=0 2024/10/25 08:57:13 2024-10-25 08:57:07,397 p=25336 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 08:57:07,398 p=25336 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:57:07,901 p=25336 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 08:57:07,901 p=25336 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:57:08,359 p=25336 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 08:57:08,359 p=25336 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:57:08,385 p=25336 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:57:08,386 p=25336 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:57:08,868 p=25336 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 08:57:08,868 p=25336 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:57:08,928 p=25336 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 08:57:08,929 p=25336 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:57:08,959 p=25336 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:57:08,959 p=25336 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:57:08,978 p=25336 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 08:57:09,747 p=25336 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 08:57:09,748 p=25336 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:57:11,356 p=25336 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Check mysql pod status] *** 2024-10-25 08:57:11,356 p=25336 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 08:57:11,356 p=25336 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:57:11,932 p=25336 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Wait until service ready for connections] *** 2024-10-25 08:57:11,932 p=25336 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:57:12,674 p=25336 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Query the inserted data] *** 2024-10-25 08:57:12,675 p=25336 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:57:13,579 p=25336 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Validate test1 file has correct md5 hash] *** 2024-10-25 08:57:13,579 p=25336 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:57:13,601 p=25336 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 08:57:13,601 p=25336 u=1002480000 n=ansible | localhost : ok=14 changed=6 unreachable=0 failed=0 skipped=12 rescued=0 ignored=0 2024/10/25 08:57:13 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 2024/10/25 08:57:13 Cleaning app [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Remove namespace test-oadp-76] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=11  changed=4  unreachable=0 failed=0 skipped=15  rescued=0 ignored=0 2024/10/25 08:57:34 2024-10-25 08:57:15,856 p=25607 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 08:57:15,856 p=25607 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:57:16,222 p=25607 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 08:57:16,222 p=25607 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:57:16,519 p=25607 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 08:57:16,520 p=25607 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:57:16,536 p=25607 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:57:16,536 p=25607 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:57:16,923 p=25607 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 08:57:16,923 p=25607 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:57:16,958 p=25607 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 08:57:16,959 p=25607 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:57:16,986 p=25607 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:57:16,987 p=25607 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:57:17,003 p=25607 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 08:57:17,597 p=25607 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 08:57:17,597 p=25607 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:57:33,744 p=25607 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-mysql : Remove namespace test-oadp-76] *** 2024-10-25 08:57:33,744 p=25607 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 08:57:33,744 p=25607 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:57:34,028 p=25607 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 08:57:34,028 p=25607 u=1002480000 n=ansible | localhost : ok=11 changed=4 unreachable=0 failed=0 skipped=15 rescued=0 ignored=0 2024/10/25 08:57:34 Cleaning setup resources for the backup • [170.188 seconds] ------------------------------ S ------------------------------ Backup restore tests Application backup [tc-id:OADP-352][bug-id:OADP-1057][skip-disconnected][interop] Django application with VSL [vsl] /alabama/cspi/e2e/cloudstorage/app_backup_vsl.go:70 2024/10/25 08:57:34 Check if VSL custom credentials exist 2024/10/25 08:57:34 Check if the cloud provider is AWS 2024/10/25 08:57:34 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 10/25/24 08:57:34.499 2024/10/25 08:57:34 vsl 2024/10/25 08:57:34 Check if VSL custom credentials exist 2024/10/25 08:57:34 Check if the cloud provider is AWS 2024/10/25 08:57:34 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "bd5cd809-a21c-4680-bb5a-cac436c8594a", "resourceVersion": "112995", "generation": 1, "creationTimestamp": "2024-10-25T08:57:34Z", "managedFields": [ { "manager": "cloudstorage.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2024-10-25T08:57:34Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {} } }, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "bucket": { "cloudStorageRef": { "name": "ci-op-jdjixbyi-interopoadp" }, "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "credentials" }, "default": true, "prefix": "velero-e2e-1981d203-92ad-11ef-8d75-0a580a81aa41" } } ], "snapshotLocations": [ { "velero": { "provider": "aws", "config": { "credentialsFile": "/tmp/credentials/openshift-adp/cloud-credentials-credentials", "enableSharedConfig": "true", "profile": "default", "region": "us-east-2" } } } ], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt" ] } }, "features": null }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 10/25/24 08:57:34.598 2024/10/25 08:57:34 Waiting for velero pod to be running 2024/10/25 08:57:34 pod: velero-56684fc7-cm94l is not yet running with status: {Succeeded [{PodReadyToStartContainers False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:57:34 +0000 UTC } {Initialized True 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:55:17 +0000 UTC PodCompleted } {Ready False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:57:34 +0000 UTC PodCompleted } {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:57:34 +0000 UTC PodCompleted } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:54:44 +0000 UTC }] 10.0.37.172 10.129.2.15 [{10.129.2.15}] 2024-10-25 08:54:44 +0000 UTC [{openshift-velero-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2024-10-25 08:55:07 +0000 UTC,FinishedAt:2024-10-25 08:55:07 +0000 UTC,ContainerID:cri-o://42d69849e17b8c356cd127687b9309a44fc490b271ad5ae97453e45386884c2d,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:a8725b48079a61e393583a3e8f0c9c70da31b08886cc9a8d115884886d623ff0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:3b66ca3d09fc4578af372c19c675c284d0eebef3daf0ef1bf748e76fa1526a08 cri-o://42d69849e17b8c356cd127687b9309a44fc490b271ad5ae97453e45386884c2d 0xc000f32d09} {velero-plugin-for-aws {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2024-10-25 08:55:13 +0000 UTC,FinishedAt:2024-10-25 08:55:13 +0000 UTC,ContainerID:cri-o://7ba8e613bc1e7e0ef75a8d33e5e487edb3684ebaa129f9f30d686e208eada616,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:55aec191293ea79b76761161ddbb9db1a68d0f98c531ea97ce543dca2e9de967 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:55aec191293ea79b76761161ddbb9db1a68d0f98c531ea97ce543dca2e9de967 cri-o://7ba8e613bc1e7e0ef75a8d33e5e487edb3684ebaa129f9f30d686e208eada616 0xc000f32d19} {kubevirt-velero-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2024-10-25 08:55:17 +0000 UTC,FinishedAt:2024-10-25 08:55:17 +0000 UTC,ContainerID:cri-o://6d6982c65cf7c9983ab2d79b3132d2798227c686d100f9750019c8072bce9141,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:342dc1fcb6a8b5f01cafa01f6c4849888a6b3f78c3e5fa94a55cdc7566e088b1 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:342dc1fcb6a8b5f01cafa01f6c4849888a6b3f78c3e5fa94a55cdc7566e088b1 cri-o://6d6982c65cf7c9983ab2d79b3132d2798227c686d100f9750019c8072bce9141 0xc000f32d29}] [{velero {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2024-10-25 08:55:23 +0000 UTC,FinishedAt:2024-10-25 08:57:34 +0000 UTC,ContainerID:cri-o://accb6085dfe95386359e6936ca323a2a52362e865b00268890693bb160dab5fc,}} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-rhel9@sha256:cf9ebc90a72bf5043ef275babb19a4eebe830db9d7194a3e973ea64910f1ae29 registry.redhat.io/oadp/oadp-velero-rhel9@sha256:cf9ebc90a72bf5043ef275babb19a4eebe830db9d7194a3e973ea64910f1ae29 cri-o://accb6085dfe95386359e6936ca323a2a52362e865b00268890693bb160dab5fc 0xc000f32d39}] Burstable []} 2024/10/25 08:57:39 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' STEP: Prepare backup resources, depending on the volumes backup type @ 10/25/24 08:57:39.694 2024/10/25 08:57:39 Checking default storage class count STEP: Installing application for case django-persistent @ 10/25/24 08:57:39.725 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Check namespace test-oadp-352] *** ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Create namespace test-oadp-352] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Create the mtc test django psql persistent template] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Create openshift django psql persistent application from openshift templates] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=14  changed=6  unreachable=0 failed=0 skipped=9  rescued=0 ignored=0 2024/10/25 08:57:45 2024-10-25 08:57:41,196 p=25806 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 08:57:41,196 p=25806 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:57:41,437 p=25806 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 08:57:41,438 p=25806 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:57:41,686 p=25806 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 08:57:41,686 p=25806 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:57:41,701 p=25806 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:57:41,702 p=25806 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:57:42,006 p=25806 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 08:57:42,006 p=25806 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:57:42,034 p=25806 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 08:57:42,034 p=25806 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:57:42,056 p=25806 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:57:42,056 p=25806 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:57:42,073 p=25806 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 08:57:42,631 p=25806 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 08:57:42,631 p=25806 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:57:43,440 p=25806 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Check namespace test-oadp-352] *** 2024-10-25 08:57:43,441 p=25806 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 08:57:43,441 p=25806 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:57:43,867 p=25806 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Create namespace test-oadp-352] *** 2024-10-25 08:57:43,867 p=25806 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:57:44,831 p=25806 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Create the mtc test django psql persistent template] *** 2024-10-25 08:57:44,831 p=25806 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:57:45,489 p=25806 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Create openshift django psql persistent application from openshift templates] *** 2024-10-25 08:57:45,489 p=25806 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:57:45,639 p=25806 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 08:57:45,640 p=25806 u=1002480000 n=ansible | localhost : ok=14 changed=6 unreachable=0 failed=0 skipped=9 rescued=0 ignored=0 STEP: Verify Application deployment @ 10/25/24 08:57:45.688 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. FAILED - RETRYING: [localhost]: Check postgresql pod status (30 retries left). FAILED - RETRYING: [localhost]: Check postgresql pod status (29 retries left). FAILED - RETRYING: [localhost]: Check postgresql pod status (28 retries left). FAILED - RETRYING: [localhost]: Check postgresql pod status (27 retries left). FAILED - RETRYING: [localhost]: Check postgresql pod status (26 retries left). FAILED - RETRYING: [localhost]: Check postgresql pod status (25 retries left). TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Check postgresql pod status] *** ok: [localhost] FAILED - RETRYING: [localhost]: Check application pod status (30 retries left). FAILED - RETRYING: [localhost]: Check application pod status (29 retries left). FAILED - RETRYING: [localhost]: Check application pod status (28 retries left). FAILED - RETRYING: [localhost]: Check application pod status (27 retries left). TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Check application pod status] *** ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Get route] *** ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Access the html file] *** ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : set_fact] *** ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Get num visits up to now] *** ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Print num of visits] *** ok: [localhost] => {  "msg": "PASS: # of visits should be 1; actual 1" } PLAY RECAP ********************************************************************* localhost : ok=17  changed=3  unreachable=0 failed=0 skipped=6  rescued=0 ignored=0 2024/10/25 08:58:51 2024-10-25 08:57:47,239 p=26090 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 08:57:47,239 p=26090 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:57:47,486 p=26090 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 08:57:47,487 p=26090 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:57:47,758 p=26090 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 08:57:47,758 p=26090 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:57:47,774 p=26090 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:57:47,774 p=26090 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:57:48,070 p=26090 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 08:57:48,070 p=26090 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:57:48,102 p=26090 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 08:57:48,102 p=26090 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:57:48,123 p=26090 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:57:48,124 p=26090 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:57:48,134 p=26090 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 08:57:48,679 p=26090 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 08:57:48,679 p=26090 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:58:24,286 p=26090 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Check postgresql pod status] *** 2024-10-25 08:58:24,286 p=26090 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 08:58:24,286 p=26090 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:58:48,547 p=26090 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Check application pod status] *** 2024-10-25 08:58:48,548 p=26090 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:58:50,303 p=26090 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Get route] *** 2024-10-25 08:58:50,303 p=26090 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:58:50,741 p=26090 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Access the html file] *** 2024-10-25 08:58:50,741 p=26090 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:58:50,765 p=26090 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : set_fact] *** 2024-10-25 08:58:50,766 p=26090 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:58:51,076 p=26090 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Get num visits up to now] *** 2024-10-25 08:58:51,077 p=26090 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:58:51,117 p=26090 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Print num of visits] *** 2024-10-25 08:58:51,117 p=26090 u=1002480000 n=ansible | ok: [localhost] => { "msg": "PASS: # of visits should be 1; actual 1" } 2024-10-25 08:58:51,129 p=26090 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 08:58:51,129 p=26090 u=1002480000 n=ansible | localhost : ok=17 changed=3 unreachable=0 failed=0 skipped=6 rescued=0 ignored=0 2024/10/25 08:58:51 {{ } { } [{{ } {postgresql test-oadp-352 9ea002bd-8d47-4b12-870e-5808914ed32b 113367 0 2024-10-25 08:57:45 +0000 UTC map[app:django-psql-persistent template:django-psql-persistent] map[openshift.io/generated-by:OpenShiftNewApp pv.kubernetes.io/bind-completed:yes pv.kubernetes.io/bound-by-controller:yes volume.beta.kubernetes.io/storage-provisioner:ebs.csi.aws.com volume.kubernetes.io/selected-node:ip-10-0-65-50.us-east-2.compute.internal volume.kubernetes.io/storage-provisioner:ebs.csi.aws.com] [] [kubernetes.io/pvc-protection] [{oc Update v1 2024-10-25 08:57:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:openshift.io/generated-by":{}},"f:labels":{".":{},"f:app":{},"f:template":{}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}}} } {kube-scheduler Update v1 2024-10-25 08:57:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:volume.kubernetes.io/selected-node":{}}}} } {kube-controller-manager Update v1 2024-10-25 08:57:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:pv.kubernetes.io/bind-completed":{},"f:pv.kubernetes.io/bound-by-controller":{},"f:volume.beta.kubernetes.io/storage-provisioner":{},"f:volume.kubernetes.io/storage-provisioner":{}}},"f:spec":{"f:volumeName":{}}} } {kube-controller-manager Update v1 2024-10-25 08:57:50 +0000 UTC FieldsV1 {"f:status":{"f:accessModes":{},"f:capacity":{".":{},"f:storage":{}},"f:phase":{}}} status}]} {[ReadWriteOnce] nil {map[] map[storage:{{1073741824 0} {} 1Gi BinarySI}]} pvc-9ea002bd-8d47-4b12-870e-5808914ed32b 0xc0014eb230 0xc0014eb250 nil nil} {Bound [ReadWriteOnce] map[storage:{{1073741824 0} {} 1Gi BinarySI}] [] map[] }}]} STEP: Creating backup django-persistent-2d0f5328-92af-11ef-8d75-0a580a81aa41 @ 10/25/24 08:58:51.194 2024/10/25 08:58:51 Wait until backup django-persistent-2d0f5328-92af-11ef-8d75-0a580a81aa41 is completed backup phase: Completed STEP: Verify backup django-persistent-2d0f5328-92af-11ef-8d75-0a580a81aa41 has completed successfully @ 10/25/24 08:59:11.271 2024/10/25 08:59:11 Backup for case django-persistent succeeded STEP: Delete the appplication resources django-persistent @ 10/25/24 08:59:11.381 STEP: Cleanup Application for case django-persistent @ 10/25/24 08:59:11.381 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Remove namespace test-oadp-352] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=11  changed=4  unreachable=0 failed=0 skipped=12  rescued=0 ignored=0 2024/10/25 08:59:30 2024-10-25 08:59:12,810 p=26479 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 08:59:12,810 p=26479 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:59:13,057 p=26479 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 08:59:13,057 p=26479 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:59:13,316 p=26479 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 08:59:13,316 p=26479 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:59:13,332 p=26479 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:59:13,332 p=26479 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:59:13,630 p=26479 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 08:59:13,630 p=26479 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:59:13,655 p=26479 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 08:59:13,656 p=26479 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:59:13,673 p=26479 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 08:59:13,673 p=26479 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:59:13,683 p=26479 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 08:59:14,226 p=26479 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 08:59:14,227 p=26479 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 08:59:30,138 p=26479 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Remove namespace test-oadp-352] *** 2024-10-25 08:59:30,138 p=26479 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 08:59:30,138 p=26479 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 08:59:30,332 p=26479 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 08:59:30,332 p=26479 u=1002480000 n=ansible | localhost : ok=11 changed=4 unreachable=0 failed=0 skipped=12 rescued=0 ignored=0 2024/10/25 08:59:30 Creating restore django-persistent-2d0f5328-92af-11ef-8d75-0a580a81aa41 for case django-persistent-2d0f5328-92af-11ef-8d75-0a580a81aa41 STEP: Create restore django-persistent-2d0f5328-92af-11ef-8d75-0a580a81aa41 from backup django-persistent-2d0f5328-92af-11ef-8d75-0a580a81aa41 @ 10/25/24 08:59:30.373 2024/10/25 08:59:30 Wait until restore django-persistent-2d0f5328-92af-11ef-8d75-0a580a81aa41 is complete restore phase: InProgress restore phase: InProgress restore phase: Completed STEP: Verify restore django-persistent-2d0f5328-92af-11ef-8d75-0a580a81aa41has completed successfully @ 10/25/24 09:00:00.492 STEP: Verify Application restore @ 10/25/24 09:00:00.508 STEP: Verify Application deployment for case django-persistent @ 10/25/24 09:00:00.508 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. FAILED - RETRYING: [localhost]: Check postgresql pod status (30 retries left). FAILED - RETRYING: [localhost]: Check postgresql pod status (29 retries left). TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Check postgresql pod status] *** ok: [localhost] FAILED - RETRYING: [localhost]: Check application pod status (30 retries left). FAILED - RETRYING: [localhost]: Check application pod status (29 retries left). FAILED - RETRYING: [localhost]: Check application pod status (28 retries left). FAILED - RETRYING: [localhost]: Check application pod status (27 retries left). FAILED - RETRYING: [localhost]: Check application pod status (26 retries left). FAILED - RETRYING: [localhost]: Check application pod status (25 retries left). TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Check application pod status] *** ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Get route] *** ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Access the html file] *** ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : set_fact] *** ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Get num visits up to now] *** ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Print num of visits] *** ok: [localhost] => {  "msg": "PASS: # of visits should be 2; actual 2" } PLAY RECAP ********************************************************************* localhost : ok=17  changed=3  unreachable=0 failed=0 skipped=6  rescued=0 ignored=0 2024/10/25 09:00:54 2024-10-25 09:00:02,523 p=26683 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 09:00:02,523 p=26683 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:00:02,766 p=26683 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 09:00:02,766 p=26683 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:00:03,014 p=26683 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 09:00:03,014 p=26683 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:00:03,031 p=26683 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:00:03,031 p=26683 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:00:03,367 p=26683 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 09:00:03,368 p=26683 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:00:03,397 p=26683 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 09:00:03,398 p=26683 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:00:03,417 p=26683 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:00:03,417 p=26683 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:00:03,428 p=26683 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 09:00:03,991 p=26683 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 09:00:03,991 p=26683 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:00:16,923 p=26683 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Check postgresql pod status] *** 2024-10-25 09:00:16,923 p=26683 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 09:00:16,923 p=26683 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:00:51,781 p=26683 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Check application pod status] *** 2024-10-25 09:00:51,781 p=26683 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:00:53,670 p=26683 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Get route] *** 2024-10-25 09:00:53,671 p=26683 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:00:54,184 p=26683 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Access the html file] *** 2024-10-25 09:00:54,184 p=26683 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:00:54,209 p=26683 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : set_fact] *** 2024-10-25 09:00:54,210 p=26683 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:00:54,545 p=26683 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Get num visits up to now] *** 2024-10-25 09:00:54,546 p=26683 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:00:54,584 p=26683 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Print num of visits] *** 2024-10-25 09:00:54,584 p=26683 u=1002480000 n=ansible | ok: [localhost] => { "msg": "PASS: # of visits should be 2; actual 2" } 2024-10-25 09:00:54,598 p=26683 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 09:00:54,598 p=26683 u=1002480000 n=ansible | localhost : ok=17 changed=3 unreachable=0 failed=0 skipped=6 rescued=0 ignored=0 2024/10/25 09:00:54 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 2024/10/25 09:00:54 Reset number of visits to 0 2024/10/25 09:00:54 Cleaning app [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Remove namespace test-oadp-352] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=11  changed=4  unreachable=0 failed=0 skipped=12  rescued=0 ignored=0 2024/10/25 09:01:14 2024-10-25 09:00:56,248 p=27047 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 09:00:56,248 p=27047 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:00:56,523 p=27047 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 09:00:56,523 p=27047 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:00:56,805 p=27047 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 09:00:56,806 p=27047 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:00:56,824 p=27047 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:00:56,824 p=27047 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:00:57,154 p=27047 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 09:00:57,155 p=27047 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:00:57,190 p=27047 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 09:00:57,190 p=27047 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:00:57,211 p=27047 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:00:57,212 p=27047 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:00:57,226 p=27047 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 09:00:57,781 p=27047 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 09:00:57,782 p=27047 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:01:13,944 p=27047 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-django : Remove namespace test-oadp-352] *** 2024-10-25 09:01:13,944 p=27047 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 09:01:13,944 p=27047 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:01:14,240 p=27047 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 09:01:14,240 p=27047 u=1002480000 n=ansible | localhost : ok=11 changed=4 unreachable=0 failed=0 skipped=12 rescued=0 ignored=0 2024/10/25 09:01:14 Cleaning setup resources for the backup • [220.221 seconds] ------------------------------ Backup restore tests Application backup [tc-id:OADP-353] [bug-id:OADP-1057][interop] [smoke] ImageStream application with VSL [vsl] /alabama/cspi/e2e/cloudstorage/app_backup_vsl.go:88 2024/10/25 09:01:14 Check if VSL custom credentials exist 2024/10/25 09:01:14 Check if the cloud provider is AWS 2024/10/25 09:01:14 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 10/25/24 09:01:14.752 2024/10/25 09:01:14 vsl 2024/10/25 09:01:14 Check if VSL custom credentials exist 2024/10/25 09:01:14 Check if the cloud provider is AWS 2024/10/25 09:01:14 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "61f7fa5d-a48c-4c9f-b22b-c1d209944746", "resourceVersion": "117540", "generation": 1, "creationTimestamp": "2024-10-25T09:01:14Z", "managedFields": [ { "manager": "cloudstorage.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2024-10-25T09:01:14Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {} } }, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "bucket": { "cloudStorageRef": { "name": "ci-op-jdjixbyi-interopoadp" }, "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "credentials" }, "default": true, "prefix": "velero-e2e-1981d203-92ad-11ef-8d75-0a580a81aa41" } } ], "snapshotLocations": [ { "velero": { "provider": "aws", "config": { "credentialsFile": "/tmp/credentials/openshift-adp/cloud-credentials-credentials", "enableSharedConfig": "true", "profile": "default", "region": "us-east-2" } } } ], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt" ] } }, "features": null }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 10/25/24 09:01:14.852 2024/10/25 09:01:14 Waiting for velero pod to be running 2024/10/25 09:01:14 pod: velero-56684fc7-xmfpd is not yet running with status: {Succeeded [{PodReadyToStartContainers False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 09:01:14 +0000 UTC } {Initialized True 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:57:37 +0000 UTC PodCompleted } {Ready False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 09:01:14 +0000 UTC PodCompleted } {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2024-10-25 09:01:14 +0000 UTC PodCompleted } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2024-10-25 08:57:34 +0000 UTC }] 10.0.65.50 10.128.2.17 [{10.128.2.17}] 2024-10-25 08:57:34 +0000 UTC [{openshift-velero-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2024-10-25 08:57:35 +0000 UTC,FinishedAt:2024-10-25 08:57:35 +0000 UTC,ContainerID:cri-o://24c3ee7f0d32d8e371296ad404c11af0f491ba9127de3d75f91c97bd39abb2a4,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:a8725b48079a61e393583a3e8f0c9c70da31b08886cc9a8d115884886d623ff0 registry.redhat.io/oadp/oadp-velero-plugin-rhel9@sha256:3b66ca3d09fc4578af372c19c675c284d0eebef3daf0ef1bf748e76fa1526a08 cri-o://24c3ee7f0d32d8e371296ad404c11af0f491ba9127de3d75f91c97bd39abb2a4 0xc000135f39} {velero-plugin-for-aws {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2024-10-25 08:57:35 +0000 UTC,FinishedAt:2024-10-25 08:57:35 +0000 UTC,ContainerID:cri-o://ed941333cebcc9c6eac51daf2a6f1a5a5ed1350baffef7aa754840c4df8730a0,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:55aec191293ea79b76761161ddbb9db1a68d0f98c531ea97ce543dca2e9de967 registry.redhat.io/oadp/oadp-velero-plugin-for-aws-rhel9@sha256:55aec191293ea79b76761161ddbb9db1a68d0f98c531ea97ce543dca2e9de967 cri-o://ed941333cebcc9c6eac51daf2a6f1a5a5ed1350baffef7aa754840c4df8730a0 0xc000135f79} {kubevirt-velero-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2024-10-25 08:57:36 +0000 UTC,FinishedAt:2024-10-25 08:57:36 +0000 UTC,ContainerID:cri-o://697104b9c929103b6a938661a9afb6fd12275584586921ca24593e193c04f967,}} {nil nil nil} true 0 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:342dc1fcb6a8b5f01cafa01f6c4849888a6b3f78c3e5fa94a55cdc7566e088b1 registry.redhat.io/oadp/oadp-kubevirt-velero-plugin-rhel9@sha256:342dc1fcb6a8b5f01cafa01f6c4849888a6b3f78c3e5fa94a55cdc7566e088b1 cri-o://697104b9c929103b6a938661a9afb6fd12275584586921ca24593e193c04f967 0xc000135f89}] [{velero {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2024-10-25 08:57:37 +0000 UTC,FinishedAt:2024-10-25 09:01:14 +0000 UTC,ContainerID:cri-o://1dd0e2015d9c4c1ef7c200f9920243397f4e90b5a974e860a54df71884f10fc7,}} {nil nil nil} false 0 registry.redhat.io/oadp/oadp-velero-rhel9@sha256:cf9ebc90a72bf5043ef275babb19a4eebe830db9d7194a3e973ea64910f1ae29 registry.redhat.io/oadp/oadp-velero-rhel9@sha256:cf9ebc90a72bf5043ef275babb19a4eebe830db9d7194a3e973ea64910f1ae29 cri-o://1dd0e2015d9c4c1ef7c200f9920243397f4e90b5a974e860a54df71884f10fc7 0xc000135fc9}] Burstable []} 2024/10/25 09:01:19 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' STEP: Prepare backup resources, depending on the volumes backup type @ 10/25/24 09:01:19.92 2024/10/25 09:01:19 Checking default storage class count STEP: Installing application for case imagestream-e2e @ 10/25/24 09:01:19.952 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Check namespace] *** ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Create namespace] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Import image to imagestream] *** changed: [localhost] => (item=oc --server https://api.ci-rosa-s-916u.wmvn.s1.devshift.org:6443 --token sha256~LrRzbl9u7Blq4ytlyvo0RCPGdwweAqpHCC1KdLwMcLY -n test-oadp-353 import-image internal-image:alp --from quay.io/migqe/alpine:3.16 --confirm) changed: [localhost] => (item=oc --server https://api.ci-rosa-s-916u.wmvn.s1.devshift.org:6443 --token sha256~LrRzbl9u7Blq4ytlyvo0RCPGdwweAqpHCC1KdLwMcLY -n test-oadp-353 import-image internal-image:busy --from quay.io/migqe/busybox:1.35 --confirm) TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Tag the imported images] *** changed: [localhost] => (item=oc --server https://api.ci-rosa-s-916u.wmvn.s1.devshift.org:6443 --token sha256~LrRzbl9u7Blq4ytlyvo0RCPGdwweAqpHCC1KdLwMcLY -n test-oadp-353 tag internal-image:alp internal-image:tag1 --alias=False) changed: [localhost] => (item=oc --server https://api.ci-rosa-s-916u.wmvn.s1.devshift.org:6443 --token sha256~LrRzbl9u7Blq4ytlyvo0RCPGdwweAqpHCC1KdLwMcLY -n test-oadp-353 tag internal-image:alp internal-image:tag2 --alias=True) changed: [localhost] => (item=oc --server https://api.ci-rosa-s-916u.wmvn.s1.devshift.org:6443 --token sha256~LrRzbl9u7Blq4ytlyvo0RCPGdwweAqpHCC1KdLwMcLY -n test-oadp-353 tag internal-image:busy internal-image:tag3 --alias=False) changed: [localhost] => (item=oc --server https://api.ci-rosa-s-916u.wmvn.s1.devshift.org:6443 --token sha256~LrRzbl9u7Blq4ytlyvo0RCPGdwweAqpHCC1KdLwMcLY -n test-oadp-353 tag internal-image:busy internal-image:latest --alias=False) changed: [localhost] => (item=oc --server https://api.ci-rosa-s-916u.wmvn.s1.devshift.org:6443 --token sha256~LrRzbl9u7Blq4ytlyvo0RCPGdwweAqpHCC1KdLwMcLY -n test-oadp-353 tag internal-image:busy internal-image:tag4 --alias=True) TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Create test pods to pull images] *** changed: [localhost] => (item=Create pod for image image-registry.openshift-image-registry.svc:5000/test-oadp-353/internal-image:alp) changed: [localhost] => (item=Create pod for image image-registry.openshift-image-registry.svc:5000/test-oadp-353/internal-image:busy) TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Create test pods to pull extra tags] *** changed: [localhost] => (item=Create pod for image image-registry.openshift-image-registry.svc:5000/test-oadp-353/internal-image:tag1) changed: [localhost] => (item=Create pod for image image-registry.openshift-image-registry.svc:5000/test-oadp-353/internal-image:tag2) changed: [localhost] => (item=Create pod for image image-registry.openshift-image-registry.svc:5000/test-oadp-353/internal-image:tag3) changed: [localhost] => (item=Create pod for image image-registry.openshift-image-registry.svc:5000/test-oadp-353/internal-image:latest) changed: [localhost] => (item=Create pod for image image-registry.openshift-image-registry.svc:5000/test-oadp-353/internal-image:tag4) PLAY RECAP ********************************************************************* localhost : ok=16  changed=8  unreachable=0 failed=0 skipped=15  rescued=0 ignored=0 2024/10/25 09:01:37 2024-10-25 09:01:21,841 p=27251 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 09:01:21,842 p=27251 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:01:22,155 p=27251 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 09:01:22,155 p=27251 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:01:22,450 p=27251 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 09:01:22,450 p=27251 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:01:22,469 p=27251 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:01:22,469 p=27251 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:01:22,835 p=27251 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 09:01:22,835 p=27251 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:01:22,867 p=27251 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 09:01:22,868 p=27251 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:01:22,891 p=27251 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:01:22,892 p=27251 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:01:22,904 p=27251 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 09:01:23,538 p=27251 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 09:01:23,538 p=27251 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:01:24,600 p=27251 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Check namespace] *** 2024-10-25 09:01:24,601 p=27251 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 09:01:24,601 p=27251 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:01:25,025 p=27251 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Create namespace] *** 2024-10-25 09:01:25,025 p=27251 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:01:29,168 p=27251 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Import image to imagestream] *** 2024-10-25 09:01:29,169 p=27251 u=1002480000 n=ansible | changed: [localhost] => (item=oc --server https://api.ci-rosa-s-916u.wmvn.s1.devshift.org:6443 --token sha256~LrRzbl9u7Blq4ytlyvo0RCPGdwweAqpHCC1KdLwMcLY -n test-oadp-353 import-image internal-image:alp --from quay.io/migqe/alpine:3.16 --confirm) 2024-10-25 09:01:30,193 p=27251 u=1002480000 n=ansible | changed: [localhost] => (item=oc --server https://api.ci-rosa-s-916u.wmvn.s1.devshift.org:6443 --token sha256~LrRzbl9u7Blq4ytlyvo0RCPGdwweAqpHCC1KdLwMcLY -n test-oadp-353 import-image internal-image:busy --from quay.io/migqe/busybox:1.35 --confirm) 2024-10-25 09:01:30,570 p=27251 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Tag the imported images] *** 2024-10-25 09:01:30,571 p=27251 u=1002480000 n=ansible | changed: [localhost] => (item=oc --server https://api.ci-rosa-s-916u.wmvn.s1.devshift.org:6443 --token sha256~LrRzbl9u7Blq4ytlyvo0RCPGdwweAqpHCC1KdLwMcLY -n test-oadp-353 tag internal-image:alp internal-image:tag1 --alias=False) 2024-10-25 09:01:30,884 p=27251 u=1002480000 n=ansible | changed: [localhost] => (item=oc --server https://api.ci-rosa-s-916u.wmvn.s1.devshift.org:6443 --token sha256~LrRzbl9u7Blq4ytlyvo0RCPGdwweAqpHCC1KdLwMcLY -n test-oadp-353 tag internal-image:alp internal-image:tag2 --alias=True) 2024-10-25 09:01:31,225 p=27251 u=1002480000 n=ansible | changed: [localhost] => (item=oc --server https://api.ci-rosa-s-916u.wmvn.s1.devshift.org:6443 --token sha256~LrRzbl9u7Blq4ytlyvo0RCPGdwweAqpHCC1KdLwMcLY -n test-oadp-353 tag internal-image:busy internal-image:tag3 --alias=False) 2024-10-25 09:01:31,548 p=27251 u=1002480000 n=ansible | changed: [localhost] => (item=oc --server https://api.ci-rosa-s-916u.wmvn.s1.devshift.org:6443 --token sha256~LrRzbl9u7Blq4ytlyvo0RCPGdwweAqpHCC1KdLwMcLY -n test-oadp-353 tag internal-image:busy internal-image:latest --alias=False) 2024-10-25 09:01:31,850 p=27251 u=1002480000 n=ansible | changed: [localhost] => (item=oc --server https://api.ci-rosa-s-916u.wmvn.s1.devshift.org:6443 --token sha256~LrRzbl9u7Blq4ytlyvo0RCPGdwweAqpHCC1KdLwMcLY -n test-oadp-353 tag internal-image:busy internal-image:tag4 --alias=True) 2024-10-25 09:01:32,730 p=27251 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Create test pods to pull images] *** 2024-10-25 09:01:32,730 p=27251 u=1002480000 n=ansible | changed: [localhost] => (item=Create pod for image image-registry.openshift-image-registry.svc:5000/test-oadp-353/internal-image:alp) 2024-10-25 09:01:33,422 p=27251 u=1002480000 n=ansible | changed: [localhost] => (item=Create pod for image image-registry.openshift-image-registry.svc:5000/test-oadp-353/internal-image:busy) 2024-10-25 09:01:34,112 p=27251 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Create test pods to pull extra tags] *** 2024-10-25 09:01:34,113 p=27251 u=1002480000 n=ansible | changed: [localhost] => (item=Create pod for image image-registry.openshift-image-registry.svc:5000/test-oadp-353/internal-image:tag1) 2024-10-25 09:01:34,870 p=27251 u=1002480000 n=ansible | changed: [localhost] => (item=Create pod for image image-registry.openshift-image-registry.svc:5000/test-oadp-353/internal-image:tag2) 2024-10-25 09:01:35,563 p=27251 u=1002480000 n=ansible | changed: [localhost] => (item=Create pod for image image-registry.openshift-image-registry.svc:5000/test-oadp-353/internal-image:tag3) 2024-10-25 09:01:36,296 p=27251 u=1002480000 n=ansible | changed: [localhost] => (item=Create pod for image image-registry.openshift-image-registry.svc:5000/test-oadp-353/internal-image:latest) 2024-10-25 09:01:36,943 p=27251 u=1002480000 n=ansible | changed: [localhost] => (item=Create pod for image image-registry.openshift-image-registry.svc:5000/test-oadp-353/internal-image:tag4) 2024-10-25 09:01:37,063 p=27251 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 09:01:37,063 p=27251 u=1002480000 n=ansible | localhost : ok=16 changed=8 unreachable=0 failed=0 skipped=15 rescued=0 ignored=0 STEP: Verify Application deployment @ 10/25/24 09:01:37.112 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Check image streams] *** ok: [localhost] => (item=Check that imagestream internal-image exists) ok: [localhost] => (item=Check that imagestream internal-image exists) TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Check initial tags] *** ok: [localhost] => (item=Check that imagestream tag internal-image:alp exists) ok: [localhost] => (item=Check that imagestream tag internal-image:busy exists) TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Check extra tags] *** ok: [localhost] => (item=Check that imagestream tag internal-image:tag1 exists) ok: [localhost] => (item=Check that imagestream tag internal-image:tag2 exists) ok: [localhost] => (item=Check that imagestream tag internal-image:tag3 exists) ok: [localhost] => (item=Check that imagestream tag internal-image:latest exists) ok: [localhost] => (item=Check that imagestream tag internal-image:tag4 exists) TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Check pods are running] *** ok: [localhost] PLAY RECAP ********************************************************************* localhost : ok=14  changed=3  unreachable=0 failed=0 skipped=17  rescued=0 ignored=0 2024/10/25 09:01:56 2024-10-25 09:01:38,564 p=27804 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 09:01:38,564 p=27804 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:01:38,793 p=27804 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 09:01:38,793 p=27804 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:01:39,027 p=27804 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 09:01:39,027 p=27804 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:01:39,043 p=27804 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:01:39,043 p=27804 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:01:39,342 p=27804 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 09:01:39,342 p=27804 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:01:39,372 p=27804 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 09:01:39,372 p=27804 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:01:39,394 p=27804 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:01:39,394 p=27804 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:01:39,406 p=27804 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 09:01:39,929 p=27804 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 09:01:39,929 p=27804 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:01:42,241 p=27804 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Check image streams] *** 2024-10-25 09:01:42,241 p=27804 u=1002480000 n=ansible | ok: [localhost] => (item=Check that imagestream internal-image exists) 2024-10-25 09:01:43,839 p=27804 u=1002480000 n=ansible | ok: [localhost] => (item=Check that imagestream internal-image exists) 2024-10-25 09:01:43,842 p=27804 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 09:01:45,448 p=27804 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Check initial tags] *** 2024-10-25 09:01:45,448 p=27804 u=1002480000 n=ansible | ok: [localhost] => (item=Check that imagestream tag internal-image:alp exists) 2024-10-25 09:01:47,175 p=27804 u=1002480000 n=ansible | ok: [localhost] => (item=Check that imagestream tag internal-image:busy exists) 2024-10-25 09:01:49,059 p=27804 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Check extra tags] *** 2024-10-25 09:01:49,060 p=27804 u=1002480000 n=ansible | ok: [localhost] => (item=Check that imagestream tag internal-image:tag1 exists) 2024-10-25 09:01:50,779 p=27804 u=1002480000 n=ansible | ok: [localhost] => (item=Check that imagestream tag internal-image:tag2 exists) 2024-10-25 09:01:52,508 p=27804 u=1002480000 n=ansible | ok: [localhost] => (item=Check that imagestream tag internal-image:tag3 exists) 2024-10-25 09:01:54,329 p=27804 u=1002480000 n=ansible | ok: [localhost] => (item=Check that imagestream tag internal-image:latest exists) 2024-10-25 09:01:56,017 p=27804 u=1002480000 n=ansible | ok: [localhost] => (item=Check that imagestream tag internal-image:tag4 exists) 2024-10-25 09:01:56,756 p=27804 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Check pods are running] *** 2024-10-25 09:01:56,756 p=27804 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:01:56,771 p=27804 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 09:01:56,772 p=27804 u=1002480000 n=ansible | localhost : ok=14 changed=3 unreachable=0 failed=0 skipped=17 rescued=0 ignored=0 2024/10/25 09:01:56 {{ } { } []} STEP: Creating backup imagestream-e2e-b0573c77-92af-11ef-8d75-0a580a81aa41 @ 10/25/24 09:01:56.849 2024/10/25 09:01:56 Wait until backup imagestream-e2e-b0573c77-92af-11ef-8d75-0a580a81aa41 is completed backup phase: Completed STEP: Verify backup imagestream-e2e-b0573c77-92af-11ef-8d75-0a580a81aa41 has completed successfully @ 10/25/24 09:02:16.904 2024/10/25 09:02:16 Backup for case imagestream-e2e succeeded STEP: Delete the appplication resources imagestream-e2e @ 10/25/24 09:02:16.999 STEP: Cleanup Application for case imagestream-e2e @ 10/25/24 09:02:16.999 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Remove namespace test-oadp-353] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Remove namespace mirror-helper] *** ok: [localhost] PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=0 skipped=19  rescued=0 ignored=0 2024/10/25 09:03:02 2024-10-25 09:02:18,432 p=28138 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 09:02:18,432 p=28138 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:02:18,673 p=28138 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 09:02:18,674 p=28138 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:02:18,911 p=28138 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 09:02:18,912 p=28138 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:02:18,927 p=28138 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:02:18,928 p=28138 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:02:19,252 p=28138 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 09:02:19,252 p=28138 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:02:19,279 p=28138 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 09:02:19,280 p=28138 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:02:19,300 p=28138 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:02:19,300 p=28138 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:02:19,310 p=28138 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 09:02:19,861 p=28138 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 09:02:19,861 p=28138 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:03:00,909 p=28138 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Remove namespace test-oadp-353] *** 2024-10-25 09:03:00,909 p=28138 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 09:03:00,910 p=28138 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:03:01,582 p=28138 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Remove namespace mirror-helper] *** 2024-10-25 09:03:01,582 p=28138 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:03:02,012 p=28138 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 09:03:02,012 p=28138 u=1002480000 n=ansible | localhost : ok=12 changed=4 unreachable=0 failed=0 skipped=19 rescued=0 ignored=0 2024/10/25 09:03:02 Creating restore imagestream-e2e-b0573c77-92af-11ef-8d75-0a580a81aa41 for case imagestream-e2e-b0573c77-92af-11ef-8d75-0a580a81aa41 STEP: Create restore imagestream-e2e-b0573c77-92af-11ef-8d75-0a580a81aa41 from backup imagestream-e2e-b0573c77-92af-11ef-8d75-0a580a81aa41 @ 10/25/24 09:03:02.059 2024/10/25 09:03:02 Wait until restore imagestream-e2e-b0573c77-92af-11ef-8d75-0a580a81aa41 is complete restore phase: InProgress restore phase: InProgress restore phase: Completed STEP: Verify restore imagestream-e2e-b0573c77-92af-11ef-8d75-0a580a81aa41has completed successfully @ 10/25/24 09:03:22.158 STEP: Verify Application restore @ 10/25/24 09:03:22.174 STEP: Verify Application deployment for case imagestream-e2e @ 10/25/24 09:03:22.174 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Check image streams] *** ok: [localhost] => (item=Check that imagestream internal-image exists) ok: [localhost] => (item=Check that imagestream internal-image exists) TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Check initial tags] *** ok: [localhost] => (item=Check that imagestream tag internal-image:alp exists) ok: [localhost] => (item=Check that imagestream tag internal-image:busy exists) TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Check extra tags] *** ok: [localhost] => (item=Check that imagestream tag internal-image:tag1 exists) ok: [localhost] => (item=Check that imagestream tag internal-image:tag2 exists) ok: [localhost] => (item=Check that imagestream tag internal-image:tag3 exists) ok: [localhost] => (item=Check that imagestream tag internal-image:latest exists) ok: [localhost] => (item=Check that imagestream tag internal-image:tag4 exists) TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Check pods are running] *** ok: [localhost] PLAY RECAP ********************************************************************* localhost : ok=14  changed=3  unreachable=0 failed=0 skipped=17  rescued=0 ignored=0 2024/10/25 09:03:42 2024-10-25 09:03:23,591 p=28369 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 09:03:23,592 p=28369 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:03:23,823 p=28369 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 09:03:23,824 p=28369 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:03:24,082 p=28369 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 09:03:24,082 p=28369 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:03:24,099 p=28369 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:03:24,099 p=28369 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:03:24,399 p=28369 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 09:03:24,399 p=28369 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:03:24,427 p=28369 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 09:03:24,427 p=28369 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:03:24,446 p=28369 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:03:24,446 p=28369 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:03:24,458 p=28369 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 09:03:24,981 p=28369 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 09:03:24,982 p=28369 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:03:27,320 p=28369 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Check image streams] *** 2024-10-25 09:03:27,320 p=28369 u=1002480000 n=ansible | ok: [localhost] => (item=Check that imagestream internal-image exists) 2024-10-25 09:03:29,181 p=28369 u=1002480000 n=ansible | ok: [localhost] => (item=Check that imagestream internal-image exists) 2024-10-25 09:03:29,183 p=28369 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 09:03:31,000 p=28369 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Check initial tags] *** 2024-10-25 09:03:31,001 p=28369 u=1002480000 n=ansible | ok: [localhost] => (item=Check that imagestream tag internal-image:alp exists) 2024-10-25 09:03:32,685 p=28369 u=1002480000 n=ansible | ok: [localhost] => (item=Check that imagestream tag internal-image:busy exists) 2024-10-25 09:03:34,521 p=28369 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Check extra tags] *** 2024-10-25 09:03:34,522 p=28369 u=1002480000 n=ansible | ok: [localhost] => (item=Check that imagestream tag internal-image:tag1 exists) 2024-10-25 09:03:36,274 p=28369 u=1002480000 n=ansible | ok: [localhost] => (item=Check that imagestream tag internal-image:tag2 exists) 2024-10-25 09:03:38,025 p=28369 u=1002480000 n=ansible | ok: [localhost] => (item=Check that imagestream tag internal-image:tag3 exists) 2024-10-25 09:03:39,870 p=28369 u=1002480000 n=ansible | ok: [localhost] => (item=Check that imagestream tag internal-image:latest exists) 2024-10-25 09:03:41,581 p=28369 u=1002480000 n=ansible | ok: [localhost] => (item=Check that imagestream tag internal-image:tag4 exists) 2024-10-25 09:03:42,301 p=28369 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Check pods are running] *** 2024-10-25 09:03:42,301 p=28369 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:03:42,316 p=28369 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 09:03:42,316 p=28369 u=1002480000 n=ansible | localhost : ok=14 changed=3 unreachable=0 failed=0 skipped=17 rescued=0 ignored=0 2024/10/25 09:03:42 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 2024/10/25 09:03:42 Cleaning app [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Remove namespace test-oadp-353] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Remove namespace mirror-helper] *** ok: [localhost] PLAY RECAP ********************************************************************* localhost : ok=12  changed=4  unreachable=0 failed=0 skipped=19  rescued=0 ignored=0 2024/10/25 09:04:32 2024-10-25 09:03:43,848 p=28703 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 09:03:43,849 p=28703 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:03:44,089 p=28703 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 09:03:44,089 p=28703 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:03:44,332 p=28703 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 09:03:44,333 p=28703 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:03:44,348 p=28703 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:03:44,348 p=28703 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:03:44,633 p=28703 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 09:03:44,633 p=28703 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:03:44,660 p=28703 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 09:03:44,661 p=28703 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:03:44,679 p=28703 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:03:44,680 p=28703 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:03:44,695 p=28703 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 09:03:45,239 p=28703 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 09:03:45,240 p=28703 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:04:31,296 p=28703 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Remove namespace test-oadp-353] *** 2024-10-25 09:04:31,297 p=28703 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 09:04:31,297 p=28703 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:04:31,967 p=28703 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-imagestreams : Remove namespace mirror-helper] *** 2024-10-25 09:04:31,968 p=28703 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:04:32,379 p=28703 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 09:04:32,380 p=28703 u=1002480000 n=ansible | localhost : ok=12 changed=4 unreachable=0 failed=0 skipped=19 rescued=0 ignored=0 2024/10/25 09:04:32 Cleaning setup resources for the backup • [198.100 seconds] ------------------------------ Backup restore tests Application backup [tc-id:OADP-357][interop] Empty-project application with VSL [vsl] /alabama/cspi/e2e/cloudstorage/app_backup_vsl.go:102 2024/10/25 09:04:32 Check if VSL custom credentials exist 2024/10/25 09:04:32 Check if the cloud provider is AWS 2024/10/25 09:04:32 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 10/25/24 09:04:32.497 2024/10/25 09:04:32 vsl 2024/10/25 09:04:32 Check if VSL custom credentials exist 2024/10/25 09:04:32 Check if the cloud provider is AWS 2024/10/25 09:04:32 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "620ae7ac-2198-40db-87e7-df92dc1e02cf", "resourceVersion": "120551", "generation": 1, "creationTimestamp": "2024-10-25T09:04:32Z", "managedFields": [ { "manager": "cloudstorage.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2024-10-25T09:04:32Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {} } }, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "bucket": { "cloudStorageRef": { "name": "ci-op-jdjixbyi-interopoadp" }, "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "credentials" }, "default": true, "prefix": "velero-e2e-1981d203-92ad-11ef-8d75-0a580a81aa41" } } ], "snapshotLocations": [ { "velero": { "provider": "aws", "config": { "credentialsFile": "/tmp/credentials/openshift-adp/cloud-credentials-credentials", "enableSharedConfig": "true", "profile": "default", "region": "us-east-2" } } } ], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt" ] } }, "features": null }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 10/25/24 09:04:32.603 2024/10/25 09:04:32 Waiting for velero pod to be running 2024/10/25 09:04:32 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' 2024/10/25 09:04:32 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "620ae7ac-2198-40db-87e7-df92dc1e02cf", "resourceVersion": "120551", "generation": 1, "creationTimestamp": "2024-10-25T09:04:32Z", "managedFields": [ { "manager": "cloudstorage.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2024-10-25T09:04:32Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {} } }, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "bucket": { "cloudStorageRef": { "name": "ci-op-jdjixbyi-interopoadp" }, "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "credentials" }, "default": true, "prefix": "velero-e2e-1981d203-92ad-11ef-8d75-0a580a81aa41" } } ], "snapshotLocations": [ { "velero": { "provider": "aws", "config": { "credentialsFile": "/tmp/credentials/openshift-adp/cloud-credentials-credentials", "enableSharedConfig": "true", "profile": "default", "region": "us-east-2" } } } ], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt" ] } }, "features": null }, "status": {} } STEP: Prepare backup resources, depending on the volumes backup type @ 10/25/24 09:04:37.662 2024/10/25 09:04:37 Checking default storage class count STEP: Installing application for case empty-project-e2e @ 10/25/24 09:04:37.693 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-project : Deploy project with labels and selectors] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=11  changed=4  unreachable=0 failed=0 skipped=3  rescued=0 ignored=0 2024/10/25 09:04:41 2024-10-25 09:04:39,120 p=28931 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 09:04:39,120 p=28931 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:04:39,365 p=28931 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 09:04:39,365 p=28931 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:04:39,618 p=28931 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 09:04:39,619 p=28931 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:04:39,635 p=28931 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:04:39,635 p=28931 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:04:39,950 p=28931 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 09:04:39,950 p=28931 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:04:39,978 p=28931 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 09:04:39,978 p=28931 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:04:39,997 p=28931 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:04:39,997 p=28931 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:04:40,007 p=28931 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 09:04:40,549 p=28931 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 09:04:40,549 p=28931 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:04:41,500 p=28931 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-project : Deploy project with labels and selectors] *** 2024-10-25 09:04:41,501 p=28931 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 09:04:41,501 p=28931 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:04:41,551 p=28931 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 09:04:41,551 p=28931 u=1002480000 n=ansible | localhost : ok=11 changed=4 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0 STEP: Verify Application deployment @ 10/25/24 09:04:41.593 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-project : Check project status] *** ok: [localhost] PLAY RECAP ********************************************************************* localhost : ok=11  changed=3  unreachable=0 failed=0 skipped=3  rescued=0 ignored=0 2024/10/25 09:04:45 2024-10-25 09:04:43,038 p=29131 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 09:04:43,038 p=29131 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:04:43,294 p=29131 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 09:04:43,294 p=29131 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:04:43,523 p=29131 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 09:04:43,524 p=29131 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:04:43,539 p=29131 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:04:43,540 p=29131 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:04:43,850 p=29131 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 09:04:43,850 p=29131 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:04:43,881 p=29131 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 09:04:43,881 p=29131 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:04:43,901 p=29131 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:04:43,901 p=29131 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:04:43,912 p=29131 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 09:04:44,440 p=29131 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 09:04:44,440 p=29131 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:04:45,240 p=29131 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-project : Check project status] *** 2024-10-25 09:04:45,240 p=29131 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 09:04:45,240 p=29131 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:04:45,273 p=29131 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 09:04:45,273 p=29131 u=1002480000 n=ansible | localhost : ok=11 changed=3 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0 2024/10/25 09:04:45 {{ } { } []} STEP: Creating backup empty-project-e2e-2633c815-92b0-11ef-8d75-0a580a81aa41 @ 10/25/24 09:04:45.328 2024/10/25 09:04:45 Wait until backup empty-project-e2e-2633c815-92b0-11ef-8d75-0a580a81aa41 is completed backup phase: Completed STEP: Verify backup empty-project-e2e-2633c815-92b0-11ef-8d75-0a580a81aa41 has completed successfully @ 10/25/24 09:05:05.38 2024/10/25 09:05:05 Backup for case empty-project-e2e succeeded STEP: Delete the appplication resources empty-project-e2e @ 10/25/24 09:05:05.475 STEP: Cleanup Application for case empty-project-e2e @ 10/25/24 09:05:05.475 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-project : Remove namespace test-oadp-357] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=11  changed=4  unreachable=0 failed=0 skipped=3  rescued=0 ignored=0 2024/10/25 09:05:19 2024-10-25 09:05:06,942 p=29332 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 09:05:06,942 p=29332 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:05:07,198 p=29332 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 09:05:07,198 p=29332 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:05:07,481 p=29332 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 09:05:07,481 p=29332 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:05:07,496 p=29332 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:05:07,496 p=29332 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:05:07,815 p=29332 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 09:05:07,815 p=29332 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:05:07,844 p=29332 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 09:05:07,844 p=29332 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:05:07,862 p=29332 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:05:07,863 p=29332 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:05:07,872 p=29332 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 09:05:08,445 p=29332 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 09:05:08,445 p=29332 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:05:19,454 p=29332 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-project : Remove namespace test-oadp-357] *** 2024-10-25 09:05:19,454 p=29332 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 09:05:19,454 p=29332 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:05:19,526 p=29332 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 09:05:19,526 p=29332 u=1002480000 n=ansible | localhost : ok=11 changed=4 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0 2024/10/25 09:05:19 Creating restore empty-project-e2e-2633c815-92b0-11ef-8d75-0a580a81aa41 for case empty-project-e2e-2633c815-92b0-11ef-8d75-0a580a81aa41 STEP: Create restore empty-project-e2e-2633c815-92b0-11ef-8d75-0a580a81aa41 from backup empty-project-e2e-2633c815-92b0-11ef-8d75-0a580a81aa41 @ 10/25/24 09:05:19.589 2024/10/25 09:05:19 Wait until restore empty-project-e2e-2633c815-92b0-11ef-8d75-0a580a81aa41 is complete restore phase: InProgress restore phase: Completed STEP: Verify restore empty-project-e2e-2633c815-92b0-11ef-8d75-0a580a81aa41has completed successfully @ 10/25/24 09:05:29.666 STEP: Verify Application restore @ 10/25/24 09:05:29.682 STEP: Verify Application deployment for case empty-project-e2e @ 10/25/24 09:05:29.682 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-project : Check project status] *** ok: [localhost] PLAY RECAP ********************************************************************* localhost : ok=11  changed=3  unreachable=0 failed=0 skipped=3  rescued=0 ignored=0 2024/10/25 09:05:33 2024-10-25 09:05:31,150 p=29528 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 09:05:31,151 p=29528 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:05:31,411 p=29528 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 09:05:31,412 p=29528 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:05:31,687 p=29528 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 09:05:31,688 p=29528 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:05:31,703 p=29528 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:05:31,703 p=29528 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:05:32,004 p=29528 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 09:05:32,004 p=29528 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:05:32,035 p=29528 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 09:05:32,035 p=29528 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:05:32,061 p=29528 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:05:32,061 p=29528 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:05:32,072 p=29528 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 09:05:32,636 p=29528 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 09:05:32,636 p=29528 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:05:33,563 p=29528 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-project : Check project status] *** 2024-10-25 09:05:33,563 p=29528 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 09:05:33,564 p=29528 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:05:33,601 p=29528 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 09:05:33,602 p=29528 u=1002480000 n=ansible | localhost : ok=11 changed=3 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0 2024/10/25 09:05:33 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 2024/10/25 09:05:33 Cleaning app [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-project : Remove namespace test-oadp-357] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=11  changed=4  unreachable=0 failed=0 skipped=3  rescued=0 ignored=0 2024/10/25 09:05:47 2024-10-25 09:05:35,178 p=29725 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 09:05:35,178 p=29725 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:05:35,427 p=29725 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 09:05:35,427 p=29725 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:05:35,687 p=29725 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 09:05:35,687 p=29725 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:05:35,704 p=29725 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:05:35,704 p=29725 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:05:36,010 p=29725 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 09:05:36,010 p=29725 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:05:36,042 p=29725 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 09:05:36,043 p=29725 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:05:36,061 p=29725 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:05:36,062 p=29725 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:05:36,072 p=29725 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 09:05:36,640 p=29725 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 09:05:36,640 p=29725 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:05:47,522 p=29725 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-project : Remove namespace test-oadp-357] *** 2024-10-25 09:05:47,522 p=29725 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 09:05:47,522 p=29725 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:05:47,590 p=29725 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 09:05:47,590 p=29725 u=1002480000 n=ansible | localhost : ok=11 changed=4 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0 2024/10/25 09:05:47 Cleaning setup resources for the backup • [75.223 seconds] ------------------------------ SSSS ------------------------------ Backup restore tests Application backup [tc-id:OADP-361][interop] Cassandra application with VSL [vsl] /alabama/cspi/e2e/cloudstorage/app_backup_vsl.go:172 2024/10/25 09:05:47 Check if VSL custom credentials exist 2024/10/25 09:05:47 Check if the cloud provider is AWS 2024/10/25 09:05:47 Delete all downloadrequest No download requests are found STEP: Create DPA CR @ 10/25/24 09:05:47.713 2024/10/25 09:05:47 vsl 2024/10/25 09:05:47 Check if VSL custom credentials exist 2024/10/25 09:05:47 Check if the cloud provider is AWS 2024/10/25 09:05:47 { "metadata": { "name": "ts-dpa", "namespace": "openshift-adp", "uid": "5aebb7d2-c8c0-48cc-9c55-a8b08e83f57c", "resourceVersion": "121601", "generation": 1, "creationTimestamp": "2024-10-25T09:05:47Z", "managedFields": [ { "manager": "cloudstorage.test", "operation": "Update", "apiVersion": "oadp.openshift.io/v1alpha1", "time": "2024-10-25T09:05:47Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:spec": { ".": {}, "f:backupLocations": {}, "f:configuration": { ".": {}, "f:velero": { ".": {}, "f:defaultPlugins": {} } }, "f:podDnsConfig": {}, "f:snapshotLocations": {} } } } ] }, "spec": { "backupLocations": [ { "bucket": { "cloudStorageRef": { "name": "ci-op-jdjixbyi-interopoadp" }, "config": { "region": "us-east-2" }, "credential": { "name": "cloud-credentials", "key": "credentials" }, "default": true, "prefix": "velero-e2e-1981d203-92ad-11ef-8d75-0a580a81aa41" } } ], "snapshotLocations": [ { "velero": { "provider": "aws", "config": { "credentialsFile": "/tmp/credentials/openshift-adp/cloud-credentials-credentials", "enableSharedConfig": "true", "profile": "default", "region": "us-east-2" } } } ], "podDnsConfig": {}, "configuration": { "velero": { "defaultPlugins": [ "openshift", "aws", "kubevirt" ] } }, "features": null }, "status": {} } Delete all the backups that remained in the phase InProgress Deleting backup CRs in progress Deletion of backup CRs in progress completed Delete all the restores that remained in the phase InProgress Deleting restore CRs in progress Deletion of restore CRs in progress completed STEP: Verify DPA CR setup @ 10/25/24 09:05:47.829 2024/10/25 09:05:47 Waiting for velero pod to be running 2024/10/25 09:05:47 Wait for DPA status.condition.reason to be 'Completed' and and message to be 'Reconcile complete' STEP: Prepare backup resources, depending on the volumes backup type @ 10/25/24 09:05:47.867 2024/10/25 09:05:47 Checking default storage class count STEP: Installing application for case cassandra-e2e @ 10/25/24 09:05:47.905 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Check namespace] *** ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Create namespace] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Add scc privileged to service account] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Create a service object required to provide network identity] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Create a statefulset with the existing yaml] *** changed: [localhost] FAILED - RETRYING: [localhost]: Check pods status (30 retries left). FAILED - RETRYING: [localhost]: Check pods status (29 retries left). FAILED - RETRYING: [localhost]: Check pods status (28 retries left). FAILED - RETRYING: [localhost]: Check pods status (27 retries left). FAILED - RETRYING: [localhost]: Check pods status (26 retries left). TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Check pods status] *** ok: [localhost] FAILED - RETRYING: [localhost]: Wait until all cassandra node are ready (Status=Up and State=Normal) (30 retries left). FAILED - RETRYING: [localhost]: Wait until all cassandra node are ready (Status=Up and State=Normal) (29 retries left). FAILED - RETRYING: [localhost]: Wait until all cassandra node are ready (Status=Up and State=Normal) (28 retries left). FAILED - RETRYING: [localhost]: Wait until all cassandra node are ready (Status=Up and State=Normal) (27 retries left). FAILED - RETRYING: [localhost]: Wait until all cassandra node are ready (Status=Up and State=Normal) (26 retries left). FAILED - RETRYING: [localhost]: Wait until all cassandra node are ready (Status=Up and State=Normal) (25 retries left). FAILED - RETRYING: [localhost]: Wait until all cassandra node are ready (Status=Up and State=Normal) (24 retries left). FAILED - RETRYING: [localhost]: Wait until all cassandra node are ready (Status=Up and State=Normal) (23 retries left). FAILED - RETRYING: [localhost]: Wait until all cassandra node are ready (Status=Up and State=Normal) (22 retries left). FAILED - RETRYING: [localhost]: Wait until all cassandra node are ready (Status=Up and State=Normal) (21 retries left). FAILED - RETRYING: [localhost]: Wait until all cassandra node are ready (Status=Up and State=Normal) (20 retries left). FAILED - RETRYING: [localhost]: Wait until all cassandra node are ready (Status=Up and State=Normal) (19 retries left). FAILED - RETRYING: [localhost]: Wait until all cassandra node are ready (Status=Up and State=Normal) (18 retries left). FAILED - RETRYING: [localhost]: Wait until all cassandra node are ready (Status=Up and State=Normal) (17 retries left). FAILED - RETRYING: [localhost]: Wait until all cassandra node are ready (Status=Up and State=Normal) (16 retries left). TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Wait until all cassandra node are ready (Status=Up and State=Normal)] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Populate the DB with some sample data] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=18  changed=9  unreachable=0 failed=0 skipped=6  rescued=0 ignored=0 2024/10/25 09:09:14 2024-10-25 09:05:49,350 p=29925 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 09:05:49,351 p=29925 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:05:49,594 p=29925 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 09:05:49,594 p=29925 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:05:49,843 p=29925 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 09:05:49,843 p=29925 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:05:49,859 p=29925 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:05:49,859 p=29925 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:05:50,166 p=29925 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 09:05:50,166 p=29925 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:05:50,200 p=29925 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 09:05:50,200 p=29925 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:05:50,219 p=29925 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:05:50,220 p=29925 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:05:50,230 p=29925 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 09:05:50,769 p=29925 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 09:05:50,769 p=29925 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:05:51,573 p=29925 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Check namespace] *** 2024-10-25 09:05:51,573 p=29925 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 09:05:51,573 p=29925 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:05:52,002 p=29925 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Create namespace] *** 2024-10-25 09:05:52,002 p=29925 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:05:52,359 p=29925 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Add scc privileged to service account] *** 2024-10-25 09:05:52,359 p=29925 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:05:53,249 p=29925 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Create a service object required to provide network identity] *** 2024-10-25 09:05:53,250 p=29925 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:05:54,019 p=29925 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Create a statefulset with the existing yaml] *** 2024-10-25 09:05:54,020 p=29925 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:06:22,790 p=29925 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Check pods status] *** 2024-10-25 09:06:22,790 p=29925 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:09:10,425 p=29925 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Wait until all cassandra node are ready (Status=Up and State=Normal)] *** 2024-10-25 09:09:10,425 p=29925 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:09:14,202 p=29925 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Populate the DB with some sample data] *** 2024-10-25 09:09:14,203 p=29925 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:09:14,305 p=29925 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 09:09:14,305 p=29925 u=1002480000 n=ansible | localhost : ok=18 changed=9 unreachable=0 failed=0 skipped=6 rescued=0 ignored=0 STEP: Verify Application deployment @ 10/25/24 09:09:14.388 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Check pods status] *** ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Wait until all cassandra node are ready (Status=Up and State=Normal)] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Verify Data was populated] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : copy] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Verify Data Integrity] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=15  changed=7  unreachable=0 failed=0 skipped=9  rescued=0 ignored=0 2024/10/25 09:09:27 2024-10-25 09:09:15,811 p=30845 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 09:09:15,811 p=30845 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:09:16,046 p=30845 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 09:09:16,046 p=30845 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:09:16,288 p=30845 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 09:09:16,288 p=30845 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:09:16,305 p=30845 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:09:16,306 p=30845 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:09:16,611 p=30845 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 09:09:16,611 p=30845 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:09:16,638 p=30845 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 09:09:16,638 p=30845 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:09:16,657 p=30845 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:09:16,657 p=30845 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:09:16,667 p=30845 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 09:09:17,208 p=30845 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 09:09:17,208 p=30845 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:09:18,208 p=30845 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Check pods status] *** 2024-10-25 09:09:18,208 p=30845 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 09:09:18,208 p=30845 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:09:25,028 p=30845 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Wait until all cassandra node are ready (Status=Up and State=Normal)] *** 2024-10-25 09:09:25,028 p=30845 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:09:26,532 p=30845 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Verify Data was populated] *** 2024-10-25 09:09:26,532 p=30845 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:09:27,084 p=30845 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : copy] *** 2024-10-25 09:09:27,084 p=30845 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:09:27,273 p=30845 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Verify Data Integrity] *** 2024-10-25 09:09:27,274 p=30845 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:09:27,287 p=30845 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 09:09:27,287 p=30845 u=1002480000 n=ansible | localhost : ok=15 changed=7 unreachable=0 failed=0 skipped=9 rescued=0 ignored=0 2024/10/25 09:09:27 {{ } { } [{{ } {cassandra-data-cassandra-0 test-oadp-361 e3d75733-c486-408d-90a6-a9728da8112c 121833 0 2024-10-25 09:05:53 +0000 UTC map[app:cassandra] map[pv.kubernetes.io/bind-completed:yes pv.kubernetes.io/bound-by-controller:yes volume.beta.kubernetes.io/storage-provisioner:ebs.csi.aws.com volume.kubernetes.io/selected-node:ip-10-0-37-172.us-east-2.compute.internal volume.kubernetes.io/storage-provisioner:ebs.csi.aws.com] [] [kubernetes.io/pvc-protection] [{kube-scheduler Update v1 2024-10-25 09:05:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volume.kubernetes.io/selected-node":{}}}} } {kube-controller-manager Update v1 2024-10-25 09:05:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:pv.kubernetes.io/bind-completed":{},"f:pv.kubernetes.io/bound-by-controller":{},"f:volume.beta.kubernetes.io/storage-provisioner":{},"f:volume.kubernetes.io/storage-provisioner":{}},"f:labels":{".":{},"f:app":{}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{},"f:volumeName":{}}} } {kube-controller-manager Update v1 2024-10-25 09:05:58 +0000 UTC FieldsV1 {"f:status":{"f:accessModes":{},"f:capacity":{".":{},"f:storage":{}},"f:phase":{}}} status}]} {[ReadWriteOnce] nil {map[] map[storage:{{1073741824 0} {} 1Gi BinarySI}]} pvc-e3d75733-c486-408d-90a6-a9728da8112c 0xc00171ece0 0xc00171ecf0 nil nil} {Bound [ReadWriteOnce] map[storage:{{1073741824 0} {} 1Gi BinarySI}] [] map[] }} {{ } {cassandra-data-cassandra-1 test-oadp-361 7f531511-7f70-4d31-8e80-d1bef8c707b1 122130 0 2024-10-25 09:06:18 +0000 UTC map[app:cassandra] map[pv.kubernetes.io/bind-completed:yes pv.kubernetes.io/bound-by-controller:yes volume.beta.kubernetes.io/storage-provisioner:ebs.csi.aws.com volume.kubernetes.io/selected-node:ip-10-0-65-50.us-east-2.compute.internal volume.kubernetes.io/storage-provisioner:ebs.csi.aws.com] [] [kubernetes.io/pvc-protection] [{kube-scheduler Update v1 2024-10-25 09:06:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volume.kubernetes.io/selected-node":{}}}} } {kube-controller-manager Update v1 2024-10-25 09:06:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:pv.kubernetes.io/bind-completed":{},"f:pv.kubernetes.io/bound-by-controller":{},"f:volume.beta.kubernetes.io/storage-provisioner":{},"f:volume.kubernetes.io/storage-provisioner":{}},"f:labels":{".":{},"f:app":{}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{},"f:volumeName":{}}} } {kube-controller-manager Update v1 2024-10-25 09:06:22 +0000 UTC FieldsV1 {"f:status":{"f:accessModes":{},"f:capacity":{".":{},"f:storage":{}},"f:phase":{}}} status}]} {[ReadWriteOnce] nil {map[] map[storage:{{1073741824 0} {} 1Gi BinarySI}]} pvc-7f531511-7f70-4d31-8e80-d1bef8c707b1 0xc00171edf0 0xc00171ee00 nil nil} {Bound [ReadWriteOnce] map[storage:{{1073741824 0} {} 1Gi BinarySI}] [] map[] }} {{ } {cassandra-data-cassandra-2 test-oadp-361 bc9d32c6-ad4f-4902-8f9c-3ec801c564b5 122421 0 2024-10-25 09:06:41 +0000 UTC map[app:cassandra] map[pv.kubernetes.io/bind-completed:yes pv.kubernetes.io/bound-by-controller:yes volume.beta.kubernetes.io/storage-provisioner:ebs.csi.aws.com volume.kubernetes.io/selected-node:ip-10-0-28-225.us-east-2.compute.internal volume.kubernetes.io/storage-provisioner:ebs.csi.aws.com] [] [kubernetes.io/pvc-protection] [{kube-scheduler Update v1 2024-10-25 09:06:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volume.kubernetes.io/selected-node":{}}}} } {kube-controller-manager Update v1 2024-10-25 09:06:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:pv.kubernetes.io/bind-completed":{},"f:pv.kubernetes.io/bound-by-controller":{},"f:volume.beta.kubernetes.io/storage-provisioner":{},"f:volume.kubernetes.io/storage-provisioner":{}},"f:labels":{".":{},"f:app":{}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{},"f:volumeName":{}}} } {kube-controller-manager Update v1 2024-10-25 09:06:45 +0000 UTC FieldsV1 {"f:status":{"f:accessModes":{},"f:capacity":{".":{},"f:storage":{}},"f:phase":{}}} status}]} {[ReadWriteOnce] nil {map[] map[storage:{{1073741824 0} {} 1Gi BinarySI}]} pvc-bc9d32c6-ad4f-4902-8f9c-3ec801c564b5 0xc00171ef00 0xc00171ef10 nil nil} {Bound [ReadWriteOnce] map[storage:{{1073741824 0} {} 1Gi BinarySI}] [] map[] }}]} STEP: Creating backup cassandra-e2e-530998a6-92b0-11ef-8d75-0a580a81aa41 @ 10/25/24 09:09:27.385 2024/10/25 09:09:27 Wait until backup cassandra-e2e-530998a6-92b0-11ef-8d75-0a580a81aa41 is completed backup phase: Completed STEP: Verify backup cassandra-e2e-530998a6-92b0-11ef-8d75-0a580a81aa41 has completed successfully @ 10/25/24 09:09:47.445 2024/10/25 09:09:47 Backup for case cassandra-e2e succeeded STEP: Delete the appplication resources cassandra-e2e @ 10/25/24 09:09:47.567 STEP: Cleanup Application for case cassandra-e2e @ 10/25/24 09:09:47.567 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Remove namespace test-oadp-361] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=11  changed=4  unreachable=0 failed=0 skipped=13  rescued=0 ignored=0 2024/10/25 09:10:21 2024-10-25 09:09:48,948 p=31153 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 09:09:48,948 p=31153 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:09:49,200 p=31153 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 09:09:49,200 p=31153 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:09:49,433 p=31153 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 09:09:49,433 p=31153 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:09:49,448 p=31153 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:09:49,448 p=31153 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:09:49,773 p=31153 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 09:09:49,773 p=31153 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:09:49,801 p=31153 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 09:09:49,802 p=31153 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:09:49,825 p=31153 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:09:49,825 p=31153 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:09:49,836 p=31153 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 09:09:50,428 p=31153 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 09:09:50,428 p=31153 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:10:21,391 p=31153 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Remove namespace test-oadp-361] *** 2024-10-25 09:10:21,391 p=31153 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 09:10:21,391 p=31153 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:10:21,607 p=31153 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 09:10:21,607 p=31153 u=1002480000 n=ansible | localhost : ok=11 changed=4 unreachable=0 failed=0 skipped=13 rescued=0 ignored=0 2024/10/25 09:10:21 Creating restore cassandra-e2e-530998a6-92b0-11ef-8d75-0a580a81aa41 for case cassandra-e2e-530998a6-92b0-11ef-8d75-0a580a81aa41 STEP: Create restore cassandra-e2e-530998a6-92b0-11ef-8d75-0a580a81aa41 from backup cassandra-e2e-530998a6-92b0-11ef-8d75-0a580a81aa41 @ 10/25/24 09:10:21.653 2024/10/25 09:10:21 Wait until restore cassandra-e2e-530998a6-92b0-11ef-8d75-0a580a81aa41 is complete restore phase: InProgress restore phase: Completed STEP: Verify restore cassandra-e2e-530998a6-92b0-11ef-8d75-0a580a81aa41has completed successfully @ 10/25/24 09:10:41.74 STEP: Verify Application restore @ 10/25/24 09:10:41.758 STEP: Verify Application deployment for case cassandra-e2e @ 10/25/24 09:10:41.758 [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Check pods status] *** ok: [localhost] FAILED - RETRYING: [localhost]: Wait until all cassandra node are ready (Status=Up and State=Normal) (30 retries left). FAILED - RETRYING: [localhost]: Wait until all cassandra node are ready (Status=Up and State=Normal) (29 retries left). TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Wait until all cassandra node are ready (Status=Up and State=Normal)] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Verify Data was populated] *** changed: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : copy] *** ok: [localhost] TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Verify Data Integrity] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=15  changed=6  unreachable=0 failed=0 skipped=9  rescued=0 ignored=0 2024/10/25 09:11:27 2024-10-25 09:10:43,162 p=31362 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 09:10:43,163 p=31362 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:10:43,392 p=31362 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 09:10:43,392 p=31362 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:10:43,639 p=31362 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 09:10:43,639 p=31362 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:10:43,655 p=31362 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:10:43,655 p=31362 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:10:43,946 p=31362 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 09:10:43,946 p=31362 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:10:43,973 p=31362 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 09:10:43,973 p=31362 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:10:43,991 p=31362 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:10:43,991 p=31362 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:10:44,001 p=31362 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 09:10:44,522 p=31362 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 09:10:44,522 p=31362 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:10:45,465 p=31362 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Check pods status] *** 2024-10-25 09:10:45,465 p=31362 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 09:10:45,465 p=31362 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:11:25,010 p=31362 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Wait until all cassandra node are ready (Status=Up and State=Normal)] *** 2024-10-25 09:11:25,010 p=31362 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:11:26,968 p=31362 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Verify Data was populated] *** 2024-10-25 09:11:26,968 p=31362 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:11:27,463 p=31362 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : copy] *** 2024-10-25 09:11:27,463 p=31362 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:11:27,628 p=31362 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Verify Data Integrity] *** 2024-10-25 09:11:27,629 p=31362 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:11:27,641 p=31362 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 09:11:27,641 p=31362 u=1002480000 n=ansible | localhost : ok=15 changed=6 unreachable=0 failed=0 skipped=9 rescued=0 ignored=0 2024/10/25 09:11:27 Using Must-gather image: registry.redhat.io/oadp/oadp-mustgather-rhel9:1.4.1 2024/10/25 09:11:27 Cleaning app [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' [WARNING]: Found variable using reserved name: namespace PLAY [localhost] *************************************************************** TASK [Gathering Facts] ********************************************************* ok: [localhost] TASK [include_vars] ************************************************************ ok: [localhost] TASK [Remove all the contents from the file] *********************************** changed: [localhost] TASK [Get cluster endpoint] **************************************************** changed: [localhost] TASK [Get current admin token] ************************************************* changed: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] TASK [Extract kubernetes minor version from cluster_info] ********************** ok: [localhost] TASK [Map kubernetes minor to ocp releases] ************************************ ok: [localhost] TASK [set_fact] **************************************************************** ok: [localhost] PLAY [Execute Task] ************************************************************ TASK [Gathering Facts] ********************************************************* ok: [localhost] [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Remove namespace test-oadp-361] *** changed: [localhost] PLAY RECAP ********************************************************************* localhost : ok=11  changed=4  unreachable=0 failed=0 skipped=13  rescued=0 ignored=0 2024/10/25 09:14:12 2024-10-25 09:11:29,129 p=31730 u=1002480000 n=ansible | TASK [Remove all the contents from the file] *********************************** 2024-10-25 09:11:29,130 p=31730 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:11:29,365 p=31730 u=1002480000 n=ansible | TASK [Get cluster endpoint] **************************************************** 2024-10-25 09:11:29,365 p=31730 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:11:29,589 p=31730 u=1002480000 n=ansible | TASK [Get current admin token] ************************************************* 2024-10-25 09:11:29,590 p=31730 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:11:29,604 p=31730 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:11:29,604 p=31730 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:11:29,911 p=31730 u=1002480000 n=ansible | TASK [Extract kubernetes minor version from cluster_info] ********************** 2024-10-25 09:11:29,912 p=31730 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:11:29,939 p=31730 u=1002480000 n=ansible | TASK [Map kubernetes minor to ocp releases] ************************************ 2024-10-25 09:11:29,939 p=31730 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:11:29,957 p=31730 u=1002480000 n=ansible | TASK [set_fact] **************************************************************** 2024-10-25 09:11:29,957 p=31730 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:11:29,969 p=31730 u=1002480000 n=ansible | PLAY [Execute Task] ************************************************************ 2024-10-25 09:11:30,513 p=31730 u=1002480000 n=ansible | TASK [Gathering Facts] ********************************************************* 2024-10-25 09:11:30,513 p=31730 u=1002480000 n=ansible | ok: [localhost] 2024-10-25 09:14:12,212 p=31730 u=1002480000 n=ansible | TASK [/alabama/cspi/sample-applications/ocpdeployer/ansible/roles/ocp-cassandra : Remove namespace test-oadp-361] *** 2024-10-25 09:14:12,212 p=31730 u=1002480000 n=ansible | [WARNING]: kubernetes<24.2.0 is not supported or tested. Some features may not work. 2024-10-25 09:14:12,212 p=31730 u=1002480000 n=ansible | changed: [localhost] 2024-10-25 09:14:12,418 p=31730 u=1002480000 n=ansible | PLAY RECAP ********************************************************************* 2024-10-25 09:14:12,418 p=31730 u=1002480000 n=ansible | localhost : ok=11 changed=4 unreachable=0 failed=0 skipped=13 rescued=0 ignored=0 2024/10/25 09:14:12 Cleaning setup resources for the backup • [504.875 seconds] ------------------------------ [AfterSuite]  /alabama/cspi/e2e/cloudstorage/cloudstorage_suite_test.go:112 2024/10/25 09:14:12 Deleting Velero CR [AfterSuite] PASSED [0.021 seconds] ------------------------------ [ReportAfterSuite] Autogenerated ReportAfterSuite for --junit-report autogenerated by Ginkgo [ReportAfterSuite] PASSED [0.005 seconds] ------------------------------ Summarizing 1 Failure: [FAIL] TOP-LEVEL [JustBeforeEach] Backup restore tests [CSI] Application backup [tc-id:OADP-122] [bug-id:OADP-1057] [interop] Django application with CSI /alabama/cspi/e2e/cloudstorage/cloudstorage_suite_test.go:120 Ran 8 of 14 Specs in 1889.902 seconds FAIL! -- 7 Passed | 1 Failed | 0 Pending | 6 Skipped --- FAIL: TestOADPE2E (1889.91s) FAIL Ginkgo ran 1 suite in 32m14.111814104s Test Suite Failed Checking for additional logs in /alabama/cspi/e2e/logs