Loading...

Type: Bug
Resolution: Done
Priority: Blocker
Fix Version/s: None
Affects Version/s: None
Component/s: E2E Automation
Labels:
None

Blocked:
False
Blocked Reason:

Hide

None

Show
None
Ready:
False
QEStatus:
ToDo
Intelligence Requested:
Market:

Severity:
Critical
Cost of Delay:
10
WSJF:
0
Risk Probability:
Very Likely
Risk Score:
0

Workstream:

None

Root Cause:
Unset
Failure Category:
Unknown

Regression:
No

SFDC Cases Links:
SFDC Cases Counter:
SFDC Cases Open:

Description of problem:

Installing volsync using the deploy_volsync script from oadp-qe-automation repo is failing on IBM Z platform.

Version-Release number of selected component (if applicable):

The installation of all volsync version on OCP version 4.12 is failing. It works on OCP 4.11

How reproducible:

always

Steps to Reproduce:
1. Fetch latest code from https://gitlab.cee.redhat.com/migrationqe/oadp-qe-automation
2. set

export VOLSYNC_REPOSITORY='redhat'

3. run

bash operator/volsync/deploy_volsync.sh

Actual results:

+++ readlink -f operator/volsync/deploy_volsync.sh
++ dirname /root/automation/oadp-qe-automation/operator/volsync/deploy_volsync.sh
+ SCRIPT_DIR=/root/automation/oadp-qe-automation/operator/volsync
++ cd /root/automation/oadp-qe-automation/operator/volsync
++ git rev-parse --show-toplevel
+ TOP_DIR=/root/automation/oadp-qe-automation
++ which oc
++ alias
++ eval declare -f
+++ declare -f
++ /usr/bin/which --tty-only --read-alias --read-functions --show-tilde --show-dot oc
+ export OC_TOOL=/usr/local/bin/oc
+ OC_TOOL=/usr/local/bin/oc
+ /usr/local/bin/oc version
Client Version: 4.12.0-rc.1
Kustomize Version: v4.5.7
Server Version: 4.12.0-rc.1
Kubernetes Version: v1.25.2+cd98eda
+ IP_APPROVAL=Manual
+ [[ -z '' ]]
+ IP_APPROVAL=Automatic
+ REPOSITORY=redhat
+ IIB_IMAGE=
+ export VOLSYNC_OPERATOR_NAME=volsync-product
+ VOLSYNC_OPERATOR_NAME=volsync-product
+ export VOLSYNC_NAMESPACE=openshift-operators
+ VOLSYNC_NAMESPACE=openshift-operators
+ source /root/automation/oadp-qe-automation/scripts/logger.sh
++ BLUE='\033[34m'
++ GREEN='\033[32m'
++ RED='\033[31m'
++ YELLOW='\033[33m'
++ CLEAR='\033[39m'
++ LOG_TARGET=
++ [[ -n '' ]]
+ source /root/automation/oadp-qe-automation/scripts/utils.sh
++++ readlink -f /root/automation/oadp-qe-automation/scripts/utils.sh
+++ dirname /root/automation/oadp-qe-automation/scripts/utils.sh
++ CURRENT_DIR=/root/automation/oadp-qe-automation/scripts
++ source /root/automation/oadp-qe-automation/scripts/logger.sh
+++ BLUE='\033[34m'
+++ GREEN='\033[32m'
+++ RED='\033[31m'
+++ YELLOW='\033[33m'
+++ CLEAR='\033[39m'
+++ LOG_TARGET=
+++ [[ -n '' ]]
+ source /root/automation/oadp-qe-automation/operator/generic/funcs.sh
++ source /root/automation/oadp-qe-automation/scripts/logger.sh
+++ BLUE='\033[34m'
+++ GREEN='\033[32m'
+++ RED='\033[31m'
+++ YELLOW='\033[33m'
+++ CLEAR='\033[39m'
+++ LOG_TARGET=
+++ [[ -n '' ]]
+++ which oc
+++ alias
+++ eval declare -f
++++ declare -f
+++ /usr/bin/which --tty-only --read-alias --read-functions --show-tilde --show-dot oc
++ OC_TOOL=/usr/local/bin/oc
++++ readlink -f /root/automation/oadp-qe-automation/operator/generic/funcs.sh
+++ dirname /root/automation/oadp-qe-automation/operator/generic/funcs.sh
++ SCRIPT_DIR=/root/automation/oadp-qe-automation/operator/generic
+++ cd /root/automation/oadp-qe-automation/operator/generic
+++ git rev-parse --show-toplevel
++ TOP_DIR=/root/automation/oadp-qe-automation
+ deploy_operator::funcs::verify_sources redhat
+ local REPOSITORY=redhat
+ OPERATOR_SOURCES=(community redhat prestage stage)
+ readonly OPERATOR_SOURCES
+ echo community redhat prestage stage
+ grep -Fqw redhat
+ VOLSYNC_CHANNEL=stable
+ VOLSYNC_STARTING_CSV=volsync-product.v
+ '[' redhat = prestage ']'
+ export VOLSYNC_CATALOGSOURCE=redhat-operators
+ VOLSYNC_CATALOGSOURCE=redhat-operators
+ scripts::logger::INFO 'Repository is redhat VOLSYNC_CATALOGSOURCE is >>>>>>> redhat-operators'
 [2023/02/07 07:24:17][scripts::logger::INFO](main) Repository is redhat VOLSYNC_CATALOGSOURCE is >>>>>>> redhat-operators 
++ deploy_operator::funcs::get_iib_per_repository redhat volsync-operator-bundle-container-v ''
++ local REPOSITORY=redhat
++ local PRESTAGE_INDEX_CONTAINS=volsync-operator-bundle-container-v
++ local IIB_IMAGE=
++ local registry_proxy_url=registry-proxy.engineering.redhat.com
+++ oc get clusterversion version -o 'jsonpath={$.status.desired.version}'
++ local OCP_VERSION=4.12.0-rc.1
+++ echo 4.12.0-rc.1
+++ cut -d . -f1,2
++ local OCP_MAJOR_MINOR=4.12
++ [[ redhat = \p\r\e\s\t\a\g\e ]]
++ [[ redhat = \s\t\a\g\e ]]
++ brew_url=brew.registry.redhat.io
+++ sed s/registry-proxy.engineering.redhat.com/brew.registry.redhat.io/
++ IIB_IMAGE=
++ echo ''
+ IIB_IMAGE=
+ deploy_operator::funcs::test_iib_image ''
+ local IIB_IMAGE=
+ [[ -z '' ]]
+ scripts::logger::INFO 'The REPOSITORY: redhat ; The IIB image to be used is >>>>>>> '
 [2023/02/07 07:24:17][scripts::logger::INFO](main) The REPOSITORY: redhat ; The IIB image to be used is >>>>>>> 
+ bash /root/automation/oadp-qe-automation/operator/generic/undeploy_operator.sh openshift-operators volsync-product redhat-operators
++ which oc
++ alias
++ eval declare -f
+++ declare -f
++ /usr/bin/which --tty-only --read-alias --read-functions --show-tilde --show-dot oc
+ readonly OC_TOOL=/usr/local/bin/oc
+ OC_TOOL=/usr/local/bin/oc
+ readonly OPERATOR_NAMESPACE=openshift-operators
+ OPERATOR_NAMESPACE=openshift-operators
+ readonly OPERATOR_NAME=volsync-product
+ OPERATOR_NAME=volsync-product
+ readonly CATALOGSOURCE=redhat-operators
+ CATALOGSOURCE=redhat-operators
+ /usr/local/bin/oc delete clusterserviceversions --namespace=openshift-operators --selector=operators.coreos.com/volsync-product.openshift-operators --ignore-not-found
No resources found
+ /usr/local/bin/oc wait deployments --namespace=openshift-operators --selector=olm.owner --for=delete --timeout=3m
+ /usr/local/bin/oc delete subscriptions --namespace=openshift-operators --all --ignore-not-found
No resources found
+ /usr/local/bin/oc delete operatorgroups --namespace=openshift-operators --field-selector 'metadata.name!=global-operators' --ignore-not-found
No resources found
+ /usr/local/bin/oc delete catalogsource --namespace=openshift-marketplace redhat-operators --ignore-not-found
catalogsource.operators.coreos.com "redhat-operators" deleted
+ /usr/local/bin/oc wait pods --namespace=openshift-marketplace --selector=olm.catalogSource=redhat-operators --for=delete --timeout=2m
pod/redhat-operators-r6qc9 condition met
error: timed out waiting for the condition on pods/redhat-operators-tpn9n
+ true
+ /usr/local/bin/oc delete project openshift-operators --ignore-not-found
project.project.openshift.io "openshift-operators" deleted
+ /usr/local/bin/oc wait ns openshift-operators --timeout=5m --for=delete
+ OPERATOR_NAMESPACE=openshift-operators
+ source /root/automation/oadp-qe-automation/operator/generic/01_namespace.sh
++ cat
++ /usr/local/bin/oc create -f -
namespace/openshift-operators created
+ sleep 60
+ brew_url=brew.registry.redhat.io
+ [[ redhat != \r\e\d\h\a\t ]]
+ deploy_operator::funcs::create_subs_n_deploy Automatic redhat-operators volsync-product openshift-operators stable volsync-product.v
+ local IP_APPROVAL=Automatic
+ local CATALOGSOURCE=redhat-operators
+ local OPERATOR_NAME=volsync-product
+ local OPERATOR_NAMESPACE=openshift-operators
+ local CHANNEL=stable
+ local STARTING_CSV=volsync-product.v
+ [[ Automatic == \A\u\t\o\m\a\t\i\c ]]
+ STARTING_CSV=
+ source /root/automation/oadp-qe-automation/operator/generic/04_subscription.sh
++ cat
++ /usr/local/bin/oc apply --namespace=openshift-operators -f -
subscription.operators.coreos.com/volsync-product created
+ install_plan=
+ retries=0
+ [[ -z '' ]]
+ (( retries == 20 ))
+ sleep 10
++ /usr/local/bin/oc get installplan -n openshift-operators '-ojsonpath={.items[?(@.metadata.ownerReferences[*].name=="volsync-product")]}'
++ jq -r ' . | select(.spec.clusterServiceVersionNames[0] | contains("")) | .metadata.name'
+ install_plan=
+ (( retries += 1 ))
+ [[ -z '' ]]
+ (( retries == 20 ))
+ sleep 10
++ /usr/local/bin/oc get installplan -n openshift-operators '-ojsonpath={.items[?(@.metadata.ownerReferences[*].name=="volsync-product")]}'
++ jq -r ' . | select(.spec.clusterServiceVersionNames[0] | contains("")) | .metadata.name'
+ install_plan='install-grq4z
install-mnml9'
+ (( retries += 1 ))
+ [[ -z install-grq4z
install-mnml9 ]]
+ /usr/local/bin/oc patch installplan 'install-grq4z
install-mnml9' --namespace=openshift-operators --type=merge '--patch={"spec":{"approved":true}}'
Error from server (NotFound): installplans.operators.coreos.com "install-grq4z\ninstall-mnml9" not found
[root@a3e18001 oadp-qe-automation]# bash operator/volsync/deploy_volsync.sh 
+++ readlink -f operator/volsync/deploy_volsync.sh
++ dirname /root/automation/oadp-qe-automation/operator/volsync/deploy_volsync.sh
+ SCRIPT_DIR=/root/automation/oadp-qe-automation/operator/volsync
++ cd /root/automation/oadp-qe-automation/operator/volsync
++ git rev-parse --show-toplevel
+ TOP_DIR=/root/automation/oadp-qe-automation
++ which oc
++ alias
++ eval declare -f
+++ declare -f
++ /usr/bin/which --tty-only --read-alias --read-functions --show-tilde --show-dot oc
+ export OC_TOOL=/usr/local/bin/oc
+ OC_TOOL=/usr/local/bin/oc
+ /usr/local/bin/oc version
Client Version: 4.12.0-rc.1
Kustomize Version: v4.5.7
Server Version: 4.12.0-rc.1
Kubernetes Version: v1.25.2+cd98eda
+ IP_APPROVAL=Manual
+ [[ -z '' ]]
+ IP_APPROVAL=Automatic
+ REPOSITORY=redhat
+ IIB_IMAGE=
+ export VOLSYNC_OPERATOR_NAME=volsync-product
+ VOLSYNC_OPERATOR_NAME=volsync-product
+ export VOLSYNC_NAMESPACE=openshift-operators
+ VOLSYNC_NAMESPACE=openshift-operators
+ source /root/automation/oadp-qe-automation/scripts/logger.sh
++ BLUE='\033[34m'
++ GREEN='\033[32m'
++ RED='\033[31m'
++ YELLOW='\033[33m'
++ CLEAR='\033[39m'
++ LOG_TARGET=
++ [[ -n '' ]]
+ source /root/automation/oadp-qe-automation/scripts/utils.sh
++++ readlink -f /root/automation/oadp-qe-automation/scripts/utils.sh
+++ dirname /root/automation/oadp-qe-automation/scripts/utils.sh
++ CURRENT_DIR=/root/automation/oadp-qe-automation/scripts
++ source /root/automation/oadp-qe-automation/scripts/logger.sh
+++ BLUE='\033[34m'
+++ GREEN='\033[32m'
+++ RED='\033[31m'
+++ YELLOW='\033[33m'
+++ CLEAR='\033[39m'
+++ LOG_TARGET=
+++ [[ -n '' ]]
+ source /root/automation/oadp-qe-automation/operator/generic/funcs.sh
++ source /root/automation/oadp-qe-automation/scripts/logger.sh
+++ BLUE='\033[34m'
+++ GREEN='\033[32m'
+++ RED='\033[31m'
+++ YELLOW='\033[33m'
+++ CLEAR='\033[39m'
+++ LOG_TARGET=
+++ [[ -n '' ]]
+++ which oc
+++ alias
+++ eval declare -f
++++ declare -f
+++ /usr/bin/which --tty-only --read-alias --read-functions --show-tilde --show-dot oc
++ OC_TOOL=/usr/local/bin/oc
++++ readlink -f /root/automation/oadp-qe-automation/operator/generic/funcs.sh
+++ dirname /root/automation/oadp-qe-automation/operator/generic/funcs.sh
++ SCRIPT_DIR=/root/automation/oadp-qe-automation/operator/generic
+++ cd /root/automation/oadp-qe-automation/operator/generic
+++ git rev-parse --show-toplevel
++ TOP_DIR=/root/automation/oadp-qe-automation
+ deploy_operator::funcs::verify_sources redhat
+ local REPOSITORY=redhat
+ OPERATOR_SOURCES=(community redhat prestage stage)
+ readonly OPERATOR_SOURCES
+ echo community redhat prestage stage
+ grep -Fqw redhat
+ VOLSYNC_CHANNEL=stable
+ VOLSYNC_STARTING_CSV=volsync-product.v
+ '[' redhat = prestage ']'
+ export VOLSYNC_CATALOGSOURCE=redhat-operators
+ VOLSYNC_CATALOGSOURCE=redhat-operators
+ scripts::logger::INFO 'Repository is redhat VOLSYNC_CATALOGSOURCE is >>>>>>> redhat-operators'
 [2023/02/07 07:46:09][scripts::logger::INFO](main) Repository is redhat VOLSYNC_CATALOGSOURCE is >>>>>>> redhat-operators 
++ deploy_operator::funcs::get_iib_per_repository redhat volsync-operator-bundle-container-v ''
++ local REPOSITORY=redhat
++ local PRESTAGE_INDEX_CONTAINS=volsync-operator-bundle-container-v
++ local IIB_IMAGE=
++ local registry_proxy_url=registry-proxy.engineering.redhat.com
+++ oc get clusterversion version -o 'jsonpath={$.status.desired.version}'
++ local OCP_VERSION=4.12.0-rc.1
+++ echo 4.12.0-rc.1
+++ cut -d . -f1,2
++ local OCP_MAJOR_MINOR=4.12
++ [[ redhat = \p\r\e\s\t\a\g\e ]]
++ [[ redhat = \s\t\a\g\e ]]
++ brew_url=brew.registry.redhat.io
+++ sed s/registry-proxy.engineering.redhat.com/brew.registry.redhat.io/
++ IIB_IMAGE=
++ echo ''
+ IIB_IMAGE=
+ deploy_operator::funcs::test_iib_image ''
+ local IIB_IMAGE=
+ [[ -z '' ]]
+ scripts::logger::INFO 'The REPOSITORY: redhat ; The IIB image to be used is >>>>>>> '
 [2023/02/07 07:46:09][scripts::logger::INFO](main) The REPOSITORY: redhat ; The IIB image to be used is >>>>>>> 
+ bash /root/automation/oadp-qe-automation/operator/generic/undeploy_operator.sh openshift-operators volsync-product redhat-operators
++ which oc
++ alias
++ eval declare -f
+++ declare -f
++ /usr/bin/which --tty-only --read-alias --read-functions --show-tilde --show-dot oc
+ readonly OC_TOOL=/usr/local/bin/oc
+ OC_TOOL=/usr/local/bin/oc
+ readonly OPERATOR_NAMESPACE=openshift-operators
+ OPERATOR_NAMESPACE=openshift-operators
+ readonly OPERATOR_NAME=volsync-product
+ OPERATOR_NAME=volsync-product
+ readonly CATALOGSOURCE=redhat-operators
+ CATALOGSOURCE=redhat-operators
+ /usr/local/bin/oc delete clusterserviceversions --namespace=openshift-operators --selector=operators.coreos.com/volsync-product.openshift-operators --ignore-not-found
clusterserviceversion.operators.coreos.com "volsync-product.v0.6.0" deleted
+ /usr/local/bin/oc wait deployments --namespace=openshift-operators --selector=olm.owner --for=delete --timeout=3m
deployment.apps/volsync-controller-manager condition met
+ /usr/local/bin/oc delete subscriptions --namespace=openshift-operators --all --ignore-not-found
subscription.operators.coreos.com "volsync-product" deleted
+ /usr/local/bin/oc delete operatorgroups --namespace=openshift-operators --field-selector 'metadata.name!=global-operators' --ignore-not-found
No resources found
+ /usr/local/bin/oc delete catalogsource --namespace=openshift-marketplace redhat-operators --ignore-not-found
catalogsource.operators.coreos.com "redhat-operators" deleted
+ /usr/local/bin/oc wait pods --namespace=openshift-marketplace --selector=olm.catalogSource=redhat-operators --for=delete --timeout=2m
pod/redhat-operators-tpn9n condition met
+ /usr/local/bin/oc delete project openshift-operators --ignore-not-found
project.project.openshift.io "openshift-operators" deleted
+ /usr/local/bin/oc wait ns openshift-operators --timeout=5m --for=delete
+ OPERATOR_NAMESPACE=openshift-operators
+ source /root/automation/oadp-qe-automation/operator/generic/01_namespace.sh
++ cat
++ /usr/local/bin/oc create -f -
namespace/openshift-operators created
+ sleep 60
+ brew_url=brew.registry.redhat.io
+ [[ redhat != \r\e\d\h\a\t ]]
+ deploy_operator::funcs::create_subs_n_deploy Automatic redhat-operators volsync-product openshift-operators stable volsync-product.v
+ local IP_APPROVAL=Automatic
+ local CATALOGSOURCE=redhat-operators
+ local OPERATOR_NAME=volsync-product
+ local OPERATOR_NAMESPACE=openshift-operators
+ local CHANNEL=stable
+ local STARTING_CSV=volsync-product.v
+ [[ Automatic == \A\u\t\o\m\a\t\i\c ]]
+ STARTING_CSV=
+ source /root/automation/oadp-qe-automation/operator/generic/04_subscription.sh
++ cat
++ /usr/local/bin/oc apply --namespace=openshift-operators -f -
subscription.operators.coreos.com/volsync-product created
+ install_plan=
+ retries=0
+ [[ -z '' ]]
+ (( retries == 20 ))
+ sleep 10
++ /usr/local/bin/oc get installplan -n openshift-operators '-ojsonpath={.items[?(@.metadata.ownerReferences[*].name=="volsync-product")]}'
++ jq -r ' . | select(.spec.clusterServiceVersionNames[0] | contains("")) | .metadata.name'
+ install_plan=
+ (( retries += 1 ))
+ [[ -z '' ]]
+ (( retries == 20 ))
+ sleep 10
++ /usr/local/bin/oc get installplan -n openshift-operators '-ojsonpath={.items[?(@.metadata.ownerReferences[*].name=="volsync-product")]}'
++ jq -r ' . | select(.spec.clusterServiceVersionNames[0] | contains("")) | .metadata.name'
+ install_plan=
+ (( retries += 1 ))
+ [[ -z '' ]]
+ (( retries == 20 ))
+ sleep 10
++ /usr/local/bin/oc get installplan -n openshift-operators '-ojsonpath={.items[?(@.metadata.ownerReferences[*].name=="volsync-product")]}'
++ jq -r ' . | select(.spec.clusterServiceVersionNames[0] | contains("")) | .metadata.name'
+ install_plan=
+ (( retries += 1 ))
+ [[ -z '' ]]
+ (( retries == 20 ))
+ sleep 10
++ /usr/local/bin/oc get installplan -n openshift-operators '-ojsonpath={.items[?(@.metadata.ownerReferences[*].name=="volsync-product")]}'
++ jq -r ' . | select(.spec.clusterServiceVersionNames[0] | contains("")) | .metadata.name'
+ install_plan=
+ (( retries += 1 ))
+ [[ -z '' ]]
+ (( retries == 20 ))
+ sleep 10
++ /usr/local/bin/oc get installplan -n openshift-operators '-ojsonpath={.items[?(@.metadata.ownerReferences[*].name=="volsync-product")]}'
++ jq -r ' . | select(.spec.clusterServiceVersionNames[0] | contains("")) | .metadata.name'
+ install_plan=
+ (( retries += 1 ))
+ [[ -z '' ]]
+ (( retries == 20 ))
+ sleep 10
++ /usr/local/bin/oc get installplan -n openshift-operators '-ojsonpath={.items[?(@.metadata.ownerReferences[*].name=="volsync-product")]}'
++ jq -r ' . | select(.spec.clusterServiceVersionNames[0] | contains("")) | .metadata.name'
+ install_plan='install-x7nhf
install-zsc68'
+ (( retries += 1 ))
+ [[ -z install-x7nhf
install-zsc68 ]]
+ /usr/local/bin/oc patch installplan 'install-x7nhf
install-zsc68' --namespace=openshift-operators --type=merge '--patch={"spec":{"approved":true}}'
Error from server (NotFound): installplans.operators.coreos.com "install-x7nhf\ninstall-zsc68" not found

Expected results:

Installation should be successful and show volsync version installed

+ /usr/local/bin/oc get csv -n openshift-operators

NAME DISPLAY VERSION REPLACES PHASE

volsync-product.v0.6.0 VolSync 0.6.0 Succeeded

Additional info:

Details

Description

Description of problem:

Actual results:

Installation should be successful and show volsync version installed

Attachments

Easy Agile Planning Poker

Activity

People

Dates