STEP-RUN-KUTTL-E2E-TESTS Logging in as kubeadmin user WARNING: Using insecure TLS client config. Setting this option is not supported! Login successful. You have access to 70 projects, the list has been suppressed. You can list all projects with 'oc projects' Using project "default". Welcome! See 'oc help' to get started. make: Entering directory '/kuttl-e2e/gitops-operator' Running GitOps Operator E2E tests... . ./test.sh all =========================== ==== RUNNING ALL TESTS ==== =========================== Running parallel test suite 2024/07/23 17:31:36 kutt-test config testdirs is overridden with args: [ /kuttl-e2e/gitops-operator/tests/parallel ] === RUN kuttl harness.go:462: starting setup harness.go:252: running tests using configured kubeconfig. I0723 17:31:37.369286 167 request.go:682] Waited for 1.012347979s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/policy/v1?timeout=32s harness.go:275: Successful connection to cluster at: https://api.test-aws-412.ocp-gitops-qe.com:6443 harness.go:360: running tests harness.go:73: going to run test suite with timeout of 720 seconds for each step harness.go:372: testsuite: /kuttl-e2e/gitops-operator/tests/parallel has 79 tests === RUN kuttl/harness === RUN kuttl/harness/1-001_alpha_to_beta_dex_conversion === PAUSE kuttl/harness/1-001_alpha_to_beta_dex_conversion === RUN kuttl/harness/1-002_alpha_to_beta_keycloak_conversion === PAUSE kuttl/harness/1-002_alpha_to_beta_keycloak_conversion === RUN kuttl/harness/1-003_alpha_to_beta_sso_conflict_conversion === PAUSE kuttl/harness/1-003_alpha_to_beta_sso_conflict_conversion === RUN kuttl/harness/1-003_validate_console_link === PAUSE kuttl/harness/1-003_validate_console_link === RUN kuttl/harness/1-004_beta_to_alpha_conversion === PAUSE kuttl/harness/1-004_beta_to_alpha_conversion === RUN kuttl/harness/1-007_validate_namespace_scoped_install === PAUSE kuttl/harness/1-007_validate_namespace_scoped_install === RUN kuttl/harness/1-008_validate-custom-argocd-namespace === PAUSE kuttl/harness/1-008_validate-custom-argocd-namespace === RUN kuttl/harness/1-009_validate-manage-other-namespace === PAUSE kuttl/harness/1-009_validate-manage-other-namespace === RUN kuttl/harness/1-012_validate-managed-by-chain === PAUSE kuttl/harness/1-012_validate-managed-by-chain === RUN kuttl/harness/1-019_validate_volume_mounts === PAUSE kuttl/harness/1-019_validate_volume_mounts === RUN kuttl/harness/1-021_validate_rolebindings === PAUSE kuttl/harness/1-021_validate_rolebindings === RUN kuttl/harness/1-023_validate_repo_server_tls === PAUSE kuttl/harness/1-023_validate_repo_server_tls === RUN kuttl/harness/1-025-validate-managed-by-change === PAUSE kuttl/harness/1-025-validate-managed-by-change === RUN kuttl/harness/1-029_validate_tls_secret_no_scale === PAUSE kuttl/harness/1-029_validate_tls_secret_no_scale === RUN kuttl/harness/1-030_validate_reencrypt === PAUSE kuttl/harness/1-030_validate_reencrypt === RUN kuttl/harness/1-032_validate_dynamic_scaling === PAUSE kuttl/harness/1-032_validate_dynamic_scaling === RUN kuttl/harness/1-032_validate_resource_inclusions === PAUSE kuttl/harness/1-032_validate_resource_inclusions === RUN kuttl/harness/1-033_validate_applicationset_tls_scm_volume_mount === PAUSE kuttl/harness/1-033_validate_applicationset_tls_scm_volume_mount === RUN kuttl/harness/1-033_validate_resource_exclusions === PAUSE kuttl/harness/1-033_validate_resource_exclusions === RUN kuttl/harness/1-034_validate_webhook_notifications === PAUSE kuttl/harness/1-034_validate_webhook_notifications === RUN kuttl/harness/1-036_validate_keycloak_resource_reqs === PAUSE kuttl/harness/1-036_validate_keycloak_resource_reqs === RUN kuttl/harness/1-037_validate_argocd_setting_replicas === PAUSE kuttl/harness/1-037_validate_argocd_setting_replicas === RUN kuttl/harness/1-038_validate_productized_images === PAUSE kuttl/harness/1-038_validate_productized_images === RUN kuttl/harness/1-039_validate_fix_argocd-tls-certs-cm === PAUSE kuttl/harness/1-039_validate_fix_argocd-tls-certs-cm === RUN kuttl/harness/1-043_validate_log_level_format === PAUSE kuttl/harness/1-043_validate_log_level_format === RUN kuttl/harness/1-044_validate_resource_limit_changes === PAUSE kuttl/harness/1-044_validate_resource_limit_changes === RUN kuttl/harness/1-045_validate_repo_exec_timeout === PAUSE kuttl/harness/1-045_validate_repo_exec_timeout === RUN kuttl/harness/1-047_validate_custom_env === PAUSE kuttl/harness/1-047_validate_custom_env === RUN kuttl/harness/1-048_validate_controller_sharding === PAUSE kuttl/harness/1-048_validate_controller_sharding === RUN kuttl/harness/1-049_validate_parallelism_limit === PAUSE kuttl/harness/1-049_validate_parallelism_limit === RUN kuttl/harness/1-050_validate_sso === PAUSE kuttl/harness/1-050_validate_sso === RUN kuttl/harness/1-051-validate_csv_permissions === PAUSE kuttl/harness/1-051-validate_csv_permissions === RUN kuttl/harness/1-052_validate_rolebinding_number === PAUSE kuttl/harness/1-052_validate_rolebinding_number === RUN kuttl/harness/1-053_validate_cluster_admin_rbac === PAUSE kuttl/harness/1-053_validate_cluster_admin_rbac === RUN kuttl/harness/1-054_validate_deploymentconfig === PAUSE kuttl/harness/1-054_validate_deploymentconfig === RUN kuttl/harness/1-055_drop_resource_customizations === PAUSE kuttl/harness/1-055_drop_resource_customizations === RUN kuttl/harness/1-055_validate_notification_controller === PAUSE kuttl/harness/1-055_validate_notification_controller === RUN kuttl/harness/1-057_validate_notifications === PAUSE kuttl/harness/1-057_validate_notifications === RUN kuttl/harness/1-058_validate_prometheus_rule === PAUSE kuttl/harness/1-058_validate_prometheus_rule === RUN kuttl/harness/1-061_validate_resource_tracking_method === PAUSE kuttl/harness/1-061_validate_resource_tracking_method === RUN kuttl/harness/1-062_validate_extra_config === PAUSE kuttl/harness/1-062_validate_extra_config === RUN kuttl/harness/1-063_validate_dex_liveness_probe === PAUSE kuttl/harness/1-063_validate_dex_liveness_probe === RUN kuttl/harness/1-063_validate_statefulset_restart === PAUSE kuttl/harness/1-063_validate_statefulset_restart === RUN kuttl/harness/1-064_validate_security_contexts === PAUSE kuttl/harness/1-064_validate_security_contexts === RUN kuttl/harness/1-065_validate_redis_ha_anti_affinity === PAUSE kuttl/harness/1-065_validate_redis_ha_anti_affinity === RUN kuttl/harness/1-066_validate_redis_secure_comm_no_autotls_no_ha === PAUSE kuttl/harness/1-066_validate_redis_secure_comm_no_autotls_no_ha === RUN kuttl/harness/1-067_validate_redis_secure_comm_no_autotls_ha === PAUSE kuttl/harness/1-067_validate_redis_secure_comm_no_autotls_ha === RUN kuttl/harness/1-068_validate_redis_secure_comm_autotls_no_ha === PAUSE kuttl/harness/1-068_validate_redis_secure_comm_autotls_no_ha === RUN kuttl/harness/1-069_validate_redis_secure_comm_autotls_ha === PAUSE kuttl/harness/1-069_validate_redis_secure_comm_autotls_ha === RUN kuttl/harness/1-070_validate_config_management_plugin === PAUSE kuttl/harness/1-070_validate_config_management_plugin === RUN kuttl/harness/1-071_validate_SCC_HA === PAUSE kuttl/harness/1-071_validate_SCC_HA === RUN kuttl/harness/1-072_validate_liveness_probe_removed === PAUSE kuttl/harness/1-072_validate_liveness_probe_removed === RUN kuttl/harness/1-073_validate_rhsso === PAUSE kuttl/harness/1-073_validate_rhsso === RUN kuttl/harness/1-074_validate_terminating_namespace_block === PAUSE kuttl/harness/1-074_validate_terminating_namespace_block === RUN kuttl/harness/1-075_validate_dex_anyuid === PAUSE kuttl/harness/1-075_validate_dex_anyuid === RUN kuttl/harness/1-077_validate_disable_dex_removed === PAUSE kuttl/harness/1-077_validate_disable_dex_removed === RUN kuttl/harness/1-079_validate_vars_for_notificaitons === PAUSE kuttl/harness/1-079_validate_vars_for_notificaitons === RUN kuttl/harness/1-080_validate_regex_support_argocd_rbac === PAUSE kuttl/harness/1-080_validate_regex_support_argocd_rbac === RUN kuttl/harness/1-081_validate_applicationset_deployment === PAUSE kuttl/harness/1-081_validate_applicationset_deployment === RUN kuttl/harness/1-082_validate_node_placement === PAUSE kuttl/harness/1-082_validate_node_placement === RUN kuttl/harness/1-083_validate_kustomize_namereference === PAUSE kuttl/harness/1-083_validate_kustomize_namereference === RUN kuttl/harness/1-083_validate_resource_customization_subkeys === PAUSE kuttl/harness/1-083_validate_resource_customization_subkeys === RUN kuttl/harness/1-084_validate_status_host_ingress === PAUSE kuttl/harness/1-084_validate_status_host_ingress === RUN kuttl/harness/1-086_validate_applicationset_extra_command_args === PAUSE kuttl/harness/1-086_validate_applicationset_extra_command_args === RUN kuttl/harness/1-087_validate_repo_server_settings === PAUSE kuttl/harness/1-087_validate_repo_server_settings === RUN kuttl/harness/1-088_validate_applicationset_add_env_vars === PAUSE kuttl/harness/1-088_validate_applicationset_add_env_vars === RUN kuttl/harness/1-089_validate_extra_repo_commands_args === PAUSE kuttl/harness/1-089_validate_extra_repo_commands_args === RUN kuttl/harness/1-090_validate_permissions === PAUSE kuttl/harness/1-090_validate_permissions === RUN kuttl/harness/1-091_argocd_app_validate-applicationset-status === PAUSE kuttl/harness/1-091_argocd_app_validate-applicationset-status === RUN kuttl/harness/1-092_validate_workload_status_monitoring_alert === PAUSE kuttl/harness/1-092_validate_workload_status_monitoring_alert === RUN kuttl/harness/1-095_validate_dex_clientsecret === PAUSE kuttl/harness/1-095_validate_dex_clientsecret === RUN kuttl/harness/1-096-validate_home_env_argocd_controller === PAUSE kuttl/harness/1-096-validate_home_env_argocd_controller === RUN kuttl/harness/1-098_validate_dex_clientsecret_deprecated === PAUSE kuttl/harness/1-098_validate_dex_clientsecret_deprecated === RUN kuttl/harness/1-099_validate_server_autoscale === PAUSE kuttl/harness/1-099_validate_server_autoscale === RUN kuttl/harness/1-102_validate_handle_terminating_namespaces === PAUSE kuttl/harness/1-102_validate_handle_terminating_namespaces === RUN kuttl/harness/1-103_argocd_alpha_to_beta_conversion === PAUSE kuttl/harness/1-103_argocd_alpha_to_beta_conversion === RUN kuttl/harness/1-104_validate_applicationset_tls_scm_volume_mount === PAUSE kuttl/harness/1-104_validate_applicationset_tls_scm_volume_mount === RUN kuttl/harness/1-107_host_attribute_sso_provider === PAUSE kuttl/harness/1-107_host_attribute_sso_provider === RUN kuttl/harness/1-108_alternate_cluster_roles_cluster_scoped_instance === PAUSE kuttl/harness/1-108_alternate_cluster_roles_cluster_scoped_instance === CONT kuttl/harness/1-001_alpha_to_beta_dex_conversion === CONT kuttl/harness/1-062_validate_extra_config === CONT kuttl/harness/1-036_validate_keycloak_resource_reqs === CONT kuttl/harness/1-021_validate_rolebindings === CONT kuttl/harness/1-083_validate_kustomize_namereference === CONT kuttl/harness/1-021_validate_rolebindings logger.go:42: 17:31:43 | 1-021_validate_rolebindings | Creating namespace: kuttl-test-causal-octopus === CONT kuttl/harness/1-001_alpha_to_beta_dex_conversion logger.go:42: 17:31:43 | 1-001_alpha_to_beta_dex_conversion | Creating namespace: kuttl-test-relieved-moose === CONT kuttl/harness/1-083_validate_kustomize_namereference logger.go:42: 17:31:43 | 1-083_validate_kustomize_namereference | Creating namespace: kuttl-test-prompt-lamprey === CONT kuttl/harness/1-036_validate_keycloak_resource_reqs logger.go:42: 17:31:43 | 1-036_validate_keycloak_resource_reqs | Creating namespace: kuttl-test-real-snake === CONT kuttl/harness/1-062_validate_extra_config logger.go:42: 17:31:43 | 1-062_validate_extra_config | Creating namespace: kuttl-test-known-whippet === CONT kuttl/harness/1-021_validate_rolebindings logger.go:42: 17:31:43 | 1-021_validate_rolebindings/1-install | starting test step 1-install === CONT kuttl/harness/1-083_validate_kustomize_namereference logger.go:42: 17:31:43 | 1-083_validate_kustomize_namereference/1-install | starting test step 1-install === CONT kuttl/harness/1-001_alpha_to_beta_dex_conversion logger.go:42: 17:31:43 | 1-001_alpha_to_beta_dex_conversion/1-argocd-dex | starting test step 1-argocd-dex === CONT kuttl/harness/1-036_validate_keycloak_resource_reqs logger.go:42: 17:31:43 | 1-036_validate_keycloak_resource_reqs/1-install | starting test step 1-install === CONT kuttl/harness/1-062_validate_extra_config logger.go:42: 17:31:43 | 1-062_validate_extra_config/1-argocd-with-extraconfig | starting test step 1-argocd-with-extraconfig === CONT kuttl/harness/1-021_validate_rolebindings logger.go:42: 17:31:46 | 1-021_validate_rolebindings/1-install | ArgoCD:kuttl-test-causal-octopus/argocd created I0723 17:31:47.412396 167 request.go:682] Waited for 1.24768274s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/monitoring.coreos.com/v1alpha1?timeout=32s === CONT kuttl/harness/1-083_validate_kustomize_namereference logger.go:42: 17:31:51 | 1-083_validate_kustomize_namereference/1-install | Namespace:/namespace-gitops-2038 created === CONT kuttl/harness/1-001_alpha_to_beta_dex_conversion logger.go:42: 17:31:51 | 1-001_alpha_to_beta_dex_conversion/1-argocd-dex | ArgoCD:kuttl-test-relieved-moose/argocd created === CONT kuttl/harness/1-083_validate_kustomize_namereference logger.go:42: 17:31:51 | 1-083_validate_kustomize_namereference/1-install | ArgoCD:namespace-gitops-2038/argocd created === CONT kuttl/harness/1-036_validate_keycloak_resource_reqs logger.go:42: 17:31:56 | 1-036_validate_keycloak_resource_reqs/1-install | ArgoCD:kuttl-test-real-snake/argocd created === CONT kuttl/harness/1-062_validate_extra_config logger.go:42: 17:31:56 | 1-062_validate_extra_config/1-argocd-with-extraconfig | ArgoCD:kuttl-test-known-whippet/example-argocd created === CONT kuttl/harness/1-021_validate_rolebindings logger.go:42: 17:32:00 | 1-021_validate_rolebindings/1-install | test step completed 1-install logger.go:42: 17:32:00 | 1-021_validate_rolebindings | skipping kubernetes event logging logger.go:42: 17:32:00 | 1-021_validate_rolebindings | Deleting namespace: kuttl-test-causal-octopus === CONT kuttl/harness/1-001_alpha_to_beta_dex_conversion logger.go:42: 17:32:03 | 1-001_alpha_to_beta_dex_conversion/1-argocd-dex | test step completed 1-argocd-dex logger.go:42: 17:32:03 | 1-001_alpha_to_beta_dex_conversion/2-delete | starting test step 2-delete I0723 17:32:04.335107 167 request.go:682] Waited for 1.046516107s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/argoproj.io/v1beta1?timeout=32s logger.go:42: 17:32:05 | 1-001_alpha_to_beta_dex_conversion/2-delete | test step completed 2-delete logger.go:42: 17:32:05 | 1-001_alpha_to_beta_dex_conversion | skipping kubernetes event logging === CONT kuttl/harness/1-083_validate_kustomize_namereference logger.go:42: 17:32:05 | 1-083_validate_kustomize_namereference/1-install | test step completed 1-install logger.go:42: 17:32:05 | 1-083_validate_kustomize_namereference/2-install-kustomize-app | starting test step 2-install-kustomize-app === CONT kuttl/harness/1-001_alpha_to_beta_dex_conversion logger.go:42: 17:32:05 | 1-001_alpha_to_beta_dex_conversion | Deleting namespace: kuttl-test-relieved-moose === CONT kuttl/harness/1-108_alternate_cluster_roles_cluster_scoped_instance 2024/07/23 17:32:06 object detected with no GVK Kind for path /kuttl-e2e/gitops-operator/tests/parallel/1-108_alternate_cluster_roles_cluster_scoped_instance/04-create-app.yaml logger.go:42: 17:32:08 | 1-108_alternate_cluster_roles_cluster_scoped_instance | Creating namespace: kuttl-test-square-whale logger.go:42: 17:32:08 | 1-108_alternate_cluster_roles_cluster_scoped_instance/1-create-ns | starting test step 1-create-ns === CONT kuttl/harness/1-083_validate_kustomize_namereference logger.go:42: 17:32:08 | 1-083_validate_kustomize_namereference/2-install-kustomize-app | Application:namespace-gitops-2038/app-kustomize created logger.go:42: 17:32:10 | 1-083_validate_kustomize_namereference/2-install-kustomize-app | test step completed 2-install-kustomize-app logger.go:42: 17:32:10 | 1-083_validate_kustomize_namereference/99-delete | starting test step 99-delete === CONT kuttl/harness/1-108_alternate_cluster_roles_cluster_scoped_instance logger.go:42: 17:32:10 | 1-108_alternate_cluster_roles_cluster_scoped_instance/1-create-ns | Namespace:/alternate-role created logger.go:42: 17:32:10 | 1-108_alternate_cluster_roles_cluster_scoped_instance/1-create-ns | test step completed 1-create-ns logger.go:42: 17:32:10 | 1-108_alternate_cluster_roles_cluster_scoped_instance/2-edit-subscription | starting test step 2-edit-subscription logger.go:42: 17:32:10 | 1-108_alternate_cluster_roles_cluster_scoped_instance/2-edit-subscription | running command: [sh -c oc patch -n openshift-operators subscription openshift-gitops-operator \ --type merge --patch '{"spec": {"config": {"env": [{"name": "ARGOCD_CLUSTER_CONFIG_NAMESPACES", "value": "openshift-gitops, alternate-role"}]}}}' ] logger.go:42: 17:32:11 | 1-108_alternate_cluster_roles_cluster_scoped_instance/2-edit-subscription | subscription.operators.coreos.com/openshift-gitops-operator patched === CONT kuttl/harness/1-107_host_attribute_sso_provider logger.go:42: 17:32:13 | 1-107_host_attribute_sso_provider | Creating namespace: kuttl-test-flying-goldfish === CONT kuttl/harness/1-108_alternate_cluster_roles_cluster_scoped_instance logger.go:42: 17:32:13 | 1-108_alternate_cluster_roles_cluster_scoped_instance/2-edit-subscription | test step completed 2-edit-subscription logger.go:42: 17:32:13 | 1-108_alternate_cluster_roles_cluster_scoped_instance/3-install | starting test step 3-install === CONT kuttl/harness/1-107_host_attribute_sso_provider logger.go:42: 17:32:13 | 1-107_host_attribute_sso_provider/1-install | starting test step 1-install I0723 17:32:15.021522 167 request.go:682] Waited for 1.047438572s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/security.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-108_alternate_cluster_roles_cluster_scoped_instance logger.go:42: 17:32:16 | 1-108_alternate_cluster_roles_cluster_scoped_instance/3-install | ArgoCD:alternate-role/argocd created === CONT kuttl/harness/1-107_host_attribute_sso_provider logger.go:42: 17:32:18 | 1-107_host_attribute_sso_provider/1-install | ArgoCD:kuttl-test-flying-goldfish/example-argocd-keycloak created I0723 17:32:25.023792 167 request.go:682] Waited for 1.184345992s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/cloudcredential.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-083_validate_kustomize_namereference logger.go:42: 17:32:26 | 1-083_validate_kustomize_namereference/99-delete | test step completed 99-delete logger.go:42: 17:32:26 | 1-083_validate_kustomize_namereference | skipping kubernetes event logging logger.go:42: 17:32:26 | 1-083_validate_kustomize_namereference | Deleting namespace: kuttl-test-prompt-lamprey === CONT kuttl/harness/1-104_validate_applicationset_tls_scm_volume_mount logger.go:42: 17:32:32 | 1-104_validate_applicationset_tls_scm_volume_mount | Creating namespace: kuttl-test-arriving-polliwog logger.go:42: 17:32:32 | 1-104_validate_applicationset_tls_scm_volume_mount/1-install | starting test step 1-install logger.go:42: 17:32:34 | 1-104_validate_applicationset_tls_scm_volume_mount/1-install | Namespace:/test-1-104-appsets-scm-tls-mount created logger.go:42: 17:32:34 | 1-104_validate_applicationset_tls_scm_volume_mount/1-install | ConfigMap:test-1-104-appsets-scm-tls-mount/test-1-104-appsets-scm-tls-cm created logger.go:42: 17:32:34 | 1-104_validate_applicationset_tls_scm_volume_mount/1-install | ArgoCD:test-1-104-appsets-scm-tls-mount/argocd created === CONT kuttl/harness/1-062_validate_extra_config logger.go:42: 17:32:42 | 1-062_validate_extra_config/1-argocd-with-extraconfig | test step completed 1-argocd-with-extraconfig logger.go:42: 17:32:42 | 1-062_validate_extra_config/2-argocd-with-firstclass-and-extraconfig copy | starting test step 2-argocd-with-firstclass-and-extraconfig copy I0723 17:32:43.859408 167 request.go:682] Waited for 1.047531827s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/imageregistry.operator.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-107_host_attribute_sso_provider logger.go:42: 17:32:45 | 1-107_host_attribute_sso_provider/1-install | test step completed 1-install logger.go:42: 17:32:45 | 1-107_host_attribute_sso_provider | skipping kubernetes event logging === CONT kuttl/harness/1-036_validate_keycloak_resource_reqs logger.go:42: 17:32:45 | 1-036_validate_keycloak_resource_reqs/1-install | test step completed 1-install logger.go:42: 17:32:45 | 1-036_validate_keycloak_resource_reqs/2-update-sso-keycloak-provider | starting test step 2-update-sso-keycloak-provider logger.go:42: 17:32:45 | 1-036_validate_keycloak_resource_reqs/2-update-sso-keycloak-provider | running command: [sh -c oc patch -n $NAMESPACE argocd/argocd --type='json' -p='[{"op": "add", "path": "/spec/sso", "value": {"provider": "keycloak"}}]' ] === CONT kuttl/harness/1-107_host_attribute_sso_provider logger.go:42: 17:32:45 | 1-107_host_attribute_sso_provider | Deleting namespace: kuttl-test-flying-goldfish === CONT kuttl/harness/1-062_validate_extra_config logger.go:42: 17:32:45 | 1-062_validate_extra_config/2-argocd-with-firstclass-and-extraconfig copy | ArgoCD:kuttl-test-known-whippet/example-argocd updated logger.go:42: 17:32:45 | 1-062_validate_extra_config/2-argocd-with-firstclass-and-extraconfig copy | test step completed 2-argocd-with-firstclass-and-extraconfig copy logger.go:42: 17:32:45 | 1-062_validate_extra_config/3-update-configmap | starting test step 3-update-configmap === CONT kuttl/harness/1-036_validate_keycloak_resource_reqs logger.go:42: 17:32:45 | 1-036_validate_keycloak_resource_reqs/2-update-sso-keycloak-provider | argocd.argoproj.io/argocd patched === CONT kuttl/harness/1-062_validate_extra_config logger.go:42: 17:32:47 | 1-062_validate_extra_config/3-update-configmap | ConfigMap:kuttl-test-known-whippet/argocd-cm updated logger.go:42: 17:32:50 | 1-062_validate_extra_config/3-update-configmap | test step completed 3-update-configmap logger.go:42: 17:32:50 | 1-062_validate_extra_config/4-create-dex-without-extraconfig | starting test step 4-create-dex-without-extraconfig === CONT kuttl/harness/1-108_alternate_cluster_roles_cluster_scoped_instance logger.go:42: 17:32:52 | 1-108_alternate_cluster_roles_cluster_scoped_instance/3-install | test step completed 3-install logger.go:42: 17:32:52 | 1-108_alternate_cluster_roles_cluster_scoped_instance/4-create-app | starting test step 4-create-app === CONT kuttl/harness/1-062_validate_extra_config logger.go:42: 17:32:52 | 1-062_validate_extra_config/4-create-dex-without-extraconfig | ArgoCD:kuttl-test-known-whippet/example-argocd updated I0723 17:32:53.884137 167 request.go:682] Waited for 1.147066964s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/network.openshift.io/v1?timeout=32s logger.go:42: 17:32:55 | 1-062_validate_extra_config/4-create-dex-without-extraconfig | test step completed 4-create-dex-without-extraconfig logger.go:42: 17:32:55 | 1-062_validate_extra_config/5-override-dex-using-extraconfig | starting test step 5-override-dex-using-extraconfig === CONT kuttl/harness/1-104_validate_applicationset_tls_scm_volume_mount logger.go:42: 17:32:55 | 1-104_validate_applicationset_tls_scm_volume_mount/1-install | test step completed 1-install logger.go:42: 17:32:55 | 1-104_validate_applicationset_tls_scm_volume_mount/99-delete | starting test step 99-delete === CONT kuttl/harness/1-108_alternate_cluster_roles_cluster_scoped_instance logger.go:42: 17:32:55 | 1-108_alternate_cluster_roles_cluster_scoped_instance/4-create-app | Application:alternate-role/clusterrole-app created === CONT kuttl/harness/1-062_validate_extra_config logger.go:42: 17:32:57 | 1-062_validate_extra_config/5-override-dex-using-extraconfig | ArgoCD:kuttl-test-known-whippet/example-argocd updated logger.go:42: 17:32:58 | 1-062_validate_extra_config/5-override-dex-using-extraconfig | test step completed 5-override-dex-using-extraconfig logger.go:42: 17:32:58 | 1-062_validate_extra_config | skipping kubernetes event logging logger.go:42: 17:32:58 | 1-062_validate_extra_config | Deleting namespace: kuttl-test-known-whippet === CONT kuttl/harness/1-108_alternate_cluster_roles_cluster_scoped_instance logger.go:42: 17:33:01 | 1-108_alternate_cluster_roles_cluster_scoped_instance/4-create-app | test step completed 4-create-app logger.go:42: 17:33:01 | 1-108_alternate_cluster_roles_cluster_scoped_instance/5-disable-role | starting test step 5-disable-role I0723 17:33:03.886340 167 request.go:682] Waited for 2.146315132s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/autoscaling.openshift.io/v1beta1?timeout=32s logger.go:42: 17:33:04 | 1-108_alternate_cluster_roles_cluster_scoped_instance/5-disable-role | ArgoCD:alternate-role/argocd updated === CONT kuttl/harness/1-103_argocd_alpha_to_beta_conversion logger.go:42: 17:33:05 | 1-103_argocd_alpha_to_beta_conversion | Creating namespace: kuttl-test-funky-weasel logger.go:42: 17:33:05 | 1-103_argocd_alpha_to_beta_conversion/1-alpha_argocd | starting test step 1-alpha_argocd logger.go:42: 17:33:07 | 1-103_argocd_alpha_to_beta_conversion/1-alpha_argocd | ArgoCD:kuttl-test-funky-weasel/argocd created === CONT kuttl/harness/1-108_alternate_cluster_roles_cluster_scoped_instance logger.go:42: 17:33:07 | 1-108_alternate_cluster_roles_cluster_scoped_instance/5-disable-role | test step completed 5-disable-role logger.go:42: 17:33:07 | 1-108_alternate_cluster_roles_cluster_scoped_instance/6-create-cluster-role | starting test step 6-create-cluster-role logger.go:42: 17:33:10 | 1-108_alternate_cluster_roles_cluster_scoped_instance/6-create-cluster-role | ClusterRole:/argocd-alternate-role-argocd-application-controller created logger.go:42: 17:33:10 | 1-108_alternate_cluster_roles_cluster_scoped_instance/6-create-cluster-role | test step completed 6-create-cluster-role logger.go:42: 17:33:10 | 1-108_alternate_cluster_roles_cluster_scoped_instance/7-create-cluster-rolebinding | starting test step 7-create-cluster-rolebinding logger.go:42: 17:33:12 | 1-108_alternate_cluster_roles_cluster_scoped_instance/7-create-cluster-rolebinding | ClusterRoleBinding:/argocd-alternate-role-argocd-application-controller created logger.go:42: 17:33:12 | 1-108_alternate_cluster_roles_cluster_scoped_instance/7-create-cluster-rolebinding | test step completed 7-create-cluster-rolebinding logger.go:42: 17:33:12 | 1-108_alternate_cluster_roles_cluster_scoped_instance/8-delete | starting test step 8-delete I0723 17:33:19.335983 167 request.go:682] Waited for 1.04538351s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/autoscaling/v2?timeout=32s logger.go:42: 17:33:20 | 1-108_alternate_cluster_roles_cluster_scoped_instance/8-delete | test step completed 8-delete logger.go:42: 17:33:20 | 1-108_alternate_cluster_roles_cluster_scoped_instance/9-check-app-message | starting test step 9-check-app-message logger.go:42: 17:33:20 | 1-108_alternate_cluster_roles_cluster_scoped_instance/9-check-app-message | running command: [sh -c sleep 60 message=$(oc get application.argoproj.io/clusterrole-app -n alternate-role -o jsonpath='{.status.operationState.message}') echo $message if ! [[ $message == *"persistentvolumes is forbidden"* ]]; then exit 1 fi ] === CONT kuttl/harness/1-103_argocd_alpha_to_beta_conversion logger.go:42: 17:33:39 | 1-103_argocd_alpha_to_beta_conversion/1-alpha_argocd | test step completed 1-alpha_argocd logger.go:42: 17:33:39 | 1-103_argocd_alpha_to_beta_conversion/2-delete | starting test step 2-delete I0723 17:33:41.222479 167 request.go:682] Waited for 1.045654428s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/route.openshift.io/v1?timeout=32s logger.go:42: 17:33:42 | 1-103_argocd_alpha_to_beta_conversion/2-delete | test step completed 2-delete logger.go:42: 17:33:42 | 1-103_argocd_alpha_to_beta_conversion | skipping kubernetes event logging logger.go:42: 17:33:42 | 1-103_argocd_alpha_to_beta_conversion | Deleting namespace: kuttl-test-funky-weasel === CONT kuttl/harness/1-102_validate_handle_terminating_namespaces logger.go:42: 17:33:49 | 1-102_validate_handle_terminating_namespaces | Creating namespace: kuttl-test-summary-panda logger.go:42: 17:33:49 | 1-102_validate_handle_terminating_namespaces/1-install | starting test step 1-install I0723 17:33:51.271643 167 request.go:682] Waited for 2.194899646s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/network.openshift.io/v1?timeout=32s logger.go:42: 17:33:51 | 1-102_validate_handle_terminating_namespaces/1-install | Namespace:/gitops-service-argocd created logger.go:42: 17:33:51 | 1-102_validate_handle_terminating_namespaces/1-install | test step completed 1-install logger.go:42: 17:33:51 | 1-102_validate_handle_terminating_namespaces/2-install | starting test step 2-install logger.go:42: 17:33:54 | 1-102_validate_handle_terminating_namespaces/2-install | ArgoCD:gitops-service-argocd/gitops-service-argocd created logger.go:42: 17:33:54 | 1-102_validate_handle_terminating_namespaces/2-install | Namespace:/jane created logger.go:42: 17:33:54 | 1-102_validate_handle_terminating_namespaces/2-install | ConfigMap:jane/my-config-map-2 created === CONT kuttl/harness/1-099_validate_server_autoscale logger.go:42: 17:34:14 | 1-099_validate_server_autoscale | Creating namespace: kuttl-test-concise-tetra logger.go:42: 17:34:14 | 1-099_validate_server_autoscale/1-install | starting test step 1-install I0723 17:34:15.412606 167 request.go:682] Waited for 1.046638253s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/network.operator.openshift.io/v1?timeout=32s logger.go:42: 17:34:16 | 1-099_validate_server_autoscale/1-install | ArgoCD:kuttl-test-concise-tetra/example-argocd created === CONT kuttl/harness/1-108_alternate_cluster_roles_cluster_scoped_instance logger.go:42: 17:34:21 | 1-108_alternate_cluster_roles_cluster_scoped_instance/9-check-app-message | one or more objects failed to apply, reason: persistentvolumes is forbidden: User "system:serviceaccount:alternate-role:argocd-argocd-application-controller" cannot create resource "persistentvolumes" in API group "" at the cluster scope. Retrying attempt #3 at 5:34PM. logger.go:42: 17:34:23 | 1-108_alternate_cluster_roles_cluster_scoped_instance/9-check-app-message | test step completed 9-check-app-message logger.go:42: 17:34:23 | 1-108_alternate_cluster_roles_cluster_scoped_instance/10-modify-clusterrole | starting test step 10-modify-clusterrole I0723 17:34:25.422935 167 request.go:682] Waited for 1.547270635s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/helm.openshift.io/v1beta1?timeout=32s logger.go:42: 17:34:26 | 1-108_alternate_cluster_roles_cluster_scoped_instance/10-modify-clusterrole | ClusterRole:/argocd-alternate-role-argocd-application-controller updated === CONT kuttl/harness/1-102_validate_handle_terminating_namespaces logger.go:42: 17:34:27 | 1-102_validate_handle_terminating_namespaces/2-install | test step completed 2-install logger.go:42: 17:34:27 | 1-102_validate_handle_terminating_namespaces/3-delete-ns | starting test step 3-delete-ns logger.go:42: 17:34:27 | 1-102_validate_handle_terminating_namespaces/3-delete-ns | running command: [kubectl delete ns jane] logger.go:42: 17:34:27 | 1-102_validate_handle_terminating_namespaces/3-delete-ns | namespace "jane" deleted === CONT kuttl/harness/1-099_validate_server_autoscale logger.go:42: 17:34:48 | 1-099_validate_server_autoscale/1-install | test step completed 1-install logger.go:42: 17:34:48 | 1-099_validate_server_autoscale/2-enable-server-autoscale | starting test step 2-enable-server-autoscale I0723 17:34:49.882705 167 request.go:682] Waited for 1.047499191s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/performance.openshift.io/v1alpha1?timeout=32s logger.go:42: 17:34:51 | 1-099_validate_server_autoscale/2-enable-server-autoscale | ArgoCD:kuttl-test-concise-tetra/example-argocd updated logger.go:42: 17:34:51 | 1-099_validate_server_autoscale/2-enable-server-autoscale | test step completed 2-enable-server-autoscale logger.go:42: 17:34:51 | 1-099_validate_server_autoscale/3-check-server-replicas | starting test step 3-check-server-replicas === CONT kuttl/harness/1-104_validate_applicationset_tls_scm_volume_mount logger.go:42: 17:34:53 | 1-104_validate_applicationset_tls_scm_volume_mount/99-delete | test step completed 99-delete logger.go:42: 17:34:53 | 1-104_validate_applicationset_tls_scm_volume_mount | skipping kubernetes event logging === CONT kuttl/harness/1-099_validate_server_autoscale logger.go:42: 17:34:53 | 1-099_validate_server_autoscale/3-check-server-replicas | running command: [sh -c sleep 45] === CONT kuttl/harness/1-104_validate_applicationset_tls_scm_volume_mount logger.go:42: 17:34:53 | 1-104_validate_applicationset_tls_scm_volume_mount | Deleting namespace: kuttl-test-arriving-polliwog === CONT kuttl/harness/1-098_validate_dex_clientsecret_deprecated === CONT kuttl/harness/1-102_validate_handle_terminating_namespaces logger.go:42: 17:34:59 | 1-102_validate_handle_terminating_namespaces/3-delete-ns | test step completed 3-delete-ns logger.go:42: 17:34:59 | 1-102_validate_handle_terminating_namespaces/4-create-ns | starting test step 4-create-ns I0723 17:35:00.887267 167 request.go:682] Waited for 1.043266356s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/network.operator.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-098_validate_dex_clientsecret_deprecated logger.go:42: 17:35:02 | 1-098_validate_dex_clientsecret_deprecated | Creating namespace: kuttl-test-meet-tick logger.go:42: 17:35:02 | 1-098_validate_dex_clientsecret_deprecated/1-install | starting test step 1-install === CONT kuttl/harness/1-102_validate_handle_terminating_namespaces logger.go:42: 17:35:02 | 1-102_validate_handle_terminating_namespaces/4-create-ns | Namespace:/john created logger.go:42: 17:35:04 | 1-102_validate_handle_terminating_namespaces/4-create-ns | test step completed 4-create-ns logger.go:42: 17:35:04 | 1-102_validate_handle_terminating_namespaces/5-create-app | starting test step 5-create-app === CONT kuttl/harness/1-098_validate_dex_clientsecret_deprecated logger.go:42: 17:35:04 | 1-098_validate_dex_clientsecret_deprecated/1-install | ArgoCD:kuttl-test-meet-tick/example-argocd created === CONT kuttl/harness/1-102_validate_handle_terminating_namespaces logger.go:42: 17:35:07 | 1-102_validate_handle_terminating_namespaces/5-create-app | Application:gitops-service-argocd/my-app created logger.go:42: 17:35:09 | 1-102_validate_handle_terminating_namespaces/5-create-app | test step completed 5-create-app logger.go:42: 17:35:09 | 1-102_validate_handle_terminating_namespaces/99-delete | starting test step 99-delete === CONT kuttl/harness/1-036_validate_keycloak_resource_reqs logger.go:42: 17:35:23 | 1-036_validate_keycloak_resource_reqs/2-update-sso-keycloak-provider | test step completed 2-update-sso-keycloak-provider logger.go:42: 17:35:23 | 1-036_validate_keycloak_resource_reqs/3-check-resources | starting test step 3-check-resources logger.go:42: 17:35:23 | 1-036_validate_keycloak_resource_reqs/3-check-resources | running command: [sh -c resources=$(oc get -n $NAMESPACE pod/keycloak-1-deploy -o jsonpath='{.spec.containers[0].resources}') if [[ -n ${resources} ]]; then if [[ "${resources}" != '{"limits":{"cpu":"500m","memory":"512Mi"},"requests":{"cpu":"250m","memory":"256Mi"}}' ]]; then echo "resources spec for pod/keycloak-1-deploy are incorrect" echo "${resources}" exit 1 fi else echo "resources spec not found in pod/keycloak-1-deploy" exit 1 fi ] I0723 17:35:25.328703 167 request.go:682] Waited for 1.025558921s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/pipelines.openshift.io/v1alpha1?timeout=32s logger.go:42: 17:35:26 | 1-036_validate_keycloak_resource_reqs/3-check-resources | test step completed 3-check-resources logger.go:42: 17:35:26 | 1-036_validate_keycloak_resource_reqs | skipping kubernetes event logging logger.go:42: 17:35:26 | 1-036_validate_keycloak_resource_reqs | Deleting namespace: kuttl-test-real-snake === CONT kuttl/harness/1-102_validate_handle_terminating_namespaces logger.go:42: 17:35:35 | 1-102_validate_handle_terminating_namespaces/99-delete | running command: [sh -c oc patch cm/my-config-map-2 -n jane --type json --patch='[ { "op": "remove", "path": "/metadata/finalizers" }]'] ] logger.go:42: 17:35:35 | 1-102_validate_handle_terminating_namespaces/99-delete | configmap/my-config-map-2 patched I0723 17:35:36.657710 167 request.go:682] Waited for 1.04318554s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/project.openshift.io/v1?timeout=32s logger.go:42: 17:35:38 | 1-102_validate_handle_terminating_namespaces/99-delete | test step completed 99-delete logger.go:42: 17:35:38 | 1-102_validate_handle_terminating_namespaces | skipping kubernetes event logging === CONT kuttl/harness/1-098_validate_dex_clientsecret_deprecated logger.go:42: 17:35:38 | 1-098_validate_dex_clientsecret_deprecated/1-install | test step completed 1-install logger.go:42: 17:35:38 | 1-098_validate_dex_clientsecret_deprecated/2-verify-clientsecret | starting test step 2-verify-clientsecret logger.go:42: 17:35:38 | 1-098_validate_dex_clientsecret_deprecated/2-verify-clientsecret | running command: [sh -c # This test validates the Dex Client Secret copied by the operator from dex serviceaccount token secret in to argocd-secret. # To verify the behavior we should first get the token secret name of the dex service account. secret=$(oc get -n $NAMESPACE sa example-argocd-argocd-dex-server -o json | jq -r '.secrets' | grep token | sed 's/ "name": "//g' | sed 's/"//g') # Extract the clientSecret expectedClientSecret=$(oc get secret $secret -n $NAMESPACE -o json | jq -r '.data.token') # actualClientSecret is the value of the secret in argocd-secret where argocd-operator should copy the secret from actualClientSecret=$(oc get secret argocd-secret -o json -n $NAMESPACE | jq -r '.data."oidc.dex.clientSecret"') # Verify if [[ ${expectedClientSecret} != ${actualClientSecret} ]]; then echo "Error: Dex Client Secret for OIDC is not valid" exit 1 fi ] === CONT kuttl/harness/1-102_validate_handle_terminating_namespaces logger.go:42: 17:35:38 | 1-102_validate_handle_terminating_namespaces | Deleting namespace: kuttl-test-summary-panda === CONT kuttl/harness/1-096-validate_home_env_argocd_controller logger.go:42: 17:35:38 | 1-096-validate_home_env_argocd_controller | Creating namespace: kuttl-test-funny-crane logger.go:42: 17:35:38 | 1-096-validate_home_env_argocd_controller/1- | starting test step 1- === CONT kuttl/harness/1-099_validate_server_autoscale logger.go:42: 17:35:38 | 1-099_validate_server_autoscale/3-check-server-replicas | running command: [sh -c set -e serverReplicas=$(kubectl get -n $NAMESPACE deployment/example-argocd-server -o jsonpath='{.spec.replicas}') if [ "$serverReplicas" -ge 4 ] && [ "$serverReplicas" -le 7 ]; then echo "server replica count matches expectation" exit 0 fi exit 1 done ] logger.go:42: 17:35:39 | 1-099_validate_server_autoscale/3-check-server-replicas | server replica count matches expectation === CONT kuttl/harness/1-096-validate_home_env_argocd_controller logger.go:42: 17:35:41 | 1-096-validate_home_env_argocd_controller/1- | test step completed 1- logger.go:42: 17:35:41 | 1-096-validate_home_env_argocd_controller | skipping kubernetes event logging logger.go:42: 17:35:41 | 1-096-validate_home_env_argocd_controller | Deleting namespace: kuttl-test-funny-crane === CONT kuttl/harness/1-099_validate_server_autoscale logger.go:42: 17:35:43 | 1-099_validate_server_autoscale/3-check-server-replicas | test step completed 3-check-server-replicas logger.go:42: 17:35:43 | 1-099_validate_server_autoscale/4-update-hpa | starting test step 4-update-hpa === CONT kuttl/harness/1-095_validate_dex_clientsecret === CONT kuttl/harness/1-092_validate_workload_status_monitoring_alert I0723 17:35:47.038819 167 request.go:682] Waited for 1.047381523s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/whereabouts.cni.cncf.io/v1alpha1?timeout=32s logger.go:42: 17:35:48 | 1-092_validate_workload_status_monitoring_alert | Creating namespace: kuttl-test-flexible-mallard === CONT kuttl/harness/1-098_validate_dex_clientsecret_deprecated logger.go:42: 17:35:48 | 1-098_validate_dex_clientsecret_deprecated/2-verify-clientsecret | test step completed 2-verify-clientsecret logger.go:42: 17:35:48 | 1-098_validate_dex_clientsecret_deprecated | skipping kubernetes event logging === CONT kuttl/harness/1-095_validate_dex_clientsecret logger.go:42: 17:35:48 | 1-095_validate_dex_clientsecret | Creating namespace: kuttl-test-sure-ox === CONT kuttl/harness/1-092_validate_workload_status_monitoring_alert logger.go:42: 17:35:48 | 1-092_validate_workload_status_monitoring_alert/1-install | starting test step 1-install === CONT kuttl/harness/1-095_validate_dex_clientsecret logger.go:42: 17:35:48 | 1-095_validate_dex_clientsecret/1-install | starting test step 1-install === CONT kuttl/harness/1-098_validate_dex_clientsecret_deprecated logger.go:42: 17:35:48 | 1-098_validate_dex_clientsecret_deprecated | Deleting namespace: kuttl-test-meet-tick === CONT kuttl/harness/1-099_validate_server_autoscale logger.go:42: 17:35:48 | 1-099_validate_server_autoscale/4-update-hpa | ArgoCD:kuttl-test-concise-tetra/example-argocd updated === CONT kuttl/harness/1-092_validate_workload_status_monitoring_alert logger.go:42: 17:35:50 | 1-092_validate_workload_status_monitoring_alert/1-install | ArgoCD:kuttl-test-flexible-mallard/argocd created === CONT kuttl/harness/1-099_validate_server_autoscale logger.go:42: 17:35:53 | 1-099_validate_server_autoscale/4-update-hpa | test step completed 4-update-hpa logger.go:42: 17:35:53 | 1-099_validate_server_autoscale | skipping kubernetes event logging === CONT kuttl/harness/1-108_alternate_cluster_roles_cluster_scoped_instance logger.go:42: 17:35:53 | 1-108_alternate_cluster_roles_cluster_scoped_instance/10-modify-clusterrole | test step completed 10-modify-clusterrole logger.go:42: 17:35:53 | 1-108_alternate_cluster_roles_cluster_scoped_instance/11-delete | starting test step 11-delete === CONT kuttl/harness/1-095_validate_dex_clientsecret logger.go:42: 17:35:53 | 1-095_validate_dex_clientsecret/1-install | ArgoCD:kuttl-test-sure-ox/example-argocd created === CONT kuttl/harness/1-099_validate_server_autoscale logger.go:42: 17:35:53 | 1-099_validate_server_autoscale | Deleting namespace: kuttl-test-concise-tetra === CONT kuttl/harness/1-091_argocd_app_validate-applicationset-status logger.go:42: 17:35:55 | 1-091_argocd_app_validate-applicationset-status | Creating namespace: kuttl-test-evolving-crab logger.go:42: 17:35:55 | 1-091_argocd_app_validate-applicationset-status/1-install | starting test step 1-install I0723 17:35:57.068752 167 request.go:682] Waited for 1.946985091s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/network.openshift.io/v1?timeout=32s logger.go:42: 17:35:57 | 1-091_argocd_app_validate-applicationset-status/1-install | ArgoCD:kuttl-test-evolving-crab/example-argocd created logger.go:42: 17:35:58 | 1-091_argocd_app_validate-applicationset-status/1-install | test step completed 1-install logger.go:42: 17:35:58 | 1-091_argocd_app_validate-applicationset-status/2-enable-appset | starting test step 2-enable-appset === CONT kuttl/harness/1-090_validate_permissions logger.go:42: 17:36:01 | 1-090_validate_permissions | Creating namespace: kuttl-test-shining-sunbird logger.go:42: 17:36:01 | 1-090_validate_permissions/1- | starting test step 1- === CONT kuttl/harness/1-091_argocd_app_validate-applicationset-status logger.go:42: 17:36:01 | 1-091_argocd_app_validate-applicationset-status/2-enable-appset | ArgoCD:kuttl-test-evolving-crab/example-argocd updated === CONT kuttl/harness/1-090_validate_permissions logger.go:42: 17:36:03 | 1-090_validate_permissions/1- | test step completed 1- logger.go:42: 17:36:03 | 1-090_validate_permissions/2- | starting test step 2- logger.go:42: 17:36:06 | 1-090_validate_permissions/2- | test step completed 2- logger.go:42: 17:36:06 | 1-090_validate_permissions | skipping kubernetes event logging logger.go:42: 17:36:06 | 1-090_validate_permissions | Deleting namespace: kuttl-test-shining-sunbird === CONT kuttl/harness/1-092_validate_workload_status_monitoring_alert logger.go:42: 17:36:06 | 1-092_validate_workload_status_monitoring_alert/1-install | test step completed 1-install logger.go:42: 17:36:06 | 1-092_validate_workload_status_monitoring_alert/2-verify-alert | starting test step 2-verify-alert === CONT kuttl/harness/1-091_argocd_app_validate-applicationset-status logger.go:42: 17:36:06 | 1-091_argocd_app_validate-applicationset-status/2-enable-appset | test step completed 2-enable-appset logger.go:42: 17:36:06 | 1-091_argocd_app_validate-applicationset-status/3-change-appset-image | starting test step 3-change-appset-image === CONT kuttl/harness/1-095_validate_dex_clientsecret logger.go:42: 17:36:06 | 1-095_validate_dex_clientsecret/1-install | test step completed 1-install logger.go:42: 17:36:06 | 1-095_validate_dex_clientsecret/2-check-clientSecret-cm | starting test step 2-check-clientSecret-cm === CONT kuttl/harness/1-092_validate_workload_status_monitoring_alert logger.go:42: 17:36:06 | 1-092_validate_workload_status_monitoring_alert/2-verify-alert | running command: [sh -c sleep 7m] I0723 17:36:07.113589 167 request.go:682] Waited for 1.094611813s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/operators.coreos.com/v2?timeout=32s === CONT kuttl/harness/1-095_validate_dex_clientsecret logger.go:42: 17:36:08 | 1-095_validate_dex_clientsecret/2-check-clientSecret-cm | running command: [sh -c clientSecret=$(oc get configmap argocd-cm -n $NAMESPACE -o jsonpath='{.data.dex\.config}' | grep clientSecret | awk -F' ' '{print $2}') if test "$clientSecret" != "\$oidc.dex.clientSecret"; then echo "Client secret exposed!" exit 1 fi exit 0 ] === CONT kuttl/harness/1-091_argocd_app_validate-applicationset-status logger.go:42: 17:36:08 | 1-091_argocd_app_validate-applicationset-status/3-change-appset-image | ArgoCD:kuttl-test-evolving-crab/example-argocd updated logger.go:42: 17:36:08 | 1-091_argocd_app_validate-applicationset-status/3-change-appset-image | test step completed 3-change-appset-image logger.go:42: 17:36:08 | 1-091_argocd_app_validate-applicationset-status/4-delete-exisiting-deployment | starting test step 4-delete-exisiting-deployment === CONT kuttl/harness/1-108_alternate_cluster_roles_cluster_scoped_instance logger.go:42: 17:36:10 | 1-108_alternate_cluster_roles_cluster_scoped_instance/11-delete | test step completed 11-delete logger.go:42: 17:36:10 | 1-108_alternate_cluster_roles_cluster_scoped_instance | skipping kubernetes event logging === CONT kuttl/harness/1-091_argocd_app_validate-applicationset-status logger.go:42: 17:36:10 | 1-091_argocd_app_validate-applicationset-status/4-delete-exisiting-deployment | running command: [kubectl delete deployment example-argocd-applicationset-controller -n kuttl-test-evolving-crab] === CONT kuttl/harness/1-108_alternate_cluster_roles_cluster_scoped_instance logger.go:42: 17:36:10 | 1-108_alternate_cluster_roles_cluster_scoped_instance | Deleting namespace: kuttl-test-square-whale === CONT kuttl/harness/1-091_argocd_app_validate-applicationset-status logger.go:42: 17:36:11 | 1-091_argocd_app_validate-applicationset-status/4-delete-exisiting-deployment | deployment.apps "example-argocd-applicationset-controller" deleted === CONT kuttl/harness/1-089_validate_extra_repo_commands_args === CONT kuttl/harness/1-095_validate_dex_clientsecret logger.go:42: 17:36:13 | 1-095_validate_dex_clientsecret/2-check-clientSecret-cm | test step completed 2-check-clientSecret-cm logger.go:42: 17:36:13 | 1-095_validate_dex_clientsecret/3-verify-clientsecret-secret | starting test step 3-verify-clientsecret-secret logger.go:42: 17:36:13 | 1-095_validate_dex_clientsecret/3-verify-clientsecret-secret | running command: [sh -c # This test validates the Dex Client Secret copied by the operator from dex serviceaccount token secret in to argocd-secret. # To verify the behavior we should first get the token secret name of the dex service account. secret=$(oc get -n $NAMESPACE sa example-argocd-argocd-dex-server -o json | jq -r '.secrets[] | select(.name | contains("token")) | .name') # Extract the clientSecret expectedClientSecret=$(oc get secret $secret -n $NAMESPACE -o json | jq -r '.data.token') # actualClientSecret is the value of the secret in argocd-secret where argocd-operator should copy the secret from actualClientSecret=$(oc get secret argocd-secret -o json -n $NAMESPACE | jq -r '.data."oidc.dex.clientSecret"') # Verify if [[ $expectedClientSecret != $actualClientSecret ]]; then echo "Error: Dex Client Secret for OIDC is not valid" exit 1 fi ] === CONT kuttl/harness/1-089_validate_extra_repo_commands_args logger.go:42: 17:36:15 | 1-089_validate_extra_repo_commands_args | Creating namespace: kuttl-test-driving-shiner === CONT kuttl/harness/1-091_argocd_app_validate-applicationset-status logger.go:42: 17:36:15 | 1-091_argocd_app_validate-applicationset-status/4-delete-exisiting-deployment | test step completed 4-delete-exisiting-deployment logger.go:42: 17:36:15 | 1-091_argocd_app_validate-applicationset-status | skipping kubernetes event logging === CONT kuttl/harness/1-089_validate_extra_repo_commands_args logger.go:42: 17:36:15 | 1-089_validate_extra_repo_commands_args/1-basic-argocd-install | starting test step 1-basic-argocd-install === CONT kuttl/harness/1-091_argocd_app_validate-applicationset-status logger.go:42: 17:36:15 | 1-091_argocd_app_validate-applicationset-status | Deleting namespace: kuttl-test-evolving-crab === CONT kuttl/harness/1-088_validate_applicationset_add_env_vars I0723 17:36:17.132774 167 request.go:682] Waited for 1.296630293s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/quota.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-095_validate_dex_clientsecret logger.go:42: 17:36:18 | 1-095_validate_dex_clientsecret/3-verify-clientsecret-secret | test step completed 3-verify-clientsecret-secret logger.go:42: 17:36:18 | 1-095_validate_dex_clientsecret | skipping kubernetes event logging logger.go:42: 17:36:18 | 1-095_validate_dex_clientsecret | Deleting namespace: kuttl-test-sure-ox === CONT kuttl/harness/1-088_validate_applicationset_add_env_vars logger.go:42: 17:36:20 | 1-088_validate_applicationset_add_env_vars | Creating namespace: kuttl-test-moving-flounder logger.go:42: 17:36:20 | 1-088_validate_applicationset_add_env_vars/1-basic-argocd-install | starting test step 1-basic-argocd-install === CONT kuttl/harness/1-089_validate_extra_repo_commands_args logger.go:42: 17:36:20 | 1-089_validate_extra_repo_commands_args/1-basic-argocd-install | Namespace:/test-1-89-repo-extra-command created logger.go:42: 17:36:20 | 1-089_validate_extra_repo_commands_args/1-basic-argocd-install | ArgoCD:test-1-89-repo-extra-command/example-argocd created === CONT kuttl/harness/1-087_validate_repo_server_settings logger.go:42: 17:36:23 | 1-087_validate_repo_server_settings | Creating namespace: kuttl-test-flying-thrush logger.go:42: 17:36:23 | 1-087_validate_repo_server_settings/1-install | starting test step 1-install === CONT kuttl/harness/1-088_validate_applicationset_add_env_vars logger.go:42: 17:36:23 | 1-088_validate_applicationset_add_env_vars/1-basic-argocd-install | ArgoCD:kuttl-test-moving-flounder/example-argocd created === CONT kuttl/harness/1-086_validate_applicationset_extra_command_args logger.go:42: 17:36:25 | 1-086_validate_applicationset_extra_command_args | Creating namespace: kuttl-test-cool-crab logger.go:42: 17:36:25 | 1-086_validate_applicationset_extra_command_args/1-basic-argocd-install | starting test step 1-basic-argocd-install === CONT kuttl/harness/1-087_validate_repo_server_settings logger.go:42: 17:36:25 | 1-087_validate_repo_server_settings/1-install | ArgoCD:kuttl-test-flying-thrush/example-argocd created I0723 17:36:27.181750 167 request.go:682] Waited for 1.49387615s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/cloud.network.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-086_validate_applicationset_extra_command_args logger.go:42: 17:36:28 | 1-086_validate_applicationset_extra_command_args/1-basic-argocd-install | Namespace:/test-1-86-appsets-extra-command created logger.go:42: 17:36:28 | 1-086_validate_applicationset_extra_command_args/1-basic-argocd-install | ArgoCD:test-1-86-appsets-extra-command/example-argocd created === CONT kuttl/harness/1-089_validate_extra_repo_commands_args logger.go:42: 17:36:33 | 1-089_validate_extra_repo_commands_args/1-basic-argocd-install | test step completed 1-basic-argocd-install logger.go:42: 17:36:33 | 1-089_validate_extra_repo_commands_args/2-add-extraRepoCommandArgs | starting test step 2-add-extraRepoCommandArgs === CONT kuttl/harness/1-088_validate_applicationset_add_env_vars logger.go:42: 17:36:35 | 1-088_validate_applicationset_add_env_vars/1-basic-argocd-install | test step completed 1-basic-argocd-install logger.go:42: 17:36:35 | 1-088_validate_applicationset_add_env_vars/2-add-env-vars | starting test step 2-add-env-vars === CONT kuttl/harness/1-089_validate_extra_repo_commands_args logger.go:42: 17:36:35 | 1-089_validate_extra_repo_commands_args/2-add-extraRepoCommandArgs | ArgoCD:test-1-89-repo-extra-command/example-argocd updated I0723 17:36:37.206540 167 request.go:682] Waited for 1.496535869s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/monitoring.coreos.com/v1alpha1?timeout=32s logger.go:42: 17:36:38 | 1-089_validate_extra_repo_commands_args/2-add-extraRepoCommandArgs | test step completed 2-add-extraRepoCommandArgs logger.go:42: 17:36:38 | 1-089_validate_extra_repo_commands_args/99-delete | starting test step 99-delete === CONT kuttl/harness/1-088_validate_applicationset_add_env_vars logger.go:42: 17:36:38 | 1-088_validate_applicationset_add_env_vars/2-add-env-vars | ArgoCD:kuttl-test-moving-flounder/example-argocd updated logger.go:42: 17:36:39 | 1-088_validate_applicationset_add_env_vars/2-add-env-vars | test step completed 2-add-env-vars logger.go:42: 17:36:39 | 1-088_validate_applicationset_add_env_vars | skipping kubernetes event logging logger.go:42: 17:36:39 | 1-088_validate_applicationset_add_env_vars | Deleting namespace: kuttl-test-moving-flounder === CONT kuttl/harness/1-084_validate_status_host_ingress logger.go:42: 17:36:45 | 1-084_validate_status_host_ingress | Creating namespace: kuttl-test-bursting-elephant logger.go:42: 17:36:45 | 1-084_validate_status_host_ingress/1-argocd-ingress | starting test step 1-argocd-ingress I0723 17:36:47.210208 167 request.go:682] Waited for 1.64660094s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/flowcontrol.apiserver.k8s.io/v1beta1?timeout=32s logger.go:42: 17:36:48 | 1-084_validate_status_host_ingress/1-argocd-ingress | ArgoCD:kuttl-test-bursting-elephant/example-argocd created === CONT kuttl/harness/1-089_validate_extra_repo_commands_args logger.go:42: 17:36:53 | 1-089_validate_extra_repo_commands_args/99-delete | test step completed 99-delete logger.go:42: 17:36:53 | 1-089_validate_extra_repo_commands_args | skipping kubernetes event logging logger.go:42: 17:36:53 | 1-089_validate_extra_repo_commands_args | Deleting namespace: kuttl-test-driving-shiner === CONT kuttl/harness/1-087_validate_repo_server_settings logger.go:42: 17:36:58 | 1-087_validate_repo_server_settings/1-install | test step completed 1-install logger.go:42: 17:36:58 | 1-087_validate_repo_server_settings/2-modify-argocd | starting test step 2-modify-argocd === CONT kuttl/harness/1-083_validate_resource_customization_subkeys I0723 17:36:59.144366 167 request.go:682] Waited for 1.040684442s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/cloudcredential.openshift.io/v1?timeout=32s logger.go:42: 17:37:00 | 1-083_validate_resource_customization_subkeys | Creating namespace: kuttl-test-amazing-monkey logger.go:42: 17:37:00 | 1-083_validate_resource_customization_subkeys/1-argocd-with-resource-customization-subkeys | starting test step 1-argocd-with-resource-customization-subkeys === CONT kuttl/harness/1-087_validate_repo_server_settings logger.go:42: 17:37:00 | 1-087_validate_repo_server_settings/2-modify-argocd | ArgoCD:kuttl-test-flying-thrush/example-argocd updated === CONT kuttl/harness/1-086_validate_applicationset_extra_command_args logger.go:42: 17:37:03 | 1-086_validate_applicationset_extra_command_args/1-basic-argocd-install | test step completed 1-basic-argocd-install logger.go:42: 17:37:03 | 1-086_validate_applicationset_extra_command_args/2-add-extraCommandArgs | starting test step 2-add-extraCommandArgs === CONT kuttl/harness/1-087_validate_repo_server_settings logger.go:42: 17:37:03 | 1-087_validate_repo_server_settings/2-modify-argocd | test step completed 2-modify-argocd logger.go:42: 17:37:03 | 1-087_validate_repo_server_settings/4-modify-serviceaccount | starting test step 4-modify-serviceaccount === CONT kuttl/harness/1-083_validate_resource_customization_subkeys logger.go:42: 17:37:03 | 1-083_validate_resource_customization_subkeys/1-argocd-with-resource-customization-subkeys | ArgoCD:kuttl-test-amazing-monkey/example-argocd created === CONT kuttl/harness/1-086_validate_applicationset_extra_command_args logger.go:42: 17:37:05 | 1-086_validate_applicationset_extra_command_args/2-add-extraCommandArgs | ArgoCD:test-1-86-appsets-extra-command/example-argocd updated logger.go:42: 17:37:07 | 1-086_validate_applicationset_extra_command_args/2-add-extraCommandArgs | test step completed 2-add-extraCommandArgs logger.go:42: 17:37:07 | 1-086_validate_applicationset_extra_command_args/99-delete | starting test step 99-delete === CONT kuttl/harness/1-087_validate_repo_server_settings logger.go:42: 17:37:07 | 1-087_validate_repo_server_settings/4-modify-serviceaccount | ServiceAccount:kuttl-test-flying-thrush/modified-default created logger.go:42: 17:37:08 | 1-087_validate_repo_server_settings/4-modify-serviceaccount | ArgoCD:kuttl-test-flying-thrush/example-argocd updated logger.go:42: 17:37:09 | 1-087_validate_repo_server_settings/4-modify-serviceaccount | test step completed 4-modify-serviceaccount logger.go:42: 17:37:09 | 1-087_validate_repo_server_settings/5-reset-repo-settings | starting test step 5-reset-repo-settings I0723 17:37:10.083123 167 request.go:682] Waited for 1.047081084s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/oauth.openshift.io/v1?timeout=32s logger.go:42: 17:37:11 | 1-087_validate_repo_server_settings/5-reset-repo-settings | ArgoCD:kuttl-test-flying-thrush/example-argocd updated logger.go:42: 17:37:12 | 1-087_validate_repo_server_settings/5-reset-repo-settings | test step completed 5-reset-repo-settings logger.go:42: 17:37:12 | 1-087_validate_repo_server_settings | skipping kubernetes event logging logger.go:42: 17:37:12 | 1-087_validate_repo_server_settings | Deleting namespace: kuttl-test-flying-thrush === CONT kuttl/harness/1-071_validate_SCC_HA logger.go:42: 17:37:18 | 1-071_validate_SCC_HA | Creating namespace: kuttl-test-main-cat logger.go:42: 17:37:18 | 1-071_validate_SCC_HA/1-create-scc | starting test step 1-create-scc I0723 17:37:20.132430 167 request.go:682] Waited for 1.196113971s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/autoscaling.openshift.io/v1beta1?timeout=32s logger.go:42: 17:37:21 | 1-071_validate_SCC_HA/1-create-scc | SecurityContextConstraints:/restricted-dropcaps created === CONT kuttl/harness/1-086_validate_applicationset_extra_command_args logger.go:42: 17:37:23 | 1-086_validate_applicationset_extra_command_args/99-delete | test step completed 99-delete logger.go:42: 17:37:23 | 1-086_validate_applicationset_extra_command_args | skipping kubernetes event logging === CONT kuttl/harness/1-084_validate_status_host_ingress logger.go:42: 17:37:23 | 1-084_validate_status_host_ingress/1-argocd-ingress | test step completed 1-argocd-ingress logger.go:42: 17:37:23 | 1-084_validate_status_host_ingress/2- | starting test step 2- === CONT kuttl/harness/1-086_validate_applicationset_extra_command_args logger.go:42: 17:37:23 | 1-086_validate_applicationset_extra_command_args | Deleting namespace: kuttl-test-cool-crab === CONT kuttl/harness/1-071_validate_SCC_HA logger.go:42: 17:37:23 | 1-071_validate_SCC_HA/1-create-scc | test step completed 1-create-scc logger.go:42: 17:37:23 | 1-071_validate_SCC_HA/2-create-argocd | starting test step 2-create-argocd === CONT kuttl/harness/1-084_validate_status_host_ingress logger.go:42: 17:37:26 | 1-084_validate_status_host_ingress/2- | test step completed 2- logger.go:42: 17:37:26 | 1-084_validate_status_host_ingress | skipping kubernetes event logging logger.go:42: 17:37:26 | 1-084_validate_status_host_ingress | Deleting namespace: kuttl-test-bursting-elephant === CONT kuttl/harness/1-071_validate_SCC_HA logger.go:42: 17:37:28 | 1-071_validate_SCC_HA/2-create-argocd | ArgoCD:kuttl-test-main-cat/argocd created === CONT kuttl/harness/1-077_validate_disable_dex_removed logger.go:42: 17:37:29 | 1-077_validate_disable_dex_removed | Creating namespace: kuttl-test-moved-gecko logger.go:42: 17:37:29 | 1-077_validate_disable_dex_removed/1-check_disable_dex_removed | starting test step 1-check_disable_dex_removed logger.go:42: 17:37:29 | 1-077_validate_disable_dex_removed/1-check_disable_dex_removed | running command: [sh -c set -e operator_name_version=$(oc get subscription/openshift-gitops-operator -n openshift-operators -o jsonpath='{.status.installedCSV}') if [ "$(oc get csv/"${operator_name_version}" -n openshift-operators -o jsonpath='{.spec.install.spec.deployments[].spec.template.spec.containers[].env}' | grep DISABLE_DEX)" != '' ]; then echo "DISABLE_DEX env var is still present in the operator CSV." exit 1 fi ] I0723 17:37:31.983089 167 request.go:682] Waited for 1.045593872s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/monitoring.coreos.com/v1beta1?timeout=32s === CONT kuttl/harness/1-075_validate_dex_anyuid === CONT kuttl/harness/1-077_validate_disable_dex_removed logger.go:42: 17:37:33 | 1-077_validate_disable_dex_removed/1-check_disable_dex_removed | test step completed 1-check_disable_dex_removed logger.go:42: 17:37:33 | 1-077_validate_disable_dex_removed | skipping kubernetes event logging logger.go:42: 17:37:33 | 1-077_validate_disable_dex_removed | Deleting namespace: kuttl-test-moved-gecko === CONT kuttl/harness/1-075_validate_dex_anyuid logger.go:42: 17:37:33 | 1-075_validate_dex_anyuid | Creating namespace: kuttl-test-trusty-hamster logger.go:42: 17:37:33 | 1-075_validate_dex_anyuid/1-install | starting test step 1-install logger.go:42: 17:37:35 | 1-075_validate_dex_anyuid/1-install | ArgoCD:kuttl-test-trusty-hamster/argocd created === CONT kuttl/harness/1-083_validate_resource_customization_subkeys logger.go:42: 17:37:36 | 1-083_validate_resource_customization_subkeys/1-argocd-with-resource-customization-subkeys | test step completed 1-argocd-with-resource-customization-subkeys logger.go:42: 17:37:36 | 1-083_validate_resource_customization_subkeys | skipping kubernetes event logging logger.go:42: 17:37:36 | 1-083_validate_resource_customization_subkeys | Deleting namespace: kuttl-test-amazing-monkey === CONT kuttl/harness/1-082_validate_node_placement logger.go:42: 17:37:39 | 1-082_validate_node_placement | Creating namespace: kuttl-test-pretty-barnacle logger.go:42: 17:37:39 | 1-082_validate_node_placement/1-default-node-selector | starting test step 1-default-node-selector logger.go:42: 17:37:41 | 1-082_validate_node_placement/1-default-node-selector | ArgoCD:kuttl-test-pretty-barnacle/example-argocd created logger.go:42: 17:37:43 | 1-082_validate_node_placement/1-default-node-selector | test step completed 1-default-node-selector logger.go:42: 17:37:43 | 1-082_validate_node_placement/2-custom-node-placement | starting test step 2-custom-node-placement === CONT kuttl/harness/1-071_validate_SCC_HA logger.go:42: 17:37:43 | 1-071_validate_SCC_HA/2-create-argocd | test step completed 2-create-argocd logger.go:42: 17:37:43 | 1-071_validate_SCC_HA/3-modify-argocd | starting test step 3-modify-argocd === CONT kuttl/harness/1-074_validate_terminating_namespace_block I0723 17:37:44.059295 167 request.go:682] Waited for 1.046874009s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/monitoring.coreos.com/v1?timeout=32s === CONT kuttl/harness/1-082_validate_node_placement logger.go:42: 17:37:45 | 1-082_validate_node_placement/2-custom-node-placement | ArgoCD:kuttl-test-pretty-barnacle/example-argocd updated === CONT kuttl/harness/1-074_validate_terminating_namespace_block logger.go:42: 17:37:47 | 1-074_validate_terminating_namespace_block | Creating namespace: kuttl-test-promoted-shark logger.go:42: 17:37:47 | 1-074_validate_terminating_namespace_block/1-install | starting test step 1-install === CONT kuttl/harness/1-071_validate_SCC_HA logger.go:42: 17:37:47 | 1-071_validate_SCC_HA/3-modify-argocd | ArgoCD:kuttl-test-main-cat/argocd updated === CONT kuttl/harness/1-082_validate_node_placement logger.go:42: 17:37:50 | 1-082_validate_node_placement/2-custom-node-placement | test step completed 2-custom-node-placement logger.go:42: 17:37:50 | 1-082_validate_node_placement/3-node-placement-with-toleration | starting test step 3-node-placement-with-toleration === CONT kuttl/harness/1-074_validate_terminating_namespace_block logger.go:42: 17:37:50 | 1-074_validate_terminating_namespace_block/1-install | Namespace:/gitops-2242-ns-main created logger.go:42: 17:37:50 | 1-074_validate_terminating_namespace_block/1-install | ArgoCD:gitops-2242-ns-main/gitops-2242-argocd created logger.go:42: 17:37:50 | 1-074_validate_terminating_namespace_block/1-install | RoleBinding:gitops-2242-ns-main/grant-argocd created logger.go:42: 17:37:53 | 1-074_validate_terminating_namespace_block/1-install | test step completed 1-install logger.go:42: 17:37:53 | 1-074_validate_terminating_namespace_block/2-managed-ns-cm | starting test step 2-managed-ns-cm === CONT kuttl/harness/1-082_validate_node_placement logger.go:42: 17:37:53 | 1-082_validate_node_placement/3-node-placement-with-toleration | ArgoCD:kuttl-test-pretty-barnacle/example-argocd updated I0723 17:37:54.061884 167 request.go:682] Waited for 1.041235762s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/tuned.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-074_validate_terminating_namespace_block logger.go:42: 17:37:55 | 1-074_validate_terminating_namespace_block/2-managed-ns-cm | Namespace:/gitops-2242-ns-first created logger.go:42: 17:37:55 | 1-074_validate_terminating_namespace_block/2-managed-ns-cm | ConfigMap:gitops-2242-ns-first/my-config-map-2 created === CONT kuttl/harness/1-082_validate_node_placement logger.go:42: 17:37:55 | 1-082_validate_node_placement/3-node-placement-with-toleration | test step completed 3-node-placement-with-toleration logger.go:42: 17:37:55 | 1-082_validate_node_placement | skipping kubernetes event logging === CONT kuttl/harness/1-074_validate_terminating_namespace_block logger.go:42: 17:37:55 | 1-074_validate_terminating_namespace_block/2-managed-ns-cm | test step completed 2-managed-ns-cm logger.go:42: 17:37:55 | 1-074_validate_terminating_namespace_block/3-delete-ns | starting test step 3-delete-ns logger.go:42: 17:37:55 | 1-074_validate_terminating_namespace_block/3-delete-ns | running command: [sh -c set -e oc delete ns gitops-2242-ns-first --wait=false exit 0 ] === CONT kuttl/harness/1-082_validate_node_placement logger.go:42: 17:37:55 | 1-082_validate_node_placement | Deleting namespace: kuttl-test-pretty-barnacle === CONT kuttl/harness/1-074_validate_terminating_namespace_block logger.go:42: 17:37:56 | 1-074_validate_terminating_namespace_block/3-delete-ns | namespace "gitops-2242-ns-first" deleted logger.go:42: 17:37:58 | 1-074_validate_terminating_namespace_block/3-delete-ns | test step completed 3-delete-ns logger.go:42: 17:37:58 | 1-074_validate_terminating_namespace_block/4-create-ns | starting test step 4-create-ns logger.go:42: 17:38:00 | 1-074_validate_terminating_namespace_block/4-create-ns | Namespace:/gitops-2242-ns-second created === CONT kuttl/harness/1-081_validate_applicationset_deployment logger.go:42: 17:38:01 | 1-081_validate_applicationset_deployment | Creating namespace: kuttl-test-content-chimp logger.go:42: 17:38:01 | 1-081_validate_applicationset_deployment/1- | starting test step 1- I0723 17:38:04.073045 167 request.go:682] Waited for 2.196953071s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/apiserver.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-074_validate_terminating_namespace_block logger.go:42: 17:38:04 | 1-074_validate_terminating_namespace_block/4-create-ns | test step completed 4-create-ns logger.go:42: 17:38:04 | 1-074_validate_terminating_namespace_block/99-cleanup | starting test step 99-cleanup logger.go:42: 17:38:04 | 1-074_validate_terminating_namespace_block/99-cleanup | running command: [sh -c set -e # Removing finalizer from cm oc patch cm/my-config-map-2 -n gitops-2242-ns-first --type json --patch='[ { "op": "remove", "path": "/metadata/finalizers" } ]' # Deleting namespaces oc delete ns gitops-2242-ns-first --wait=false oc delete ns gitops-2242-ns-second --wait=false oc delete ns gitops-2242-ns-main --wait=false exit 0 ] === CONT kuttl/harness/1-081_validate_applicationset_deployment logger.go:42: 17:38:04 | 1-081_validate_applicationset_deployment/1- | test step completed 1- logger.go:42: 17:38:04 | 1-081_validate_applicationset_deployment | skipping kubernetes event logging logger.go:42: 17:38:04 | 1-081_validate_applicationset_deployment | Deleting namespace: kuttl-test-content-chimp === CONT kuttl/harness/1-074_validate_terminating_namespace_block logger.go:42: 17:38:04 | 1-074_validate_terminating_namespace_block/99-cleanup | configmap/my-config-map-2 patched logger.go:42: 17:38:05 | 1-074_validate_terminating_namespace_block/99-cleanup | namespace "gitops-2242-ns-first" deleted logger.go:42: 17:38:06 | 1-074_validate_terminating_namespace_block/99-cleanup | namespace "gitops-2242-ns-second" deleted logger.go:42: 17:38:06 | 1-074_validate_terminating_namespace_block/99-cleanup | namespace "gitops-2242-ns-main" deleted logger.go:42: 17:38:09 | 1-074_validate_terminating_namespace_block/99-cleanup | test step completed 99-cleanup logger.go:42: 17:38:09 | 1-074_validate_terminating_namespace_block | skipping kubernetes event logging === CONT kuttl/harness/1-075_validate_dex_anyuid logger.go:42: 17:38:09 | 1-075_validate_dex_anyuid/1-install | test step completed 1-install logger.go:42: 17:38:09 | 1-075_validate_dex_anyuid/2-change_dex_scc_to_anyuid | starting test step 2-change_dex_scc_to_anyuid logger.go:42: 17:38:09 | 1-075_validate_dex_anyuid/2-change_dex_scc_to_anyuid | running command: [sh -c set -e oc adm policy add-scc-to-user anyuid -z argocd-argocd-dex-server -n $NAMESPACE ] === CONT kuttl/harness/1-074_validate_terminating_namespace_block logger.go:42: 17:38:09 | 1-074_validate_terminating_namespace_block | Deleting namespace: kuttl-test-promoted-shark === CONT kuttl/harness/1-075_validate_dex_anyuid logger.go:42: 17:38:09 | 1-075_validate_dex_anyuid/2-change_dex_scc_to_anyuid | clusterrole.rbac.authorization.k8s.io/system:openshift:scc:anyuid added: "argocd-argocd-dex-server" === CONT kuttl/harness/1-073_validate_rhsso === CONT kuttl/harness/1-075_validate_dex_anyuid logger.go:42: 17:38:12 | 1-075_validate_dex_anyuid/2-change_dex_scc_to_anyuid | test step completed 2-change_dex_scc_to_anyuid logger.go:42: 17:38:12 | 1-075_validate_dex_anyuid/3-restart_dex_pod | starting test step 3-restart_dex_pod === CONT kuttl/harness/1-073_validate_rhsso logger.go:42: 17:38:12 | 1-073_validate_rhsso | Creating namespace: kuttl-test-sure-dory === CONT kuttl/harness/1-075_validate_dex_anyuid logger.go:42: 17:38:12 | 1-075_validate_dex_anyuid/3-restart_dex_pod | running command: [sh -c set -e oc rollout restart deployment/argocd-dex-server -n $NAMESPACE ] === CONT kuttl/harness/1-073_validate_rhsso logger.go:42: 17:38:12 | 1-073_validate_rhsso/1-argocd-rhsso | starting test step 1-argocd-rhsso === CONT kuttl/harness/1-075_validate_dex_anyuid logger.go:42: 17:38:12 | 1-075_validate_dex_anyuid/3-restart_dex_pod | deployment.apps/argocd-dex-server restarted I0723 17:38:14.106544 167 request.go:682] Waited for 1.993077404s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/security.internal.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-073_validate_rhsso logger.go:42: 17:38:14 | 1-073_validate_rhsso/1-argocd-rhsso | ArgoCD:kuttl-test-sure-dory/example-argocd-keycloak created === CONT kuttl/harness/1-080_validate_regex_support_argocd_rbac logger.go:42: 17:38:17 | 1-080_validate_regex_support_argocd_rbac | Creating namespace: kuttl-test-expert-gibbon logger.go:42: 17:38:17 | 1-080_validate_regex_support_argocd_rbac/1-install | starting test step 1-install === CONT kuttl/harness/1-075_validate_dex_anyuid logger.go:42: 17:38:19 | 1-075_validate_dex_anyuid/3-restart_dex_pod | test step completed 3-restart_dex_pod logger.go:42: 17:38:19 | 1-075_validate_dex_anyuid | skipping kubernetes event logging === CONT kuttl/harness/1-080_validate_regex_support_argocd_rbac logger.go:42: 17:38:19 | 1-080_validate_regex_support_argocd_rbac/1-install | ArgoCD:kuttl-test-expert-gibbon/example-argocd created === CONT kuttl/harness/1-075_validate_dex_anyuid logger.go:42: 17:38:19 | 1-075_validate_dex_anyuid | Deleting namespace: kuttl-test-trusty-hamster === CONT kuttl/harness/1-072_validate_liveness_probe_removed logger.go:42: 17:38:26 | 1-072_validate_liveness_probe_removed | Creating namespace: kuttl-test-firm-mongoose logger.go:42: 17:38:26 | 1-072_validate_liveness_probe_removed/1-install | starting test step 1-install I0723 17:38:27.135797 167 request.go:682] Waited for 1.046571694s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/flowcontrol.apiserver.k8s.io/v1beta2?timeout=32s logger.go:42: 17:38:28 | 1-072_validate_liveness_probe_removed/1-install | ArgoCD:kuttl-test-firm-mongoose/argocd created === CONT kuttl/harness/1-080_validate_regex_support_argocd_rbac logger.go:42: 17:38:51 | 1-080_validate_regex_support_argocd_rbac/1-install | test step completed 1-install logger.go:42: 17:38:51 | 1-080_validate_regex_support_argocd_rbac/2-modify-argocd | starting test step 2-modify-argocd I0723 17:38:52.806218 167 request.go:682] Waited for 1.046905238s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/performance.openshift.io/v1alpha1?timeout=32s logger.go:42: 17:38:54 | 1-080_validate_regex_support_argocd_rbac/2-modify-argocd | ArgoCD:kuttl-test-expert-gibbon/example-argocd updated logger.go:42: 17:38:55 | 1-080_validate_regex_support_argocd_rbac/2-modify-argocd | test step completed 2-modify-argocd logger.go:42: 17:38:55 | 1-080_validate_regex_support_argocd_rbac/3-try-modify | starting test step 3-try-modify logger.go:42: 17:38:57 | 1-080_validate_regex_support_argocd_rbac/3-try-modify | ConfigMap:kuttl-test-expert-gibbon/argocd-rbac-cm updated logger.go:42: 17:38:58 | 1-080_validate_regex_support_argocd_rbac/3-try-modify | test step completed 3-try-modify logger.go:42: 17:38:58 | 1-080_validate_regex_support_argocd_rbac/4-modify-argocd | starting test step 4-modify-argocd === CONT kuttl/harness/1-072_validate_liveness_probe_removed logger.go:42: 17:39:01 | 1-072_validate_liveness_probe_removed/1-install | test step completed 1-install logger.go:42: 17:39:01 | 1-072_validate_liveness_probe_removed/2-check_liveness_probe_removed | starting test step 2-check_liveness_probe_removed logger.go:42: 17:39:01 | 1-072_validate_liveness_probe_removed/2-check_liveness_probe_removed | running command: [sh -c set -e if ! test -z $(oc get statefulset.apps/argocd-application-controller -n $NAMESPACE -o jsonpath='{.spec.template.spec.containers[].livenessProbe}'); then echo "livenessProbe should not exist on statefulset.apps/argocd-application-controller." exit 1 fi ] === CONT kuttl/harness/1-080_validate_regex_support_argocd_rbac logger.go:42: 17:39:01 | 1-080_validate_regex_support_argocd_rbac/4-modify-argocd | ArgoCD:kuttl-test-expert-gibbon/example-argocd updated I0723 17:39:03.057963 167 request.go:682] Waited for 1.046523373s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/machine.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-072_validate_liveness_probe_removed logger.go:42: 17:39:04 | 1-072_validate_liveness_probe_removed/2-check_liveness_probe_removed | test step completed 2-check_liveness_probe_removed logger.go:42: 17:39:04 | 1-072_validate_liveness_probe_removed | skipping kubernetes event logging === CONT kuttl/harness/1-080_validate_regex_support_argocd_rbac logger.go:42: 17:39:04 | 1-080_validate_regex_support_argocd_rbac/4-modify-argocd | test step completed 4-modify-argocd logger.go:42: 17:39:04 | 1-080_validate_regex_support_argocd_rbac | skipping kubernetes event logging === CONT kuttl/harness/1-072_validate_liveness_probe_removed logger.go:42: 17:39:04 | 1-072_validate_liveness_probe_removed | Deleting namespace: kuttl-test-firm-mongoose === CONT kuttl/harness/1-080_validate_regex_support_argocd_rbac logger.go:42: 17:39:04 | 1-080_validate_regex_support_argocd_rbac | Deleting namespace: kuttl-test-expert-gibbon === CONT kuttl/harness/1-079_validate_vars_for_notificaitons logger.go:42: 17:39:10 | 1-079_validate_vars_for_notificaitons | Creating namespace: kuttl-test-fresh-locust logger.go:42: 17:39:10 | 1-079_validate_vars_for_notificaitons/1-install | starting test step 1-install === CONT kuttl/harness/1-050_validate_sso logger.go:42: 17:39:12 | 1-050_validate_sso | Creating namespace: kuttl-test-top-sawfish logger.go:42: 17:39:12 | 1-050_validate_sso/1-install | starting test step 1-install === CONT kuttl/harness/1-079_validate_vars_for_notificaitons logger.go:42: 17:39:13 | 1-079_validate_vars_for_notificaitons/1-install | ArgoCD:kuttl-test-fresh-locust/example-argocd created I0723 17:39:14.035707 167 request.go:682] Waited for 1.044024826s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/packages.operators.coreos.com/v1?timeout=32s === CONT kuttl/harness/1-050_validate_sso logger.go:42: 17:39:15 | 1-050_validate_sso/1-install | ArgoCD:kuttl-test-top-sawfish/argocd created logger.go:42: 17:39:27 | 1-050_validate_sso/1-install | test step completed 1-install logger.go:42: 17:39:27 | 1-050_validate_sso/2-remove-spec-sso | starting test step 2-remove-spec-sso logger.go:42: 17:39:27 | 1-050_validate_sso/2-remove-spec-sso | running command: [sh -c oc patch -n $NAMESPACE argocd/argocd --type='json' -p='[{"op": "remove", "path": "/spec/sso"}]' ] logger.go:42: 17:39:28 | 1-050_validate_sso/2-remove-spec-sso | argocd.argoproj.io/argocd patched logger.go:42: 17:39:28 | 1-050_validate_sso/2-remove-spec-sso | running command: [sh -c sleep 10] I0723 17:39:39.459227 167 request.go:682] Waited for 1.046092044s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/whereabouts.cni.cncf.io/v1alpha1?timeout=32s logger.go:42: 17:39:40 | 1-050_validate_sso/2-remove-spec-sso | test step completed 2-remove-spec-sso logger.go:42: 17:39:40 | 1-050_validate_sso/7-provider-keycloak-dex-spec | starting test step 7-provider-keycloak-dex-spec logger.go:42: 17:39:43 | 1-050_validate_sso/7-provider-keycloak-dex-spec | ArgoCD:kuttl-test-top-sawfish/argocd updated logger.go:42: 17:39:44 | 1-050_validate_sso/7-provider-keycloak-dex-spec | test step completed 7-provider-keycloak-dex-spec logger.go:42: 17:39:44 | 1-050_validate_sso/8-remove-spec-dex | starting test step 8-remove-spec-dex logger.go:42: 17:39:44 | 1-050_validate_sso/8-remove-spec-dex | running command: [sh -c oc patch -n $NAMESPACE argocd/argocd --type='json' -p='[{"op": "remove", "path": "/spec/sso/dex"}]' ] logger.go:42: 17:39:44 | 1-050_validate_sso/8-remove-spec-dex | argocd.argoproj.io/argocd patched === CONT kuttl/harness/1-079_validate_vars_for_notificaitons logger.go:42: 17:39:47 | 1-079_validate_vars_for_notificaitons/1-install | test step completed 1-install logger.go:42: 17:39:47 | 1-079_validate_vars_for_notificaitons/2-modify-argocd | starting test step 2-modify-argocd I0723 17:39:49.504876 167 request.go:682] Waited for 1.895361392s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/machineconfiguration.openshift.io/v1?timeout=32s logger.go:42: 17:39:50 | 1-079_validate_vars_for_notificaitons/2-modify-argocd | ArgoCD:kuttl-test-fresh-locust/example-argocd updated logger.go:42: 17:39:54 | 1-079_validate_vars_for_notificaitons/2-modify-argocd | test step completed 2-modify-argocd logger.go:42: 17:39:54 | 1-079_validate_vars_for_notificaitons | skipping kubernetes event logging logger.go:42: 17:39:54 | 1-079_validate_vars_for_notificaitons | Deleting namespace: kuttl-test-fresh-locust === CONT kuttl/harness/1-007_validate_namespace_scoped_install logger.go:42: 17:40:00 | 1-007_validate_namespace_scoped_install | Creating namespace: kuttl-test-proud-rabbit logger.go:42: 17:40:00 | 1-007_validate_namespace_scoped_install/1-install | starting test step 1-install I0723 17:40:01.482280 167 request.go:682] Waited for 1.047329051s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/cloud.network.openshift.io/v1?timeout=32s logger.go:42: 17:40:02 | 1-007_validate_namespace_scoped_install/1-install | ArgoCD:kuttl-test-proud-rabbit/argocd created === CONT kuttl/harness/1-073_validate_rhsso logger.go:42: 17:40:29 | 1-073_validate_rhsso/1-argocd-rhsso | test step completed 1-argocd-rhsso logger.go:42: 17:40:29 | 1-073_validate_rhsso/2-sleep | starting test step 2-sleep logger.go:42: 17:40:29 | 1-073_validate_rhsso/2-sleep | running command: [sh -c sleep 60] === CONT kuttl/harness/1-007_validate_namespace_scoped_install logger.go:42: 17:40:34 | 1-007_validate_namespace_scoped_install/1-install | test step completed 1-install logger.go:42: 17:40:34 | 1-007_validate_namespace_scoped_install/2-check-cluster-secret | starting test step 2-check-cluster-secret logger.go:42: 17:40:34 | 1-007_validate_namespace_scoped_install/2-check-cluster-secret | running command: [sh -c namespaces=$(oc get secret -n $NAMESPACE argocd-default-cluster-config -o jsonpath='{.data.namespaces}' | base64 -d) if test "$namespaces" != "$NAMESPACE"; then echo "Assertion for cluster secret failed!" exit 1 fi exit 0 ] I0723 17:40:36.609283 167 request.go:682] Waited for 1.00536571s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/snapshot.storage.k8s.io/v1?timeout=32s logger.go:42: 17:40:37 | 1-007_validate_namespace_scoped_install/2-check-cluster-secret | test step completed 2-check-cluster-secret logger.go:42: 17:40:37 | 1-007_validate_namespace_scoped_install | skipping kubernetes event logging logger.go:42: 17:40:37 | 1-007_validate_namespace_scoped_install | Deleting namespace: kuttl-test-proud-rabbit === CONT kuttl/harness/1-019_validate_volume_mounts logger.go:42: 17:40:44 | 1-019_validate_volume_mounts | Creating namespace: kuttl-test-humble-antelope logger.go:42: 17:40:44 | 1-019_validate_volume_mounts/1-install | starting test step 1-install logger.go:42: 17:40:46 | 1-019_validate_volume_mounts/1-install | ArgoCD:kuttl-test-humble-antelope/argocd created logger.go:42: 17:41:18 | 1-019_validate_volume_mounts/1-install | test step completed 1-install logger.go:42: 17:41:18 | 1-019_validate_volume_mounts | skipping kubernetes event logging logger.go:42: 17:41:18 | 1-019_validate_volume_mounts | Deleting namespace: kuttl-test-humble-antelope === CONT kuttl/harness/1-061_validate_resource_tracking_method logger.go:42: 17:41:24 | 1-061_validate_resource_tracking_method | Creating namespace: kuttl-test-enormous-raccoon logger.go:42: 17:41:24 | 1-061_validate_resource_tracking_method/1-install | starting test step 1-install I0723 17:41:25.753431 167 request.go:682] Waited for 1.041671517s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/network.operator.openshift.io/v1?timeout=32s logger.go:42: 17:41:27 | 1-061_validate_resource_tracking_method/1-install | ArgoCD:kuttl-test-enormous-raccoon/argocd created === CONT kuttl/harness/1-073_validate_rhsso logger.go:42: 17:41:31 | 1-073_validate_rhsso/2-sleep | test step completed 2-sleep logger.go:42: 17:41:31 | 1-073_validate_rhsso/3-verify-oidc | starting test step 3-verify-oidc logger.go:42: 17:41:31 | 1-073_validate_rhsso/3-verify-oidc | running command: [sh -c issuer=$(kubectl get configmap argocd-cm -o jsonpath='{.data.oidc\.config}' -n $NAMESPACE | grep issuer | awk -F' ' '{print $2}') keycloakRoute=$(kubectl get route keycloak -n $NAMESPACE -o jsonpath='{.spec.host}') if [[ "${issuer}" == "https://${keycloakRoute}/auth/realms/argocd" ]]; then echo "issuer matched" else echo "issuer mismatched" echo "${issuer} not equals ${keycloakRoute}/auth/realms/argocd" exit 1 fi ] logger.go:42: 17:41:33 | 1-073_validate_rhsso/3-verify-oidc | issuer matched logger.go:42: 17:41:33 | 1-073_validate_rhsso/3-verify-oidc | running command: [sh -c clientid=$(kubectl get configmap argocd-cm -o jsonpath='{.data.oidc\.config}' -n $NAMESPACE | grep clientid | awk -F' ' '{print $2}') name=$(kubectl get configmap argocd-cm -o jsonpath='{.data.oidc\.config}' -n $NAMESPACE | grep name | awk -F' ' '{print $2}') rootCA=$(kubectl get configmap argocd-cm -o jsonpath='{.data.oidc\.config}' -n $NAMESPACE | grep rootca | awk -F' ' '{print $2}') if [[ "${clientid}" == "argocd" && "${name}" == "Keycloak" && "${rootCA}" == "'---BEGIN---END---'" ]]; then echo "oidc name, clientid and rootCA matched" else echo "oidc name, clientid or rootCA mismatched" echo "${clientid}, ${name} and ${rootCA}" exit 1 fi ] logger.go:42: 17:41:34 | 1-073_validate_rhsso/3-verify-oidc | oidc name, clientid and rootCA matched I0723 17:41:35.758033 167 request.go:682] Waited for 1.046584732s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/imageregistry.operator.openshift.io/v1?timeout=32s logger.go:42: 17:41:37 | 1-073_validate_rhsso/3-verify-oidc | test step completed 3-verify-oidc logger.go:42: 17:41:37 | 1-073_validate_rhsso/4-verifyRealmCreation | starting test step 4-verifyRealmCreation logger.go:42: 17:41:37 | 1-073_validate_rhsso/4-verifyRealmCreation | running command: [sh -c # Set the needed parameter for the authorization KEYCLOAK_URL=$(oc get route keycloak -n $NAMESPACE -o jsonpath='{.spec.host}') tenant=argocd USER=$(oc get secret keycloak-secret -n $NAMESPACE -o jsonpath='{.data.SSO_USERNAME}' | base64 --decode) PASSWORD=$(oc get secret keycloak-secret -n $NAMESPACE -o jsonpath='{.data.SSO_PASSWORD}' | base64 --decode) GRANT_TYPE=password CLIENT_ID=admin-cli # Execute the CURL command to request the access-token access_token=$(curl -d "client_id=$CLIENT_ID" -d "username=$USER" -d "password=$PASSWORD" -d "grant_type=$GRANT_TYPE" "https://$KEYCLOAK_URL/auth/realms/master/protocol/openid-connect/token" -k | sed -n 's|.*"access_token":"\([^"]*\)".*|\1|p') # Execute the CURL command to verify the realm and client creation clientFound=$(curl -H "Content-Type: application/json" -H "Authorization: bearer $access_token" "https://$KEYCLOAK_URL/auth/admin/realms/$tenant/clients" -k | grep '"clientId":"argocd"') if ! [ "$clientFound" = "" ]; then echo "argocd realm and client creation verified" else echo "argocd client not found" exit 1 fi # Verify OpenShift-v4 IdP creation IdPFound=$(curl -H "Content-Type: application/json" -H "Authorization: bearer $access_token" "https://$KEYCLOAK_URL/auth/admin/realms/$tenant/identity-provider/instances" -k | grep -i OpenShift-v4) if ! [ "$IdPFound" = "" ]; then echo "OpenShift-v4 IdP creation verified" else echo "OpenShift-v4 IdP not found" exit 1 fi # Verify OpenShift-v4 IdP creation syncModeVerified=$(curl -H "Content-Type: application/json" -H "Authorization: bearer $access_token" "https://$KEYCLOAK_URL/auth/admin/realms/$tenant/identity-provider/instances" -k | grep -i '"syncMode":"FORCE"') if ! [ "$syncModeVerified" = "" ]; then echo "syncMode set to Force, verified" else echo "syncMode not set to Force" exit 1 fi ] logger.go:42: 17:41:39 | 1-073_validate_rhsso/4-verifyRealmCreation | % Total % Received % Xferd Average Speed Time Time Time Current logger.go:42: 17:41:39 | 1-073_validate_rhsso/4-verifyRealmCreation | Dload Upload Total Spent Left Speed logger.go:42: 17:41:39 | 1-073_validate_rhsso/4-verifyRealmCreation | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 2125 100 2050 100 75 18303 669 --:--:-- --:--:-- --:--:-- 18973 logger.go:42: 17:41:39 | 1-073_validate_rhsso/4-verifyRealmCreation | % Total % Received % Xferd Average Speed Time Time Time Current logger.go:42: 17:41:39 | 1-073_validate_rhsso/4-verifyRealmCreation | Dload Upload Total Spent Left Speed logger.go:42: 17:41:39 | 1-073_validate_rhsso/4-verifyRealmCreation | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 6424 100 6424 0 0 16140 0 --:--:-- --:--:-- --:--:-- 16181 logger.go:42: 17:41:39 | 1-073_validate_rhsso/4-verifyRealmCreation | argocd realm and client creation verified logger.go:42: 17:41:39 | 1-073_validate_rhsso/4-verifyRealmCreation | % Total % Received % Xferd Average Speed Time Time Time Current logger.go:42: 17:41:39 | 1-073_validate_rhsso/4-verifyRealmCreation | Dload Upload Total Spent Left Speed logger.go:42: 17:41:39 | 1-073_validate_rhsso/4-verifyRealmCreation | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 552 100 552 0 0 14526 0 --:--:-- --:--:-- --:--:-- 14526 logger.go:42: 17:41:39 | 1-073_validate_rhsso/4-verifyRealmCreation | OpenShift-v4 IdP creation verified logger.go:42: 17:41:39 | 1-073_validate_rhsso/4-verifyRealmCreation | % Total % Received % Xferd Average Speed Time Time Time Current logger.go:42: 17:41:39 | 1-073_validate_rhsso/4-verifyRealmCreation | Dload Upload Total Spent Left Speed logger.go:42: 17:41:39 | 1-073_validate_rhsso/4-verifyRealmCreation | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 552 100 552 0 0 13463 0 --:--:-- --:--:-- --:--:-- 13463 logger.go:42: 17:41:39 | 1-073_validate_rhsso/4-verifyRealmCreation | syncMode set to Force, verified logger.go:42: 17:41:42 | 1-073_validate_rhsso/4-verifyRealmCreation | test step completed 4-verifyRealmCreation logger.go:42: 17:41:42 | 1-073_validate_rhsso/99-delete | starting test step 99-delete logger.go:42: 17:41:42 | 1-073_validate_rhsso/99-delete | running command: [oc delete argocd example-argocd-keycloak -n kuttl-test-sure-dory] logger.go:42: 17:41:42 | 1-073_validate_rhsso/99-delete | argocd.argoproj.io "example-argocd-keycloak" deleted logger.go:42: 17:41:45 | 1-073_validate_rhsso/99-delete | test step completed 99-delete logger.go:42: 17:41:45 | 1-073_validate_rhsso | skipping kubernetes event logging logger.go:42: 17:41:45 | 1-073_validate_rhsso | Deleting namespace: kuttl-test-sure-dory === CONT kuttl/harness/1-058_validate_prometheus_rule logger.go:42: 17:41:51 | 1-058_validate_prometheus_rule | Creating namespace: kuttl-test-on-owl logger.go:42: 17:41:51 | 1-058_validate_prometheus_rule/1-install | starting test step 1-install I0723 17:41:52.900287 167 request.go:682] Waited for 1.039341634s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/machineconfiguration.openshift.io/v1?timeout=32s logger.go:42: 17:41:54 | 1-058_validate_prometheus_rule/1-install | ArgoCD:kuttl-test-on-owl/example-argocd created === CONT kuttl/harness/1-061_validate_resource_tracking_method logger.go:42: 17:41:59 | 1-061_validate_resource_tracking_method/1-install | test step completed 1-install logger.go:42: 17:41:59 | 1-061_validate_resource_tracking_method/2-patch_cm_annotation | starting test step 2-patch_cm_annotation logger.go:42: 17:41:59 | 1-061_validate_resource_tracking_method/2-patch_cm_annotation | running command: [sh -c set -e oc patch argocds.argoproj.io argocd --type=merge -p '{"spec":{"resourceTrackingMethod":"annotation"}}' -n $NAMESPACE ] logger.go:42: 17:42:00 | 1-061_validate_resource_tracking_method/2-patch_cm_annotation | argocd.argoproj.io/argocd patched logger.go:42: 17:42:02 | 1-061_validate_resource_tracking_method/2-patch_cm_annotation | test step completed 2-patch_cm_annotation logger.go:42: 17:42:02 | 1-061_validate_resource_tracking_method/3-patch_cm_annotation_label | starting test step 3-patch_cm_annotation_label logger.go:42: 17:42:02 | 1-061_validate_resource_tracking_method/3-patch_cm_annotation_label | running command: [sh -c set -e oc patch argocds.argoproj.io argocd --type=merge -p '{"spec":{"resourceTrackingMethod":"annotation+label"}}' -n $NAMESPACE ] === CONT kuttl/harness/1-050_validate_sso logger.go:42: 17:42:02 | 1-050_validate_sso/8-remove-spec-dex | test step completed 8-remove-spec-dex logger.go:42: 17:42:02 | 1-050_validate_sso | skipping kubernetes event logging logger.go:42: 17:42:02 | 1-050_validate_sso | Deleting namespace: kuttl-test-top-sawfish === CONT kuttl/harness/1-061_validate_resource_tracking_method logger.go:42: 17:42:03 | 1-061_validate_resource_tracking_method/3-patch_cm_annotation_label | argocd.argoproj.io/argocd patched I0723 17:42:04.771860 167 request.go:682] Waited for 1.046983349s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/route.openshift.io/v1?timeout=32s logger.go:42: 17:42:06 | 1-061_validate_resource_tracking_method/3-patch_cm_annotation_label | test step completed 3-patch_cm_annotation_label logger.go:42: 17:42:06 | 1-061_validate_resource_tracking_method/4-patch_cm_invalid_method | starting test step 4-patch_cm_invalid_method logger.go:42: 17:42:06 | 1-061_validate_resource_tracking_method/4-patch_cm_invalid_method | running command: [sh -c set -e oc patch argocds.argoproj.io argocd --type=merge -p '{"spec":{"resourceTrackingMethod":"invalid_method"}}' -n $NAMESPACE ] logger.go:42: 17:42:07 | 1-061_validate_resource_tracking_method/4-patch_cm_invalid_method | argocd.argoproj.io/argocd patched === CONT kuttl/harness/1-058_validate_prometheus_rule logger.go:42: 17:42:09 | 1-058_validate_prometheus_rule/1-install | test step completed 1-install logger.go:42: 17:42:09 | 1-058_validate_prometheus_rule/2-wait | starting test step 2-wait logger.go:42: 17:42:09 | 1-058_validate_prometheus_rule/2-wait | running command: [sh -c function wait_until_pods_running() { echo -n "Waiting until all pods in namespace $1 are up" for i in {1..150}; do # timeout after 5 minutes local pods="$(oc get pods --no-headers -n $1 2>/dev/null)" # All pods must be running local not_running=$(echo "${pods}" | grep -v Running | grep -v Completed | wc -l) if [[ -n "${pods}" && ${not_running} -eq 0 ]]; then local all_ready=1 while read pod ; do local status=(`echo -n ${pod} | cut -f2 -d' ' | tr '/' ' '`) # All containers must be ready [[ -z ${status[0]} ]] && all_ready=0 && break [[ -z ${status[1]} ]] && all_ready=0 && break [[ ${status[0]} -lt 1 ]] && all_ready=0 && break [[ ${status[1]} -lt 1 ]] && all_ready=0 && break [[ ${status[0]} -ne ${status[1]} ]] && all_ready=0 && break done <<< $(echo "${pods}" | grep -v Completed) if (( all_ready )); then echo -e "\nAll pods are up:\n${pods}" return 0 fi fi echo -n "." sleep 2 done echo -e "\n\nERROR: timeout waiting for pods to come up\n${pods}" return 1 } wait_until_pods_running $NAMESPACE ] === CONT kuttl/harness/1-061_validate_resource_tracking_method logger.go:42: 17:42:09 | 1-061_validate_resource_tracking_method/4-patch_cm_invalid_method | test step completed 4-patch_cm_invalid_method logger.go:42: 17:42:09 | 1-061_validate_resource_tracking_method | skipping kubernetes event logging logger.go:42: 17:42:09 | 1-061_validate_resource_tracking_method | Deleting namespace: kuttl-test-enormous-raccoon === CONT kuttl/harness/1-058_validate_prometheus_rule logger.go:42: 17:42:10 | 1-058_validate_prometheus_rule/2-wait | Waiting until all pods in namespace kuttl-test-on-owl are up logger.go:42: 17:42:10 | 1-058_validate_prometheus_rule/2-wait | All pods are up: logger.go:42: 17:42:10 | 1-058_validate_prometheus_rule/2-wait | example-argocd-application-controller-0 1/1 Running 0 14s logger.go:42: 17:42:10 | 1-058_validate_prometheus_rule/2-wait | example-argocd-redis-6d79595d67-g5xp2 1/1 Running 0 14s logger.go:42: 17:42:10 | 1-058_validate_prometheus_rule/2-wait | example-argocd-repo-server-6f6b8dbc6-rc8tc 1/1 Running 0 14s logger.go:42: 17:42:10 | 1-058_validate_prometheus_rule/2-wait | example-argocd-server-57b755996c-m8l9q 1/1 Running 0 14s logger.go:42: 17:42:12 | 1-058_validate_prometheus_rule/2-wait | test step completed 2-wait logger.go:42: 17:42:12 | 1-058_validate_prometheus_rule/3-create_application | starting test step 3-create_application logger.go:42: 17:42:12 | 1-058_validate_prometheus_rule/3-create_application | running command: [sh -c set -e cat << EOF | oc apply -f - apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: test-1-58-custom namespace: ${NAMESPACE} spec: project: default source: repoURL: https://github.com/jaideepr97/gitops-examples path: bgd-k8s targetRevision: "HEAD" destination: server: "https://kubernetes.default.svc" namespace: ${NAMESPACE} syncPolicy: automated: {} EOF ] logger.go:42: 17:42:14 | 1-058_validate_prometheus_rule/3-create_application | application.argoproj.io/test-1-58-custom created === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 17:42:14 | 1-012_validate-managed-by-chain | Ignoring README.md as it does not match file name regexp: ^(\d+)-(?:[^\.]+)(?:\.yaml)?$ I0723 17:42:15.358152 167 request.go:682] Waited for 1.037068809s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/certificates.k8s.io/v1?timeout=32s === CONT kuttl/harness/1-057_validate_notifications === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 17:42:16 | 1-012_validate-managed-by-chain | Creating namespace: kuttl-test-cosmic-stallion === CONT kuttl/harness/1-057_validate_notifications logger.go:42: 17:42:16 | 1-057_validate_notifications | Creating namespace: kuttl-test-top-walleye === CONT kuttl/harness/1-058_validate_prometheus_rule logger.go:42: 17:42:16 | 1-058_validate_prometheus_rule/3-create_application | test step completed 3-create_application logger.go:42: 17:42:16 | 1-058_validate_prometheus_rule | skipping kubernetes event logging === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 17:42:16 | 1-012_validate-managed-by-chain/1-install | starting test step 1-install === CONT kuttl/harness/1-057_validate_notifications logger.go:42: 17:42:16 | 1-057_validate_notifications/1-install | starting test step 1-install === CONT kuttl/harness/1-058_validate_prometheus_rule logger.go:42: 17:42:16 | 1-058_validate_prometheus_rule | Deleting namespace: kuttl-test-on-owl === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 17:42:19 | 1-012_validate-managed-by-chain/1-install | Namespace:/test-1-12-custom created logger.go:42: 17:42:19 | 1-012_validate-managed-by-chain/1-install | Namespace:/test-1-12-custom2 created logger.go:42: 17:42:19 | 1-012_validate-managed-by-chain/1-install | ArgoCD:kuttl-test-cosmic-stallion/argocd created === CONT kuttl/harness/1-057_validate_notifications logger.go:42: 17:42:21 | 1-057_validate_notifications/1-install | ArgoCD:kuttl-test-top-walleye/argocd created logger.go:42: 17:42:21 | 1-057_validate_notifications/1-install | Service:kuttl-test-top-walleye/smtp4dev created logger.go:42: 17:42:21 | 1-057_validate_notifications/1-install | Deployment:kuttl-test-top-walleye/smtp4dev created === CONT kuttl/harness/1-055_validate_notification_controller logger.go:42: 17:42:23 | 1-055_validate_notification_controller | Creating namespace: kuttl-test-choice-crow logger.go:42: 17:42:23 | 1-055_validate_notification_controller/1-install | starting test step 1-install I0723 17:42:25.394739 167 request.go:682] Waited for 2.147081108s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/tuned.openshift.io/v1?timeout=32s logger.go:42: 17:42:25 | 1-055_validate_notification_controller/1-install | ArgoCD:kuttl-test-choice-crow/example-argocd created === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 17:42:51 | 1-012_validate-managed-by-chain/1-install | test step completed 1-install logger.go:42: 17:42:51 | 1-012_validate-managed-by-chain/2-label-namespace | starting test step 2-label-namespace logger.go:42: 17:42:51 | 1-012_validate-managed-by-chain/2-label-namespace | running command: [sh -c kubectl label ns test-1-12-custom argocd.argoproj.io/managed-by=$NAMESPACE] logger.go:42: 17:42:51 | 1-012_validate-managed-by-chain/2-label-namespace | namespace/test-1-12-custom labeled I0723 17:42:52.854193 167 request.go:682] Waited for 1.046429075s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/autoscaling.openshift.io/v1beta1?timeout=32s logger.go:42: 17:42:54 | 1-012_validate-managed-by-chain/2-label-namespace | test step completed 2-label-namespace logger.go:42: 17:42:54 | 1-012_validate-managed-by-chain/3-check-secret | starting test step 3-check-secret logger.go:42: 17:42:54 | 1-012_validate-managed-by-chain/3-check-secret | running command: [sh -c namespaces=$(oc get secret -n $NAMESPACE argocd-default-cluster-config -o jsonpath='{.data.namespaces}' | base64 -d) if test "$namespaces" != "$NAMESPACE,test-1-12-custom"; then echo "Assertion for cluster secret failed!" exit 1 fi exit 0 ] === CONT kuttl/harness/1-057_validate_notifications logger.go:42: 17:42:54 | 1-057_validate_notifications/1-install | test step completed 1-install logger.go:42: 17:42:54 | 1-057_validate_notifications/2-update-notifications-cm | starting test step 2-update-notifications-cm logger.go:42: 17:42:54 | 1-057_validate_notifications/2-update-notifications-cm | running command: [sh -c sleep 15] === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 17:42:57 | 1-012_validate-managed-by-chain/3-check-secret | test step completed 3-check-secret logger.go:42: 17:42:57 | 1-012_validate-managed-by-chain/4-create-application | starting test step 4-create-application === CONT kuttl/harness/1-055_validate_notification_controller logger.go:42: 17:43:00 | 1-055_validate_notification_controller/1-install | test step completed 1-install logger.go:42: 17:43:00 | 1-055_validate_notification_controller/2-enable_notification | starting test step 2-enable_notification === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 17:43:00 | 1-012_validate-managed-by-chain/4-create-application | Application:kuttl-test-cosmic-stallion/test-1-12-custom created === CONT kuttl/harness/1-055_validate_notification_controller logger.go:42: 17:43:02 | 1-055_validate_notification_controller/2-enable_notification | ArgoCD:kuttl-test-choice-crow/example-argocd updated logger.go:42: 17:43:05 | 1-055_validate_notification_controller/2-enable_notification | test step completed 2-enable_notification logger.go:42: 17:43:05 | 1-055_validate_notification_controller/3-add-sleep | starting test step 3-add-sleep logger.go:42: 17:43:05 | 1-055_validate_notification_controller/3-add-sleep | running command: [sh -c sleep 20] === CONT kuttl/harness/1-092_validate_workload_status_monitoring_alert logger.go:42: 17:43:06 | 1-092_validate_workload_status_monitoring_alert/2-verify-alert | running command: [sh -c set -eo pipefail cleanup() { rm alerts.json } trap cleanup INT TERM EXIT oc whoami jq '.data.alerts' <<< "$(oc -n openshift-monitoring exec -c prometheus prometheus-k8s-0 -- curl -s 'http://localhost:9090/api/v1/alerts')" > alerts.json result=$(jq --arg NAMESPACE "$NAMESPACE" ' .[] | select( .labels.deployment == "argocd-applicationset-controller" and .labels.namespace == $NAMESPACE and .labels.alertname == "ApplicationSetControllerNotReady" and .state == "firing")' alerts.json) [[ -n "$result" ]] && exit 0 || exit 1 ] logger.go:42: 17:43:06 | 1-092_validate_workload_status_monitoring_alert/2-verify-alert | kube:admin I0723 17:43:08.561037 167 request.go:682] Waited for 1.046633685s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/operators.coreos.com/v2?timeout=32s === CONT kuttl/harness/1-057_validate_notifications logger.go:42: 17:43:09 | 1-057_validate_notifications/2-update-notifications-cm | running command: [sh -c set -e kubectl patch notificationsconfiguration default-notifications-configuration -n $NAMESPACE --type='json' -p='[{"op": "add", "path": "/spec/services", "value": {"service.email.gmail": "{host: smtp4dev, port: 2525, from: fake@email.com }" } }]' ] logger.go:42: 17:43:09 | 1-057_validate_notifications/2-update-notifications-cm | notificationsconfiguration.argoproj.io/default-notifications-configuration patched === CONT kuttl/harness/1-092_validate_workload_status_monitoring_alert logger.go:42: 17:43:10 | 1-092_validate_workload_status_monitoring_alert/2-verify-alert | test step completed 2-verify-alert logger.go:42: 17:43:10 | 1-092_validate_workload_status_monitoring_alert/3-disable-monitoring | starting test step 3-disable-monitoring === CONT kuttl/harness/1-057_validate_notifications logger.go:42: 17:43:10 | 1-057_validate_notifications/2-update-notifications-cm | running command: [sh -c sleep 5] === CONT kuttl/harness/1-092_validate_workload_status_monitoring_alert logger.go:42: 17:43:12 | 1-092_validate_workload_status_monitoring_alert/3-disable-monitoring | ArgoCD:kuttl-test-flexible-mallard/argocd updated logger.go:42: 17:43:13 | 1-092_validate_workload_status_monitoring_alert/3-disable-monitoring | test step completed 3-disable-monitoring logger.go:42: 17:43:13 | 1-092_validate_workload_status_monitoring_alert | skipping kubernetes event logging logger.go:42: 17:43:13 | 1-092_validate_workload_status_monitoring_alert | Deleting namespace: kuttl-test-flexible-mallard === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 17:43:14 | 1-012_validate-managed-by-chain/4-create-application | test step completed 4-create-application logger.go:42: 17:43:14 | 1-012_validate-managed-by-chain/7-label-namespace | starting test step 7-label-namespace logger.go:42: 17:43:14 | 1-012_validate-managed-by-chain/7-label-namespace | running command: [sh -c kubectl label ns test-1-12-custom2 argocd.argoproj.io/managed-by=$NAMESPACE] logger.go:42: 17:43:14 | 1-012_validate-managed-by-chain/7-label-namespace | namespace/test-1-12-custom2 labeled I0723 17:43:18.561847 167 request.go:682] Waited for 1.196293118s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/operators.coreos.com/v1alpha1?timeout=32s === CONT kuttl/harness/1-055_drop_resource_customizations === CONT kuttl/harness/1-057_validate_notifications logger.go:42: 17:43:19 | 1-057_validate_notifications/2-update-notifications-cm | test step completed 2-update-notifications-cm logger.go:42: 17:43:19 | 1-057_validate_notifications/3-create-app | starting test step 3-create-app logger.go:42: 17:43:19 | 1-057_validate_notifications/3-create-app | running command: [sh -c set -e cat << EOF | oc apply -f - apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: my-app-3 namespace: $NAMESPACE annotations: "notifications.argoproj.io/subscribe.on-created.gmail": "jdfake@email.com" spec: destination: namespace: $NAMESPACE server: https://kubernetes.default.svc project: default source: repoURL: https://github.com/redhat-developer/gitops-operator path: test/examples/nginx targetRevision: HEAD EOF ] === CONT kuttl/harness/1-055_drop_resource_customizations logger.go:42: 17:43:19 | 1-055_drop_resource_customizations | Creating namespace: kuttl-test-absolute-crayfish logger.go:42: 17:43:19 | 1-055_drop_resource_customizations/1-alpha-argocd | starting test step 1-alpha-argocd === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 17:43:19 | 1-012_validate-managed-by-chain/7-label-namespace | test step completed 7-label-namespace logger.go:42: 17:43:19 | 1-012_validate-managed-by-chain/8-create-application | starting test step 8-create-application === CONT kuttl/harness/1-057_validate_notifications logger.go:42: 17:43:21 | 1-057_validate_notifications/3-create-app | application.argoproj.io/my-app-3 created logger.go:42: 17:43:21 | 1-057_validate_notifications/3-create-app | running command: [sh -c sleep 5] === CONT kuttl/harness/1-055_drop_resource_customizations logger.go:42: 17:43:22 | 1-055_drop_resource_customizations/1-alpha-argocd | ArgoCD:kuttl-test-absolute-crayfish/argocd created === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 17:43:24 | 1-012_validate-managed-by-chain/8-create-application | Application:kuttl-test-cosmic-stallion/test-1-12-custom2 created === CONT kuttl/harness/1-055_validate_notification_controller logger.go:42: 17:43:28 | 1-055_validate_notification_controller/3-add-sleep | ArgoCD:kuttl-test-choice-crow/example-argocd updated I0723 17:43:29.513300 167 request.go:682] Waited for 1.045154446s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/security.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-057_validate_notifications logger.go:42: 17:43:30 | 1-057_validate_notifications/3-create-app | test step completed 3-create-app logger.go:42: 17:43:30 | 1-057_validate_notifications/4-delete-app | starting test step 4-delete-app logger.go:42: 17:43:30 | 1-057_validate_notifications/4-delete-app | running command: [sh -c set -e kubectl delete -n $NAMESPACE application.argoproj.io my-app-3 ] === CONT kuttl/harness/1-055_validate_notification_controller logger.go:42: 17:43:31 | 1-055_validate_notification_controller/3-add-sleep | test step completed 3-add-sleep logger.go:42: 17:43:31 | 1-055_validate_notification_controller | skipping kubernetes event logging logger.go:42: 17:43:31 | 1-055_validate_notification_controller | Deleting namespace: kuttl-test-choice-crow === CONT kuttl/harness/1-057_validate_notifications logger.go:42: 17:43:31 | 1-057_validate_notifications/4-delete-app | application.argoproj.io "my-app-3" deleted logger.go:42: 17:43:31 | 1-057_validate_notifications/4-delete-app | running command: [sh -c sleep 5] === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 17:43:31 | 1-012_validate-managed-by-chain/8-create-application | test step completed 8-create-application logger.go:42: 17:43:31 | 1-012_validate-managed-by-chain/9-delete-app-and-namespace | starting test step 9-delete-app-and-namespace === CONT kuttl/harness/1-055_drop_resource_customizations logger.go:42: 17:43:34 | 1-055_drop_resource_customizations/1-alpha-argocd | test step completed 1-alpha-argocd logger.go:42: 17:43:34 | 1-055_drop_resource_customizations/2- | starting test step 2- === CONT kuttl/harness/1-054_validate_deploymentconfig === CONT kuttl/harness/1-055_drop_resource_customizations logger.go:42: 17:43:37 | 1-055_drop_resource_customizations/2- | test step completed 2- logger.go:42: 17:43:37 | 1-055_drop_resource_customizations | skipping kubernetes event logging logger.go:42: 17:43:37 | 1-055_drop_resource_customizations | Deleting namespace: kuttl-test-absolute-crayfish I0723 17:43:39.559346 167 request.go:682] Waited for 2.146651962s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/template.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-054_validate_deploymentconfig logger.go:42: 17:43:39 | 1-054_validate_deploymentconfig | Creating namespace: kuttl-test-full-bird logger.go:42: 17:43:39 | 1-054_validate_deploymentconfig/1-install | starting test step 1-install logger.go:42: 17:43:42 | 1-054_validate_deploymentconfig/1-install | ArgoCD:kuttl-test-full-bird/argocd created === CONT kuttl/harness/1-057_validate_notifications logger.go:42: 17:43:42 | 1-057_validate_notifications/4-delete-app | test step completed 4-delete-app logger.go:42: 17:43:42 | 1-057_validate_notifications/5-verify-email | starting test step 5-verify-email logger.go:42: 17:43:42 | 1-057_validate_notifications/5-verify-email | running command: [sh -c set -e smtp4dev_pod=$(kubectl get pod -l=app=smtp4dev -o NAME -n $NAMESPACE) exit_code=$(kubectl -n $NAMESPACE exec -i --stdin "${smtp4dev_pod}" -- /bin/bash \ -c 'if [[ $(grep -rnw /tmp -e "Subject: Application my-app-3 has been created.") ]]; then exit 0; else exit 1; fi') if [ $exit_code=0 ]; then exit 0 else exit 1 fi ] === CONT kuttl/harness/1-053_validate_cluster_admin_rbac === CONT kuttl/harness/1-057_validate_notifications logger.go:42: 17:43:47 | 1-057_validate_notifications/5-verify-email | test step completed 5-verify-email logger.go:42: 17:43:47 | 1-057_validate_notifications | skipping kubernetes event logging logger.go:42: 17:43:47 | 1-057_validate_notifications | Deleting namespace: kuttl-test-top-walleye I0723 17:43:49.579880 167 request.go:682] Waited for 2.347010737s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/autoscaling/v1?timeout=32s === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 17:43:49 | 1-012_validate-managed-by-chain/9-delete-app-and-namespace | test step completed 9-delete-app-and-namespace logger.go:42: 17:43:49 | 1-012_validate-managed-by-chain/10-check-secret | starting test step 10-check-secret logger.go:42: 17:43:49 | 1-012_validate-managed-by-chain/10-check-secret | running command: [sh -c should="$NAMESPACE,test-1-12-custom2" namespaces=$(oc get secret -n $NAMESPACE argocd-default-cluster-config -o jsonpath='{.data.namespaces}' | base64 -d) if test "$namespaces" != "$should"; then echo "Assertion for cluster secret failed! '$namespaces' != '$should'" # 1.2.0 doesn't reconcile the cluster secret on namespace deletion if test GITOPS_TARGET_VERSION = "1.2.0"; then exit 0 else exit 1 fi fi exit 0 ] === CONT kuttl/harness/1-053_validate_cluster_admin_rbac logger.go:42: 17:43:49 | 1-053_validate_cluster_admin_rbac | Creating namespace: kuttl-test-major-bull logger.go:42: 17:43:49 | 1-053_validate_cluster_admin_rbac/1- | starting test step 1- logger.go:42: 17:43:52 | 1-053_validate_cluster_admin_rbac/1- | test step completed 1- logger.go:42: 17:43:52 | 1-053_validate_cluster_admin_rbac/2- | starting test step 2- === CONT kuttl/harness/1-009_validate-manage-other-namespace logger.go:42: 17:43:53 | 1-009_validate-manage-other-namespace | Ignoring README.md as it does not match file name regexp: ^(\d+)-(?:[^\.]+)(?:\.yaml)?$ logger.go:42: 17:43:53 | 1-009_validate-manage-other-namespace | Ignoring errors.yaml as it does not match file name regexp: ^(\d+)-(?:[^\.]+)(?:\.yaml)?$ === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 17:43:54 | 1-012_validate-managed-by-chain/10-check-secret | test step completed 10-check-secret logger.go:42: 17:43:54 | 1-012_validate-managed-by-chain/11-recreate-applications | starting test step 11-recreate-applications === CONT kuttl/harness/1-009_validate-manage-other-namespace logger.go:42: 17:43:57 | 1-009_validate-manage-other-namespace | Creating namespace: kuttl-test-handy-raccoon logger.go:42: 17:43:57 | 1-009_validate-manage-other-namespace/1-install | starting test step 1-install === CONT kuttl/harness/1-054_validate_deploymentconfig logger.go:42: 17:43:57 | 1-054_validate_deploymentconfig/1-install | test step completed 1-install logger.go:42: 17:43:57 | 1-054_validate_deploymentconfig/2-install-deploymentconfig | starting test step 2-install-deploymentconfig === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 17:43:57 | 1-012_validate-managed-by-chain/11-recreate-applications | Namespace:/test-1-12-custom created logger.go:42: 17:43:57 | 1-012_validate-managed-by-chain/11-recreate-applications | Application:kuttl-test-cosmic-stallion/test-1-12-custom created logger.go:42: 17:43:57 | 1-012_validate-managed-by-chain/11-recreate-applications | Application:kuttl-test-cosmic-stallion/test-1-12-custom2 created === CONT kuttl/harness/1-053_validate_cluster_admin_rbac logger.go:42: 17:43:59 | 1-053_validate_cluster_admin_rbac/2- | test step completed 2- logger.go:42: 17:43:59 | 1-053_validate_cluster_admin_rbac | skipping kubernetes event logging logger.go:42: 17:43:59 | 1-053_validate_cluster_admin_rbac | Deleting namespace: kuttl-test-major-bull I0723 17:44:00.615118 167 request.go:682] Waited for 1.011918359s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/network.operator.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-054_validate_deploymentconfig logger.go:42: 17:44:02 | 1-054_validate_deploymentconfig/2-install-deploymentconfig | running command: [sh -c set -eo pipefail # Install ArgoCD Application with 2 replicas cat << EOF | oc apply -f - apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: app-deploymentconfig namespace: ${NAMESPACE} spec: project: default source: repoURL: https://github.com/redhat-developer/gitops-operator path: test/examples/deploymentconfig-example targetRevision: "HEAD" destination: server: https://kubernetes.default.svc namespace: ${NAMESPACE} syncPolicy: automated: {} EOF # Give some time to Application before asserting sleep 10 exit 0 ] === CONT kuttl/harness/1-009_validate-manage-other-namespace logger.go:42: 17:44:02 | 1-009_validate-manage-other-namespace/1-install | Namespace:/test-1-9-custom created === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 17:44:02 | 1-012_validate-managed-by-chain/11-recreate-applications | test step completed 11-recreate-applications logger.go:42: 17:44:02 | 1-012_validate-managed-by-chain/99-delete | starting test step 99-delete === CONT kuttl/harness/1-009_validate-manage-other-namespace logger.go:42: 17:44:02 | 1-009_validate-manage-other-namespace/1-install | ArgoCD:kuttl-test-handy-raccoon/argocd created === CONT kuttl/harness/1-054_validate_deploymentconfig logger.go:42: 17:44:04 | 1-054_validate_deploymentconfig/2-install-deploymentconfig | application.argoproj.io/app-deploymentconfig created === CONT kuttl/harness/1-052_validate_rolebinding_number logger.go:42: 17:44:05 | 1-052_validate_rolebinding_number | Creating namespace: kuttl-test-expert-tiger logger.go:42: 17:44:05 | 1-052_validate_rolebinding_number/1-label_namespace | starting test step 1-label_namespace logger.go:42: 17:44:05 | 1-052_validate_rolebinding_number/1-label_namespace | running command: [sh -c oc label namespace $NAMESPACE argocd.argoproj.io/managed-by=openshift-gitops ] logger.go:42: 17:44:06 | 1-052_validate_rolebinding_number/1-label_namespace | namespace/kuttl-test-expert-tiger labeled logger.go:42: 17:44:08 | 1-052_validate_rolebinding_number/1-label_namespace | test step completed 1-label_namespace logger.go:42: 17:44:08 | 1-052_validate_rolebinding_number/2-check_rolebindings | starting test step 2-check_rolebindings logger.go:42: 17:44:08 | 1-052_validate_rolebinding_number/2-check_rolebindings | running command: [sh -c set -eo pipefail # Expected and Current RoleBindings expected_rb=( "openshift-gitops-argocd-application-controller" "openshift-gitops-argocd-server" ) current_rb=( $(oc get rolebindings -n "${NAMESPACE}" | awk '/gitops/ {print $1}') ) # Check that the required RoleBindings exist: for rb in "${expected_rb[@]}" do oc get rolebinding "${rb}" -n "${NAMESPACE}" > /dev/null done # Check that there are only two RoleBindings echo "Current RoleBindings: ${current_rb[*]}" [[ "${#current_rb[@]}" == "2" ]] ] logger.go:42: 17:44:10 | 1-052_validate_rolebinding_number/2-check_rolebindings | Current RoleBindings: openshift-gitops-argocd-application-controller openshift-gitops-argocd-server I0723 17:44:11.758794 167 request.go:682] Waited for 1.044696304s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/samples.operator.openshift.io/v1?timeout=32s logger.go:42: 17:44:13 | 1-052_validate_rolebinding_number/2-check_rolebindings | test step completed 2-check_rolebindings logger.go:42: 17:44:13 | 1-052_validate_rolebinding_number | skipping kubernetes event logging logger.go:42: 17:44:13 | 1-052_validate_rolebinding_number | Deleting namespace: kuttl-test-expert-tiger === CONT kuttl/harness/1-012_validate-managed-by-chain logger.go:42: 17:44:16 | 1-012_validate-managed-by-chain/99-delete | test step completed 99-delete logger.go:42: 17:44:16 | 1-012_validate-managed-by-chain | skipping kubernetes event logging logger.go:42: 17:44:16 | 1-012_validate-managed-by-chain | Deleting namespace: kuttl-test-cosmic-stallion === CONT kuttl/harness/1-054_validate_deploymentconfig logger.go:42: 17:44:18 | 1-054_validate_deploymentconfig/2-install-deploymentconfig | test step completed 2-install-deploymentconfig logger.go:42: 17:44:18 | 1-054_validate_deploymentconfig/3- | starting test step 3- === CONT kuttl/harness/1-008_validate-custom-argocd-namespace logger.go:42: 17:44:21 | 1-008_validate-custom-argocd-namespace | Creating namespace: kuttl-test-hopeful-redbird logger.go:42: 17:44:21 | 1-008_validate-custom-argocd-namespace/1-install | starting test step 1-install I0723 17:44:22.470268 167 request.go:682] Waited for 1.046495942s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/samples.operator.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-051-validate_csv_permissions logger.go:42: 17:44:23 | 1-051-validate_csv_permissions | Creating namespace: kuttl-test-great-coral logger.go:42: 17:44:23 | 1-051-validate_csv_permissions/1-validate | starting test step 1-validate logger.go:42: 17:44:23 | 1-051-validate_csv_permissions/1-validate | running command: [sh -c set -e res=$(oc auth can-i delete resourcequotas -n openshift-gitops --as system:serviceaccount:openshift-operators:openshift-gitops-operator-controller-manager) if test "$res" != "yes"; then echo "Can't delete resourcequotas" exit 1 fi ] === CONT kuttl/harness/1-008_validate-custom-argocd-namespace logger.go:42: 17:44:23 | 1-008_validate-custom-argocd-namespace/1-install | Namespace:/test-1-8-custom created === CONT kuttl/harness/1-054_validate_deploymentconfig logger.go:42: 17:44:23 | 1-054_validate_deploymentconfig/3- | test step completed 3- logger.go:42: 17:44:23 | 1-054_validate_deploymentconfig/4-scaledown-deploymentconfig | starting test step 4-scaledown-deploymentconfig === CONT kuttl/harness/1-008_validate-custom-argocd-namespace logger.go:42: 17:44:23 | 1-008_validate-custom-argocd-namespace/1-install | ArgoCD:test-1-8-custom/argocd created === CONT kuttl/harness/1-054_validate_deploymentconfig logger.go:42: 17:44:26 | 1-054_validate_deploymentconfig/4-scaledown-deploymentconfig | Application:kuttl-test-full-bird/app-deploymentconfig updated === CONT kuttl/harness/1-051-validate_csv_permissions logger.go:42: 17:44:28 | 1-051-validate_csv_permissions/1-validate | test step completed 1-validate logger.go:42: 17:44:28 | 1-051-validate_csv_permissions | skipping kubernetes event logging logger.go:42: 17:44:28 | 1-051-validate_csv_permissions | Deleting namespace: kuttl-test-great-coral === CONT kuttl/harness/1-054_validate_deploymentconfig logger.go:42: 17:44:28 | 1-054_validate_deploymentconfig/4-scaledown-deploymentconfig | test step completed 4-scaledown-deploymentconfig logger.go:42: 17:44:28 | 1-054_validate_deploymentconfig | skipping kubernetes event logging logger.go:42: 17:44:28 | 1-054_validate_deploymentconfig | Deleting namespace: kuttl-test-full-bird === CONT kuttl/harness/1-009_validate-manage-other-namespace logger.go:42: 17:44:33 | 1-009_validate-manage-other-namespace/1-install | test step completed 1-install logger.go:42: 17:44:33 | 1-009_validate-manage-other-namespace/2-label-namespace | starting test step 2-label-namespace logger.go:42: 17:44:33 | 1-009_validate-manage-other-namespace/2-label-namespace | running command: [sh -c kubectl label ns test-1-9-custom argocd.argoproj.io/managed-by=$NAMESPACE] logger.go:42: 17:44:34 | 1-009_validate-manage-other-namespace/2-label-namespace | namespace/test-1-9-custom labeled === CONT kuttl/harness/1-044_validate_resource_limit_changes I0723 17:44:35.458158 167 request.go:682] Waited for 1.046782738s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/machine.openshift.io/v1beta1?timeout=32s logger.go:42: 17:44:36 | 1-044_validate_resource_limit_changes | Creating namespace: kuttl-test-notable-bulldog logger.go:42: 17:44:36 | 1-044_validate_resource_limit_changes/1-install | starting test step 1-install === CONT kuttl/harness/1-009_validate-manage-other-namespace logger.go:42: 17:44:36 | 1-009_validate-manage-other-namespace/2-label-namespace | test step completed 2-label-namespace logger.go:42: 17:44:36 | 1-009_validate-manage-other-namespace/3-check-secret | starting test step 3-check-secret logger.go:42: 17:44:39 | 1-009_validate-manage-other-namespace/3-check-secret | running command: [sh -c namespaces=$(oc get secret -n $NAMESPACE argocd-default-cluster-config -o jsonpath='{.data.namespaces}' | base64 -d) if test "$namespaces" != "$NAMESPACE,test-1-9-custom"; then echo "Assertion for cluster secret failed!" exit 1 fi exit 0 ] === CONT kuttl/harness/1-044_validate_resource_limit_changes logger.go:42: 17:44:39 | 1-044_validate_resource_limit_changes/1-install | ArgoCD:kuttl-test-notable-bulldog/argocd created === CONT kuttl/harness/1-009_validate-manage-other-namespace logger.go:42: 17:44:42 | 1-009_validate-manage-other-namespace/3-check-secret | test step completed 3-check-secret logger.go:42: 17:44:42 | 1-009_validate-manage-other-namespace/4-create-application | starting test step 4-create-application logger.go:42: 17:44:44 | 1-009_validate-manage-other-namespace/4-create-application | Application:kuttl-test-handy-raccoon/test-1-9-custom created logger.go:42: 17:44:52 | 1-009_validate-manage-other-namespace/4-create-application | test step completed 4-create-application logger.go:42: 17:44:52 | 1-009_validate-manage-other-namespace/5-unlabel-namespace | starting test step 5-unlabel-namespace logger.go:42: 17:44:52 | 1-009_validate-manage-other-namespace/5-unlabel-namespace | running command: [sh -c kubectl label ns test-1-9-custom argocd.argoproj.io/managed-by-] logger.go:42: 17:44:53 | 1-009_validate-manage-other-namespace/5-unlabel-namespace | namespace/test-1-9-custom unlabeled logger.go:42: 17:44:53 | 1-009_validate-manage-other-namespace/5-unlabel-namespace | running command: [sh -c sleep 5] === CONT kuttl/harness/1-008_validate-custom-argocd-namespace logger.go:42: 17:44:55 | 1-008_validate-custom-argocd-namespace/1-install | test step completed 1-install logger.go:42: 17:44:55 | 1-008_validate-custom-argocd-namespace/2-wait | starting test step 2-wait logger.go:42: 17:44:55 | 1-008_validate-custom-argocd-namespace/2-wait | running command: [sh -c function wait_until_pods_running() { echo -n "Waiting until all pods in namespace $1 are up" for i in {1..150}; do # timeout after 5 minutes local pods="$(oc get pods --no-headers -n $1 2>/dev/null)" # All pods must be running local not_running=$(echo "${pods}" | grep -v Running | grep -v Completed | wc -l) if [[ -n "${pods}" && ${not_running} -eq 0 ]]; then local all_ready=1 while read pod ; do local status=(`echo -n ${pod} | cut -f2 -d' ' | tr '/' ' '`) # All containers must be ready [[ -z ${status[0]} ]] && all_ready=0 && break [[ -z ${status[1]} ]] && all_ready=0 && break [[ ${status[0]} -lt 1 ]] && all_ready=0 && break [[ ${status[1]} -lt 1 ]] && all_ready=0 && break [[ ${status[0]} -ne ${status[1]} ]] && all_ready=0 && break done <<< $(echo "${pods}" | grep -v Completed) if (( all_ready )); then echo -e "\nAll pods are up:\n${pods}" return 0 fi fi echo -n "." sleep 2 done echo -e "\n\nERROR: timeout waiting for pods to come up\n${pods}" return 1 } wait_until_pods_running "test-1-8-custom" ] logger.go:42: 17:44:56 | 1-008_validate-custom-argocd-namespace/2-wait | Waiting until all pods in namespace test-1-8-custom are up logger.go:42: 17:44:56 | 1-008_validate-custom-argocd-namespace/2-wait | All pods are up: logger.go:42: 17:44:56 | 1-008_validate-custom-argocd-namespace/2-wait | argocd-application-controller-0 1/1 Running 0 32s logger.go:42: 17:44:56 | 1-008_validate-custom-argocd-namespace/2-wait | argocd-redis-5d6dd5f55-n5x79 1/1 Running 0 32s logger.go:42: 17:44:56 | 1-008_validate-custom-argocd-namespace/2-wait | argocd-repo-server-86f8486f4d-qszsh 1/1 Running 0 32s logger.go:42: 17:44:56 | 1-008_validate-custom-argocd-namespace/2-wait | argocd-server-577c75669c-4mzw6 1/1 Running 0 32s I0723 17:44:57.474167 167 request.go:682] Waited for 1.043158329s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/cloudcredential.openshift.io/v1?timeout=32s logger.go:42: 17:44:58 | 1-008_validate-custom-argocd-namespace/2-wait | test step completed 2-wait logger.go:42: 17:44:58 | 1-008_validate-custom-argocd-namespace/3-create-app | starting test step 3-create-app === CONT kuttl/harness/1-009_validate-manage-other-namespace logger.go:42: 17:45:03 | 1-009_validate-manage-other-namespace/5-unlabel-namespace | test step completed 5-unlabel-namespace logger.go:42: 17:45:03 | 1-009_validate-manage-other-namespace/6-check-secret | starting test step 6-check-secret logger.go:42: 17:45:03 | 1-009_validate-manage-other-namespace/6-check-secret | running command: [sh -c namespaces=$(oc get secret -n $NAMESPACE argocd-default-cluster-config -o jsonpath='{.data.namespaces}' | base64 -d) if test "$namespaces" != "$NAMESPACE"; then echo "Assertion for cluster secret failed!" exit 1 fi exit 0 ] === CONT kuttl/harness/1-008_validate-custom-argocd-namespace logger.go:42: 17:45:03 | 1-008_validate-custom-argocd-namespace/3-create-app | Application:test-1-8-custom/validate-custom-argocd created === CONT kuttl/harness/1-009_validate-manage-other-namespace logger.go:42: 17:45:07 | 1-009_validate-manage-other-namespace/6-check-secret | test step completed 6-check-secret logger.go:42: 17:45:07 | 1-009_validate-manage-other-namespace/7-check | starting test step 7-check I0723 17:45:08.132537 167 request.go:682] Waited for 1.047244203s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/controlplane.operator.openshift.io/v1alpha1?timeout=32s logger.go:42: 17:45:09 | 1-009_validate-manage-other-namespace/7-check | test step completed 7-check logger.go:42: 17:45:09 | 1-009_validate-manage-other-namespace/99-delete | starting test step 99-delete === CONT kuttl/harness/1-044_validate_resource_limit_changes logger.go:42: 17:45:11 | 1-044_validate_resource_limit_changes/1-install | test step completed 1-install logger.go:42: 17:45:11 | 1-044_validate_resource_limit_changes/2-change-limits | starting test step 2-change-limits logger.go:42: 17:45:14 | 1-044_validate_resource_limit_changes/2-change-limits | ArgoCD:kuttl-test-notable-bulldog/argocd updated logger.go:42: 17:45:14 | 1-044_validate_resource_limit_changes/2-change-limits | test step completed 2-change-limits logger.go:42: 17:45:14 | 1-044_validate_resource_limit_changes/4-check-workloads | starting test step 4-check-workloads logger.go:42: 17:45:14 | 1-044_validate_resource_limit_changes/4-check-workloads | running command: [sh -c sleep 10] === CONT kuttl/harness/1-032_validate_dynamic_scaling logger.go:42: 17:45:14 | 1-032_validate_dynamic_scaling | Creating namespace: kuttl-test-cheerful-albacore logger.go:42: 17:45:14 | 1-032_validate_dynamic_scaling/1-install | starting test step 1-install logger.go:42: 17:45:14 | 1-032_validate_dynamic_scaling/1-install | running command: [sleep 30s] === CONT kuttl/harness/1-008_validate-custom-argocd-namespace logger.go:42: 17:45:15 | 1-008_validate-custom-argocd-namespace/3-create-app | test step completed 3-create-app logger.go:42: 17:45:15 | 1-008_validate-custom-argocd-namespace/99-delete | starting test step 99-delete I0723 17:45:22.459334 167 request.go:682] Waited for 1.045920476s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/storage.k8s.io/v1?timeout=32s === CONT kuttl/harness/1-009_validate-manage-other-namespace logger.go:42: 17:45:23 | 1-009_validate-manage-other-namespace/99-delete | test step completed 99-delete logger.go:42: 17:45:23 | 1-009_validate-manage-other-namespace | skipping kubernetes event logging logger.go:42: 17:45:23 | 1-009_validate-manage-other-namespace | Deleting namespace: kuttl-test-handy-raccoon === CONT kuttl/harness/1-044_validate_resource_limit_changes logger.go:42: 17:45:24 | 1-044_validate_resource_limit_changes/4-check-workloads | running command: [sh -c set -e set -o pipefail for wl in deployments/argocd-server deployments/argocd-repo-server statefulsets/argocd-application-controller; do res=$(oc get -n $NAMESPACE $wl -o jsonpath='{.spec.template.spec.containers[0].resources.limits.cpu}') if test "$res" != "2"; then echo "Reconciliation of resources for $wl failed: Should be 2, is '$res'" exit 1 fi done ] logger.go:42: 17:45:28 | 1-044_validate_resource_limit_changes/4-check-workloads | test step completed 4-check-workloads logger.go:42: 17:45:28 | 1-044_validate_resource_limit_changes | skipping kubernetes event logging logger.go:42: 17:45:28 | 1-044_validate_resource_limit_changes | Deleting namespace: kuttl-test-notable-bulldog === CONT kuttl/harness/1-034_validate_webhook_notifications === CONT kuttl/harness/1-008_validate-custom-argocd-namespace logger.go:42: 17:45:30 | 1-008_validate-custom-argocd-namespace/99-delete | test step completed 99-delete logger.go:42: 17:45:30 | 1-008_validate-custom-argocd-namespace | skipping kubernetes event logging === CONT kuttl/harness/1-034_validate_webhook_notifications logger.go:42: 17:45:30 | 1-034_validate_webhook_notifications | Creating namespace: kuttl-test-stirring-griffon logger.go:42: 17:45:30 | 1-034_validate_webhook_notifications/0-setup-webhook-server | starting test step 0-setup-webhook-server === CONT kuttl/harness/1-008_validate-custom-argocd-namespace logger.go:42: 17:45:30 | 1-008_validate-custom-argocd-namespace | Deleting namespace: kuttl-test-hopeful-redbird I0723 17:45:32.480519 167 request.go:682] Waited for 1.746686817s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/machineconfiguration.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-034_validate_webhook_notifications logger.go:42: 17:45:33 | 1-034_validate_webhook_notifications/0-setup-webhook-server | Service:kuttl-test-stirring-griffon/webhook created logger.go:42: 17:45:33 | 1-034_validate_webhook_notifications/0-setup-webhook-server | Deployment:kuttl-test-stirring-griffon/webhook created logger.go:42: 17:45:33 | 1-034_validate_webhook_notifications/0-setup-webhook-server | ConfigMap:kuttl-test-stirring-griffon/webhook-config created logger.go:42: 17:45:33 | 1-034_validate_webhook_notifications/0-setup-webhook-server | Secret:kuttl-test-stirring-griffon/webhook-tls created === CONT kuttl/harness/1-049_validate_parallelism_limit logger.go:42: 17:45:34 | 1-049_validate_parallelism_limit | Creating namespace: kuttl-test-enormous-rat logger.go:42: 17:45:34 | 1-049_validate_parallelism_limit/1-install | starting test step 1-install === CONT kuttl/harness/1-033_validate_resource_exclusions logger.go:42: 17:45:37 | 1-033_validate_resource_exclusions | Creating namespace: kuttl-test-more-lionfish logger.go:42: 17:45:37 | 1-033_validate_resource_exclusions/1-install | starting test step 1-install === CONT kuttl/harness/1-049_validate_parallelism_limit logger.go:42: 17:45:37 | 1-049_validate_parallelism_limit/1-install | ArgoCD:kuttl-test-enormous-rat/argocd created === CONT kuttl/harness/1-034_validate_webhook_notifications logger.go:42: 17:45:39 | 1-034_validate_webhook_notifications/0-setup-webhook-server | test step completed 0-setup-webhook-server logger.go:42: 17:45:39 | 1-034_validate_webhook_notifications/1-install-argocd | starting test step 1-install-argocd === CONT kuttl/harness/1-033_validate_resource_exclusions logger.go:42: 17:45:39 | 1-033_validate_resource_exclusions/1-install | ArgoCD:kuttl-test-more-lionfish/argocd created === CONT kuttl/harness/1-034_validate_webhook_notifications logger.go:42: 17:45:42 | 1-034_validate_webhook_notifications/1-install-argocd | ArgoCD:kuttl-test-stirring-griffon/argocd created I0723 17:45:45.634425 167 request.go:682] Waited for 1.046516482s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/performance.openshift.io/v2?timeout=32s === CONT kuttl/harness/1-032_validate_dynamic_scaling logger.go:42: 17:45:47 | 1-032_validate_dynamic_scaling/1-install | ArgoCD:kuttl-test-cheerful-albacore/argocd created === CONT kuttl/harness/1-033_validate_resource_exclusions logger.go:42: 17:45:52 | 1-033_validate_resource_exclusions/1-install | test step completed 1-install logger.go:42: 17:45:52 | 1-033_validate_resource_exclusions/2-add_resource_exclusions | starting test step 2-add_resource_exclusions logger.go:42: 17:45:54 | 1-033_validate_resource_exclusions/2-add_resource_exclusions | ArgoCD:kuttl-test-more-lionfish/argocd updated logger.go:42: 17:45:55 | 1-033_validate_resource_exclusions/2-add_resource_exclusions | test step completed 2-add_resource_exclusions logger.go:42: 17:45:55 | 1-033_validate_resource_exclusions | skipping kubernetes event logging logger.go:42: 17:45:55 | 1-033_validate_resource_exclusions | Deleting namespace: kuttl-test-more-lionfish === CONT kuttl/harness/1-048_validate_controller_sharding logger.go:42: 17:46:01 | 1-048_validate_controller_sharding | Creating namespace: kuttl-test-electric-ladybug logger.go:42: 17:46:01 | 1-048_validate_controller_sharding/1-install | starting test step 1-install I0723 17:46:02.809357 167 request.go:682] Waited for 1.037773614s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/monitoring.coreos.com/v1?timeout=32s logger.go:42: 17:46:04 | 1-048_validate_controller_sharding/1-install | ArgoCD:kuttl-test-electric-ladybug/argocd created === CONT kuttl/harness/1-049_validate_parallelism_limit logger.go:42: 17:46:09 | 1-049_validate_parallelism_limit/1-install | test step completed 1-install logger.go:42: 17:46:09 | 1-049_validate_parallelism_limit/2-check-deployment | starting test step 2-check-deployment logger.go:42: 17:46:09 | 1-049_validate_parallelism_limit/2-check-deployment | running command: [sh -c set -e set -o pipefail expected=10 wlCommand=$(oc get -n $NAMESPACE statefulset/argocd-application-controller -o jsonpath='{.spec.template.spec.containers[0].command}'| jq -r '.[]' ) if ! echo "$wlCommand" | grep -qPz -- "--kubectl-parallelism-limit\\n${expected}(\$|\\n)"; then echo "Incorrect or missing --kubectl-parallelism-limit detected." echo "$wlCommand" exit 1 fi ] logger.go:42: 17:46:12 | 1-049_validate_parallelism_limit/2-check-deployment | test step completed 2-check-deployment logger.go:42: 17:46:12 | 1-049_validate_parallelism_limit/3-change-limit | starting test step 3-change-limit I0723 17:46:13.512151 167 request.go:682] Waited for 1.046421152s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/tuned.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-034_validate_webhook_notifications logger.go:42: 17:46:14 | 1-034_validate_webhook_notifications/1-install-argocd | test step completed 1-install-argocd logger.go:42: 17:46:14 | 1-034_validate_webhook_notifications/2-add-notification-confs | starting test step 2-add-notification-confs === CONT kuttl/harness/1-049_validate_parallelism_limit logger.go:42: 17:46:14 | 1-049_validate_parallelism_limit/3-change-limit | ArgoCD:kuttl-test-enormous-rat/argocd updated logger.go:42: 17:46:14 | 1-049_validate_parallelism_limit/3-change-limit | test step completed 3-change-limit logger.go:42: 17:46:14 | 1-049_validate_parallelism_limit/4-check-deployment | starting test step 4-check-deployment logger.go:42: 17:46:17 | 1-049_validate_parallelism_limit/4-check-deployment | running command: [sh -c sleep 5] === CONT kuttl/harness/1-048_validate_controller_sharding logger.go:42: 17:46:17 | 1-048_validate_controller_sharding/1-install | test step completed 1-install logger.go:42: 17:46:17 | 1-048_validate_controller_sharding/2-change-sharding | starting test step 2-change-sharding === CONT kuttl/harness/1-034_validate_webhook_notifications logger.go:42: 17:46:17 | 1-034_validate_webhook_notifications/2-add-notification-confs | NotificationsConfiguration:kuttl-test-stirring-griffon/default-notifications-configuration updated logger.go:42: 17:46:17 | 1-034_validate_webhook_notifications/2-add-notification-confs | test step completed 2-add-notification-confs logger.go:42: 17:46:17 | 1-034_validate_webhook_notifications/3-sleep | starting test step 3-sleep logger.go:42: 17:46:19 | 1-034_validate_webhook_notifications/3-sleep | running command: [sh -c sleep 20] === CONT kuttl/harness/1-032_validate_dynamic_scaling logger.go:42: 17:46:19 | 1-032_validate_dynamic_scaling/1-install | test step completed 1-install logger.go:42: 17:46:19 | 1-032_validate_dynamic_scaling/2-add-cluster-secrets | starting test step 2-add-cluster-secrets logger.go:42: 17:46:19 | 1-032_validate_dynamic_scaling/2-add-cluster-secrets | running command: [sleep 30s] === CONT kuttl/harness/1-048_validate_controller_sharding logger.go:42: 17:46:19 | 1-048_validate_controller_sharding/2-change-sharding | ArgoCD:kuttl-test-electric-ladybug/argocd updated === CONT kuttl/harness/1-049_validate_parallelism_limit logger.go:42: 17:46:22 | 1-049_validate_parallelism_limit/4-check-deployment | running command: [sh -c set -e set -o pipefail expected=20 wlCommand=$(oc get -n $NAMESPACE statefulset/argocd-application-controller -o jsonpath='{.spec.template.spec.containers[0].command}'| jq -r '.[]' ) if ! echo "$wlCommand" | grep -qPz -- "--kubectl-parallelism-limit\\n${expected}(\$|\\n)"; then echo "Incorrect or missing --kubectl-parallelism-limit detected." echo "$wlCommand" exit 1 fi ] I0723 17:46:24.358854 167 request.go:682] Waited for 1.047159859s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/autoscaling.openshift.io/v1?timeout=32s logger.go:42: 17:46:25 | 1-049_validate_parallelism_limit/4-check-deployment | test step completed 4-check-deployment logger.go:42: 17:46:25 | 1-049_validate_parallelism_limit/5-change-back-to-default | starting test step 5-change-back-to-default logger.go:42: 17:46:28 | 1-049_validate_parallelism_limit/5-change-back-to-default | ArgoCD:kuttl-test-enormous-rat/argocd updated logger.go:42: 17:46:28 | 1-049_validate_parallelism_limit/5-change-back-to-default | test step completed 5-change-back-to-default logger.go:42: 17:46:28 | 1-049_validate_parallelism_limit/6-check-deployment | starting test step 6-check-deployment logger.go:42: 17:46:28 | 1-049_validate_parallelism_limit/6-check-deployment | running command: [sh -c set -e oc patch -n $NAMESPACE argocds/argocd --type=json --patch '[{"op": "remove", "path": "/spec/controller/parallelismLimit"}]' ] logger.go:42: 17:46:28 | 1-049_validate_parallelism_limit/6-check-deployment | argocd.argoproj.io/argocd patched logger.go:42: 17:46:28 | 1-049_validate_parallelism_limit/6-check-deployment | running command: [sh -c sleep 5] logger.go:42: 17:46:33 | 1-049_validate_parallelism_limit/6-check-deployment | running command: [sh -c set -e set -o pipefail expected=10 wlCommand=$(oc get -n $NAMESPACE statefulset/argocd-application-controller -o jsonpath='{.spec.template.spec.containers[0].command}'| jq -r '.[]' ) if ! echo "$wlCommand" | grep -qPz -- "--kubectl-parallelism-limit\\n${expected}(\$|\\n)"; then echo "Incorrect or missing --kubectl-parallelism-limit detected." echo "$wlCommand" exit 1 fi ] I0723 17:46:35.359049 167 request.go:682] Waited for 1.046615654s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/argoproj.io/v1beta1?timeout=32s logger.go:42: 17:46:36 | 1-049_validate_parallelism_limit/6-check-deployment | test step completed 6-check-deployment logger.go:42: 17:46:36 | 1-049_validate_parallelism_limit | skipping kubernetes event logging logger.go:42: 17:46:36 | 1-049_validate_parallelism_limit | Deleting namespace: kuttl-test-enormous-rat === CONT kuttl/harness/1-034_validate_webhook_notifications logger.go:42: 17:46:42 | 1-034_validate_webhook_notifications/3-sleep | test step completed 3-sleep logger.go:42: 17:46:42 | 1-034_validate_webhook_notifications/4-create-app | starting test step 4-create-app logger.go:42: 17:46:42 | 1-034_validate_webhook_notifications/4-create-app | running command: [sh -c set -e cat << EOF | kubectl apply -f - apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: my-app-3 namespace: $NAMESPACE annotations: "notifications.argoproj.io/subscribe.test-on-created.test-webhook": "" spec: destination: namespace: $NAMESPACE server: https://kubernetes.default.svc project: default source: repoURL: https://github.com/redhat-developer/gitops-operator path: test/examples/nginx targetRevision: HEAD EOF ] === CONT kuttl/harness/1-033_validate_applicationset_tls_scm_volume_mount logger.go:42: 17:46:43 | 1-033_validate_applicationset_tls_scm_volume_mount | Creating namespace: kuttl-test-aware-dog logger.go:42: 17:46:43 | 1-033_validate_applicationset_tls_scm_volume_mount/1-install | starting test step 1-install === CONT kuttl/harness/1-034_validate_webhook_notifications logger.go:42: 17:46:44 | 1-034_validate_webhook_notifications/4-create-app | application.argoproj.io/my-app-3 created logger.go:42: 17:46:45 | 1-034_validate_webhook_notifications/4-create-app | running command: [sh -c sleep 10] I0723 17:46:45.375163 167 request.go:682] Waited for 2.345086768s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/whereabouts.cni.cncf.io/v1alpha1?timeout=32s === CONT kuttl/harness/1-033_validate_applicationset_tls_scm_volume_mount logger.go:42: 17:46:45 | 1-033_validate_applicationset_tls_scm_volume_mount/1-install | Namespace:/test-1-32-appsets-scm-tls-mount created logger.go:42: 17:46:45 | 1-033_validate_applicationset_tls_scm_volume_mount/1-install | ConfigMap:test-1-32-appsets-scm-tls-mount/test-1-32-appsets-scm-tls-cm created logger.go:42: 17:46:45 | 1-033_validate_applicationset_tls_scm_volume_mount/1-install | ArgoCD:test-1-32-appsets-scm-tls-mount/argocd created === CONT kuttl/harness/1-032_validate_dynamic_scaling logger.go:42: 17:46:52 | 1-032_validate_dynamic_scaling/2-add-cluster-secrets | Secret:kuttl-test-cheerful-albacore/cluster-1 created === CONT kuttl/harness/1-048_validate_controller_sharding logger.go:42: 17:46:52 | 1-048_validate_controller_sharding/2-change-sharding | test step completed 2-change-sharding logger.go:42: 17:46:52 | 1-048_validate_controller_sharding/3-check-env | starting test step 3-check-env logger.go:42: 17:46:52 | 1-048_validate_controller_sharding/3-check-env | running command: [sh -c replicas=$(oc get -n $NAMESPACE statefulset argocd-application-controller -o json \ | jq -r '.spec.template.spec.containers[0].env[]|select(.name=="ARGOCD_CONTROLLER_REPLICAS").value') if test "$replicas" != "3"; then echo "Environment ARGOCD_CONTROLLER_REPLICAS not correct. Should '3', is '$replicas'" exit 1 fi ] === CONT kuttl/harness/1-032_validate_dynamic_scaling logger.go:42: 17:46:52 | 1-032_validate_dynamic_scaling/2-add-cluster-secrets | Secret:kuttl-test-cheerful-albacore/cluster-2 created === CONT kuttl/harness/1-071_validate_SCC_HA logger.go:42: 17:46:52 | 1-071_validate_SCC_HA/3-modify-argocd | test step completed 3-modify-argocd logger.go:42: 17:46:52 | 1-071_validate_SCC_HA/99-delete | starting test step 99-delete I0723 17:46:55.401757 167 request.go:682] Waited for 2.446166478s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/apiserver.openshift.io/v1?timeout=32s logger.go:42: 17:46:55 | 1-071_validate_SCC_HA/99-delete | test step completed 99-delete logger.go:42: 17:46:55 | 1-071_validate_SCC_HA | skipping kubernetes event logging logger.go:42: 17:46:55 | 1-071_validate_SCC_HA | Deleting namespace: kuttl-test-main-cat === CONT kuttl/harness/1-048_validate_controller_sharding logger.go:42: 17:46:57 | 1-048_validate_controller_sharding/3-check-env | test step completed 3-check-env logger.go:42: 17:46:57 | 1-048_validate_controller_sharding/4-change-sharding | starting test step 4-change-sharding === CONT kuttl/harness/1-034_validate_webhook_notifications logger.go:42: 17:47:02 | 1-034_validate_webhook_notifications/4-create-app | test step completed 4-create-app logger.go:42: 17:47:02 | 1-034_validate_webhook_notifications/5-verify-webhook-notification | starting test step 5-verify-webhook-notification logger.go:42: 17:47:02 | 1-034_validate_webhook_notifications/5-verify-webhook-notification | running command: [sh -c #!/bin/bash kubectl -n $NAMESPACE logs deployment.apps/example-notifications-controller | grep "x509" PREV_CMD=$? if [ $PREV_CMD -eq 1 ]; then exit 0 else exit 1 fi ] === CONT kuttl/harness/1-048_validate_controller_sharding logger.go:42: 17:47:02 | 1-048_validate_controller_sharding/4-change-sharding | ArgoCD:kuttl-test-electric-ladybug/argocd updated === CONT kuttl/harness/1-034_validate_webhook_notifications logger.go:42: 17:47:03 | 1-034_validate_webhook_notifications/5-verify-webhook-notification | Error from server (NotFound): deployments.apps "example-notifications-controller" not found logger.go:42: 17:47:03 | 1-034_validate_webhook_notifications/5-verify-webhook-notification | running command: [sh -c #!/bin/bash # delays could occur in receiving notification, hence retry before failing for i in {1..5} do sleep 5 kubectl -n $NAMESPACE logs deployment.apps/webhook | grep '{"created":"my-app-3","type":"Directory"}' && break done ] === CONT kuttl/harness/1-048_validate_controller_sharding logger.go:42: 17:47:05 | 1-048_validate_controller_sharding/4-change-sharding | test step completed 4-change-sharding logger.go:42: 17:47:05 | 1-048_validate_controller_sharding/5-check-env | starting test step 5-check-env logger.go:42: 17:47:05 | 1-048_validate_controller_sharding/5-check-env | running command: [sh -c # we do expect error from jq here if env is not set at all replicas=$(oc get -n $NAMESPACE statefulset argocd-application-controller -o json \ | jq -r '.spec.template.spec.containers[0].env[]|select(.name=="ARGOCD_CONTROLLER_REPLICASE").value') if test "$replicas" != ""; then echo "Environment ARGOCD_CONTROLLER_REPLICAS not correct. Should '', is '$replicas'" exit 1 fi ] I0723 17:47:07.458878 167 request.go:682] Waited for 1.045459553s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/apps.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-034_validate_webhook_notifications logger.go:42: 17:47:08 | 1-034_validate_webhook_notifications/5-verify-webhook-notification | [webhook] 2024/07/23 17:46:47 [776fdd] executing /usr/bin/date (/usr/bin/date) with arguments ["/usr/bin/date"] and environment [PAYLOAD={"created":"my-app-3","type":"Directory"}] using /var/webhook as cwd === CONT kuttl/harness/1-048_validate_controller_sharding logger.go:42: 17:47:08 | 1-048_validate_controller_sharding/5-check-env | test step completed 5-check-env logger.go:42: 17:47:08 | 1-048_validate_controller_sharding | skipping kubernetes event logging logger.go:42: 17:47:08 | 1-048_validate_controller_sharding | Deleting namespace: kuttl-test-electric-ladybug === CONT kuttl/harness/1-034_validate_webhook_notifications logger.go:42: 17:47:11 | 1-034_validate_webhook_notifications/5-verify-webhook-notification | test step completed 5-verify-webhook-notification logger.go:42: 17:47:11 | 1-034_validate_webhook_notifications | skipping kubernetes event logging logger.go:42: 17:47:11 | 1-034_validate_webhook_notifications | Deleting namespace: kuttl-test-stirring-griffon === CONT kuttl/harness/1-047_validate_custom_env logger.go:42: 17:47:15 | 1-047_validate_custom_env | Creating namespace: kuttl-test-grown-camel logger.go:42: 17:47:15 | 1-047_validate_custom_env/1-install | starting test step 1-install I0723 17:47:17.467292 167 request.go:682] Waited for 2.24671709s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/oauth.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-033_validate_applicationset_tls_scm_volume_mount logger.go:42: 17:47:17 | 1-033_validate_applicationset_tls_scm_volume_mount/1-install | test step completed 1-install logger.go:42: 17:47:17 | 1-033_validate_applicationset_tls_scm_volume_mount | skipping kubernetes event logging === CONT kuttl/harness/1-047_validate_custom_env logger.go:42: 17:47:17 | 1-047_validate_custom_env/1-install | ArgoCD:kuttl-test-grown-camel/argocd created === CONT kuttl/harness/1-033_validate_applicationset_tls_scm_volume_mount logger.go:42: 17:47:17 | 1-033_validate_applicationset_tls_scm_volume_mount | Deleting namespace: kuttl-test-aware-dog === CONT kuttl/harness/1-032_validate_resource_inclusions logger.go:42: 17:47:17 | 1-032_validate_resource_inclusions | Creating namespace: kuttl-test-wired-dove logger.go:42: 17:47:17 | 1-032_validate_resource_inclusions/1-install | starting test step 1-install logger.go:42: 17:47:20 | 1-032_validate_resource_inclusions/1-install | ArgoCD:kuttl-test-wired-dove/argocd created === CONT kuttl/harness/1-045_validate_repo_exec_timeout logger.go:42: 17:47:23 | 1-045_validate_repo_exec_timeout | Creating namespace: kuttl-test-dashing-guppy logger.go:42: 17:47:23 | 1-045_validate_repo_exec_timeout/1-install | starting test step 1-install logger.go:42: 17:47:26 | 1-045_validate_repo_exec_timeout/1-install | ArgoCD:kuttl-test-dashing-guppy/argocd created === CONT kuttl/harness/1-029_validate_tls_secret_no_scale logger.go:42: 17:47:41 | 1-029_validate_tls_secret_no_scale | Creating namespace: kuttl-test-probable-coral logger.go:42: 17:47:41 | 1-029_validate_tls_secret_no_scale/1-install | starting test step 1-install I0723 17:47:42.211276 167 request.go:682] Waited for 1.04657552s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/machineconfiguration.openshift.io/v1?timeout=32s logger.go:42: 17:47:43 | 1-029_validate_tls_secret_no_scale/1-install | ArgoCD:kuttl-test-probable-coral/argocd created logger.go:42: 17:49:57 | 1-029_validate_tls_secret_no_scale/1-install | test step completed 1-install logger.go:42: 17:49:57 | 1-029_validate_tls_secret_no_scale/2-create-tls-secret | starting test step 2-create-tls-secret logger.go:42: 17:49:57 | 1-029_validate_tls_secret_no_scale/2-create-tls-secret | running command: [sh -c openssl req -x509 -newkey rsa:4096 -keyout /tmp/test-029-key.pem -out /tmp/test-029-cert.pem -subj '/CN=test-029-cert' -days 365 -nodes cert=$(cat /tmp/test-029-cert.pem | base64 -w 0) key=$(cat /tmp/test-029-key.pem | base64 -w 0) # Dirty hack to replace argocd-tls cat <<_EOF_ | oc replace -n $NAMESPACE secret argocd-tls -f - apiVersion: v1 kind: Secret type: kubernetes.io/tls metadata: name: argocd-tls namespace: $NAMESPACE data: tls.key: $key tls.crt: $cert _EOF_ ] logger.go:42: 17:49:57 | 1-029_validate_tls_secret_no_scale/2-create-tls-secret | Generating a RSA private key logger.go:42: 17:49:58 | 1-029_validate_tls_secret_no_scale/2-create-tls-secret | ..................................................................++++ logger.go:42: 17:49:58 | 1-029_validate_tls_secret_no_scale/2-create-tls-secret | ..........................++++ logger.go:42: 17:49:58 | 1-029_validate_tls_secret_no_scale/2-create-tls-secret | writing new private key to '/tmp/test-029-key.pem' logger.go:42: 17:49:58 | 1-029_validate_tls_secret_no_scale/2-create-tls-secret | ----- logger.go:42: 17:49:58 | 1-029_validate_tls_secret_no_scale/2-create-tls-secret | secret/argocd-tls replaced I0723 17:49:59.757844 167 request.go:682] Waited for 1.046359783s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/operators.coreos.com/v1alpha2?timeout=32s logger.go:42: 17:50:01 | 1-029_validate_tls_secret_no_scale/2-create-tls-secret | test step completed 2-create-tls-secret logger.go:42: 17:50:01 | 1-029_validate_tls_secret_no_scale/3-sleep-and-recheck | starting test step 3-sleep-and-recheck logger.go:42: 17:50:01 | 1-029_validate_tls_secret_no_scale/3-sleep-and-recheck | running command: [sh -c sleep 10] I0723 17:50:12.222287 167 request.go:682] Waited for 1.046287213s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/network.openshift.io/v1?timeout=32s logger.go:42: 17:50:13 | 1-029_validate_tls_secret_no_scale/3-sleep-and-recheck | test step completed 3-sleep-and-recheck logger.go:42: 17:50:13 | 1-029_validate_tls_secret_no_scale | skipping kubernetes event logging logger.go:42: 17:50:13 | 1-029_validate_tls_secret_no_scale | Deleting namespace: kuttl-test-probable-coral === CONT kuttl/harness/1-032_validate_dynamic_scaling logger.go:42: 17:50:15 | 1-032_validate_dynamic_scaling/2-add-cluster-secrets | test step completed 2-add-cluster-secrets logger.go:42: 17:50:15 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | starting test step 3-cluster-secrets-exceed-maxShards logger.go:42: 17:50:15 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | running command: [sleep 60s] === CONT kuttl/harness/1-045_validate_repo_exec_timeout logger.go:42: 17:50:16 | 1-045_validate_repo_exec_timeout/1-install | test step completed 1-install logger.go:42: 17:50:16 | 1-045_validate_repo_exec_timeout/2-change-exec-timeout | starting test step 2-change-exec-timeout logger.go:42: 17:50:19 | 1-045_validate_repo_exec_timeout/2-change-exec-timeout | ArgoCD:kuttl-test-dashing-guppy/argocd updated logger.go:42: 17:50:19 | 1-045_validate_repo_exec_timeout/2-change-exec-timeout | test step completed 2-change-exec-timeout logger.go:42: 17:50:19 | 1-045_validate_repo_exec_timeout/4-check-workload-env | starting test step 4-check-workload-env logger.go:42: 17:50:19 | 1-045_validate_repo_exec_timeout/4-check-workload-env | running command: [sh -c sleep 10] === CONT kuttl/harness/1-066_validate_redis_secure_comm_no_autotls_no_ha logger.go:42: 17:50:19 | 1-066_validate_redis_secure_comm_no_autotls_no_ha | Creating namespace: kuttl-test-sound-turtle logger.go:42: 17:50:19 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/1-install | starting test step 1-install I0723 17:50:22.251024 167 request.go:682] Waited for 2.447687211s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/route.openshift.io/v1?timeout=32s logger.go:42: 17:50:22 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/1-install | ArgoCD:kuttl-test-sound-turtle/argocd created === CONT kuttl/harness/1-045_validate_repo_exec_timeout logger.go:42: 17:50:29 | 1-045_validate_repo_exec_timeout/4-check-workload-env | running command: [sh -c timeout=$(oc get -n $NAMESPACE deployment argocd-repo-server -o json \ | jq -r '.spec.template.spec.containers[0].env[]|select(.name=="ARGOCD_EXEC_TIMEOUT").value') if test "$timeout" != "300s"; then echo "Assertion failed. Timeout should be 300, is '$timeout'" exit 1 fi ] logger.go:42: 17:50:32 | 1-045_validate_repo_exec_timeout/4-check-workload-env | test step completed 4-check-workload-env logger.go:42: 17:50:32 | 1-045_validate_repo_exec_timeout | skipping kubernetes event logging === CONT kuttl/harness/1-047_validate_custom_env logger.go:42: 17:50:32 | 1-047_validate_custom_env/1-install | test step completed 1-install logger.go:42: 17:50:32 | 1-047_validate_custom_env/2-change-env-vars | starting test step 2-change-env-vars === CONT kuttl/harness/1-045_validate_repo_exec_timeout logger.go:42: 17:50:32 | 1-045_validate_repo_exec_timeout | Deleting namespace: kuttl-test-dashing-guppy I0723 17:50:33.119272 167 request.go:682] Waited for 1.047188485s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/operator.openshift.io/v1alpha1?timeout=32s === CONT kuttl/harness/1-032_validate_resource_inclusions logger.go:42: 17:50:34 | 1-032_validate_resource_inclusions/1-install | test step completed 1-install logger.go:42: 17:50:34 | 1-032_validate_resource_inclusions/2-add_resource_inclusions | starting test step 2-add_resource_inclusions === CONT kuttl/harness/1-047_validate_custom_env logger.go:42: 17:50:34 | 1-047_validate_custom_env/2-change-env-vars | ArgoCD:kuttl-test-grown-camel/argocd updated logger.go:42: 17:50:34 | 1-047_validate_custom_env/2-change-env-vars | test step completed 2-change-env-vars logger.go:42: 17:50:34 | 1-047_validate_custom_env/3-check-workloads | starting test step 3-check-workloads logger.go:42: 17:50:36 | 1-047_validate_custom_env/3-check-workloads | running command: [sh -c sleep 10] === CONT kuttl/harness/1-032_validate_resource_inclusions logger.go:42: 17:50:37 | 1-032_validate_resource_inclusions/2-add_resource_inclusions | ArgoCD:kuttl-test-wired-dove/argocd updated logger.go:42: 17:50:38 | 1-032_validate_resource_inclusions/2-add_resource_inclusions | test step completed 2-add_resource_inclusions logger.go:42: 17:50:38 | 1-032_validate_resource_inclusions | skipping kubernetes event logging logger.go:42: 17:50:38 | 1-032_validate_resource_inclusions | Deleting namespace: kuttl-test-wired-dove === CONT kuttl/harness/1-030_validate_reencrypt logger.go:42: 17:50:38 | 1-030_validate_reencrypt | Creating namespace: kuttl-test-relative-fox logger.go:42: 17:50:38 | 1-030_validate_reencrypt/1-install | starting test step 1-install logger.go:42: 17:50:40 | 1-030_validate_reencrypt/1-install | Namespace:/test-1-30-argo1 created logger.go:42: 17:50:40 | 1-030_validate_reencrypt/1-install | ArgoCD:test-1-30-argo1/argocd created === CONT kuttl/harness/1-070_validate_config_management_plugin logger.go:42: 17:50:44 | 1-070_validate_config_management_plugin | Creating namespace: kuttl-test-guided-kingfish logger.go:42: 17:50:44 | 1-070_validate_config_management_plugin/1-install | starting test step 1-install I0723 17:50:45.529851 167 request.go:682] Waited for 1.04730477s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/policy/v1?timeout=32s logger.go:42: 17:50:46 | 1-070_validate_config_management_plugin/1-install | Namespace:/argocd created logger.go:42: 17:50:46 | 1-070_validate_config_management_plugin/1-install | ConfigMap:argocd/cmp-plugin created === CONT kuttl/harness/1-047_validate_custom_env logger.go:42: 17:50:46 | 1-047_validate_custom_env/3-check-workloads | running command: [sh -c set -e set -o pipefail for wl in deployment/argocd-server deployment/argocd-repo-server statefulset/argocd-application-controller; do val=$(oc get -n $NAMESPACE $wl -o json | jq -r '.spec.template.spec.containers[0].env[]|select(.name=="FOO").value') if test "$val" != "bar"; then echo "Environment for $wl was not set correctly." exit 1 fi done ] === CONT kuttl/harness/1-070_validate_config_management_plugin logger.go:42: 17:50:46 | 1-070_validate_config_management_plugin/1-install | ArgoCD:argocd/argocd created === CONT kuttl/harness/1-047_validate_custom_env logger.go:42: 17:50:50 | 1-047_validate_custom_env/3-check-workloads | test step completed 3-check-workloads logger.go:42: 17:50:50 | 1-047_validate_custom_env | skipping kubernetes event logging logger.go:42: 17:50:50 | 1-047_validate_custom_env | Deleting namespace: kuttl-test-grown-camel === CONT kuttl/harness/1-066_validate_redis_secure_comm_no_autotls_no_ha logger.go:42: 17:50:55 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/1-install | test step completed 1-install logger.go:42: 17:50:55 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/2-generate_cert | starting test step 2-generate_cert logger.go:42: 17:50:55 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/2-generate_cert | running command: [sh -c set -e echo -e "\n[SAN]\nsubjectAltName=DNS:argocd-redis.$NAMESPACE.svc.cluster.local\n[req]\ndistinguished_name=req" > ${PWD}/openssl_test.cnf openssl req -new -x509 -sha256 \ -subj "/C=XX/ST=XX/O=Testing/CN=redis" \ -reqexts SAN -extensions SAN \ -config ${PWD}/openssl_test.cnf \ -keyout ${PWD}/redis.key \ -out ${PWD}/redis.crt \ -newkey rsa:4096 \ -nodes \ -sha256 \ -days 10 ] logger.go:42: 17:50:55 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/2-generate_cert | Generating a RSA private key logger.go:42: 17:50:55 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/2-generate_cert | .......................................................................................................++++ logger.go:42: 17:50:55 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/2-generate_cert | .....++++ logger.go:42: 17:50:55 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/2-generate_cert | writing new private key to '/kuttl-e2e/gitops-operator/tests/parallel/1-066_validate_redis_secure_comm_no_autotls_no_ha/redis.key' logger.go:42: 17:50:55 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/2-generate_cert | ----- I0723 17:50:56.807074 167 request.go:682] Waited for 1.000774878s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/autoscaling/v2?timeout=32s === CONT kuttl/harness/1-064_validate_security_contexts === CONT kuttl/harness/1-066_validate_redis_secure_comm_no_autotls_no_ha logger.go:42: 17:50:58 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/2-generate_cert | test step completed 2-generate_cert logger.go:42: 17:50:58 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/3-create_secret | starting test step 3-create_secret === CONT kuttl/harness/1-064_validate_security_contexts logger.go:42: 17:50:58 | 1-064_validate_security_contexts | Creating namespace: kuttl-test-needed-pika === CONT kuttl/harness/1-066_validate_redis_secure_comm_no_autotls_no_ha logger.go:42: 17:50:58 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/3-create_secret | running command: [sh -c set -e # clean up key, cert and config file cleanup() { rm -rf ${PWD}/redis.crt && rm -rf ${PWD}/redis.key && rm -rf ${PWD}/openssl_test.cnf } trap cleanup INT TERM EXIT oc create secret tls argocd-operator-redis-tls --key=${PWD}/redis.key --cert=${PWD}/redis.crt -n $NAMESPACE sleep 10 ] === CONT kuttl/harness/1-064_validate_security_contexts logger.go:42: 17:50:58 | 1-064_validate_security_contexts/1-install | starting test step 1-install === CONT kuttl/harness/1-066_validate_redis_secure_comm_no_autotls_no_ha logger.go:42: 17:50:58 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/3-create_secret | secret/argocd-operator-redis-tls created === CONT kuttl/harness/1-070_validate_config_management_plugin logger.go:42: 17:51:00 | 1-070_validate_config_management_plugin/1-install | test step completed 1-install logger.go:42: 17:51:00 | 1-070_validate_config_management_plugin/2-create-app | starting test step 2-create-app === CONT kuttl/harness/1-064_validate_security_contexts logger.go:42: 17:51:00 | 1-064_validate_security_contexts/1-install | ArgoCD:kuttl-test-needed-pika/argocd created === CONT kuttl/harness/1-070_validate_config_management_plugin logger.go:42: 17:51:03 | 1-070_validate_config_management_plugin/2-create-app | Application:argocd/guestbook created === CONT kuttl/harness/1-064_validate_security_contexts logger.go:42: 17:51:03 | 1-064_validate_security_contexts/1-install | test step completed 1-install logger.go:42: 17:51:03 | 1-064_validate_security_contexts/2-check-controller-manager | starting test step 2-check-controller-manager logger.go:42: 17:51:03 | 1-064_validate_security_contexts/2-check-controller-manager | running command: [sh -c set -eo pipefail ocp_major_version=$(oc version -o json | jq -r '.openshiftVersion' | cut -d'.' -f 1) ocp_minor_version=$(oc version -o json | jq -r '.openshiftVersion' | cut -d'.' -f 2) security_context=$(oc get deployment/openshift-gitops-operator-controller-manager -n openshift-operators -o jsonpath='{.spec.template.spec.containers[].securityContext}') expected_security_context='{"allowPrivilegeEscalation":false,"capabilities":{"drop":["ALL"]},"readOnlyRootFilesystem":true,"runAsNonRoot":true}' if [ "${ocp_major_version}" -eq 4 ] && [ "${ocp_minor_version}" -ge 12 ]; then if [ "${security_context}" = "${expected_security_context}" ]; then exit 0 else echo "Error: deployment/openshift-gitops-operator-controller-manager container's security context is wrong" exit 1 fi else echo "Info: This test is only applicable to OCP version 4.12 and higher." exit 0 fi ] I0723 17:51:06.854422 167 request.go:682] Waited for 2.346688991s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/helm.openshift.io/v1beta1?timeout=32s logger.go:42: 17:51:06 | 1-064_validate_security_contexts/2-check-controller-manager | test step completed 2-check-controller-manager logger.go:42: 17:51:06 | 1-064_validate_security_contexts | skipping kubernetes event logging === CONT kuttl/harness/1-070_validate_config_management_plugin logger.go:42: 17:51:06 | 1-070_validate_config_management_plugin/2-create-app | test step completed 2-create-app logger.go:42: 17:51:06 | 1-070_validate_config_management_plugin/99-delete | starting test step 99-delete === CONT kuttl/harness/1-064_validate_security_contexts logger.go:42: 17:51:06 | 1-064_validate_security_contexts | Deleting namespace: kuttl-test-needed-pika === CONT kuttl/harness/1-066_validate_redis_secure_comm_no_autotls_no_ha logger.go:42: 17:51:11 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/3-create_secret | test step completed 3-create_secret logger.go:42: 17:51:11 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/4-create_annotation | starting test step 4-create_annotation logger.go:42: 17:51:11 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/4-create_annotation | running command: [sh -c set -e oc annotate secret argocd-operator-redis-tls argocds.argoproj.io/name=argocd -n $NAMESPACE sleep 30 ] logger.go:42: 17:51:11 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/4-create_annotation | secret/argocd-operator-redis-tls annotated === CONT kuttl/harness/1-030_validate_reencrypt logger.go:42: 17:51:13 | 1-030_validate_reencrypt/1-install | test step completed 1-install logger.go:42: 17:51:13 | 1-030_validate_reencrypt/2-wait | starting test step 2-wait logger.go:42: 17:51:13 | 1-030_validate_reencrypt/2-wait | running command: [sh -c function wait_until_pods_running() { echo -n "Waiting until all pods in namespace $1 are up" for i in {1..150}; do # timeout after 5 minutes local pods="$(oc get pods --no-headers -n $1 2>/dev/null)" # All pods must be running local not_running=$(echo "${pods}" | grep -v Running | grep -v Completed | wc -l) if [[ -n "${pods}" && ${not_running} -eq 0 ]]; then local all_ready=1 while read pod ; do local status=(`echo -n ${pod} | cut -f2 -d' ' | tr '/' ' '`) # All containers must be ready [[ -z ${status[0]} ]] && all_ready=0 && break [[ -z ${status[1]} ]] && all_ready=0 && break [[ ${status[0]} -lt 1 ]] && all_ready=0 && break [[ ${status[1]} -lt 1 ]] && all_ready=0 && break [[ ${status[0]} -ne ${status[1]} ]] && all_ready=0 && break done <<< $(echo "${pods}" | grep -v Completed) if (( all_ready )); then echo -e "\nAll pods are up:\n${pods}" return 0 fi fi echo -n "." sleep 2 done echo -e "\n\nERROR: timeout waiting for pods to come up\n${pods}" return 1 } wait_until_pods_running "test-1-30-argo1" ] === CONT kuttl/harness/1-065_validate_redis_ha_anti_affinity logger.go:42: 17:51:13 | 1-065_validate_redis_ha_anti_affinity | Creating namespace: kuttl-test-boss-goblin logger.go:42: 17:51:13 | 1-065_validate_redis_ha_anti_affinity/1-install | starting test step 1-install === CONT kuttl/harness/1-030_validate_reencrypt logger.go:42: 17:51:13 | 1-030_validate_reencrypt/2-wait | Waiting until all pods in namespace test-1-30-argo1 are up logger.go:42: 17:51:13 | 1-030_validate_reencrypt/2-wait | All pods are up: logger.go:42: 17:51:13 | 1-030_validate_reencrypt/2-wait | argocd-application-controller-0 1/1 Running 0 32s logger.go:42: 17:51:13 | 1-030_validate_reencrypt/2-wait | argocd-redis-5d6dd5f55-5qknw 1/1 Running 0 32s logger.go:42: 17:51:13 | 1-030_validate_reencrypt/2-wait | argocd-repo-server-5698b76558-z6bpq 1/1 Running 0 32s logger.go:42: 17:51:13 | 1-030_validate_reencrypt/2-wait | argocd-server-676b7c87c9-22kp6 1/1 Running 0 32s === CONT kuttl/harness/1-065_validate_redis_ha_anti_affinity logger.go:42: 17:51:16 | 1-065_validate_redis_ha_anti_affinity/1-install | ArgoCD:kuttl-test-boss-goblin/argocd created logger.go:42: 17:51:16 | 1-065_validate_redis_ha_anti_affinity/1-install | test step completed 1-install logger.go:42: 17:51:16 | 1-065_validate_redis_ha_anti_affinity/2- | starting test step 2- I0723 17:51:17.022569 167 request.go:682] Waited for 1.046770454s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/certificates.k8s.io/v1?timeout=32s === CONT kuttl/harness/1-030_validate_reencrypt logger.go:42: 17:51:18 | 1-030_validate_reencrypt/2-wait | test step completed 2-wait logger.go:42: 17:51:18 | 1-030_validate_reencrypt/3-check-route | starting test step 3-check-route logger.go:42: 17:51:18 | 1-030_validate_reencrypt/3-check-route | running command: [sh -c routeURL=$(oc -n test-1-30-argo1 get route argocd-server -o jsonpath='{.status.ingress[0].host}') if ! curl --silent -k https://${routeURL} | grep -q "Your browser does not support JavaScript."; then echo "Route not configured properly" exit 1 fi ] === CONT kuttl/harness/1-032_validate_dynamic_scaling logger.go:42: 17:51:20 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | Secret:kuttl-test-cheerful-albacore/cluster-3 created logger.go:42: 17:51:20 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | Secret:kuttl-test-cheerful-albacore/cluster-4 created logger.go:42: 17:51:20 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | Secret:kuttl-test-cheerful-albacore/cluster-5 created logger.go:42: 17:51:20 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | Secret:kuttl-test-cheerful-albacore/cluster-6 created === CONT kuttl/harness/1-030_validate_reencrypt logger.go:42: 17:51:25 | 1-030_validate_reencrypt/3-check-route | test step completed 3-check-route logger.go:42: 17:51:25 | 1-030_validate_reencrypt | skipping kubernetes event logging === CONT kuttl/harness/1-032_validate_dynamic_scaling logger.go:42: 17:51:25 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | running command: [sh -c stsReplicas=$(kubectl get sts argocd-application-controller -n $NAMESPACE -o jsonpath='{.status.replicas}') echo "$stsReplicas" if test "$stsReplicas" != "4"; then echo "FAILED! Number of replicas not equal to maxShards" exit 1 fi exit 0 ] === CONT kuttl/harness/1-030_validate_reencrypt logger.go:42: 17:51:25 | 1-030_validate_reencrypt | Deleting namespace: kuttl-test-relative-fox === CONT kuttl/harness/1-032_validate_dynamic_scaling logger.go:42: 17:51:26 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | 4 logger.go:42: 17:51:27 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | running command: [sh -c stsReplicas=$(kubectl get sts argocd-application-controller -n $NAMESPACE -o jsonpath='{.status.replicas}') echo "$stsReplicas" if test "$stsReplicas" != "4"; then echo "FAILED! Number of replicas not equal to maxShards" exit 1 fi exit 0 ] logger.go:42: 17:51:27 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | 4 logger.go:42: 17:51:28 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | running command: [sh -c stsReplicas=$(kubectl get sts argocd-application-controller -n $NAMESPACE -o jsonpath='{.status.replicas}') echo "$stsReplicas" if test "$stsReplicas" != "4"; then echo "FAILED! Number of replicas not equal to maxShards" exit 1 fi exit 0 ] logger.go:42: 17:51:29 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | 4 logger.go:42: 17:51:30 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | running command: [sh -c stsReplicas=$(kubectl get sts argocd-application-controller -n $NAMESPACE -o jsonpath='{.status.replicas}') echo "$stsReplicas" if test "$stsReplicas" != "4"; then echo "FAILED! Number of replicas not equal to maxShards" exit 1 fi exit 0 ] logger.go:42: 17:51:30 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | 4 logger.go:42: 17:51:31 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | running command: [sh -c stsReplicas=$(kubectl get sts argocd-application-controller -n $NAMESPACE -o jsonpath='{.status.replicas}') echo "$stsReplicas" if test "$stsReplicas" != "4"; then echo "FAILED! Number of replicas not equal to maxShards" exit 1 fi exit 0 ] === CONT kuttl/harness/1-025-validate-managed-by-change logger.go:42: 17:51:31 | 1-025-validate-managed-by-change | Creating namespace: kuttl-test-master-grub logger.go:42: 17:51:31 | 1-025-validate-managed-by-change/1-install | starting test step 1-install === CONT kuttl/harness/1-032_validate_dynamic_scaling logger.go:42: 17:51:32 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | 4 logger.go:42: 17:51:32 | 1-032_validate_dynamic_scaling/3-cluster-secrets-exceed-maxShards | test step completed 3-cluster-secrets-exceed-maxShards logger.go:42: 17:51:32 | 1-032_validate_dynamic_scaling/4-delete-cluster-secret | starting test step 4-delete-cluster-secret I0723 17:51:32.900436 167 request.go:682] Waited for 1.046315852s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/k8s.cni.cncf.io/v1?timeout=32s logger.go:42: 17:51:34 | 1-032_validate_dynamic_scaling/4-delete-cluster-secret | running command: [sleep 60s] === CONT kuttl/harness/1-025-validate-managed-by-change logger.go:42: 17:51:34 | 1-025-validate-managed-by-change/1-install | Namespace:/test-1-25-target created logger.go:42: 17:51:34 | 1-025-validate-managed-by-change/1-install | Namespace:/test-1-25-argo1 created logger.go:42: 17:51:34 | 1-025-validate-managed-by-change/1-install | Namespace:/test-1-25-argo2 created logger.go:42: 17:51:34 | 1-025-validate-managed-by-change/1-install | ArgoCD:test-1-25-argo1/argocd created logger.go:42: 17:51:34 | 1-025-validate-managed-by-change/1-install | ArgoCD:test-1-25-argo2/argocd created I0723 17:51:42.910280 167 request.go:682] Waited for 1.296267223s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/template.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-066_validate_redis_secure_comm_no_autotls_no_ha logger.go:42: 17:51:44 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/4-create_annotation | test step completed 4-create_annotation logger.go:42: 17:51:44 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/5-check_deployments | starting test step 5-check_deployments logger.go:42: 17:51:44 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/5-check_deployments | running command: [sh -c set -e if test "$(oc get deployments.apps argocd-redis -n $NAMESPACE --template '{{range .spec.template.spec.containers}}{{.args}}{{"\n"}}{{end}}')" != \ "[redis-server \ --protected-mode no \ --save \ --appendonly no \ --requirepass \$(REDIS_PASSWORD) \ --tls-port 6379 \ --port 0 \ --tls-cert-file /app/config/redis/tls/tls.crt \ --tls-key-file /app/config/redis/tls/tls.key \ --tls-auth-clients no]"; then echo "TLS .spec.template.spec.containers.args for argocd-redis deployment are wrong" exit 1 fi ] logger.go:42: 17:51:44 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/5-check_deployments | running command: [sh -c set -e if test "$(oc get deployments.apps argocd-repo-server -n $NAMESPACE --template '{{range .spec.template.spec.containers}}{{.command}}{{"\n"}}{{end}}')" != \ "[uid_entrypoint.sh \ argocd-repo-server \ --redis argocd-redis.$NAMESPACE.svc.cluster.local:6379 \ --redis-use-tls \ --redis-ca-certificate /app/config/reposerver/tls/redis/tls.crt \ --loglevel info \ --logformat text]"; then echo "TLS .spec.template.spec.containers.command for argocd-repo-server deployment is wrong" exit 1 fi ] logger.go:42: 17:51:45 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/5-check_deployments | running command: [sh -c set -e if test "$(oc get deployments.apps argocd-server -n $NAMESPACE --template '{{range .spec.template.spec.containers}}{{.command}}{{"\n"}}{{end}}')" != \ "[argocd-server \ --staticassets /shared/app \ --dex-server https://argocd-dex-server.$NAMESPACE.svc.cluster.local:5556 \ --repo-server argocd-repo-server.$NAMESPACE.svc.cluster.local:8081 \ --redis argocd-redis.$NAMESPACE.svc.cluster.local:6379 \ --redis-use-tls \ --redis-ca-certificate /app/config/server/tls/redis/tls.crt \ --loglevel info \ --logformat text]"; then echo "TLS .spec.template.spec.containers.command for argocd-server deployment is wrong" exit 1 fi ] === CONT kuttl/harness/1-070_validate_config_management_plugin logger.go:42: 17:51:46 | 1-070_validate_config_management_plugin/99-delete | test step completed 99-delete logger.go:42: 17:51:46 | 1-070_validate_config_management_plugin | skipping kubernetes event logging logger.go:42: 17:51:46 | 1-070_validate_config_management_plugin | Deleting namespace: kuttl-test-guided-kingfish === CONT kuttl/harness/1-066_validate_redis_secure_comm_no_autotls_no_ha logger.go:42: 17:51:49 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/5-check_deployments | test step completed 5-check_deployments logger.go:42: 17:51:49 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/6-check_statefulset | starting test step 6-check_statefulset logger.go:42: 17:51:49 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/6-check_statefulset | running command: [sh -c set -e if test "$(oc get statefulsets.apps argocd-application-controller -n $NAMESPACE --template '{{range .spec.template.spec.containers}}{{.command}}{{"\n"}}{{end}}')" != \ "[argocd-application-controller \ --operation-processors 10 \ --redis argocd-redis.$NAMESPACE.svc.cluster.local:6379 \ --redis-use-tls \ --redis-ca-certificate /app/config/controller/tls/redis/tls.crt \ --repo-server argocd-repo-server.$NAMESPACE.svc.cluster.local:8081 \ --status-processors 20 \ --kubectl-parallelism-limit 10 \ --loglevel info \ --logformat text]"; then echo "TLS .spec.template.spec.containers.command for argocd-application-controller statefulsets is wrong" exit 1 fi ] logger.go:42: 17:51:52 | 1-066_validate_redis_secure_comm_no_autotls_no_ha/6-check_statefulset | test step completed 6-check_statefulset logger.go:42: 17:51:52 | 1-066_validate_redis_secure_comm_no_autotls_no_ha | skipping kubernetes event logging logger.go:42: 17:51:52 | 1-066_validate_redis_secure_comm_no_autotls_no_ha | Deleting namespace: kuttl-test-sound-turtle === CONT kuttl/harness/1-069_validate_redis_secure_comm_autotls_ha logger.go:42: 17:51:52 | 1-069_validate_redis_secure_comm_autotls_ha | Creating namespace: kuttl-test-sacred-tomcat logger.go:42: 17:51:52 | 1-069_validate_redis_secure_comm_autotls_ha/1-install | starting test step 1-install I0723 17:51:53.851197 167 request.go:682] Waited for 1.038728246s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/cloudcredential.openshift.io/v1?timeout=32s logger.go:42: 17:51:55 | 1-069_validate_redis_secure_comm_autotls_ha/1-install | ArgoCD:kuttl-test-sacred-tomcat/argocd created === CONT kuttl/harness/1-068_validate_redis_secure_comm_autotls_no_ha logger.go:42: 17:51:58 | 1-068_validate_redis_secure_comm_autotls_no_ha | Creating namespace: kuttl-test-polished-poodle logger.go:42: 17:51:58 | 1-068_validate_redis_secure_comm_autotls_no_ha/1-install | starting test step 1-install logger.go:42: 17:52:01 | 1-068_validate_redis_secure_comm_autotls_no_ha/1-install | ArgoCD:kuttl-test-polished-poodle/argocd created === CONT kuttl/harness/1-025-validate-managed-by-change logger.go:42: 17:52:08 | 1-025-validate-managed-by-change/1-install | test step completed 1-install logger.go:42: 17:52:08 | 1-025-validate-managed-by-change/2-install-app | starting test step 2-install-app logger.go:42: 17:52:08 | 1-025-validate-managed-by-change/2-install-app | running command: [sh -c function wait_until_pods_running() { echo -n "Waiting until all pods in namespace $1 are up" for i in {1..150}; do # timeout after 5 minutes local pods="$(oc get pods --no-headers -n $1 2>/dev/null)" # All pods must be running local not_running=$(echo "${pods}" | grep -v Running | grep -v Completed | wc -l) if [[ -n "${pods}" && ${not_running} -eq 0 ]]; then local all_ready=1 while read pod ; do local status=(`echo -n ${pod} | cut -f2 -d' ' | tr '/' ' '`) # All containers must be ready [[ -z ${status[0]} ]] && all_ready=0 && break [[ -z ${status[1]} ]] && all_ready=0 && break [[ ${status[0]} -lt 1 ]] && all_ready=0 && break [[ ${status[1]} -lt 1 ]] && all_ready=0 && break [[ ${status[0]} -ne ${status[1]} ]] && all_ready=0 && break done <<< $(echo "${pods}" | grep -v Completed) if (( all_ready )); then echo -e "\nAll pods are up:\n${pods}" return 0 fi fi echo -n "." sleep 2 done echo -e "\n\nERROR: timeout waiting for pods to come up\n${pods}" return 1 } wait_until_pods_running "test-1-25-argo1" wait_until_pods_running "test-1-25-argo2" ] logger.go:42: 17:52:09 | 1-025-validate-managed-by-change/2-install-app | Waiting until all pods in namespace test-1-25-argo1 are up logger.go:42: 17:52:09 | 1-025-validate-managed-by-change/2-install-app | All pods are up: logger.go:42: 17:52:09 | 1-025-validate-managed-by-change/2-install-app | argocd-application-controller-0 1/1 Running 0 33s logger.go:42: 17:52:09 | 1-025-validate-managed-by-change/2-install-app | argocd-redis-5d6dd5f55-ppf77 1/1 Running 0 33s logger.go:42: 17:52:09 | 1-025-validate-managed-by-change/2-install-app | argocd-repo-server-7d8b6db5d8-4vjp4 1/1 Running 0 33s logger.go:42: 17:52:09 | 1-025-validate-managed-by-change/2-install-app | argocd-server-5964578ddf-jpxkw 1/1 Running 0 33s logger.go:42: 17:52:09 | 1-025-validate-managed-by-change/2-install-app | Waiting until all pods in namespace test-1-25-argo2 are up logger.go:42: 17:52:09 | 1-025-validate-managed-by-change/2-install-app | All pods are up: logger.go:42: 17:52:09 | 1-025-validate-managed-by-change/2-install-app | argocd-application-controller-0 1/1 Running 0 33s logger.go:42: 17:52:09 | 1-025-validate-managed-by-change/2-install-app | argocd-redis-5d6dd5f55-wv7nw 1/1 Running 0 33s logger.go:42: 17:52:09 | 1-025-validate-managed-by-change/2-install-app | argocd-repo-server-56d6b488bf-kr56w 1/1 Running 0 33s logger.go:42: 17:52:09 | 1-025-validate-managed-by-change/2-install-app | argocd-server-7bc9ff6486-hmnm7 1/1 Running 0 33s I0723 17:52:10.663915 167 request.go:682] Waited for 1.045545233s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/template.openshift.io/v1?timeout=32s logger.go:42: 17:52:12 | 1-025-validate-managed-by-change/2-install-app | Application:test-1-25-argo1/guestbook created === CONT kuttl/harness/1-068_validate_redis_secure_comm_autotls_no_ha logger.go:42: 17:52:13 | 1-068_validate_redis_secure_comm_autotls_no_ha/1-install | test step completed 1-install logger.go:42: 17:52:13 | 1-068_validate_redis_secure_comm_autotls_no_ha/2-enable_autotls | starting test step 2-enable_autotls logger.go:42: 17:52:13 | 1-068_validate_redis_secure_comm_autotls_no_ha/2-enable_autotls | running command: [sh -c set -e oc patch argocds.argoproj.io argocd --type=merge -p '{"spec":{"redis":{"autotls":"openshift"}}}' -n $NAMESPACE ] logger.go:42: 17:52:13 | 1-068_validate_redis_secure_comm_autotls_no_ha/2-enable_autotls | argocd.argoproj.io/argocd patched === CONT kuttl/harness/1-025-validate-managed-by-change logger.go:42: 17:52:22 | 1-025-validate-managed-by-change/2-install-app | test step completed 2-install-app logger.go:42: 17:52:22 | 1-025-validate-managed-by-change/3-update-namespace | starting test step 3-update-namespace I0723 17:52:23.267386 167 request.go:682] Waited for 1.046862245s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/network.openshift.io/v1?timeout=32s logger.go:42: 17:52:24 | 1-025-validate-managed-by-change/3-update-namespace | Namespace:/test-1-25-target updated logger.go:42: 17:52:24 | 1-025-validate-managed-by-change/3-update-namespace | Application:test-1-25-argo1/guestbook updated logger.go:42: 17:52:24 | 1-025-validate-managed-by-change/3-update-namespace | Application:test-1-25-argo2/guestbook created logger.go:42: 17:52:27 | 1-025-validate-managed-by-change/3-update-namespace | test step completed 3-update-namespace logger.go:42: 17:52:27 | 1-025-validate-managed-by-change/99-delete | starting test step 99-delete === CONT kuttl/harness/1-032_validate_dynamic_scaling logger.go:42: 17:52:34 | 1-032_validate_dynamic_scaling/4-delete-cluster-secret | running command: [kubectl delete Secret cluster-2 cluster-3 cluster-4 cluster-5 cluster-6 -n kuttl-test-cheerful-albacore] logger.go:42: 17:52:34 | 1-032_validate_dynamic_scaling/4-delete-cluster-secret | secret "cluster-2" deleted logger.go:42: 17:52:34 | 1-032_validate_dynamic_scaling/4-delete-cluster-secret | secret "cluster-3" deleted logger.go:42: 17:52:34 | 1-032_validate_dynamic_scaling/4-delete-cluster-secret | secret "cluster-4" deleted logger.go:42: 17:52:34 | 1-032_validate_dynamic_scaling/4-delete-cluster-secret | secret "cluster-5" deleted logger.go:42: 17:52:34 | 1-032_validate_dynamic_scaling/4-delete-cluster-secret | secret "cluster-6" deleted logger.go:42: 17:52:35 | 1-032_validate_dynamic_scaling/4-delete-cluster-secret | running command: [sleep 30s] I0723 17:52:41.186582 167 request.go:682] Waited for 1.045169852s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/operator.openshift.io/v1alpha1?timeout=32s === CONT kuttl/harness/1-025-validate-managed-by-change logger.go:42: 17:52:42 | 1-025-validate-managed-by-change/99-delete | test step completed 99-delete logger.go:42: 17:52:42 | 1-025-validate-managed-by-change | skipping kubernetes event logging logger.go:42: 17:52:42 | 1-025-validate-managed-by-change | Deleting namespace: kuttl-test-master-grub === CONT kuttl/harness/1-068_validate_redis_secure_comm_autotls_no_ha logger.go:42: 17:52:45 | 1-068_validate_redis_secure_comm_autotls_no_ha/2-enable_autotls | test step completed 2-enable_autotls logger.go:42: 17:52:45 | 1-068_validate_redis_secure_comm_autotls_no_ha/3-check_secret | starting test step 3-check_secret logger.go:42: 17:52:45 | 1-068_validate_redis_secure_comm_autotls_no_ha/3-check_secret | running command: [sh -c set -e secret_type="$(oc get secrets argocd-operator-redis-tls -n $NAMESPACE --template '{{.type}}')" secret_len="$(oc get secrets argocd-operator-redis-tls -n $NAMESPACE --template '{{len .data}}')" expected_secret_type="kubernetes.io/tls" expected_secret_len=2 if test ${secret_type} != ${expected_secret_type}; then echo "argocd-operator-redis-tls secret type is ${secret_type} and should be ${expected_secret_type}" exit 1 fi if test ${secret_len} != ${expected_secret_len}; then echo "argocd-operator-redis-tls secret length is ${secret_len} and should be ${expected_secret_len}" exit 1 fi ] === CONT kuttl/harness/1-003_alpha_to_beta_sso_conflict_conversion === CONT kuttl/harness/1-068_validate_redis_secure_comm_autotls_no_ha logger.go:42: 17:52:49 | 1-068_validate_redis_secure_comm_autotls_no_ha/3-check_secret | test step completed 3-check_secret logger.go:42: 17:52:49 | 1-068_validate_redis_secure_comm_autotls_no_ha/4- | starting test step 4- I0723 17:52:51.211380 167 request.go:682] Waited for 1.946616684s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/monitoring.coreos.com/v1alpha1?timeout=32s === CONT kuttl/harness/1-003_alpha_to_beta_sso_conflict_conversion logger.go:42: 17:52:51 | 1-003_alpha_to_beta_sso_conflict_conversion | Creating namespace: kuttl-test-crack-wasp logger.go:42: 17:52:51 | 1-003_alpha_to_beta_sso_conflict_conversion/1-argocd-dex-keycloak-conflict | starting test step 1-argocd-dex-keycloak-conflict === CONT kuttl/harness/1-068_validate_redis_secure_comm_autotls_no_ha logger.go:42: 17:52:54 | 1-068_validate_redis_secure_comm_autotls_no_ha/4- | test step completed 4- logger.go:42: 17:52:54 | 1-068_validate_redis_secure_comm_autotls_no_ha/5-check_crt_files | starting test step 5-check_crt_files logger.go:42: 17:52:54 | 1-068_validate_redis_secure_comm_autotls_no_ha/5-check_crt_files | running command: [sh -c set -e oc exec -i $(oc get pod -l app.kubernetes.io/name=argocd-server -n $NAMESPACE -o=NAME) -n $NAMESPACE -- ls /app/config/server/tls/redis/tls.crt oc exec -i $(oc get pod -l app.kubernetes.io/name=argocd-repo-server -n $NAMESPACE -o=NAME) -n $NAMESPACE -- ls /app/config/reposerver/tls/redis/tls.crt oc exec -i $(oc get pod -l app.kubernetes.io/name=argocd-redis -n $NAMESPACE -o=NAME) -n $NAMESPACE -- ls /app/config/redis/tls/tls.crt oc exec -i $(oc get pod -l app.kubernetes.io/name=argocd-application-controller -n $NAMESPACE -o=NAME) -n $NAMESPACE -- ls /app/config/controller/tls/redis/tls.crt ] === CONT kuttl/harness/1-003_alpha_to_beta_sso_conflict_conversion logger.go:42: 17:52:54 | 1-003_alpha_to_beta_sso_conflict_conversion/1-argocd-dex-keycloak-conflict | ArgoCD:kuttl-test-crack-wasp/argocd created === CONT kuttl/harness/1-068_validate_redis_secure_comm_autotls_no_ha logger.go:42: 17:52:55 | 1-068_validate_redis_secure_comm_autotls_no_ha/5-check_crt_files | /app/config/server/tls/redis/tls.crt logger.go:42: 17:52:56 | 1-068_validate_redis_secure_comm_autotls_no_ha/5-check_crt_files | Defaulted container "argocd-repo-server" out of: argocd-repo-server, copyutil (init) logger.go:42: 17:52:56 | 1-068_validate_redis_secure_comm_autotls_no_ha/5-check_crt_files | /app/config/reposerver/tls/redis/tls.crt logger.go:42: 17:52:57 | 1-068_validate_redis_secure_comm_autotls_no_ha/5-check_crt_files | /app/config/redis/tls/tls.crt logger.go:42: 17:52:59 | 1-068_validate_redis_secure_comm_autotls_no_ha/5-check_crt_files | /app/config/controller/tls/redis/tls.crt I0723 17:53:01.221777 167 request.go:682] Waited for 2.038371335s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/controlplane.operator.openshift.io/v1alpha1?timeout=32s logger.go:42: 17:53:01 | 1-068_validate_redis_secure_comm_autotls_no_ha/5-check_crt_files | test step completed 5-check_crt_files logger.go:42: 17:53:01 | 1-068_validate_redis_secure_comm_autotls_no_ha | skipping kubernetes event logging logger.go:42: 17:53:01 | 1-068_validate_redis_secure_comm_autotls_no_ha | Deleting namespace: kuttl-test-polished-poodle === CONT kuttl/harness/1-032_validate_dynamic_scaling logger.go:42: 17:53:07 | 1-032_validate_dynamic_scaling/4-delete-cluster-secret | test step completed 4-delete-cluster-secret logger.go:42: 17:53:07 | 1-032_validate_dynamic_scaling | skipping kubernetes event logging logger.go:42: 17:53:07 | 1-032_validate_dynamic_scaling | Deleting namespace: kuttl-test-cheerful-albacore === CONT kuttl/harness/1-067_validate_redis_secure_comm_no_autotls_ha logger.go:42: 17:53:08 | 1-067_validate_redis_secure_comm_no_autotls_ha | Creating namespace: kuttl-test-key-reindeer logger.go:42: 17:53:08 | 1-067_validate_redis_secure_comm_no_autotls_ha/1-install | starting test step 1-install logger.go:42: 17:53:10 | 1-067_validate_redis_secure_comm_no_autotls_ha/1-install | ArgoCD:kuttl-test-key-reindeer/argocd created === CONT kuttl/harness/1-004_beta_to_alpha_conversion logger.go:42: 17:53:13 | 1-004_beta_to_alpha_conversion | Creating namespace: kuttl-test-loved-alpaca logger.go:42: 17:53:13 | 1-004_beta_to_alpha_conversion/1-argocd-dex | starting test step 1-argocd-dex I0723 17:53:14.961288 167 request.go:682] Waited for 1.045102336s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/machine.openshift.io/v1?timeout=32s logger.go:42: 17:53:16 | 1-004_beta_to_alpha_conversion/1-argocd-dex | ArgoCD:kuttl-test-loved-alpaca/argocd created === CONT kuttl/harness/1-003_alpha_to_beta_sso_conflict_conversion logger.go:42: 17:53:26 | 1-003_alpha_to_beta_sso_conflict_conversion/1-argocd-dex-keycloak-conflict | test step completed 1-argocd-dex-keycloak-conflict logger.go:42: 17:53:26 | 1-003_alpha_to_beta_sso_conflict_conversion/2-delete | starting test step 2-delete I0723 17:53:27.633026 167 request.go:682] Waited for 1.045670421s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/migration.k8s.io/v1alpha1?timeout=32s logger.go:42: 17:53:29 | 1-003_alpha_to_beta_sso_conflict_conversion/2-delete | test step completed 2-delete logger.go:42: 17:53:29 | 1-003_alpha_to_beta_sso_conflict_conversion | skipping kubernetes event logging logger.go:42: 17:53:29 | 1-003_alpha_to_beta_sso_conflict_conversion | Deleting namespace: kuttl-test-crack-wasp === CONT kuttl/harness/1-004_beta_to_alpha_conversion logger.go:42: 17:53:29 | 1-004_beta_to_alpha_conversion/1-argocd-dex | test step completed 1-argocd-dex logger.go:42: 17:53:29 | 1-004_beta_to_alpha_conversion/2-delete | starting test step 2-delete logger.go:42: 17:53:31 | 1-004_beta_to_alpha_conversion/2-delete | test step completed 2-delete logger.go:42: 17:53:31 | 1-004_beta_to_alpha_conversion | skipping kubernetes event logging logger.go:42: 17:53:31 | 1-004_beta_to_alpha_conversion | Deleting namespace: kuttl-test-loved-alpaca === CONT kuttl/harness/1-063_validate_statefulset_restart logger.go:42: 17:53:35 | 1-063_validate_statefulset_restart | Creating namespace: kuttl-test-bright-sawfly logger.go:42: 17:53:35 | 1-063_validate_statefulset_restart/1-install | starting test step 1-install I0723 17:53:37.679896 167 request.go:682] Waited for 2.195701703s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/monitoring.coreos.com/v1?timeout=32s === CONT kuttl/harness/1-039_validate_fix_argocd-tls-certs-cm logger.go:42: 17:53:37 | 1-039_validate_fix_argocd-tls-certs-cm | Creating namespace: kuttl-test-ample-panda logger.go:42: 17:53:37 | 1-039_validate_fix_argocd-tls-certs-cm/1-install | starting test step 1-install === CONT kuttl/harness/1-063_validate_statefulset_restart logger.go:42: 17:53:37 | 1-063_validate_statefulset_restart/1-install | ArgoCD:kuttl-test-bright-sawfly/example-argocd created === CONT kuttl/harness/1-039_validate_fix_argocd-tls-certs-cm logger.go:42: 17:53:40 | 1-039_validate_fix_argocd-tls-certs-cm/1-install | ArgoCD:kuttl-test-ample-panda/example-argocd created === CONT kuttl/harness/1-063_validate_statefulset_restart logger.go:42: 17:54:09 | 1-063_validate_statefulset_restart/1-install | test step completed 1-install logger.go:42: 17:54:09 | 1-063_validate_statefulset_restart/2-change_to_invalid_image | starting test step 2-change_to_invalid_image logger.go:42: 17:54:09 | 1-063_validate_statefulset_restart/2-change_to_invalid_image | running command: [sh -c set -e oc patch statefulset/example-argocd-application-controller \ -n $NAMESPACE \ --type "json" \ -p '[{"op":"replace","path":"/spec/template/spec/containers/0/image","value":"invalid_image"}]' sleep 10 ] logger.go:42: 17:54:10 | 1-063_validate_statefulset_restart/2-change_to_invalid_image | statefulset.apps/example-argocd-application-controller patched === CONT kuttl/harness/1-039_validate_fix_argocd-tls-certs-cm logger.go:42: 17:54:12 | 1-039_validate_fix_argocd-tls-certs-cm/1-install | test step completed 1-install logger.go:42: 17:54:12 | 1-039_validate_fix_argocd-tls-certs-cm/2-modify_configmap | starting test step 2-modify_configmap I0723 17:54:13.826363 167 request.go:682] Waited for 1.046915984s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/argoproj.io/v1alpha1?timeout=32s logger.go:42: 17:54:15 | 1-039_validate_fix_argocd-tls-certs-cm/2-modify_configmap | ConfigMap:kuttl-test-ample-panda/argocd-tls-certs-cm updated logger.go:42: 17:54:15 | 1-039_validate_fix_argocd-tls-certs-cm/2-modify_configmap | test step completed 2-modify_configmap logger.go:42: 17:54:15 | 1-039_validate_fix_argocd-tls-certs-cm/3-modify_argocd_cr | starting test step 3-modify_argocd_cr logger.go:42: 17:54:17 | 1-039_validate_fix_argocd-tls-certs-cm/3-modify_argocd_cr | ArgoCD:kuttl-test-ample-panda/example-argocd updated logger.go:42: 17:54:17 | 1-039_validate_fix_argocd-tls-certs-cm/3-modify_argocd_cr | test step completed 3-modify_argocd_cr logger.go:42: 17:54:17 | 1-039_validate_fix_argocd-tls-certs-cm/4- | starting test step 4- logger.go:42: 17:54:20 | 1-039_validate_fix_argocd-tls-certs-cm/4- | test step completed 4- logger.go:42: 17:54:20 | 1-039_validate_fix_argocd-tls-certs-cm | skipping kubernetes event logging logger.go:42: 17:54:20 | 1-039_validate_fix_argocd-tls-certs-cm | Deleting namespace: kuttl-test-ample-panda === CONT kuttl/harness/1-063_validate_statefulset_restart logger.go:42: 17:54:23 | 1-063_validate_statefulset_restart/2-change_to_invalid_image | test step completed 2-change_to_invalid_image logger.go:42: 17:54:23 | 1-063_validate_statefulset_restart/3-check_image_after_change | starting test step 3-check_image_after_change logger.go:42: 17:54:23 | 1-063_validate_statefulset_restart/3-check_image_after_change | running command: [sh -c set -e image=$(oc get statefulset/example-argocd-application-controller -o jsonpath='{.spec.template.spec.containers[].image}' -n $NAMESPACE) if test "${image}" == "invalid_image"; then echo "The application-controller statefulset is using 'invalid_image'" exit 1 fi exit 0 ] I0723 17:54:24.758509 167 request.go:682] Waited for 1.045787231s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/node.k8s.io/v1?timeout=32s logger.go:42: 17:54:26 | 1-063_validate_statefulset_restart/3-check_image_after_change | test step completed 3-check_image_after_change logger.go:42: 17:54:26 | 1-063_validate_statefulset_restart | skipping kubernetes event logging logger.go:42: 17:54:26 | 1-063_validate_statefulset_restart | Deleting namespace: kuttl-test-bright-sawfly === CONT kuttl/harness/1-043_validate_log_level_format logger.go:42: 17:54:26 | 1-043_validate_log_level_format | Creating namespace: kuttl-test-devoted-bedbug logger.go:42: 17:54:26 | 1-043_validate_log_level_format/1-install | starting test step 1-install logger.go:42: 17:54:28 | 1-043_validate_log_level_format/1-install | ArgoCD:kuttl-test-devoted-bedbug/argocd created === CONT kuttl/harness/1-063_validate_dex_liveness_probe logger.go:42: 17:54:32 | 1-063_validate_dex_liveness_probe | Creating namespace: kuttl-test-deciding-mallard logger.go:42: 17:54:32 | 1-063_validate_dex_liveness_probe/1- | starting test step 1- logger.go:42: 17:54:34 | 1-063_validate_dex_liveness_probe/1- | test step completed 1- logger.go:42: 17:54:34 | 1-063_validate_dex_liveness_probe | skipping kubernetes event logging logger.go:42: 17:54:34 | 1-063_validate_dex_liveness_probe | Deleting namespace: kuttl-test-deciding-mallard === CONT kuttl/harness/1-002_alpha_to_beta_keycloak_conversion logger.go:42: 17:54:40 | 1-002_alpha_to_beta_keycloak_conversion | Creating namespace: kuttl-test-poetic-sheep logger.go:42: 17:54:40 | 1-002_alpha_to_beta_keycloak_conversion/1-argocd-keycloak | starting test step 1-argocd-keycloak I0723 17:54:41.729950 167 request.go:682] Waited for 1.047696174s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/performance.openshift.io/v1alpha1?timeout=32s === CONT kuttl/harness/1-043_validate_log_level_format logger.go:42: 17:54:43 | 1-043_validate_log_level_format/1-install | test step completed 1-install logger.go:42: 17:54:43 | 1-043_validate_log_level_format/2-change-loglevel | starting test step 2-change-loglevel === CONT kuttl/harness/1-002_alpha_to_beta_keycloak_conversion logger.go:42: 17:54:43 | 1-002_alpha_to_beta_keycloak_conversion/1-argocd-keycloak | ArgoCD:kuttl-test-poetic-sheep/argocd created === CONT kuttl/harness/1-043_validate_log_level_format logger.go:42: 17:54:45 | 1-043_validate_log_level_format/2-change-loglevel | ArgoCD:kuttl-test-devoted-bedbug/argocd updated logger.go:42: 17:54:45 | 1-043_validate_log_level_format/2-change-loglevel | test step completed 2-change-loglevel logger.go:42: 17:54:45 | 1-043_validate_log_level_format/3-check-loglevel | starting test step 3-check-loglevel logger.go:42: 17:54:45 | 1-043_validate_log_level_format/3-check-loglevel | running command: [sh -c sleep 15] logger.go:42: 17:55:00 | 1-043_validate_log_level_format/3-check-loglevel | running command: [sh -c set -e set -o pipefail for wl in deployment/argocd-server deployment/argocd-repo-server statefulset/argocd-application-controller; do wlCommand=$(oc get -n $NAMESPACE $wl -o jsonpath='{.spec.template.spec.containers[0].command}'| jq -r '.[]' ) level="debug" format="json" if ! echo "$wlCommand" | grep -qPz -- "--loglevel\\n$level(\$|\\n)"; then echo "logLevel was not set correctly for $wl" echo "CWD: $wlCommand" exit 1 fi if ! echo "$wlCommand" | grep -qPz -- "--logformat\\n$format(\$|\\n)"; then echo "logFormat was not set correctly for $wl" echo "CWD: $wlCommand" exit 1 fi done ] I0723 17:55:03.259993 167 request.go:682] Waited for 1.046086698s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/performance.openshift.io/v1?timeout=32s logger.go:42: 17:55:04 | 1-043_validate_log_level_format/3-check-loglevel | test step completed 3-check-loglevel logger.go:42: 17:55:04 | 1-043_validate_log_level_format | skipping kubernetes event logging logger.go:42: 17:55:04 | 1-043_validate_log_level_format | Deleting namespace: kuttl-test-devoted-bedbug === CONT kuttl/harness/1-038_validate_productized_images logger.go:42: 17:55:11 | 1-038_validate_productized_images | Creating namespace: kuttl-test-normal-gull logger.go:42: 17:55:11 | 1-038_validate_productized_images/1-install | starting test step 1-install I0723 17:55:13.268354 167 request.go:682] Waited for 2.244486877s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/operators.coreos.com/v2?timeout=32s logger.go:42: 17:55:13 | 1-038_validate_productized_images/1-install | ArgoCD:kuttl-test-normal-gull/argocd created === CONT kuttl/harness/1-002_alpha_to_beta_keycloak_conversion logger.go:42: 17:55:15 | 1-002_alpha_to_beta_keycloak_conversion/1-argocd-keycloak | test step completed 1-argocd-keycloak logger.go:42: 17:55:15 | 1-002_alpha_to_beta_keycloak_conversion/2-delete | starting test step 2-delete logger.go:42: 17:55:18 | 1-002_alpha_to_beta_keycloak_conversion/2-delete | test step completed 2-delete logger.go:42: 17:55:18 | 1-002_alpha_to_beta_keycloak_conversion | skipping kubernetes event logging logger.go:42: 17:55:18 | 1-002_alpha_to_beta_keycloak_conversion | Deleting namespace: kuttl-test-poetic-sheep === CONT kuttl/harness/1-037_validate_argocd_setting_replicas logger.go:42: 17:55:24 | 1-037_validate_argocd_setting_replicas | Creating namespace: kuttl-test-pleasant-kangaroo logger.go:42: 17:55:24 | 1-037_validate_argocd_setting_replicas/1-install | starting test step 1-install I0723 17:55:25.287270 167 request.go:682] Waited for 1.038328901s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/operators.coreos.com/v1alpha2?timeout=32s logger.go:42: 17:55:26 | 1-037_validate_argocd_setting_replicas/1-install | ArgoCD:kuttl-test-pleasant-kangaroo/argocd created === CONT kuttl/harness/1-038_validate_productized_images logger.go:42: 17:55:44 | 1-038_validate_productized_images/1-install | test step completed 1-install logger.go:42: 17:55:44 | 1-038_validate_productized_images/2-check-images | starting test step 2-check-images logger.go:42: 17:55:44 | 1-038_validate_productized_images/2-check-images | running command: [sh -c for wl in deployment/argocd-server deployment/argocd-repo-server statefulset/argocd-application-controller; do image=$(oc -n $NAMESPACE get ${wl} -o jsonpath='{.spec.template.spec.containers[0].image}' | awk -F'@' '{print $1}') if test "$image" != "registry.redhat.io/openshift-gitops-1/argocd-rhel8"; then echo "Non-productized image in workload $wl detected." exit 1 fi done ] I0723 17:55:47.458024 167 request.go:682] Waited for 1.046004979s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/operators.coreos.com/v1alpha2?timeout=32s logger.go:42: 17:55:48 | 1-038_validate_productized_images/2-check-images | test step completed 2-check-images logger.go:42: 17:55:48 | 1-038_validate_productized_images | skipping kubernetes event logging logger.go:42: 17:55:48 | 1-038_validate_productized_images | Deleting namespace: kuttl-test-normal-gull === CONT kuttl/harness/1-023_validate_repo_server_tls logger.go:42: 17:55:55 | 1-023_validate_repo_server_tls | Creating namespace: kuttl-test-sensible-alpaca logger.go:42: 17:55:55 | 1-023_validate_repo_server_tls/1-install | starting test step 1-install I0723 17:55:57.464771 167 request.go:682] Waited for 2.14627312s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/pipelines.openshift.io/v1alpha1?timeout=32s logger.go:42: 17:55:57 | 1-023_validate_repo_server_tls/1-install | Namespace:/test-1-23-custom created logger.go:42: 17:55:57 | 1-023_validate_repo_server_tls/1-install | ArgoCD:test-1-23-custom/argocd created === CONT kuttl/harness/1-037_validate_argocd_setting_replicas logger.go:42: 17:55:59 | 1-037_validate_argocd_setting_replicas/1-install | test step completed 1-install logger.go:42: 17:55:59 | 1-037_validate_argocd_setting_replicas/2-scale_out_server | starting test step 2-scale_out_server logger.go:42: 17:55:59 | 1-037_validate_argocd_setting_replicas/2-scale_out_server | running command: [sh -c oc patch argocd argocd \ -n $NAMESPACE \ --type='json' \ -p='[{"op": "replace", "path": "/spec/server/replicas", "value": 3 }]' exit 0 ] logger.go:42: 17:56:00 | 1-037_validate_argocd_setting_replicas/2-scale_out_server | argocd.argoproj.io/argocd patched logger.go:42: 17:56:02 | 1-037_validate_argocd_setting_replicas/2-scale_out_server | test step completed 2-scale_out_server logger.go:42: 17:56:02 | 1-037_validate_argocd_setting_replicas/3-scale_out_repo | starting test step 3-scale_out_repo logger.go:42: 17:56:02 | 1-037_validate_argocd_setting_replicas/3-scale_out_repo | running command: [sh -c oc patch argocd argocd \ -n $NAMESPACE \ --type='json' \ -p='[{"op": "replace", "path": "/spec/repo/replicas", "value": 3 }]' exit 0 ] logger.go:42: 17:56:03 | 1-037_validate_argocd_setting_replicas/3-scale_out_repo | argocd.argoproj.io/argocd patched logger.go:42: 17:56:05 | 1-037_validate_argocd_setting_replicas/3-scale_out_repo | test step completed 3-scale_out_repo logger.go:42: 17:56:05 | 1-037_validate_argocd_setting_replicas/4- | starting test step 4- I0723 17:56:07.512972 167 request.go:682] Waited for 1.747063756s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/networking.k8s.io/v1?timeout=32s === CONT kuttl/harness/1-023_validate_repo_server_tls logger.go:42: 17:56:21 | 1-023_validate_repo_server_tls/1-install | test step completed 1-install logger.go:42: 17:56:21 | 1-023_validate_repo_server_tls/2-change-argocd | starting test step 2-change-argocd I0723 17:56:22.467936 167 request.go:682] Waited for 1.044937536s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/flowcontrol.apiserver.k8s.io/v1beta1?timeout=32s logger.go:42: 17:56:23 | 1-023_validate_repo_server_tls/2-change-argocd | ArgoCD:test-1-23-custom/argocd updated logger.go:42: 17:56:24 | 1-023_validate_repo_server_tls/2-change-argocd | test step completed 2-change-argocd logger.go:42: 17:56:24 | 1-023_validate_repo_server_tls/4-install-app | starting test step 4-install-app logger.go:42: 17:56:27 | 1-023_validate_repo_server_tls/4-install-app | Application:test-1-23-custom/guestbook created === CONT kuttl/harness/1-037_validate_argocd_setting_replicas logger.go:42: 17:56:31 | 1-037_validate_argocd_setting_replicas/4- | test step completed 4- logger.go:42: 17:56:31 | 1-037_validate_argocd_setting_replicas/5-scale_in_server | starting test step 5-scale_in_server logger.go:42: 17:56:31 | 1-037_validate_argocd_setting_replicas/5-scale_in_server | running command: [sh -c oc patch argocd argocd \ -n $NAMESPACE \ --type='json' \ -p='[{"op": "replace", "path": "/spec/server/replicas", "value": 1 }]' exit 0 ] logger.go:42: 17:56:32 | 1-037_validate_argocd_setting_replicas/5-scale_in_server | argocd.argoproj.io/argocd patched I0723 17:56:33.457542 167 request.go:682] Waited for 1.047402679s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/apiserver.openshift.io/v1?timeout=32s logger.go:42: 17:56:34 | 1-037_validate_argocd_setting_replicas/5-scale_in_server | test step completed 5-scale_in_server logger.go:42: 17:56:34 | 1-037_validate_argocd_setting_replicas/6-scale_in_repo | starting test step 6-scale_in_repo logger.go:42: 17:56:34 | 1-037_validate_argocd_setting_replicas/6-scale_in_repo | running command: [sh -c oc patch argocd argocd \ -n $NAMESPACE \ --type='json' \ -p='[{"op": "replace", "path": "/spec/repo/replicas", "value": 1 }]' exit 0 ] logger.go:42: 17:56:35 | 1-037_validate_argocd_setting_replicas/6-scale_in_repo | argocd.argoproj.io/argocd patched logger.go:42: 17:56:37 | 1-037_validate_argocd_setting_replicas/6-scale_in_repo | test step completed 6-scale_in_repo logger.go:42: 17:56:37 | 1-037_validate_argocd_setting_replicas/7- | starting test step 7- logger.go:42: 17:56:40 | 1-037_validate_argocd_setting_replicas/7- | test step completed 7- logger.go:42: 17:56:40 | 1-037_validate_argocd_setting_replicas | skipping kubernetes event logging logger.go:42: 17:56:40 | 1-037_validate_argocd_setting_replicas | Deleting namespace: kuttl-test-pleasant-kangaroo === CONT kuttl/harness/1-023_validate_repo_server_tls logger.go:42: 17:56:46 | 1-023_validate_repo_server_tls/4-install-app | test step completed 4-install-app logger.go:42: 17:56:46 | 1-023_validate_repo_server_tls/99-delete | starting test step 99-delete === CONT kuttl/harness/1-003_validate_console_link logger.go:42: 17:56:46 | 1-003_validate_console_link | Creating namespace: kuttl-test-vast-labrador logger.go:42: 17:56:46 | 1-003_validate_console_link/1-check-console-link | starting test step 1-check-console-link I0723 17:56:47.715670 167 request.go:682] Waited for 1.046281548s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/packages.operators.coreos.com/v1?timeout=32s logger.go:42: 17:56:49 | 1-003_validate_console_link/1-check-console-link | test step completed 1-check-console-link logger.go:42: 17:56:49 | 1-003_validate_console_link | skipping kubernetes event logging logger.go:42: 17:56:49 | 1-003_validate_console_link | Deleting namespace: kuttl-test-vast-labrador I0723 17:56:59.933932 167 request.go:682] Waited for 1.046136083s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/scheduling.k8s.io/v1?timeout=32s === CONT kuttl/harness/1-023_validate_repo_server_tls logger.go:42: 17:57:01 | 1-023_validate_repo_server_tls/99-delete | test step completed 99-delete logger.go:42: 17:57:01 | 1-023_validate_repo_server_tls | skipping kubernetes event logging logger.go:42: 17:57:01 | 1-023_validate_repo_server_tls | Deleting namespace: kuttl-test-sensible-alpaca === CONT kuttl/harness/1-065_validate_redis_ha_anti_affinity logger.go:42: 18:00:19 | 1-065_validate_redis_ha_anti_affinity/2- | test step completed 2- logger.go:42: 18:00:19 | 1-065_validate_redis_ha_anti_affinity | skipping kubernetes event logging logger.go:42: 18:00:19 | 1-065_validate_redis_ha_anti_affinity | Deleting namespace: kuttl-test-boss-goblin === CONT kuttl/harness/1-069_validate_redis_secure_comm_autotls_ha logger.go:42: 18:00:58 | 1-069_validate_redis_secure_comm_autotls_ha/1-install | test step completed 1-install logger.go:42: 18:00:58 | 1-069_validate_redis_secure_comm_autotls_ha/2-enable_autotls | starting test step 2-enable_autotls logger.go:42: 18:00:58 | 1-069_validate_redis_secure_comm_autotls_ha/2-enable_autotls | running command: [sh -c set -e oc patch argocds.argoproj.io argocd --type=merge -p '{"spec":{"redis":{"autotls":"openshift"}}}' -n $NAMESPACE ] logger.go:42: 18:00:59 | 1-069_validate_redis_secure_comm_autotls_ha/2-enable_autotls | argocd.argoproj.io/argocd patched I0723 18:01:00.555078 167 request.go:682] Waited for 1.044801907s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/autoscaling.openshift.io/v1?timeout=32s === CONT kuttl/harness/1-067_validate_redis_secure_comm_no_autotls_ha logger.go:42: 18:02:14 | 1-067_validate_redis_secure_comm_no_autotls_ha/1-install | test step completed 1-install logger.go:42: 18:02:14 | 1-067_validate_redis_secure_comm_no_autotls_ha/2-generate_cert | starting test step 2-generate_cert logger.go:42: 18:02:14 | 1-067_validate_redis_secure_comm_no_autotls_ha/2-generate_cert | running command: [sh -c set -e echo -e "\n[SAN]\nsubjectAltName=DNS:argocd-redis.$NAMESPACE.svc.cluster.local\n[req]\ndistinguished_name=req" > ${PWD}/openssl_test.cnf openssl req -new -x509 -sha256 \ -subj "/C=XX/ST=XX/O=Testing/CN=redis" \ -reqexts SAN -extensions SAN \ -config ${PWD}/openssl_test.cnf \ -keyout ${PWD}/redis-ha.key \ -out ${PWD}/redis-ha.crt \ -newkey rsa:4096 \ -nodes \ -sha256 \ -days 10 ] logger.go:42: 18:02:14 | 1-067_validate_redis_secure_comm_no_autotls_ha/2-generate_cert | Generating a RSA private key logger.go:42: 18:02:14 | 1-067_validate_redis_secure_comm_no_autotls_ha/2-generate_cert | .........++++ logger.go:42: 18:02:14 | 1-067_validate_redis_secure_comm_no_autotls_ha/2-generate_cert | .................................................++++ logger.go:42: 18:02:14 | 1-067_validate_redis_secure_comm_no_autotls_ha/2-generate_cert | writing new private key to '/kuttl-e2e/gitops-operator/tests/parallel/1-067_validate_redis_secure_comm_no_autotls_ha/redis-ha.key' logger.go:42: 18:02:14 | 1-067_validate_redis_secure_comm_no_autotls_ha/2-generate_cert | ----- I0723 18:02:15.680336 167 request.go:682] Waited for 1.046895903s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/tuned.openshift.io/v1?timeout=32s logger.go:42: 18:02:17 | 1-067_validate_redis_secure_comm_no_autotls_ha/2-generate_cert | test step completed 2-generate_cert logger.go:42: 18:02:17 | 1-067_validate_redis_secure_comm_no_autotls_ha/3-create_secret | starting test step 3-create_secret logger.go:42: 18:02:17 | 1-067_validate_redis_secure_comm_no_autotls_ha/3-create_secret | running command: [sh -c set -e # clean up key and cert and config file cleanup() { rm -rf ${PWD}/redis-ha.crt && rm -rf ${PWD}/redis-ha.key && rm -rf ${PWD}/openssl_test.cnf } trap cleanup INT TERM EXIT oc create secret tls argocd-operator-redis-tls --key=${PWD}/redis-ha.key --cert=${PWD}/redis-ha.crt -n $NAMESPACE ] logger.go:42: 18:02:17 | 1-067_validate_redis_secure_comm_no_autotls_ha/3-create_secret | secret/argocd-operator-redis-tls created logger.go:42: 18:02:19 | 1-067_validate_redis_secure_comm_no_autotls_ha/3-create_secret | test step completed 3-create_secret logger.go:42: 18:02:19 | 1-067_validate_redis_secure_comm_no_autotls_ha/4-create_annotation | starting test step 4-create_annotation logger.go:42: 18:02:19 | 1-067_validate_redis_secure_comm_no_autotls_ha/4-create_annotation | running command: [sh -c set -e oc annotate secret argocd-operator-redis-tls argocds.argoproj.io/name=argocd -n $NAMESPACE ] logger.go:42: 18:02:20 | 1-067_validate_redis_secure_comm_no_autotls_ha/4-create_annotation | secret/argocd-operator-redis-tls annotated === CONT kuttl/harness/1-069_validate_redis_secure_comm_autotls_ha logger.go:42: 18:10:03 | 1-069_validate_redis_secure_comm_autotls_ha/2-enable_autotls | test step completed 2-enable_autotls logger.go:42: 18:10:03 | 1-069_validate_redis_secure_comm_autotls_ha/3-check_secret | starting test step 3-check_secret logger.go:42: 18:10:03 | 1-069_validate_redis_secure_comm_autotls_ha/3-check_secret | running command: [sh -c set -e secret_type="$(oc get secrets argocd-operator-redis-tls -n $NAMESPACE --template '{{.type}}')" secret_len="$(oc get secrets argocd-operator-redis-tls -n $NAMESPACE --template '{{len .data}}')" expected_secret_type="kubernetes.io/tls" expected_secret_len=2 if test ${secret_type} != ${expected_secret_type}; then echo "argocd-operator-redis-tls secret type is ${secret_type} and should be ${expected_secret_type}" exit 1 fi if test ${secret_len} != ${expected_secret_len}; then echo "argocd-operator-redis-tls secret length is ${secret_len} and should be ${expected_secret_len}" exit 1 fi ] I0723 18:10:05.653872 167 request.go:682] Waited for 1.045999987s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/network.openshift.io/v1?timeout=32s logger.go:42: 18:10:07 | 1-069_validate_redis_secure_comm_autotls_ha/3-check_secret | test step completed 3-check_secret logger.go:42: 18:10:07 | 1-069_validate_redis_secure_comm_autotls_ha | skipping kubernetes event logging logger.go:42: 18:10:07 | 1-069_validate_redis_secure_comm_autotls_ha | Deleting namespace: kuttl-test-sacred-tomcat === CONT kuttl/harness/1-067_validate_redis_secure_comm_no_autotls_ha logger.go:42: 18:11:24 | 1-067_validate_redis_secure_comm_no_autotls_ha/4-create_annotation | test step completed 4-create_annotation logger.go:42: 18:11:24 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | starting test step 5-check_redis_config logger.go:42: 18:11:24 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | running command: [sh -c set -e expected_redis_config=( 'port 0' 'tls-port 6379' 'tls-cert-file /app/config/redis/tls/tls.crt' 'tls-ca-cert-file /app/config/redis/tls/tls.crt' 'tls-key-file /app/config/redis/tls/tls.key' 'tls-replication yes' 'tls-auth-clients no' ) expected_sentinel_config=( 'port 0' 'tls-port 26379' 'tls-cert-file "/app/config/redis/tls/tls.crt"' 'tls-ca-cert-file "/app/config/redis/tls/tls.crt"' 'tls-key-file "/app/config/redis/tls/tls.key"' 'tls-replication yes' 'tls-auth-clients no' ) for config in "${expected_redis_config[@]}" do oc exec -i pod/argocd-redis-ha-server-0 -n $NAMESPACE -c redis -- grep "${config}" /data/conf/redis.conf done for config in "${expected_sentinel_config[@]}" do oc exec -i pod/argocd-redis-ha-server-0 -n $NAMESPACE -c redis -- grep "${config}" /data/conf/sentinel.conf done ] logger.go:42: 18:11:24 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | port 0 logger.go:42: 18:11:25 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | tls-port 6379 logger.go:42: 18:11:26 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | tls-cert-file /app/config/redis/tls/tls.crt logger.go:42: 18:11:26 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | tls-ca-cert-file /app/config/redis/tls/tls.crt logger.go:42: 18:11:27 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | tls-key-file /app/config/redis/tls/tls.key logger.go:42: 18:11:27 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | tls-replication yes logger.go:42: 18:11:28 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | tls-auth-clients no logger.go:42: 18:11:28 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | port 0 logger.go:42: 18:11:29 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | tls-port 26379 logger.go:42: 18:11:30 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | tls-cert-file "/app/config/redis/tls/tls.crt" logger.go:42: 18:11:30 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | tls-ca-cert-file "/app/config/redis/tls/tls.crt" logger.go:42: 18:11:31 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | tls-key-file "/app/config/redis/tls/tls.key" logger.go:42: 18:11:31 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | tls-replication yes logger.go:42: 18:11:32 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | tls-auth-clients no I0723 18:11:33.358948 167 request.go:682] Waited for 1.04576133s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/operator.openshift.io/v1alpha1?timeout=32s logger.go:42: 18:11:34 | 1-067_validate_redis_secure_comm_no_autotls_ha/5-check_redis_config | test step completed 5-check_redis_config logger.go:42: 18:11:34 | 1-067_validate_redis_secure_comm_no_autotls_ha/6-check_deployments | starting test step 6-check_deployments logger.go:42: 18:11:34 | 1-067_validate_redis_secure_comm_no_autotls_ha/6-check_deployments | running command: [sh -c set -e if test "$(oc get deployments.apps argocd-repo-server -n $NAMESPACE --template \ '{{range .spec.template.spec.containers}}{{.command}}{{"\n"}}{{end}}')" != \ "[uid_entrypoint.sh \ argocd-repo-server \ --redis argocd-redis-ha-haproxy.$NAMESPACE.svc.cluster.local:6379 \ --redis-use-tls \ --redis-ca-certificate /app/config/reposerver/tls/redis/tls.crt \ --loglevel info \ --logformat text]"; then echo "TLS .spec.template.spec.containers.command for argocd-repo-server deployment is wrong" exit 1 fi ] logger.go:42: 18:11:35 | 1-067_validate_redis_secure_comm_no_autotls_ha/6-check_deployments | running command: [sh -c set -e if test "$(oc get deployments.apps argocd-server -n $NAMESPACE --template \ '{{range .spec.template.spec.containers}}{{.command}}{{"\n"}}{{end}}')" != \ "[argocd-server \ --staticassets /shared/app \ --dex-server https://argocd-dex-server.$NAMESPACE.svc.cluster.local:5556 \ --repo-server argocd-repo-server.$NAMESPACE.svc.cluster.local:8081 \ --redis argocd-redis-ha-haproxy.$NAMESPACE.svc.cluster.local:6379 \ --redis-use-tls \ --redis-ca-certificate /app/config/server/tls/redis/tls.crt \ --loglevel info \ --logformat text]"; then echo "TLS .spec.template.spec.containers.command for argocd-server deployment is wrong" exit 1 fi ] logger.go:42: 18:11:38 | 1-067_validate_redis_secure_comm_no_autotls_ha/6-check_deployments | test step completed 6-check_deployments logger.go:42: 18:11:38 | 1-067_validate_redis_secure_comm_no_autotls_ha/7-check_statefulset | starting test step 7-check_statefulset logger.go:42: 18:11:38 | 1-067_validate_redis_secure_comm_no_autotls_ha/7-check_statefulset | running command: [sh -c set -e if test "$(oc get statefulsets.apps argocd-application-controller -n $NAMESPACE --template \ '{{range .spec.template.spec.containers}}{{.command}}{{"\n"}}{{end}}')" != \ "[argocd-application-controller \ --operation-processors 10 \ --redis argocd-redis-ha-haproxy.$NAMESPACE.svc.cluster.local:6379 \ --redis-use-tls \ --redis-ca-certificate /app/config/controller/tls/redis/tls.crt \ --repo-server argocd-repo-server.$NAMESPACE.svc.cluster.local:8081 \ --status-processors 20 \ --kubectl-parallelism-limit 10 \ --loglevel info \ --logformat text]"; then echo "TLS .spec.template.spec.containers.command for argocd-application-controller statefulsets is wrong" exit 1 fi ] logger.go:42: 18:11:41 | 1-067_validate_redis_secure_comm_no_autotls_ha/7-check_statefulset | test step completed 7-check_statefulset logger.go:42: 18:11:41 | 1-067_validate_redis_secure_comm_no_autotls_ha | skipping kubernetes event logging logger.go:42: 18:11:41 | 1-067_validate_redis_secure_comm_no_autotls_ha | Deleting namespace: kuttl-test-key-reindeer === CONT kuttl harness.go:405: run tests finished harness.go:513: cleaning up harness.go:570: removing temp folder: "" --- PASS: kuttl (2450.68s) --- PASS: kuttl/harness (0.00s) --- PASS: kuttl/harness/1-021_validate_rolebindings (22.54s) --- PASS: kuttl/harness/1-001_alpha_to_beta_dex_conversion (28.67s) --- PASS: kuttl/harness/1-083_validate_kustomize_namereference (48.43s) --- PASS: kuttl/harness/1-062_validate_extra_config (81.43s) --- PASS: kuttl/harness/1-103_argocd_alpha_to_beta_conversion (43.92s) --- PASS: kuttl/harness/1-107_host_attribute_sso_provider (121.98s) --- PASS: kuttl/harness/1-104_validate_applicationset_tls_scm_volume_mount (147.60s) --- PASS: kuttl/harness/1-036_validate_keycloak_resource_reqs (234.91s) --- PASS: kuttl/harness/1-102_validate_handle_terminating_namespaces (114.86s) --- PASS: kuttl/harness/1-096-validate_home_env_argocd_controller (8.21s) --- PASS: kuttl/harness/1-098_validate_dex_clientsecret_deprecated (55.38s) --- PASS: kuttl/harness/1-099_validate_server_autoscale (106.26s) --- PASS: kuttl/harness/1-090_validate_permissions (11.32s) --- PASS: kuttl/harness/1-108_alternate_cluster_roles_cluster_scoped_instance (250.89s) --- PASS: kuttl/harness/1-091_argocd_app_validate-applicationset-status (27.28s) --- PASS: kuttl/harness/1-095_validate_dex_clientsecret (41.01s) --- PASS: kuttl/harness/1-088_validate_applicationset_add_env_vars (28.42s) --- PASS: kuttl/harness/1-089_validate_extra_repo_commands_args (46.84s) --- PASS: kuttl/harness/1-087_validate_repo_server_settings (56.52s) --- PASS: kuttl/harness/1-086_validate_applicationset_extra_command_args (64.95s) --- PASS: kuttl/harness/1-084_validate_status_host_ingress (47.11s) --- PASS: kuttl/harness/1-077_validate_disable_dex_removed (9.47s) --- PASS: kuttl/harness/1-083_validate_resource_customization_subkeys (44.55s) --- PASS: kuttl/harness/1-082_validate_node_placement (22.52s) --- PASS: kuttl/harness/1-081_validate_applicationset_deployment (8.50s) --- PASS: kuttl/harness/1-074_validate_terminating_namespace_block (31.90s) --- PASS: kuttl/harness/1-075_validate_dex_anyuid (53.41s) --- PASS: kuttl/harness/1-072_validate_liveness_probe_removed (44.44s) --- PASS: kuttl/harness/1-080_validate_regex_support_argocd_rbac (55.49s) --- PASS: kuttl/harness/1-079_validate_vars_for_notificaitons (49.92s) --- PASS: kuttl/harness/1-007_validate_namespace_scoped_install (43.68s) --- PASS: kuttl/harness/1-019_validate_volume_mounts (40.59s) --- PASS: kuttl/harness/1-073_validate_rhsso (221.47s) --- PASS: kuttl/harness/1-050_validate_sso (183.91s) --- PASS: kuttl/harness/1-061_validate_resource_tracking_method (51.13s) --- PASS: kuttl/harness/1-058_validate_prometheus_rule (31.40s) --- PASS: kuttl/harness/1-092_validate_workload_status_monitoring_alert (452.72s) --- PASS: kuttl/harness/1-055_validate_notification_controller (73.94s) --- PASS: kuttl/harness/1-055_drop_resource_customizations (26.01s) --- PASS: kuttl/harness/1-057_validate_notifications (97.78s) --- PASS: kuttl/harness/1-053_validate_cluster_admin_rbac (20.17s) --- PASS: kuttl/harness/1-052_validate_rolebinding_number (13.26s) --- PASS: kuttl/harness/1-012_validate-managed-by-chain (128.15s) --- PASS: kuttl/harness/1-051-validate_csv_permissions (11.95s) --- PASS: kuttl/harness/1-054_validate_deploymentconfig (97.40s) --- PASS: kuttl/harness/1-009_validate-manage-other-namespace (96.52s) --- PASS: kuttl/harness/1-044_validate_resource_limit_changes (59.79s) --- PASS: kuttl/harness/1-008_validate-custom-argocd-namespace (77.59s) --- PASS: kuttl/harness/1-033_validate_resource_exclusions (25.17s) --- PASS: kuttl/harness/1-049_validate_parallelism_limit (68.50s) --- PASS: kuttl/harness/1-048_validate_controller_sharding (73.46s) --- PASS: kuttl/harness/1-034_validate_webhook_notifications (107.71s) --- PASS: kuttl/harness/1-033_validate_applicationset_tls_scm_volume_mount (40.62s) --- PASS: kuttl/harness/1-071_validate_SCC_HA (622.24s) --- PASS: kuttl/harness/1-029_validate_tls_secret_no_scale (158.63s) --- PASS: kuttl/harness/1-045_validate_repo_exec_timeout (194.57s) --- PASS: kuttl/harness/1-032_validate_resource_inclusions (206.62s) --- PASS: kuttl/harness/1-047_validate_custom_env (222.01s) --- PASS: kuttl/harness/1-064_validate_security_contexts (16.30s) --- PASS: kuttl/harness/1-030_validate_reencrypt (53.63s) --- PASS: kuttl/harness/1-070_validate_config_management_plugin (68.33s) --- PASS: kuttl/harness/1-066_validate_redis_secure_comm_no_autotls_no_ha (99.03s) --- PASS: kuttl/harness/1-025-validate-managed-by-change (76.61s) --- PASS: kuttl/harness/1-068_validate_redis_secure_comm_autotls_no_ha (69.27s) --- PASS: kuttl/harness/1-032_validate_dynamic_scaling (479.33s) --- PASS: kuttl/harness/1-003_alpha_to_beta_sso_conflict_conversion (47.02s) --- PASS: kuttl/harness/1-004_beta_to_alpha_conversion (24.00s) --- PASS: kuttl/harness/1-039_validate_fix_argocd-tls-certs-cm (48.46s) --- PASS: kuttl/harness/1-063_validate_statefulset_restart (56.75s) --- PASS: kuttl/harness/1-063_validate_dex_liveness_probe (8.44s) --- PASS: kuttl/harness/1-043_validate_log_level_format (44.65s) --- PASS: kuttl/harness/1-002_alpha_to_beta_keycloak_conversion (43.56s) --- PASS: kuttl/harness/1-038_validate_productized_images (44.29s) --- PASS: kuttl/harness/1-037_validate_argocd_setting_replicas (82.44s) --- PASS: kuttl/harness/1-003_validate_console_link (8.29s) --- PASS: kuttl/harness/1-023_validate_repo_server_tls (71.97s) --- PASS: kuttl/harness/1-065_validate_redis_ha_anti_affinity (591.23s) --- PASS: kuttl/harness/1-069_validate_redis_secure_comm_autotls_ha (1139.12s) --- PASS: kuttl/harness/1-067_validate_redis_secure_comm_no_autotls_ha (1158.92s) PASS Running sequential test suite 2024/07/23 18:12:27 kutt-test config testdirs is overridden with args: [ /kuttl-e2e/gitops-operator/tests/sequential ] === RUN kuttl harness.go:462: starting setup harness.go:252: running tests using configured kubeconfig. I0723 18:12:32.563448 5681 request.go:682] Waited for 1.047682623s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/machine.openshift.io/v1beta1?timeout=32s harness.go:275: Successful connection to cluster at: https://api.test-aws-412.ocp-gitops-qe.com:6443 harness.go:360: running tests harness.go:73: going to run test suite with timeout of 600 seconds for each step harness.go:372: testsuite: /kuttl-e2e/gitops-operator/tests/sequential has 36 tests === RUN kuttl/harness === RUN kuttl/harness/1-001_validate_kam_service === PAUSE kuttl/harness/1-001_validate_kam_service === RUN kuttl/harness/1-002_validate_backend_service === PAUSE kuttl/harness/1-002_validate_backend_service === RUN kuttl/harness/1-003_validate_cluster_config === PAUSE kuttl/harness/1-003_validate_cluster_config === RUN kuttl/harness/1-004_validate_argocd_installation === PAUSE kuttl/harness/1-004_validate_argocd_installation === RUN kuttl/harness/1-005_validate_metrics === PAUSE kuttl/harness/1-005_validate_metrics === RUN kuttl/harness/1-006_validate_machine_config === PAUSE kuttl/harness/1-006_validate_machine_config === RUN kuttl/harness/1-010_validate-ootb-manage-other-namespace === PAUSE kuttl/harness/1-010_validate-ootb-manage-other-namespace === RUN kuttl/harness/1-018_validate_disable_default_instance === PAUSE kuttl/harness/1-018_validate_disable_default_instance === RUN kuttl/harness/1-020_validate_redis_ha_nonha === PAUSE kuttl/harness/1-020_validate_redis_ha_nonha === RUN kuttl/harness/1-026-validate_backend_service_permissions === PAUSE kuttl/harness/1-026-validate_backend_service_permissions === RUN kuttl/harness/1-027_validate_operand_from_git === PAUSE kuttl/harness/1-027_validate_operand_from_git === RUN kuttl/harness/1-028_validate_run_on_infra === PAUSE kuttl/harness/1-028_validate_run_on_infra === RUN kuttl/harness/1-031_validate_toolchain === PAUSE kuttl/harness/1-031_validate_toolchain === RUN kuttl/harness/1-034_validate_custom_roles === PAUSE kuttl/harness/1-034_validate_custom_roles === RUN kuttl/harness/1-035_validate_argocd_secret_repopulate === PAUSE kuttl/harness/1-035_validate_argocd_secret_repopulate === RUN kuttl/harness/1-036_validate_role_rolebinding_for_source_namespace === PAUSE kuttl/harness/1-036_validate_role_rolebinding_for_source_namespace === RUN kuttl/harness/1-037_validate_applicationset_in_any_namespace === PAUSE kuttl/harness/1-037_validate_applicationset_in_any_namespace === RUN kuttl/harness/1-040_validate_quoted_RBAC_group_names === PAUSE kuttl/harness/1-040_validate_quoted_RBAC_group_names === RUN kuttl/harness/1-041_validate_argocd_sync_alert === PAUSE kuttl/harness/1-041_validate_argocd_sync_alert === RUN kuttl/harness/1-042_validate_status_host === PAUSE kuttl/harness/1-042_validate_status_host === RUN kuttl/harness/1-050_validate_sso === PAUSE kuttl/harness/1-050_validate_sso === RUN kuttl/harness/1-056_validate_managed-by === PAUSE kuttl/harness/1-056_validate_managed-by === RUN kuttl/harness/1-064_validate_tcp_reset_error === PAUSE kuttl/harness/1-064_validate_tcp_reset_error === RUN kuttl/harness/1-071_validate_node_selectors === PAUSE kuttl/harness/1-071_validate_node_selectors === RUN kuttl/harness/1-078_validate_default_argocd_consoleLink === PAUSE kuttl/harness/1-078_validate_default_argocd_consoleLink === RUN kuttl/harness/1-083_validate_apps_in_any_namespace === PAUSE kuttl/harness/1-083_validate_apps_in_any_namespace === RUN kuttl/harness/1-084_validate_prune_templates === PAUSE kuttl/harness/1-084_validate_prune_templates === RUN kuttl/harness/1-085_validate_dynamic_plugin_installation === PAUSE kuttl/harness/1-085_validate_dynamic_plugin_installation === RUN kuttl/harness/1-086_validate_default_argocd_role === PAUSE kuttl/harness/1-086_validate_default_argocd_role === RUN kuttl/harness/1-100_validate_rollouts_resources_creation === PAUSE kuttl/harness/1-100_validate_rollouts_resources_creation === RUN kuttl/harness/1-101_validate_rollout_policyrules === PAUSE kuttl/harness/1-101_validate_rollout_policyrules === RUN kuttl/harness/1-104_validate_prometheus_alert === PAUSE kuttl/harness/1-104_validate_prometheus_alert === RUN kuttl/harness/1-105_validate_label_selector === PAUSE kuttl/harness/1-105_validate_label_selector === RUN kuttl/harness/1-106_validate_argocd_metrics_controller === PAUSE kuttl/harness/1-106_validate_argocd_metrics_controller === RUN kuttl/harness/1-109_validate_reencrypt_termination_policy === PAUSE kuttl/harness/1-109_validate_reencrypt_termination_policy === RUN kuttl/harness/1-110_validate_podsecurity_alerts === PAUSE kuttl/harness/1-110_validate_podsecurity_alerts === CONT kuttl/harness/1-001_validate_kam_service logger.go:42: 18:12:38 | 1-001_validate_kam_service | Creating namespace: kuttl-test-enabling-antelope logger.go:42: 18:12:38 | 1-001_validate_kam_service/1-check-kam-service | starting test step 1-check-kam-service logger.go:42: 18:12:41 | 1-001_validate_kam_service/1-check-kam-service | test step completed 1-check-kam-service logger.go:42: 18:12:41 | 1-001_validate_kam_service/2-check-route | starting test step 2-check-route logger.go:42: 18:12:41 | 1-001_validate_kam_service/2-check-route | running command: [sh -c status=$(oc get -n openshift-gitops route kam -o json | jq -r .status) route_admitted=$(echo "$status" | jq -r '.ingress[0].conditions[0].type') route_status=$(echo "$status" | jq -r '.ingress[0].conditions[0].status') route_host=$(echo "$status" | jq -r '.ingress[0].host') router_name=$(echo "$status" | jq -r '.ingress[0].routerName') route_wildcard_policy=$(echo "$status" | jq -r '.ingress[0].wildcardPolicy') if test "$route_admitted" != "Admitted" -o "$route_status" != "True"; then echo "Route not ready" exit 1 fi if test "$route_host" = ""; then echo "Route has no host" exit 1 fi if test "$router_name" != "default"; then echo "Router is not default" exit 1 fi if test "$route_wildcard_policy" != "None"; then echo "Route wildcard policy is not None" exit 1 fi exit 0 ] I0723 18:12:42.926107 5681 request.go:682] Waited for 1.022435068s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/autoscaling.openshift.io/v1beta1?timeout=32s logger.go:42: 18:12:44 | 1-001_validate_kam_service/2-check-route | test step completed 2-check-route logger.go:42: 18:12:44 | 1-001_validate_kam_service/3-check-download | starting test step 3-check-download logger.go:42: 18:12:44 | 1-001_validate_kam_service/3-check-download | running command: [sh -c set -e base_url=$(oc get -n openshift-gitops route kam -o json | jq -r '.status.ingress[0].host') curl -k --silent -o/dev/null "${base_url}/kam/kam_linux_amd64" curl -k --silent -o/dev/null "${base_url}/kam/kam_darwin_amd64" curl -k --silent -o/dev/null "${base_url}/kam/kam_windows_amd64.exe" ] logger.go:42: 18:12:47 | 1-001_validate_kam_service/3-check-download | test step completed 3-check-download logger.go:42: 18:12:47 | 1-001_validate_kam_service | skipping kubernetes event logging logger.go:42: 18:12:47 | 1-001_validate_kam_service | Deleting namespace: kuttl-test-enabling-antelope === CONT kuttl/harness/1-041_validate_argocd_sync_alert logger.go:42: 18:12:53 | 1-041_validate_argocd_sync_alert | Creating namespace: kuttl-test-evolved-weevil logger.go:42: 18:12:53 | 1-041_validate_argocd_sync_alert/1- | starting test step 1- I0723 18:12:54.170357 5681 request.go:682] Waited for 1.04457834s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/network.operator.openshift.io/v1?timeout=32s logger.go:42: 18:12:55 | 1-041_validate_argocd_sync_alert/1- | test step completed 1- logger.go:42: 18:12:55 | 1-041_validate_argocd_sync_alert | skipping kubernetes event logging logger.go:42: 18:12:55 | 1-041_validate_argocd_sync_alert | Deleting namespace: kuttl-test-evolved-weevil === CONT kuttl/harness/1-110_validate_podsecurity_alerts logger.go:42: 18:13:01 | 1-110_validate_podsecurity_alerts | Creating namespace: kuttl-test-enormous-cockatoo logger.go:42: 18:13:01 | 1-110_validate_podsecurity_alerts/1- | starting test step 1- logger.go:42: 18:13:03 | 1-110_validate_podsecurity_alerts/1- | test step completed 1- logger.go:42: 18:13:03 | 1-110_validate_podsecurity_alerts | skipping kubernetes event logging logger.go:42: 18:13:03 | 1-110_validate_podsecurity_alerts | Deleting namespace: kuttl-test-enormous-cockatoo === CONT kuttl/harness/1-109_validate_reencrypt_termination_policy logger.go:42: 18:13:09 | 1-109_validate_reencrypt_termination_policy | Creating namespace: kuttl-test-moved-finch logger.go:42: 18:13:09 | 1-109_validate_reencrypt_termination_policy/1- | starting test step 1- I0723 18:13:10.652856 5681 request.go:682] Waited for 1.045271147s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/flowcontrol.apiserver.k8s.io/v1beta1?timeout=32s logger.go:42: 18:13:12 | 1-109_validate_reencrypt_termination_policy/1- | test step completed 1- logger.go:42: 18:13:12 | 1-109_validate_reencrypt_termination_policy/2-check-certificate | starting test step 2-check-certificate logger.go:42: 18:13:12 | 1-109_validate_reencrypt_termination_policy/2-check-certificate | running command: [sh -c route=https://$(oc get route -n openshift-gitops openshift-gitops-server -o jsonpath='{.spec.host}') issuer=$(curl --insecure -v "$route" 2>&1 | awk '/issuer:/') if [ $issuer != *"root-ca"* ] && [ $issuer != *"ingress-operator"* ]; then echo "TLS connections rely on the default ingress operator certificate or root-ca" exit 1 fi ] logger.go:42: 18:13:12 | 1-109_validate_reencrypt_termination_policy/2-check-certificate | sh: line 4: [: too many arguments logger.go:42: 18:13:15 | 1-109_validate_reencrypt_termination_policy/2-check-certificate | test step completed 2-check-certificate logger.go:42: 18:13:15 | 1-109_validate_reencrypt_termination_policy | skipping kubernetes event logging logger.go:42: 18:13:15 | 1-109_validate_reencrypt_termination_policy | Deleting namespace: kuttl-test-moved-finch === CONT kuttl/harness/1-083_validate_apps_in_any_namespace logger.go:42: 18:13:20 | 1-083_validate_apps_in_any_namespace | Ignoring errors.yaml as it does not match file name regexp: ^(\d+)-(?:[^\.]+)(?:\.yaml)?$ logger.go:42: 18:13:20 | 1-083_validate_apps_in_any_namespace | Creating namespace: kuttl-test-touched-lark logger.go:42: 18:13:20 | 1-083_validate_apps_in_any_namespace/1-create-ns-managed-by | starting test step 1-create-ns-managed-by I0723 18:13:21.999643 5681 request.go:682] Waited for 1.042437933s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/rbac.authorization.k8s.io/v1?timeout=32s logger.go:42: 18:13:23 | 1-083_validate_apps_in_any_namespace/1-create-ns-managed-by | Namespace:/test-1-24-custom created logger.go:42: 18:13:24 | 1-083_validate_apps_in_any_namespace/1-create-ns-managed-by | test step completed 1-create-ns-managed-by logger.go:42: 18:13:24 | 1-083_validate_apps_in_any_namespace/2-add-source-ns | starting test step 2-add-source-ns logger.go:42: 18:13:27 | 1-083_validate_apps_in_any_namespace/2-add-source-ns | ArgoCD:openshift-gitops/openshift-gitops updated logger.go:42: 18:13:27 | 1-083_validate_apps_in_any_namespace/2-add-source-ns | test step completed 2-add-source-ns logger.go:42: 18:13:27 | 1-083_validate_apps_in_any_namespace/3-delete | starting test step 3-delete I0723 18:13:34.044644 5681 request.go:682] Waited for 1.039542251s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/operators.coreos.com/v1alpha2?timeout=32s logger.go:42: 18:13:35 | 1-083_validate_apps_in_any_namespace/3-delete | test step completed 3-delete logger.go:42: 18:13:35 | 1-083_validate_apps_in_any_namespace/4-change-source-namespaces | starting test step 4-change-source-namespaces logger.go:42: 18:13:37 | 1-083_validate_apps_in_any_namespace/4-change-source-namespaces | ArgoCD:openshift-gitops/openshift-gitops updated logger.go:42: 18:13:38 | 1-083_validate_apps_in_any_namespace/4-change-source-namespaces | test step completed 4-change-source-namespaces logger.go:42: 18:13:38 | 1-083_validate_apps_in_any_namespace/5-create-ns-and-add-source-ns | starting test step 5-create-ns-and-add-source-ns logger.go:42: 18:13:41 | 1-083_validate_apps_in_any_namespace/5-create-ns-and-add-source-ns | Namespace:/test-1-24-custom created logger.go:42: 18:13:41 | 1-083_validate_apps_in_any_namespace/5-create-ns-and-add-source-ns | ArgoCD:openshift-gitops/openshift-gitops updated logger.go:42: 18:13:50 | 1-083_validate_apps_in_any_namespace/5-create-ns-and-add-source-ns | test step completed 5-create-ns-and-add-source-ns logger.go:42: 18:13:50 | 1-083_validate_apps_in_any_namespace/6-add-managed-by | starting test step 6-add-managed-by I0723 18:13:51.909703 5681 request.go:682] Waited for 1.048132112s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/quota.openshift.io/v1?timeout=32s logger.go:42: 18:13:53 | 1-083_validate_apps_in_any_namespace/6-add-managed-by | Namespace:/test-1-24-custom updated logger.go:42: 18:13:54 | 1-083_validate_apps_in_any_namespace/6-add-managed-by | test step completed 6-add-managed-by logger.go:42: 18:13:54 | 1-083_validate_apps_in_any_namespace/7-remove-managed-by | starting test step 7-remove-managed-by logger.go:42: 18:13:54 | 1-083_validate_apps_in_any_namespace/7-remove-managed-by | running command: [sh -c oc label ns test-1-24-custom argocd.argoproj.io/managed-by-] logger.go:42: 18:13:54 | 1-083_validate_apps_in_any_namespace/7-remove-managed-by | namespace/test-1-24-custom unlabeled logger.go:42: 18:13:54 | 1-083_validate_apps_in_any_namespace/7-remove-managed-by | running command: [sh -c oc rollout restart deployment.apps/openshift-gitops-server -n openshift-gitops] logger.go:42: 18:13:55 | 1-083_validate_apps_in_any_namespace/7-remove-managed-by | deployment.apps/openshift-gitops-server restarted logger.go:42: 18:13:55 | 1-083_validate_apps_in_any_namespace/7-remove-managed-by | running command: [sh -c oc rollout restart statefulset.apps/openshift-gitops-application-controller -n openshift-gitops] logger.go:42: 18:13:55 | 1-083_validate_apps_in_any_namespace/7-remove-managed-by | statefulset.apps/openshift-gitops-application-controller restarted logger.go:42: 18:13:56 | 1-083_validate_apps_in_any_namespace/7-remove-managed-by | running command: [sh -c sleep 5] I0723 18:14:02.061772 5681 request.go:682] Waited for 1.047311852s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/performance.openshift.io/v2?timeout=32s logger.go:42: 18:14:26 | 1-083_validate_apps_in_any_namespace/7-remove-managed-by | test step completed 7-remove-managed-by logger.go:42: 18:14:26 | 1-083_validate_apps_in_any_namespace/8-change-source-namespaces | starting test step 8-change-source-namespaces I0723 18:14:27.893376 5681 request.go:682] Waited for 1.048300948s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/packages.operators.coreos.com/v1?timeout=32s logger.go:42: 18:14:29 | 1-083_validate_apps_in_any_namespace/8-change-source-namespaces | ArgoCD:openshift-gitops/openshift-gitops updated logger.go:42: 18:14:30 | 1-083_validate_apps_in_any_namespace/8-change-source-namespaces | test step completed 8-change-source-namespaces logger.go:42: 18:14:30 | 1-083_validate_apps_in_any_namespace/99-delete | starting test step 99-delete I0723 18:14:37.932484 5681 request.go:682] Waited for 1.848796109s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/snapshot.storage.k8s.io/v1?timeout=32s logger.go:42: 18:14:38 | 1-083_validate_apps_in_any_namespace/99-delete | test step completed 99-delete logger.go:42: 18:14:38 | 1-083_validate_apps_in_any_namespace | skipping kubernetes event logging logger.go:42: 18:14:39 | 1-083_validate_apps_in_any_namespace | Deleting namespace: kuttl-test-touched-lark === CONT kuttl/harness/1-078_validate_default_argocd_consoleLink logger.go:42: 18:14:45 | 1-078_validate_default_argocd_consoleLink | Creating namespace: kuttl-test-verified-unicorn logger.go:42: 18:14:45 | 1-078_validate_default_argocd_consoleLink/1- | starting test step 1- logger.go:42: 18:14:47 | 1-078_validate_default_argocd_consoleLink/1- | test step completed 1- logger.go:42: 18:14:47 | 1-078_validate_default_argocd_consoleLink/2-add_env | starting test step 2-add_env logger.go:42: 18:14:47 | 1-078_validate_default_argocd_consoleLink/2-add_env | running command: [sh -c oc patch -n openshift-operators subscription openshift-gitops-operator \ --type merge --patch '{"spec": {"config": {"env": [{"name": "DISABLE_DEFAULT_ARGOCD_CONSOLELINK", "value": "true"}]}}}' ] logger.go:42: 18:14:47 | 1-078_validate_default_argocd_consoleLink/2-add_env | subscription.operators.coreos.com/openshift-gitops-operator patched I0723 18:14:49.057434 5681 request.go:682] Waited for 1.047865547s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/network.openshift.io/v1?timeout=32s logger.go:42: 18:15:16 | 1-078_validate_default_argocd_consoleLink/2-add_env | test step completed 2-add_env logger.go:42: 18:15:16 | 1-078_validate_default_argocd_consoleLink/3-disable-env | starting test step 3-disable-env logger.go:42: 18:15:16 | 1-078_validate_default_argocd_consoleLink/3-disable-env | running command: [sh -c oc patch -n openshift-operators subscription openshift-gitops-operator \ --type merge --patch '{"spec": {"config": {"env": [{"name": "DISABLE_DEFAULT_ARGOCD_CONSOLELINK", "value": "false"}]}}}' ] logger.go:42: 18:15:18 | 1-078_validate_default_argocd_consoleLink/3-disable-env | subscription.operators.coreos.com/openshift-gitops-operator patched I0723 18:15:19.294412 5681 request.go:682] Waited for 1.047210992s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/autoscaling.openshift.io/v1?timeout=32s logger.go:42: 18:15:46 | 1-078_validate_default_argocd_consoleLink/3-disable-env | test step completed 3-disable-env logger.go:42: 18:15:46 | 1-078_validate_default_argocd_consoleLink/4-empty-valued-env | starting test step 4-empty-valued-env logger.go:42: 18:15:46 | 1-078_validate_default_argocd_consoleLink/4-empty-valued-env | running command: [sh -c oc patch -n openshift-operators subscription openshift-gitops-operator \ --type merge --patch '{"spec": {"config": {"env": [{"name": "DISABLE_DEFAULT_ARGOCD_CONSOLELINK", "value": ""}]}}}' ] logger.go:42: 18:15:47 | 1-078_validate_default_argocd_consoleLink/4-empty-valued-env | subscription.operators.coreos.com/openshift-gitops-operator patched I0723 18:15:48.386277 5681 request.go:682] Waited for 1.04866963s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/apiextensions.k8s.io/v1?timeout=32s logger.go:42: 18:15:49 | 1-078_validate_default_argocd_consoleLink/4-empty-valued-env | test step completed 4-empty-valued-env logger.go:42: 18:15:49 | 1-078_validate_default_argocd_consoleLink/5-remove-env | starting test step 5-remove-env logger.go:42: 18:15:49 | 1-078_validate_default_argocd_consoleLink/5-remove-env | running command: [sh -c oc -n openshift-operators patch subscription openshift-gitops-operator --type='json' -p='[{"op": "remove", "path": "/spec/config" }]' ] logger.go:42: 18:15:50 | 1-078_validate_default_argocd_consoleLink/5-remove-env | subscription.operators.coreos.com/openshift-gitops-operator patched logger.go:42: 18:15:52 | 1-078_validate_default_argocd_consoleLink/5-remove-env | test step completed 5-remove-env logger.go:42: 18:15:52 | 1-078_validate_default_argocd_consoleLink | skipping kubernetes event logging logger.go:42: 18:15:52 | 1-078_validate_default_argocd_consoleLink | Deleting namespace: kuttl-test-verified-unicorn === CONT kuttl/harness/1-071_validate_node_selectors logger.go:42: 18:15:58 | 1-071_validate_node_selectors | Creating namespace: kuttl-test-main-dolphin logger.go:42: 18:15:58 | 1-071_validate_node_selectors/1- | starting test step 1- I0723 18:15:59.951404 5681 request.go:682] Waited for 1.043343519s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/quota.openshift.io/v1?timeout=32s logger.go:42: 18:16:01 | 1-071_validate_node_selectors/1- | test step completed 1- logger.go:42: 18:16:01 | 1-071_validate_node_selectors/2-custom-node-selector | starting test step 2-custom-node-selector logger.go:42: 18:16:01 | 1-071_validate_node_selectors/2-custom-node-selector | running command: [sh -c oc patch gitopsservice cluster --type=merge --patch='{"spec": {"nodeSelector": {"key1": "value1"}}}' ] logger.go:42: 18:16:01 | 1-071_validate_node_selectors/2-custom-node-selector | gitopsservice.pipelines.openshift.io/cluster patched logger.go:42: 18:16:20 | 1-071_validate_node_selectors/2-custom-node-selector | test step completed 2-custom-node-selector logger.go:42: 18:16:20 | 1-071_validate_node_selectors/3-patch-run-on-infra | starting test step 3-patch-run-on-infra logger.go:42: 18:16:20 | 1-071_validate_node_selectors/3-patch-run-on-infra | running command: [sh -c oc patch gitopsservice cluster --type=merge --patch='{"spec": {"runOnInfra": true, "tolerations": [{"effect": "NoSchedule", "key": "infra", "value": "reserved"}]}}' ] logger.go:42: 18:16:21 | 1-071_validate_node_selectors/3-patch-run-on-infra | gitopsservice.pipelines.openshift.io/cluster patched I0723 18:16:22.093656 5681 request.go:682] Waited for 1.036627874s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/security.internal.openshift.io/v1?timeout=32s logger.go:42: 18:16:23 | 1-071_validate_node_selectors/3-patch-run-on-infra | test step completed 3-patch-run-on-infra logger.go:42: 18:16:23 | 1-071_validate_node_selectors/4-change-back | starting test step 4-change-back logger.go:42: 18:16:23 | 1-071_validate_node_selectors/4-change-back | running command: [sh -c oc patch gitopsservice cluster --type=json --patch='[{"op": "remove", "path": "/spec/tolerations"}, {"op": "remove", "path": "/spec/runOnInfra"}, {"op": "remove", "path": "/spec/nodeSelector"}]' oc patch statefulset openshift-gitops-application-controller -n openshift-gitops -p '{"spec":{"replicas":0}}' ] logger.go:42: 18:16:24 | 1-071_validate_node_selectors/4-change-back | gitopsservice.pipelines.openshift.io/cluster patched logger.go:42: 18:16:24 | 1-071_validate_node_selectors/4-change-back | statefulset.apps/openshift-gitops-application-controller patched logger.go:42: 18:16:27 | 1-071_validate_node_selectors/4-change-back | test step completed 4-change-back logger.go:42: 18:16:27 | 1-071_validate_node_selectors | skipping kubernetes event logging logger.go:42: 18:16:27 | 1-071_validate_node_selectors | Deleting namespace: kuttl-test-main-dolphin === CONT kuttl/harness/1-084_validate_prune_templates logger.go:42: 18:16:32 | 1-084_validate_prune_templates | Creating namespace: kuttl-test-uncommon-drake logger.go:42: 18:16:32 | 1-084_validate_prune_templates/1-create-ns-managed-by | starting test step 1-create-ns-managed-by I0723 18:16:33.906083 5681 request.go:682] Waited for 1.048239326s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/project.openshift.io/v1?timeout=32s logger.go:42: 18:16:35 | 1-084_validate_prune_templates/1-create-ns-managed-by | Namespace:/namespace-gitops-1672 created logger.go:42: 18:16:35 | 1-084_validate_prune_templates/1-create-ns-managed-by | test step completed 1-create-ns-managed-by logger.go:42: 18:16:35 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | starting test step 2-prune-duplicateGVK-resource logger.go:42: 18:16:35 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | running command: [sh -c set -eou pipefail DIR="$(pwd)" cleanup() { argocd logout "${api_server}" oc delete application app-kustomize -n openshift-gitops rm -rf $WORK_DIR } WORK_DIR=`mktemp -d -p "$DIR"` # check if tmp dir was created if [[ ! "$WORK_DIR" || ! -d "$WORK_DIR" ]]; then echo "Could not create temp dir" exit 1 fi trap cleanup INT TERM EXIT # Login to apiserver api_server=$(oc get routes -n openshift-gitops --field-selector metadata.name=openshift-gitops-server -o jsonpath="{.items[*]['spec.host']}") password=$(oc get secret openshift-gitops-cluster -n openshift-gitops -o jsonpath='{.data.admin\.password}' | base64 -d) output=$(argocd login $api_server --username admin --password $password --insecure) # ensure namespace exists oc get ns namespace-gitops-1672 2>/dev/null || { oc new-project namespace-gitops-1672 oc label namespace namespace-gitops-1672 argocd.argoproj.io/managed-by=openshift-gitops } # ensure template cleanup, if exists in namespace before we start tests oc delete template redis-template-gitops -n namespace-gitops-1672 --ignore-not-found=true # Adding duplicate GVK to check prune functionality against issue #GITOPS-1672 cat > $WORK_DIR/app-template.yaml << EOF apiVersion: template.openshift.io/v1 kind: Template metadata: name: redis-template-gitops annotations: description: "Description" iconClass: "icon-redis" tags: "database,nosql" objects: - apiVersion: v1 kind: Pod metadata: name: redis-master spec: containers: - env: - name: REDIS_PASSWORD value: xyz1234s image: dockerfile/redis name: master ports: - containerPort: 6379 protocol: TCP parameters: - description: Password used for Redis authentication from: '[A-Z0-9]{8}' generate: expression name: REDIS_PASSWORD labels: redis: master EOF cat > $WORK_DIR/app-template2.yaml << EOF apiVersion: template.openshift.io/v1 kind: Template metadata: name: redis-template-gitops2 annotations: description: "Description" iconClass: "icon-redis" tags: "database,nosql" objects: - apiVersion: v1 kind: Pod metadata: name: redis-master spec: containers: - env: - name: REDIS_PASSWORD value: xyz1234s image: dockerfile/redis name: master ports: - containerPort: 6379 protocol: TCP parameters: - description: Password used for Redis authentication from: '[A-Z0-9]{8}' generate: expression name: REDIS_PASSWORD labels: redis: master EOF # create application.yaml cat </dev/null; then echo "Application listing failed" exit 1 fi if ! argocd app sync app-kustomize --local $WORK_DIR --timeout 100 2>/dev/null; then echo "App didn't sync well with local flag" exit 1 fi # Now delete resource from git source and try to resync back rm -rf $WORK_DIR/app-template.yaml sleep 5 # Now sync application with prune and PruneLast options enabled if ! argocd app sync app-kustomize --local $WORK_DIR --prune --timeout 100 2>/dev/null; then echo "App didn't prune or synced well with local flag" exit 1 fi if oc get template redis-template-gitops -n namespace-gitops-1672 --no-headers 2>/dev/null; then echo "Templete is not pruned properly." exit 1 else echo "Test is successful, no templates found" exit 0 fi ] logger.go:42: 18:16:36 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | time="2024-07-23T18:16:36Z" level=warning msg="Failed to invoke grpc call. Use flag --grpc-web in grpc calls. To avoid this warning message, use flag --grpc-web." logger.go:42: 18:16:37 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | NAME STATUS AGE logger.go:42: 18:16:37 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | namespace-gitops-1672 Active 2s logger.go:42: 18:16:40 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | application.argoproj.io/app-kustomize created logger.go:42: 18:16:45 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | NAME CLUSTER NAMESPACE PROJECT STATUS HEALTH SYNCPOLICY CONDITIONS REPO PATH TARGET logger.go:42: 18:16:45 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | openshift-gitops/app-kustomize https://kubernetes.default.svc namespace-gitops-1672 default Unknown Healthy Manual ComparisonError file:///kuttl-e2e/gitops-operator/tests/sequential/1-084_validate_prune_templates/tmp.giHAIcobVp.git . HEAD logger.go:42: 18:16:45 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | openshift-gitops/builds-config https://kubernetes.default.svc openshift-gitops default Synced Healthy Auto-Prune https://github.com/openshift/openshift-gitops-examples.git argocd/builds/base HEAD logger.go:42: 18:16:45 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | openshift-gitops/console-config https://kubernetes.default.svc openshift-gitops default Synced Healthy Auto-Prune https://github.com/openshift/openshift-gitops-examples.git argocd/console HEAD logger.go:42: 18:16:45 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | openshift-gitops/image-config https://kubernetes.default.svc openshift-gitops default Synced Healthy Auto-Prune https://github.com/openshift/openshift-gitops-examples.git argocd/image HEAD logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | TIMESTAMP GROUP KIND NAMESPACE NAME STATUS HEALTH HOOK MESSAGE logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | 2024-07-23T18:16:46+00:00 template.openshift.io Template namespace-gitops-1672 redis-template-gitops Running Synced template.template.openshift.io/redis-template-gitops created logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | 2024-07-23T18:16:46+00:00 template.openshift.io Template namespace-gitops-1672 redis-template-gitops2 Running Synced template.template.openshift.io/redis-template-gitops2 created logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | 2024-07-23T18:16:46+00:00 template.openshift.io Template namespace-gitops-1672 redis-template-gitops Synced Synced template.template.openshift.io/redis-template-gitops created logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | 2024-07-23T18:16:46+00:00 template.openshift.io Template namespace-gitops-1672 redis-template-gitops2 OutOfSync Missing template.template.openshift.io/redis-template-gitops2 created logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Name: openshift-gitops/app-kustomize logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Project: default logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Server: https://kubernetes.default.svc logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Namespace: namespace-gitops-1672 logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | URL: https://openshift-gitops-server-openshift-gitops.apps.test-aws-412.ocp-gitops-qe.com/applications/app-kustomize logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Source: logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | - Repo: file:///kuttl-e2e/gitops-operator/tests/sequential/1-084_validate_prune_templates/tmp.giHAIcobVp.git logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Target: HEAD logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Path: . logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | SyncWindow: Sync Allowed logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Sync Policy: Manual logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Sync Status: Synced to HEAD logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Health Status: Healthy logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Operation: Sync logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Sync Revision: logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Phase: Succeeded logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Start: 2024-07-23 18:16:46 +0000 UTC logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Finished: 2024-07-23 18:16:46 +0000 UTC logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Duration: 0s logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Message: successfully synced (all tasks run) logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | GROUP KIND NAMESPACE NAME STATUS HEALTH HOOK MESSAGE logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | template.openshift.io Template namespace-gitops-1672 redis-template-gitops Synced template.template.openshift.io/redis-template-gitops created logger.go:42: 18:16:46 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | template.openshift.io Template namespace-gitops-1672 redis-template-gitops2 Synced template.template.openshift.io/redis-template-gitops2 created logger.go:42: 18:16:52 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | TIMESTAMP GROUP KIND NAMESPACE NAME STATUS HEALTH HOOK MESSAGE logger.go:42: 18:16:52 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | 2024-07-23T18:16:52+00:00 template.openshift.io Template namespace-gitops-1672 redis-template-gitops Synced logger.go:42: 18:16:52 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | 2024-07-23T18:16:52+00:00 template.openshift.io Template namespace-gitops-1672 redis-template-gitops2 Synced logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | 2024-07-23T18:16:54+00:00 template.openshift.io Template namespace-gitops-1672 redis-template-gitops2 Synced template.template.openshift.io/redis-template-gitops2 unchanged logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | 2024-07-23T18:16:54+00:00 template.openshift.io Template namespace-gitops-1672 redis-template-gitops Synced pruned logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | 2024-07-23T18:16:54+00:00 template.openshift.io Template namespace-gitops-1672 redis-template-gitops OutOfSync Progressing pruned logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Name: openshift-gitops/app-kustomize logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Project: default logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Server: https://kubernetes.default.svc logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Namespace: namespace-gitops-1672 logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | URL: https://openshift-gitops-server-openshift-gitops.apps.test-aws-412.ocp-gitops-qe.com/applications/app-kustomize logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Source: logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | - Repo: file:///kuttl-e2e/gitops-operator/tests/sequential/1-084_validate_prune_templates/tmp.giHAIcobVp.git logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Target: HEAD logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Path: . logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | SyncWindow: Sync Allowed logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Sync Policy: Manual logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Sync Status: Synced to HEAD logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Health Status: Healthy logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Operation: Sync logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Sync Revision: logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Phase: Succeeded logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Start: 2024-07-23 18:16:52 +0000 UTC logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Finished: 2024-07-23 18:16:54 +0000 UTC logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Duration: 2s logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Message: successfully synced (all tasks run) logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | GROUP KIND NAMESPACE NAME STATUS HEALTH HOOK MESSAGE logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | template.openshift.io Template namespace-gitops-1672 redis-template-gitops2 Synced template.template.openshift.io/redis-template-gitops2 unchanged logger.go:42: 18:16:54 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | template.openshift.io Template namespace-gitops-1672 redis-template-gitops Succeeded Pruned pruned logger.go:42: 18:16:55 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Test is successful, no templates found logger.go:42: 18:16:55 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | Logged out from 'openshift-gitops-server-openshift-gitops.apps.test-aws-412.ocp-gitops-qe.com' logger.go:42: 18:16:55 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | application.argoproj.io "app-kustomize" deleted I0723 18:16:56.990132 5681 request.go:682] Waited for 1.047038843s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/argoproj.io/v1beta1?timeout=32s logger.go:42: 18:16:58 | 1-084_validate_prune_templates/2-prune-duplicateGVK-resource | test step completed 2-prune-duplicateGVK-resource logger.go:42: 18:16:58 | 1-084_validate_prune_templates/3- | starting test step 3- logger.go:42: 18:17:00 | 1-084_validate_prune_templates/3- | test step completed 3- logger.go:42: 18:17:00 | 1-084_validate_prune_templates/4-delete | starting test step 4-delete I0723 18:17:07.827044 5681 request.go:682] Waited for 1.046983487s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/autoscaling/v1?timeout=32s logger.go:42: 18:17:09 | 1-084_validate_prune_templates/4-delete | test step completed 4-delete logger.go:42: 18:17:09 | 1-084_validate_prune_templates | skipping kubernetes event logging logger.go:42: 18:17:09 | 1-084_validate_prune_templates | Deleting namespace: kuttl-test-uncommon-drake === CONT kuttl/harness/1-064_validate_tcp_reset_error logger.go:42: 18:17:15 | 1-064_validate_tcp_reset_error | Creating namespace: kuttl-test-useful-yak logger.go:42: 18:17:15 | 1-064_validate_tcp_reset_error/1- | starting test step 1- logger.go:42: 18:17:17 | 1-064_validate_tcp_reset_error/1- | test step completed 1- logger.go:42: 18:17:17 | 1-064_validate_tcp_reset_error/2-install | starting test step 2-install I0723 18:17:18.678702 5681 request.go:682] Waited for 1.047992483s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/security.openshift.io/v1?timeout=32s logger.go:42: 18:17:20 | 1-064_validate_tcp_reset_error/2-install | Application:openshift-gitops/1-27-argocd created logger.go:42: 18:17:52 | 1-064_validate_tcp_reset_error/2-install | test step completed 2-install logger.go:42: 18:17:52 | 1-064_validate_tcp_reset_error/3-create-app | starting test step 3-create-app I0723 18:17:53.745700 5681 request.go:682] Waited for 1.04873743s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/imageregistry.operator.openshift.io/v1?timeout=32s logger.go:42: 18:17:55 | 1-064_validate_tcp_reset_error/3-create-app | Application:test-1-27-custom/guestbook created logger.go:42: 18:18:03 | 1-064_validate_tcp_reset_error/3-create-app | test step completed 3-create-app logger.go:42: 18:18:03 | 1-064_validate_tcp_reset_error/4-check-app-manifest | starting test step 4-check-app-manifest logger.go:42: 18:18:03 | 1-064_validate_tcp_reset_error/4-check-app-manifest | running command: [sh -c set -e cleanup() { argocd logout "${api_server}" unset api_server unset password } trap cleanup INT TERM EXIT # Get credentials api_server=$(oc get routes -n openshift-gitops --field-selector metadata.name=openshift-gitops-server -o jsonpath="{.items[*]['spec.host']}") password=$(oc get secret openshift-gitops-cluster -n openshift-gitops -o jsonpath='{.data.admin\.password}' | base64 -d) # Login into ArgoCD login=$(argocd login $api_server --username admin --password $password --insecure) if ! [[ "${login}" =~ "'admin:login' logged in successfully" ]]; then echo "ERROR: unable to login to ArgoCD remote api server" exit 1 fi # Execute 'argocd app manifest' and check the 'read tcp' error app_manifests=$(argocd app manifests 1-27-argocd --source git --revision HEAD) if [[ "${app_manifests}" =~ 'Original error: read tcp' ]]; then echo "ERROR: TCP reset error is present in this code" exit 2 fi exit 0 ] logger.go:42: 18:18:04 | 1-064_validate_tcp_reset_error/4-check-app-manifest | time="2024-07-23T18:18:04Z" level=warning msg="Failed to invoke grpc call. Use flag --grpc-web in grpc calls. To avoid this warning message, use flag --grpc-web." logger.go:42: 18:18:05 | 1-064_validate_tcp_reset_error/4-check-app-manifest | time="2024-07-23T18:18:05Z" level=warning msg="Failed to invoke grpc call. Use flag --grpc-web in grpc calls. To avoid this warning message, use flag --grpc-web." logger.go:42: 18:18:05 | 1-064_validate_tcp_reset_error/4-check-app-manifest | Logged out from 'openshift-gitops-server-openshift-gitops.apps.test-aws-412.ocp-gitops-qe.com' I0723 18:18:06.865283 5681 request.go:682] Waited for 1.046973553s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/operator.openshift.io/v1?timeout=32s logger.go:42: 18:18:08 | 1-064_validate_tcp_reset_error/4-check-app-manifest | test step completed 4-check-app-manifest logger.go:42: 18:18:08 | 1-064_validate_tcp_reset_error/99-delete | starting test step 99-delete I0723 18:18:21.552858 5681 request.go:682] Waited for 1.033344125s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/apiserver.openshift.io/v1?timeout=32s logger.go:42: 18:18:22 | 1-064_validate_tcp_reset_error/99-delete | test step completed 99-delete logger.go:42: 18:18:22 | 1-064_validate_tcp_reset_error | skipping kubernetes event logging logger.go:42: 18:18:22 | 1-064_validate_tcp_reset_error | Deleting namespace: kuttl-test-useful-yak === CONT kuttl/harness/1-056_validate_managed-by logger.go:42: 18:18:28 | 1-056_validate_managed-by | Creating namespace: kuttl-test-moved-mayfly logger.go:42: 18:18:29 | 1-056_validate_managed-by/1-install | starting test step 1-install logger.go:42: 18:18:31 | 1-056_validate_managed-by/1-install | Namespace:/test-1-56-target created logger.go:42: 18:18:31 | 1-056_validate_managed-by/1-install | Namespace:/test-1-56-custom created logger.go:42: 18:18:31 | 1-056_validate_managed-by/1-install | ArgoCD:test-1-56-target/example-argocd created logger.go:42: 18:18:44 | 1-056_validate_managed-by/1-install | test step completed 1-install logger.go:42: 18:18:44 | 1-056_validate_managed-by/2-delete | starting test step 2-delete I0723 18:18:45.461050 5681 request.go:682] Waited for 1.046331696s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/argoproj.io/v1alpha1?timeout=32s logger.go:42: 18:18:46 | 1-056_validate_managed-by/2-delete | test step completed 2-delete logger.go:42: 18:18:46 | 1-056_validate_managed-by/3-check | starting test step 3-check logger.go:42: 18:18:46 | 1-056_validate_managed-by/3-check | running command: [sh -c set -e label_value=$(oc get ns/test-1-56-custom -o go-template=$'{{index .metadata.labels "argocd.argoproj.io/managed-by"}}') echo "label value: ${label_value}" if [[ "${label_value}" == "test-1-56-target" ]]; then echo "managed-by label exists, set REMOVE_MANAGED_BY_LABEL_ON_ARGOCD_DELETION variable to delete the label" exit 0 fi exit 1 ] logger.go:42: 18:18:47 | 1-056_validate_managed-by/3-check | label value: test-1-56-target logger.go:42: 18:18:47 | 1-056_validate_managed-by/3-check | managed-by label exists, set REMOVE_MANAGED_BY_LABEL_ON_ARGOCD_DELETION variable to delete the label logger.go:42: 18:18:49 | 1-056_validate_managed-by/3-check | test step completed 3-check logger.go:42: 18:18:49 | 1-056_validate_managed-by/4-add_env | starting test step 4-add_env logger.go:42: 18:18:49 | 1-056_validate_managed-by/4-add_env | running command: [sh -c set -e oc patch -n openshift-operators subscription openshift-gitops-operator \ --type merge --patch '{"spec": {"config": {"env": [{"name": "REMOVE_MANAGED_BY_LABEL_ON_ARGOCD_DELETION", "value": "true"}]}}}' ] logger.go:42: 18:18:50 | 1-056_validate_managed-by/4-add_env | subscription.operators.coreos.com/openshift-gitops-operator patched logger.go:42: 18:18:52 | 1-056_validate_managed-by/4-add_env | test step completed 4-add_env logger.go:42: 18:18:52 | 1-056_validate_managed-by/5-check | starting test step 5-check logger.go:42: 18:18:52 | 1-056_validate_managed-by/5-check | running command: [sh -c set -e label_value=$(oc get ns/test-1-56-custom -o go-template=$'{{index .metadata.labels "argocd.argoproj.io/managed-by"}}') echo "label value: ${label_value}" if [[ "${label_value}" == "test-1-56-target" ]]; then echo "managed-by label exists, set REMOVE_MANAGED_BY_LABEL_ON_ARGOCD_DELETION variable to delete the label" exit 0 fi exit 1 ] logger.go:42: 18:18:53 | 1-056_validate_managed-by/5-check | label value: test-1-56-target logger.go:42: 18:18:53 | 1-056_validate_managed-by/5-check | managed-by label exists, set REMOVE_MANAGED_BY_LABEL_ON_ARGOCD_DELETION variable to delete the label I0723 18:18:55.506212 5681 request.go:682] Waited for 2.297367829s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/operator.openshift.io/v1alpha1?timeout=32s logger.go:42: 18:18:55 | 1-056_validate_managed-by/5-check | test step completed 5-check logger.go:42: 18:18:55 | 1-056_validate_managed-by/6-install | starting test step 6-install logger.go:42: 18:18:58 | 1-056_validate_managed-by/6-install | Namespace:/test-1-56-target-2 created logger.go:42: 18:18:58 | 1-056_validate_managed-by/6-install | Namespace:/test-1-56-custom-2 created logger.go:42: 18:18:58 | 1-056_validate_managed-by/6-install | ArgoCD:test-1-56-target-2/example-argocd-2 created logger.go:42: 18:19:30 | 1-056_validate_managed-by/6-install | test step completed 6-install logger.go:42: 18:19:30 | 1-056_validate_managed-by/7-delete | starting test step 7-delete I0723 18:19:32.033854 5681 request.go:682] Waited for 1.046636094s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/migration.k8s.io/v1alpha1?timeout=32s logger.go:42: 18:19:33 | 1-056_validate_managed-by/7-delete | test step completed 7-delete logger.go:42: 18:19:33 | 1-056_validate_managed-by/8-remove_env | starting test step 8-remove_env logger.go:42: 18:19:33 | 1-056_validate_managed-by/8-remove_env | running command: [sh -c oc patch -n openshift-operators subscription openshift-gitops-operator \ --type json --patch '[{"op": "remove", "path": "/spec/config"}]' ] logger.go:42: 18:19:33 | 1-056_validate_managed-by/8-remove_env | subscription.operators.coreos.com/openshift-gitops-operator patched logger.go:42: 18:19:36 | 1-056_validate_managed-by/8-remove_env | test step completed 8-remove_env logger.go:42: 18:19:36 | 1-056_validate_managed-by/99-delete | starting test step 99-delete I0723 18:19:44.159805 5681 request.go:682] Waited for 1.042483128s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/project.openshift.io/v1?timeout=32s logger.go:42: 18:19:45 | 1-056_validate_managed-by/99-delete | test step completed 99-delete logger.go:42: 18:19:45 | 1-056_validate_managed-by | skipping kubernetes event logging logger.go:42: 18:19:45 | 1-056_validate_managed-by | Deleting namespace: kuttl-test-moved-mayfly === CONT kuttl/harness/1-106_validate_argocd_metrics_controller logger.go:42: 18:19:51 | 1-106_validate_argocd_metrics_controller | Creating namespace: kuttl-test-aware-dory logger.go:42: 18:19:51 | 1-106_validate_argocd_metrics_controller/1- | starting test step 1- logger.go:42: 18:19:54 | 1-106_validate_argocd_metrics_controller/1- | test step completed 1- logger.go:42: 18:19:54 | 1-106_validate_argocd_metrics_controller/2-disable-metrics | starting test step 2-disable-metrics logger.go:42: 18:19:54 | 1-106_validate_argocd_metrics_controller/2-disable-metrics | running command: [sh -c oc patch argocd openshift-gitops -n openshift-gitops --type='json' -p='[{"op": "replace", "path": "/spec/monitoring/disableMetrics", "value": true}]' ] logger.go:42: 18:19:54 | 1-106_validate_argocd_metrics_controller/2-disable-metrics | argocd.argoproj.io/openshift-gitops patched I0723 18:19:55.557970 5681 request.go:682] Waited for 1.047182631s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/operators.coreos.com/v1?timeout=32s logger.go:42: 18:20:06 | 1-106_validate_argocd_metrics_controller/2-disable-metrics | test step completed 2-disable-metrics logger.go:42: 18:20:06 | 1-106_validate_argocd_metrics_controller/3-enable-metrics | starting test step 3-enable-metrics logger.go:42: 18:20:06 | 1-106_validate_argocd_metrics_controller/3-enable-metrics | running command: [sh -c oc patch argocd openshift-gitops -n openshift-gitops --type='json' -p='[{"op": "replace", "path": "/spec/monitoring/disableMetrics", "value": false}]' ] logger.go:42: 18:20:06 | 1-106_validate_argocd_metrics_controller/3-enable-metrics | argocd.argoproj.io/openshift-gitops patched I0723 18:20:07.758089 5681 request.go:682] Waited for 1.048189793s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/scheduling.k8s.io/v1?timeout=32s logger.go:42: 18:20:09 | 1-106_validate_argocd_metrics_controller/3-enable-metrics | test step completed 3-enable-metrics logger.go:42: 18:20:09 | 1-106_validate_argocd_metrics_controller | skipping kubernetes event logging logger.go:42: 18:20:09 | 1-106_validate_argocd_metrics_controller | Deleting namespace: kuttl-test-aware-dory === CONT kuttl/harness/1-050_validate_sso logger.go:42: 18:20:15 | 1-050_validate_sso | Creating namespace: kuttl-test-capable-calf logger.go:42: 18:20:15 | 1-050_validate_sso/1-install | starting test step 1-install logger.go:42: 18:20:17 | 1-050_validate_sso/1-install | ArgoCD:kuttl-test-capable-calf/argocd created logger.go:42: 18:20:29 | 1-050_validate_sso/1-install | test step completed 1-install logger.go:42: 18:20:29 | 1-050_validate_sso/2-remove-spec-sso | starting test step 2-remove-spec-sso logger.go:42: 18:20:29 | 1-050_validate_sso/2-remove-spec-sso | running command: [sh -c oc patch -n $NAMESPACE argocd/argocd --type='json' -p='[{"op": "remove", "path": "/spec/sso"}]' ] logger.go:42: 18:20:30 | 1-050_validate_sso/2-remove-spec-sso | argocd.argoproj.io/argocd patched logger.go:42: 18:20:30 | 1-050_validate_sso/2-remove-spec-sso | running command: [sh -c sleep 10] I0723 18:20:41.457203 5681 request.go:682] Waited for 1.025463569s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/template.openshift.io/v1?timeout=32s logger.go:42: 18:20:42 | 1-050_validate_sso/2-remove-spec-sso | test step completed 2-remove-spec-sso logger.go:42: 18:20:42 | 1-050_validate_sso/7-provider-keycloak-dex-spec | starting test step 7-provider-keycloak-dex-spec logger.go:42: 18:20:45 | 1-050_validate_sso/7-provider-keycloak-dex-spec | ArgoCD:kuttl-test-capable-calf/argocd updated logger.go:42: 18:20:46 | 1-050_validate_sso/7-provider-keycloak-dex-spec | test step completed 7-provider-keycloak-dex-spec logger.go:42: 18:20:46 | 1-050_validate_sso/8-remove-spec-dex | starting test step 8-remove-spec-dex logger.go:42: 18:20:46 | 1-050_validate_sso/8-remove-spec-dex | running command: [sh -c oc patch -n $NAMESPACE argocd/argocd --type='json' -p='[{"op": "remove", "path": "/spec/sso/dex"}]' ] logger.go:42: 18:20:46 | 1-050_validate_sso/8-remove-spec-dex | argocd.argoproj.io/argocd patched logger.go:42: 18:23:02 | 1-050_validate_sso/8-remove-spec-dex | test step completed 8-remove-spec-dex logger.go:42: 18:23:02 | 1-050_validate_sso | skipping kubernetes event logging logger.go:42: 18:23:02 | 1-050_validate_sso | Deleting namespace: kuttl-test-capable-calf === CONT kuttl/harness/1-100_validate_rollouts_resources_creation logger.go:42: 18:23:14 | 1-100_validate_rollouts_resources_creation | Creating namespace: kuttl-test-absolute-duckling logger.go:42: 18:23:14 | 1-100_validate_rollouts_resources_creation/1-install-rolloutmanager | starting test step 1-install-rolloutmanager I0723 18:23:15.560777 5681 request.go:682] Waited for 1.039977382s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/quota.openshift.io/v1?timeout=32s logger.go:42: 18:23:16 | 1-100_validate_rollouts_resources_creation/1-install-rolloutmanager | RolloutManager:kuttl-test-absolute-duckling/example-rollout-manager created logger.go:42: 18:23:37 | 1-100_validate_rollouts_resources_creation/1-install-rolloutmanager | test step completed 1-install-rolloutmanager logger.go:42: 18:23:37 | 1-100_validate_rollouts_resources_creation/2- | starting test step 2- I0723 18:23:38.875027 5681 request.go:682] Waited for 1.048245808s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/build.openshift.io/v1?timeout=32s logger.go:42: 18:23:40 | 1-100_validate_rollouts_resources_creation/2- | test step completed 2- logger.go:42: 18:23:40 | 1-100_validate_rollouts_resources_creation | skipping kubernetes event logging logger.go:42: 18:23:40 | 1-100_validate_rollouts_resources_creation | Deleting namespace: kuttl-test-absolute-duckling === CONT kuttl/harness/1-042_validate_status_host logger.go:42: 18:23:46 | 1-042_validate_status_host | Creating namespace: kuttl-test-engaged-bluebird logger.go:42: 18:23:46 | 1-042_validate_status_host/1-install | starting test step 1-install logger.go:42: 18:23:48 | 1-042_validate_status_host/1-install | ArgoCD:kuttl-test-engaged-bluebird/example-argocd created logger.go:42: 18:24:20 | 1-042_validate_status_host/1-install | test step completed 1-install logger.go:42: 18:24:20 | 1-042_validate_status_host/2-check_URL | starting test step 2-check_URL logger.go:42: 18:24:20 | 1-042_validate_status_host/2-check_URL | running command: [sh -c until [[ $i -eq 20 ]] do route_URL=$(oc get routes -o jsonpath="{.items[*]['spec.host']}" --field-selector metadata.name=example-argocd-server -n $NAMESPACE) status_URL=$(oc get argocd -o jsonpath="{.items[*]['status.host']}" -n $NAMESPACE) echo -e "route_URL:${route_URL}\nstatus_URL:${status_URL}" if ! [[ "${status_URL}" == "${route_URL}" ]]; then i=$((i+1)) sleep 6 else exit 0 fi done echo "Error: Route does not match" exit 1 ] logger.go:42: 18:24:21 | 1-042_validate_status_host/2-check_URL | route_URL:example-argocd-server-kuttl-test-engaged-bluebird.apps.test-aws-412.ocp-gitops-qe.com logger.go:42: 18:24:21 | 1-042_validate_status_host/2-check_URL | status_URL:example-argocd-server-kuttl-test-engaged-bluebird.apps.test-aws-412.ocp-gitops-qe.com I0723 18:24:22.360271 5681 request.go:682] Waited for 1.048183213s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/monitoring.coreos.com/v1alpha1?timeout=32s logger.go:42: 18:24:23 | 1-042_validate_status_host/2-check_URL | test step completed 2-check_URL logger.go:42: 18:24:23 | 1-042_validate_status_host/3-modify_route_URL | starting test step 3-modify_route_URL logger.go:42: 18:24:23 | 1-042_validate_status_host/3-modify_route_URL | running command: [sh -c oc patch route example-argocd-server -n $NAMESPACE --type='json' -p='[{"op": "replace", "path": "/spec/host", "value":"modified-route"}]' ] logger.go:42: 18:24:24 | 1-042_validate_status_host/3-modify_route_URL | route.route.openshift.io/example-argocd-server patched logger.go:42: 18:24:26 | 1-042_validate_status_host/3-modify_route_URL | test step completed 3-modify_route_URL logger.go:42: 18:24:26 | 1-042_validate_status_host/4-disable_route | starting test step 4-disable_route logger.go:42: 18:24:29 | 1-042_validate_status_host/4-disable_route | ArgoCD:kuttl-test-engaged-bluebird/example-argocd updated logger.go:42: 18:24:29 | 1-042_validate_status_host/4-disable_route | test step completed 4-disable_route logger.go:42: 18:24:29 | 1-042_validate_status_host/5-check_URL_gone | starting test step 5-check_URL_gone logger.go:42: 18:24:29 | 1-042_validate_status_host/5-check_URL_gone | running command: [sh -c sleep 5 status_URL=$(oc get argocd -o jsonpath="{.items[*]['status.host']}" -n $NAMESPACE) echo "${status_URL}" if test "${status_URL}" != ""; then echo "Error: status.host should not be present in argocd" exit 1 fi exit 0 ] logger.go:42: 18:24:34 | 1-042_validate_status_host/5-check_URL_gone | I0723 18:24:35.757558 5681 request.go:682] Waited for 1.047656138s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/coordination.k8s.io/v1?timeout=32s logger.go:42: 18:24:37 | 1-042_validate_status_host/5-check_URL_gone | test step completed 5-check_URL_gone logger.go:42: 18:24:37 | 1-042_validate_status_host | skipping kubernetes event logging logger.go:42: 18:24:37 | 1-042_validate_status_host | Deleting namespace: kuttl-test-engaged-bluebird === CONT kuttl/harness/1-086_validate_default_argocd_role logger.go:42: 18:24:43 | 1-086_validate_default_argocd_role | Creating namespace: kuttl-test-brave-ape logger.go:42: 18:24:43 | 1-086_validate_default_argocd_role/1- | starting test step 1- logger.go:42: 18:24:45 | 1-086_validate_default_argocd_role/1- | test step completed 1- logger.go:42: 18:24:45 | 1-086_validate_default_argocd_role/2-install | starting test step 2-install I0723 18:24:46.720518 5681 request.go:682] Waited for 1.047730552s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/config.openshift.io/v1?timeout=32s logger.go:42: 18:24:48 | 1-086_validate_default_argocd_role/2-install | Namespace:/test-1-086-custom created logger.go:42: 18:24:48 | 1-086_validate_default_argocd_role/2-install | Namespace:/test-1-086-custom2 created logger.go:42: 18:24:48 | 1-086_validate_default_argocd_role/2-install | Namespace:/test-1-086-custom3 created logger.go:42: 18:24:48 | 1-086_validate_default_argocd_role/2-install | ArgoCD:test-1-086-custom/argocd-default-policy created logger.go:42: 18:24:48 | 1-086_validate_default_argocd_role/2-install | ArgoCD:test-1-086-custom2/argocd-default-policy-empty created logger.go:42: 18:24:48 | 1-086_validate_default_argocd_role/2-install | ArgoCD:test-1-086-custom3/argocd-default-policy-admin created logger.go:42: 18:25:20 | 1-086_validate_default_argocd_role/2-install | test step completed 2-install logger.go:42: 18:25:20 | 1-086_validate_default_argocd_role | skipping kubernetes event logging logger.go:42: 18:25:20 | 1-086_validate_default_argocd_role | Deleting namespace: kuttl-test-brave-ape === CONT kuttl/harness/1-085_validate_dynamic_plugin_installation logger.go:42: 18:25:26 | 1-085_validate_dynamic_plugin_installation | Creating namespace: kuttl-test-genuine-toucan logger.go:42: 18:25:26 | 1-085_validate_dynamic_plugin_installation/1-install-dynamic-plugin | starting test step 1-install-dynamic-plugin logger.go:42: 18:25:26 | 1-085_validate_dynamic_plugin_installation/1-install-dynamic-plugin | running command: [sh -c set -eo pipefail # Get CSV name and also OCP version. export namespace=openshift-operators export csv_name=$(oc get csv -n openshift-operators | awk '/gitops-operator/ {print $1}') if [ -z "$csv_name" ]; then export csv_name=$(oc get csv -n openshift-gitops-operator | awk '/gitops-operator/ {print $1}') export namespace=openshift-gitops-operator fi ocp_version=$(oc version | awk '/Server Version/ {split($3,a,"-"); print a[1]}') if [[ ${ocp_version} =~ "4.15" ]]; then exit 0 fi # Patch the CSV to add the necessary env key:value for Dynamic Plugin oc patch csv "${csv_name}" -n $namespace \ --type='json' \ -p='[{ "op": "add", "path": "/spec/install/spec/deployments/0/spec/template/spec/containers/0/env/-", "value": { "name": "DYNAMIC_PLUGIN_START_OCP_VERSION", "value": "'${ocp_version}'" } }]' # Sleep 10 seconds, so the resources are launched before next assert step. sleep 5 exit 0 ] logger.go:42: 18:25:27 | 1-085_validate_dynamic_plugin_installation/1-install-dynamic-plugin | clusterserviceversion.operators.coreos.com/openshift-gitops-operator.v1.13.1 patched I0723 18:25:33.955606 5681 request.go:682] Waited for 1.047532625s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/batch/v1?timeout=32s logger.go:42: 18:25:35 | 1-085_validate_dynamic_plugin_installation/1-install-dynamic-plugin | test step completed 1-install-dynamic-plugin logger.go:42: 18:25:35 | 1-085_validate_dynamic_plugin_installation/2-install-dynamic-plugin | starting test step 2-install-dynamic-plugin logger.go:42: 18:25:35 | 1-085_validate_dynamic_plugin_installation/2-install-dynamic-plugin | running command: [sh -c set -eo pipefail # Get CSV name and also OCP version. csv_name=$(oc get csv -n openshift-operators | awk '/openshift-gitops-operator/ {print $1}') ocp_version=$(oc version | awk '/Server Version/ {split($3,a,"-"); print a[1]}') # Patch the CSV to add the necessary env key:value for Dynamic Plugin oc patch csv "${csv_name}" -n openshift-operators \ --type='json' \ -p='[{ "op": "add", "path": "/spec/install/spec/deployments/0/spec/template/spec/containers/0/env/-", "value": { "name": "DYNAMIC_PLUGIN_START_OCP_VERSION", "value": "'${ocp_version}'" } }]' # Sleep 10 seconds, so the resources are launched before next assert step. sleep 5 exit 0 ] logger.go:42: 18:25:36 | 1-085_validate_dynamic_plugin_installation/2-install-dynamic-plugin | clusterserviceversion.operators.coreos.com/openshift-gitops-operator.v1.13.1 patched I0723 18:25:43.961490 5681 request.go:682] Waited for 2.046039568s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/monitoring.coreos.com/v1?timeout=32s logger.go:42: 18:26:01 | 1-085_validate_dynamic_plugin_installation/2-install-dynamic-plugin | test step completed 2-install-dynamic-plugin logger.go:42: 18:26:01 | 1-085_validate_dynamic_plugin_installation | skipping kubernetes event logging logger.go:42: 18:26:01 | 1-085_validate_dynamic_plugin_installation | Deleting namespace: kuttl-test-genuine-toucan === CONT kuttl/harness/1-101_validate_rollout_policyrules logger.go:42: 18:26:07 | 1-101_validate_rollout_policyrules | Creating namespace: kuttl-test-correct-badger logger.go:42: 18:26:07 | 1-101_validate_rollout_policyrules/1-install-rollloutmanager | starting test step 1-install-rollloutmanager I0723 18:26:08.702824 5681 request.go:682] Waited for 1.046178079s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/pipelines.openshift.io/v1alpha1?timeout=32s logger.go:42: 18:26:10 | 1-101_validate_rollout_policyrules/1-install-rollloutmanager | RolloutManager:kuttl-test-correct-badger/example-rollout-manager created logger.go:42: 18:26:10 | 1-101_validate_rollout_policyrules/1-install-rollloutmanager | test step completed 1-install-rollloutmanager logger.go:42: 18:26:10 | 1-101_validate_rollout_policyrules | skipping kubernetes event logging logger.go:42: 18:26:10 | 1-101_validate_rollout_policyrules | Deleting namespace: kuttl-test-correct-badger === CONT kuttl/harness/1-105_validate_label_selector logger.go:42: 18:26:16 | 1-105_validate_label_selector | Creating namespace: kuttl-test-fit-firefly logger.go:42: 18:26:16 | 1-105_validate_label_selector/1-add-label-selector | starting test step 1-add-label-selector logger.go:42: 18:26:16 | 1-105_validate_label_selector/1-add-label-selector | running command: [sh -c if ! [ -z $NON_OLM ]; then oc set env deployment openshift-gitops-operator-controller-manager -n openshift-gitops-operator ARGOCD_LABEL_SELECTOR=foo=bar -c manager elif ! [ -z $CI ]; then oc patch -n openshift-gitops-operator subscription openshift-gitops-operator \ --type merge --patch '{"spec": {"config": {"env": [{"name": "ARGOCD_LABEL_SELECTOR", "value": "foo=bar"}]}}}' else oc patch -n openshift-operators subscription `subscription=gitops-operator && oc get subscription --all-namespaces | grep $subscription | head -1 | awk '{print $2}'` \ --type merge --patch '{"spec": {"config": {"env": [{"name": "ARGOCD_LABEL_SELECTOR", "value": "foo=bar"}]}}}' fi ] logger.go:42: 18:26:17 | 1-105_validate_label_selector/1-add-label-selector | subscription.operators.coreos.com/openshift-gitops-operator patched logger.go:42: 18:26:17 | 1-105_validate_label_selector/1-add-label-selector | running command: [sh -c sleep 120] I0723 18:28:18.322095 5681 request.go:682] Waited for 1.045394196s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/admissionregistration.k8s.io/v1?timeout=32s logger.go:42: 18:28:19 | 1-105_validate_label_selector/1-add-label-selector | test step completed 1-add-label-selector logger.go:42: 18:28:19 | 1-105_validate_label_selector/2-create-ns | starting test step 2-create-ns logger.go:42: 18:28:22 | 1-105_validate_label_selector/2-create-ns | Namespace:/test-argocd created logger.go:42: 18:28:22 | 1-105_validate_label_selector/2-create-ns | ArgoCD:test-argocd/argocd created logger.go:42: 18:28:22 | 1-105_validate_label_selector/2-create-ns | ArgoCD:test-argocd/test1 created logger.go:42: 18:28:22 | 1-105_validate_label_selector/2-create-ns | test step completed 2-create-ns logger.go:42: 18:28:22 | 1-105_validate_label_selector/3-add-label | starting test step 3-add-label logger.go:42: 18:28:22 | 1-105_validate_label_selector/3-add-label | running command: [sh -c kubectl label argocd test1 foo=bar -n test-argocd ] logger.go:42: 18:28:22 | 1-105_validate_label_selector/3-add-label | argocd.argoproj.io/test1 labeled logger.go:42: 18:28:22 | 1-105_validate_label_selector/3-add-label | running command: [sh -c sleep 30] I0723 18:28:53.715281 5681 request.go:682] Waited for 1.046663627s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/console.openshift.io/v1?timeout=32s logger.go:42: 18:28:55 | 1-105_validate_label_selector/3-add-label | test step completed 3-add-label logger.go:42: 18:28:55 | 1-105_validate_label_selector/4-install | starting test step 4-install logger.go:42: 18:28:57 | 1-105_validate_label_selector/4-install | ArgoCD:test-argocd/test1 updated logger.go:42: 18:28:58 | 1-105_validate_label_selector/4-install | test step completed 4-install logger.go:42: 18:28:58 | 1-105_validate_label_selector/5-remove-label | starting test step 5-remove-label logger.go:42: 18:28:58 | 1-105_validate_label_selector/5-remove-label | running command: [sh -c kubectl label argocd test1 foo- -n test-argocd ] logger.go:42: 18:28:59 | 1-105_validate_label_selector/5-remove-label | argocd.argoproj.io/test1 unlabeled logger.go:42: 18:28:59 | 1-105_validate_label_selector/5-remove-label | running command: [sh -c sleep 30] I0723 18:29:30.160933 5681 request.go:682] Waited for 1.046852695s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/autoscaling.openshift.io/v1beta1?timeout=32s logger.go:42: 18:29:31 | 1-105_validate_label_selector/5-remove-label | test step completed 5-remove-label logger.go:42: 18:29:31 | 1-105_validate_label_selector/6-install | starting test step 6-install logger.go:42: 18:29:34 | 1-105_validate_label_selector/6-install | ArgoCD:test-argocd/test1 updated logger.go:42: 18:29:34 | 1-105_validate_label_selector/6-install | test step completed 6-install logger.go:42: 18:29:34 | 1-105_validate_label_selector/7-remove-label-selector | starting test step 7-remove-label-selector logger.go:42: 18:29:34 | 1-105_validate_label_selector/7-remove-label-selector | running command: [sh -c if ! [ -z $NON_OLM ]; then oc set env deployment openshift-gitops-operator-controller-manager -n openshift-gitops-operator ARGOCD_LABEL_SELECTOR- -c manager elif ! [ -z $CI ]; then oc patch -n openshift-gitops-operator subscription openshift-gitops-operator \ --type json --patch '[{"op": "remove", "path": "/spec/config"}]' else oc patch -n openshift-operators subscription `subscription=gitops-operator && oc get subscription --all-namespaces | grep $subscription | head -1 | awk '{print $2}'` \ --type json --patch '[{"op": "remove", "path": "/spec/config"}]' fi ] logger.go:42: 18:29:35 | 1-105_validate_label_selector/7-remove-label-selector | subscription.operators.coreos.com/openshift-gitops-operator patched logger.go:42: 18:29:35 | 1-105_validate_label_selector/7-remove-label-selector | running command: [sh -c sleep 120] I0723 18:31:36.307575 5681 request.go:682] Waited for 1.047768745s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/storage.k8s.io/v1beta1?timeout=32s logger.go:42: 18:31:37 | 1-105_validate_label_selector/7-remove-label-selector | test step completed 7-remove-label-selector logger.go:42: 18:31:37 | 1-105_validate_label_selector/99-delete | starting test step 99-delete I0723 18:31:51.946939 5681 request.go:682] Waited for 1.046269914s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/argoproj.io/v1beta1?timeout=32s logger.go:42: 18:31:53 | 1-105_validate_label_selector/99-delete | test step completed 99-delete logger.go:42: 18:31:53 | 1-105_validate_label_selector | skipping kubernetes event logging logger.go:42: 18:31:53 | 1-105_validate_label_selector | Deleting namespace: kuttl-test-fit-firefly === CONT kuttl/harness/1-104_validate_prometheus_alert logger.go:42: 18:31:59 | 1-104_validate_prometheus_alert | Creating namespace: kuttl-test-talented-jawfish logger.go:42: 18:31:59 | 1-104_validate_prometheus_alert/1- | starting test step 1- logger.go:42: 18:32:01 | 1-104_validate_prometheus_alert/1- | test step completed 1- logger.go:42: 18:32:01 | 1-104_validate_prometheus_alert | skipping kubernetes event logging logger.go:42: 18:32:01 | 1-104_validate_prometheus_alert | Deleting namespace: kuttl-test-talented-jawfish === CONT kuttl/harness/1-006_validate_machine_config logger.go:42: 18:32:08 | 1-006_validate_machine_config | Creating namespace: kuttl-test-complete-camel logger.go:42: 18:32:08 | 1-006_validate_machine_config/1-install | starting test step 1-install I0723 18:32:09.147291 5681 request.go:682] Waited for 1.047785917s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/console.openshift.io/v1alpha1?timeout=32s logger.go:42: 18:32:10 | 1-006_validate_machine_config/1-install | ArgoCD:openshift-gitops/openshift-gitops updated logger.go:42: 18:32:10 | 1-006_validate_machine_config/1-install | Application:openshift-gitops/validate-machine-config created logger.go:42: 18:32:22 | 1-006_validate_machine_config/1-install | test step completed 1-install logger.go:42: 18:32:22 | 1-006_validate_machine_config/2- | starting test step 2- I0723 18:32:23.568013 5681 request.go:682] Waited for 1.047336307s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/network.openshift.io/v1?timeout=32s logger.go:42: 18:32:24 | 1-006_validate_machine_config/2- | test step completed 2- logger.go:42: 18:32:24 | 1-006_validate_machine_config/3-scale-down-repo-server | starting test step 3-scale-down-repo-server logger.go:42: 18:32:27 | 1-006_validate_machine_config/3-scale-down-repo-server | ArgoCD:openshift-gitops/openshift-gitops updated logger.go:42: 18:32:28 | 1-006_validate_machine_config/3-scale-down-repo-server | test step completed 3-scale-down-repo-server logger.go:42: 18:32:28 | 1-006_validate_machine_config/99-delete | starting test step 99-delete logger.go:42: 18:32:30 | 1-006_validate_machine_config/99-delete | test step completed 99-delete logger.go:42: 18:32:30 | 1-006_validate_machine_config | skipping kubernetes event logging logger.go:42: 18:32:30 | 1-006_validate_machine_config | Deleting namespace: kuttl-test-complete-camel === CONT kuttl/harness/1-026-validate_backend_service_permissions logger.go:42: 18:32:37 | 1-026-validate_backend_service_permissions | Creating namespace: kuttl-test-oriented-hippo logger.go:42: 18:32:37 | 1-026-validate_backend_service_permissions/1-check-backend-service | starting test step 1-check-backend-service I0723 18:32:38.270037 5681 request.go:682] Waited for 1.046131208s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/cloudcredential.openshift.io/v1?timeout=32s logger.go:42: 18:32:39 | 1-026-validate_backend_service_permissions/1-check-backend-service | test step completed 1-check-backend-service logger.go:42: 18:32:39 | 1-026-validate_backend_service_permissions | skipping kubernetes event logging logger.go:42: 18:32:39 | 1-026-validate_backend_service_permissions | Deleting namespace: kuttl-test-oriented-hippo === CONT kuttl/harness/1-020_validate_redis_ha_nonha logger.go:42: 18:32:53 | 1-020_validate_redis_ha_nonha | Creating namespace: kuttl-test-wise-turtle logger.go:42: 18:32:53 | 1-020_validate_redis_ha_nonha/1- | starting test step 1- I0723 18:32:54.942862 5681 request.go:682] Waited for 1.0418437s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/k8s.cni.cncf.io/v1?timeout=32s logger.go:42: 18:32:56 | 1-020_validate_redis_ha_nonha/1- | test step completed 1- logger.go:42: 18:32:56 | 1-020_validate_redis_ha_nonha/2-enable-ha | starting test step 2-enable-ha logger.go:42: 18:32:59 | 1-020_validate_redis_ha_nonha/2-enable-ha | ArgoCD:openshift-gitops/openshift-gitops updated logger.go:42: 18:42:02 | 1-020_validate_redis_ha_nonha/2-enable-ha | test step completed 2-enable-ha logger.go:42: 18:42:02 | 1-020_validate_redis_ha_nonha/3-update-ha-resources | starting test step 3-update-ha-resources I0723 18:42:03.734560 5681 request.go:682] Waited for 1.047695758s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/helm.openshift.io/v1beta1?timeout=32s logger.go:42: 18:42:05 | 1-020_validate_redis_ha_nonha/3-update-ha-resources | ArgoCD:openshift-gitops/openshift-gitops updated logger.go:42: 18:42:06 | 1-020_validate_redis_ha_nonha/3-update-ha-resources | test step completed 3-update-ha-resources logger.go:42: 18:42:06 | 1-020_validate_redis_ha_nonha/4-disable-ha | starting test step 4-disable-ha logger.go:42: 18:42:08 | 1-020_validate_redis_ha_nonha/4-disable-ha | ArgoCD:openshift-gitops/openshift-gitops updated logger.go:42: 18:42:21 | 1-020_validate_redis_ha_nonha/4-disable-ha | test step completed 4-disable-ha logger.go:42: 18:42:21 | 1-020_validate_redis_ha_nonha | skipping kubernetes event logging logger.go:42: 18:42:21 | 1-020_validate_redis_ha_nonha | Deleting namespace: kuttl-test-wise-turtle === CONT kuttl/harness/1-040_validate_quoted_RBAC_group_names logger.go:42: 18:42:27 | 1-040_validate_quoted_RBAC_group_names | Creating namespace: kuttl-test-known-katydid logger.go:42: 18:42:27 | 1-040_validate_quoted_RBAC_group_names/1-login_argocd_api_server | starting test step 1-login_argocd_api_server logger.go:42: 18:42:27 | 1-040_validate_quoted_RBAC_group_names/1-login_argocd_api_server | running command: [sh -c api_server=$(oc get routes -n openshift-gitops --field-selector metadata.name=openshift-gitops-server -o jsonpath="{.items[*]['spec.host']}") password=$(oc get secret openshift-gitops-cluster -n openshift-gitops -o jsonpath='{.data.admin\.password}' | base64 -d) output=$(argocd login $api_server --username admin --password $password --insecure) if ! [[ "${output}" =~ "'admin:login' logged in successfully" ]]; then echo $output exit 1 fi exit 0 ] logger.go:42: 18:42:28 | 1-040_validate_quoted_RBAC_group_names/1-login_argocd_api_server | time="2024-07-23T18:42:28Z" level=warning msg="Failed to invoke grpc call. Use flag --grpc-web in grpc calls. To avoid this warning message, use flag --grpc-web." I0723 18:42:30.722126 5681 request.go:682] Waited for 1.046234686s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/security.openshift.io/v1?timeout=32s logger.go:42: 18:42:32 | 1-040_validate_quoted_RBAC_group_names/1-login_argocd_api_server | test step completed 1-login_argocd_api_server logger.go:42: 18:42:32 | 1-040_validate_quoted_RBAC_group_names/2-create_role | starting test step 2-create_role logger.go:42: 18:42:32 | 1-040_validate_quoted_RBAC_group_names/2-create_role | running command: [sh -c output=$(argocd proj role create default somerole) if ! [[ "$output" =~ "Role 'somerole' created" ]]; then echo "Error creating role" exit 1 fi exit 0 ] logger.go:42: 18:42:32 | 1-040_validate_quoted_RBAC_group_names/2-create_role | time="2024-07-23T18:42:32Z" level=warning msg="Failed to invoke grpc call. Use flag --grpc-web in grpc calls. To avoid this warning message, use flag --grpc-web." logger.go:42: 18:42:34 | 1-040_validate_quoted_RBAC_group_names/2-create_role | test step completed 2-create_role logger.go:42: 18:42:34 | 1-040_validate_quoted_RBAC_group_names/3-specify_group_names | starting test step 3-specify_group_names logger.go:42: 18:42:34 | 1-040_validate_quoted_RBAC_group_names/3-specify_group_names | running command: [sh -c output=$(argocd proj role add-group default somerole '"CN=foo,OU=bar,O=baz"') cleanup() { argocd proj role delete default somerole } trap cleanup INT TERM EXIT if ! [[ "${output}" =~ "added to role 'somerole'" ]]; then echo "Contains comma characters!" exit 1 fi ] logger.go:42: 18:42:34 | 1-040_validate_quoted_RBAC_group_names/3-specify_group_names | time="2024-07-23T18:42:34Z" level=warning msg="Failed to invoke grpc call. Use flag --grpc-web in grpc calls. To avoid this warning message, use flag --grpc-web." logger.go:42: 18:42:35 | 1-040_validate_quoted_RBAC_group_names/3-specify_group_names | time="2024-07-23T18:42:35Z" level=warning msg="Failed to invoke grpc call. Use flag --grpc-web in grpc calls. To avoid this warning message, use flag --grpc-web." logger.go:42: 18:42:35 | 1-040_validate_quoted_RBAC_group_names/3-specify_group_names | Role 'somerole' deleted logger.go:42: 18:42:37 | 1-040_validate_quoted_RBAC_group_names/3-specify_group_names | test step completed 3-specify_group_names logger.go:42: 18:42:37 | 1-040_validate_quoted_RBAC_group_names | skipping kubernetes event logging logger.go:42: 18:42:37 | 1-040_validate_quoted_RBAC_group_names | Deleting namespace: kuttl-test-known-katydid === CONT kuttl/harness/1-018_validate_disable_default_instance logger.go:42: 18:42:43 | 1-018_validate_disable_default_instance | Ignoring errors.yaml as it does not match file name regexp: ^(\d+)-(?:[^\.]+)(?:\.yaml)?$ logger.go:42: 18:42:43 | 1-018_validate_disable_default_instance | Creating namespace: kuttl-test-trusting-octopus logger.go:42: 18:42:43 | 1-018_validate_disable_default_instance/1- | starting test step 1- I0723 18:42:44.834280 5681 request.go:682] Waited for 1.046676899s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/quota.openshift.io/v1?timeout=32s logger.go:42: 18:42:46 | 1-018_validate_disable_default_instance/1- | test step completed 1- logger.go:42: 18:42:46 | 1-018_validate_disable_default_instance/2-patch-subscription | starting test step 2-patch-subscription logger.go:42: 18:42:46 | 1-018_validate_disable_default_instance/2-patch-subscription | running command: [sh -c oc patch -n openshift-operators subscription openshift-gitops-operator \ --type merge --patch '{"spec": {"config": {"env": [{"name": "DISABLE_DEFAULT_ARGOCD_INSTANCE", "value": "true"}]}}}' ] logger.go:42: 18:42:46 | 1-018_validate_disable_default_instance/2-patch-subscription | subscription.operators.coreos.com/openshift-gitops-operator patched logger.go:42: 18:42:46 | 1-018_validate_disable_default_instance/2-patch-subscription | running command: [sh -c sleep 20] I0723 18:43:07.860128 5681 request.go:682] Waited for 1.037503335s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/build.openshift.io/v1?timeout=32s logger.go:42: 18:43:09 | 1-018_validate_disable_default_instance/2-patch-subscription | test step completed 2-patch-subscription logger.go:42: 18:43:09 | 1-018_validate_disable_default_instance/3-check | starting test step 3-check logger.go:42: 18:53:12 | 1-018_validate_disable_default_instance/3-check | test step failed 3-check case.go:364: failed in step 3-check case.go:366: resource argoproj.io/v1alpha1, Kind=ArgoCD openshift-gitops matched error assertion case.go:366: resource apps/v1, Kind=Deployment openshift-gitops-server matched error assertion logger.go:42: 18:53:12 | 1-018_validate_disable_default_instance | skipping kubernetes event logging logger.go:42: 18:53:12 | 1-018_validate_disable_default_instance | Deleting namespace: kuttl-test-trusting-octopus === CONT kuttl/harness/1-010_validate-ootb-manage-other-namespace logger.go:42: 18:53:18 | 1-010_validate-ootb-manage-other-namespace | Ignoring README.md as it does not match file name regexp: ^(\d+)-(?:[^\.]+)(?:\.yaml)?$ logger.go:42: 18:53:18 | 1-010_validate-ootb-manage-other-namespace | Ignoring errors.yaml as it does not match file name regexp: ^(\d+)-(?:[^\.]+)(?:\.yaml)?$ logger.go:42: 18:53:18 | 1-010_validate-ootb-manage-other-namespace | Creating namespace: kuttl-test-possible-guinea logger.go:42: 18:53:18 | 1-010_validate-ootb-manage-other-namespace/1-install | starting test step 1-install I0723 18:53:19.776604 5681 request.go:682] Waited for 1.046574945s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/monitoring.coreos.com/v1beta1?timeout=32s logger.go:42: 18:53:21 | 1-010_validate-ootb-manage-other-namespace/1-install | Namespace:/test-1-10-custom created logger.go:42: 18:53:21 | 1-010_validate-ootb-manage-other-namespace/1-install | test step completed 1-install logger.go:42: 18:53:21 | 1-010_validate-ootb-manage-other-namespace/2- | starting test step 2- logger.go:42: 18:53:23 | 1-010_validate-ootb-manage-other-namespace/2- | test step completed 2- logger.go:42: 18:53:23 | 1-010_validate-ootb-manage-other-namespace/3-create-application | starting test step 3-create-application logger.go:42: 18:53:26 | 1-010_validate-ootb-manage-other-namespace/3-create-application | Application:openshift-gitops/test-1-10-custom created logger.go:42: 18:53:35 | 1-010_validate-ootb-manage-other-namespace/3-create-application | test step completed 3-create-application logger.go:42: 18:53:35 | 1-010_validate-ootb-manage-other-namespace/4-unlabel-namespace | starting test step 4-unlabel-namespace logger.go:42: 18:53:35 | 1-010_validate-ootb-manage-other-namespace/4-unlabel-namespace | running command: [sh -c kubectl label ns test-1-10-custom argocd.argoproj.io/managed-by-] logger.go:42: 18:53:36 | 1-010_validate-ootb-manage-other-namespace/4-unlabel-namespace | namespace/test-1-10-custom unlabeled logger.go:42: 18:53:36 | 1-010_validate-ootb-manage-other-namespace/4-unlabel-namespace | running command: [sh -c sleep 5] I0723 18:53:42.163181 5681 request.go:682] Waited for 1.046712093s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/snapshot.storage.k8s.io/v1?timeout=32s logger.go:42: 18:53:43 | 1-010_validate-ootb-manage-other-namespace/4-unlabel-namespace | test step completed 4-unlabel-namespace logger.go:42: 18:53:43 | 1-010_validate-ootb-manage-other-namespace/5-check-rbac | starting test step 5-check-rbac logger.go:42: 18:53:46 | 1-010_validate-ootb-manage-other-namespace/5-check-rbac | test step completed 5-check-rbac logger.go:42: 18:53:46 | 1-010_validate-ootb-manage-other-namespace/99-delete | starting test step 99-delete I0723 18:53:59.343267 5681 request.go:682] Waited for 1.048138398s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/monitoring.coreos.com/v1beta1?timeout=32s logger.go:42: 18:54:00 | 1-010_validate-ootb-manage-other-namespace/99-delete | test step completed 99-delete logger.go:42: 18:54:00 | 1-010_validate-ootb-manage-other-namespace | skipping kubernetes event logging logger.go:42: 18:54:00 | 1-010_validate-ootb-manage-other-namespace | Deleting namespace: kuttl-test-possible-guinea === CONT kuttl/harness/1-037_validate_applicationset_in_any_namespace logger.go:42: 18:54:06 | 1-037_validate_applicationset_in_any_namespace | Creating namespace: kuttl-test-sought-coyote logger.go:42: 18:54:06 | 1-037_validate_applicationset_in_any_namespace/0-prepare | starting test step 0-prepare I0723 18:54:09.365103 5681 request.go:682] Waited for 2.394985508s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/operator.openshift.io/v1alpha1?timeout=32s logger.go:42: 18:54:09 | 1-037_validate_applicationset_in_any_namespace/0-prepare | Namespace:/appset-argocd created logger.go:42: 18:54:09 | 1-037_validate_applicationset_in_any_namespace/0-prepare | Namespace:/appset-old-ns created logger.go:42: 18:54:09 | 1-037_validate_applicationset_in_any_namespace/0-prepare | Namespace:/appset-new-ns created logger.go:42: 18:54:09 | 1-037_validate_applicationset_in_any_namespace/0-prepare | test step completed 0-prepare logger.go:42: 18:54:09 | 1-037_validate_applicationset_in_any_namespace/1-argocd | starting test step 1-argocd logger.go:42: 18:54:12 | 1-037_validate_applicationset_in_any_namespace/1-argocd | ArgoCD:appset-argocd/example created logger.go:42: 18:54:44 | 1-037_validate_applicationset_in_any_namespace/1-argocd | test step completed 1-argocd logger.go:42: 18:54:44 | 1-037_validate_applicationset_in_any_namespace/2-enable-appset-in-any-ns | starting test step 2-enable-appset-in-any-ns I0723 18:54:45.677875 5681 request.go:682] Waited for 1.044862485s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/autoscaling.openshift.io/v1?timeout=32s logger.go:42: 18:54:47 | 1-037_validate_applicationset_in_any_namespace/2-enable-appset-in-any-ns | ArgoCD:appset-argocd/example updated logger.go:42: 18:54:47 | 1-037_validate_applicationset_in_any_namespace/2-enable-appset-in-any-ns | test step completed 2-enable-appset-in-any-ns logger.go:42: 18:54:47 | 1-037_validate_applicationset_in_any_namespace/3-add-new-appset-ns | starting test step 3-add-new-appset-ns logger.go:42: 18:54:49 | 1-037_validate_applicationset_in_any_namespace/3-add-new-appset-ns | ArgoCD:appset-argocd/example updated logger.go:42: 18:54:49 | 1-037_validate_applicationset_in_any_namespace/3-add-new-appset-ns | test step completed 3-add-new-appset-ns logger.go:42: 18:54:49 | 1-037_validate_applicationset_in_any_namespace/4-enable-apps-in-new-ns | starting test step 4-enable-apps-in-new-ns logger.go:42: 18:54:52 | 1-037_validate_applicationset_in_any_namespace/4-enable-apps-in-new-ns | ArgoCD:appset-argocd/example updated logger.go:42: 18:54:53 | 1-037_validate_applicationset_in_any_namespace/4-enable-apps-in-new-ns | test step completed 4-enable-apps-in-new-ns logger.go:42: 18:54:53 | 1-037_validate_applicationset_in_any_namespace/5-enable-apps-in-old-ns | starting test step 5-enable-apps-in-old-ns I0723 18:54:55.712927 5681 request.go:682] Waited for 2.440762943s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/performance.openshift.io/v1alpha1?timeout=32s logger.go:42: 18:54:55 | 1-037_validate_applicationset_in_any_namespace/5-enable-apps-in-old-ns | ArgoCD:appset-argocd/example updated logger.go:42: 18:54:56 | 1-037_validate_applicationset_in_any_namespace/5-enable-apps-in-old-ns | test step completed 5-enable-apps-in-old-ns logger.go:42: 18:54:56 | 1-037_validate_applicationset_in_any_namespace/6-disable-appset-in-new-ns | starting test step 6-disable-appset-in-new-ns logger.go:42: 18:54:59 | 1-037_validate_applicationset_in_any_namespace/6-disable-appset-in-new-ns | ArgoCD:appset-argocd/example updated logger.go:42: 18:55:00 | 1-037_validate_applicationset_in_any_namespace/6-disable-appset-in-new-ns | test step completed 6-disable-appset-in-new-ns logger.go:42: 18:55:00 | 1-037_validate_applicationset_in_any_namespace/7-disable-apps-in-all-ns | starting test step 7-disable-apps-in-all-ns logger.go:42: 18:55:02 | 1-037_validate_applicationset_in_any_namespace/7-disable-apps-in-all-ns | ArgoCD:appset-argocd/example updated logger.go:42: 18:55:04 | 1-037_validate_applicationset_in_any_namespace/7-disable-apps-in-all-ns | test step completed 7-disable-apps-in-all-ns logger.go:42: 18:55:04 | 1-037_validate_applicationset_in_any_namespace/99-cleanup | starting test step 99-cleanup I0723 18:55:38.974024 5681 request.go:682] Waited for 1.033939146s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/authorization.openshift.io/v1?timeout=32s logger.go:42: 18:55:40 | 1-037_validate_applicationset_in_any_namespace/99-cleanup | test step completed 99-cleanup logger.go:42: 18:55:40 | 1-037_validate_applicationset_in_any_namespace | skipping kubernetes event logging logger.go:42: 18:55:40 | 1-037_validate_applicationset_in_any_namespace | Deleting namespace: kuttl-test-sought-coyote === CONT kuttl/harness/1-036_validate_role_rolebinding_for_source_namespace logger.go:42: 18:55:46 | 1-036_validate_role_rolebinding_for_source_namespace | Creating namespace: kuttl-test-sweeping-bug logger.go:42: 18:55:46 | 1-036_validate_role_rolebinding_for_source_namespace/1-sourcenamespace_without_wildcard | starting test step 1-sourcenamespace_without_wildcard I0723 18:55:48.976212 5681 request.go:682] Waited for 2.387172707s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/performance.openshift.io/v1?timeout=32s logger.go:42: 18:55:49 | 1-036_validate_role_rolebinding_for_source_namespace/1-sourcenamespace_without_wildcard | Namespace:/test created logger.go:42: 18:55:49 | 1-036_validate_role_rolebinding_for_source_namespace/1-sourcenamespace_without_wildcard | ArgoCD:default/example-argocd created logger.go:42: 18:55:50 | 1-036_validate_role_rolebinding_for_source_namespace/1-sourcenamespace_without_wildcard | test step completed 1-sourcenamespace_without_wildcard logger.go:42: 18:55:50 | 1-036_validate_role_rolebinding_for_source_namespace/2-sourceNamespace_with_wildcard_pattern | starting test step 2-sourceNamespace_with_wildcard_pattern logger.go:42: 18:55:52 | 1-036_validate_role_rolebinding_for_source_namespace/2-sourceNamespace_with_wildcard_pattern | Namespace:/test-1 created logger.go:42: 18:55:52 | 1-036_validate_role_rolebinding_for_source_namespace/2-sourceNamespace_with_wildcard_pattern | Namespace:/dev created logger.go:42: 18:55:52 | 1-036_validate_role_rolebinding_for_source_namespace/2-sourceNamespace_with_wildcard_pattern | ArgoCD:default/example-argocd updated logger.go:42: 18:55:53 | 1-036_validate_role_rolebinding_for_source_namespace/2-sourceNamespace_with_wildcard_pattern | test step completed 2-sourceNamespace_with_wildcard_pattern logger.go:42: 18:55:53 | 1-036_validate_role_rolebinding_for_source_namespace/3-new_namespace_with_match_pattern | starting test step 3-new_namespace_with_match_pattern logger.go:42: 18:55:56 | 1-036_validate_role_rolebinding_for_source_namespace/3-new_namespace_with_match_pattern | Namespace:/test-2 created logger.go:42: 18:55:57 | 1-036_validate_role_rolebinding_for_source_namespace/3-new_namespace_with_match_pattern | test step completed 3-new_namespace_with_match_pattern logger.go:42: 18:55:57 | 1-036_validate_role_rolebinding_for_source_namespace/4-sourceNamespace_with_wildcard | starting test step 4-sourceNamespace_with_wildcard I0723 18:55:59.019088 5681 request.go:682] Waited for 1.646976569s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/argoproj.io/v1beta1?timeout=32s logger.go:42: 18:55:59 | 1-036_validate_role_rolebinding_for_source_namespace/4-sourceNamespace_with_wildcard | Namespace:/test-3 created logger.go:42: 18:55:59 | 1-036_validate_role_rolebinding_for_source_namespace/4-sourceNamespace_with_wildcard | ArgoCD:default/example-argocd updated logger.go:42: 18:56:04 | 1-036_validate_role_rolebinding_for_source_namespace/4-sourceNamespace_with_wildcard | test step completed 4-sourceNamespace_with_wildcard logger.go:42: 18:56:04 | 1-036_validate_role_rolebinding_for_source_namespace/5-multiple_sourcenamespace | starting test step 5-multiple_sourcenamespace logger.go:42: 18:56:06 | 1-036_validate_role_rolebinding_for_source_namespace/5-multiple_sourcenamespace | Namespace:/test-ns-1 created logger.go:42: 18:56:06 | 1-036_validate_role_rolebinding_for_source_namespace/5-multiple_sourcenamespace | Namespace:/dev-ns-1 created logger.go:42: 18:56:06 | 1-036_validate_role_rolebinding_for_source_namespace/5-multiple_sourcenamespace | Namespace:/other-ns created logger.go:42: 18:56:06 | 1-036_validate_role_rolebinding_for_source_namespace/5-multiple_sourcenamespace | ArgoCD:default/example-argocd updated logger.go:42: 18:56:08 | 1-036_validate_role_rolebinding_for_source_namespace/5-multiple_sourcenamespace | test step completed 5-multiple_sourcenamespace logger.go:42: 18:56:08 | 1-036_validate_role_rolebinding_for_source_namespace/6-validate_labels_role_rolebinding_after_removing_namespace_from_sourcenamespace | starting test step 6-validate_labels_role_rolebinding_after_removing_namespace_from_sourcenamespace I0723 18:56:09.098292 5681 request.go:682] Waited for 1.047383667s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/imageregistry.operator.openshift.io/v1?timeout=32s logger.go:42: 18:56:10 | 1-036_validate_role_rolebinding_for_source_namespace/6-validate_labels_role_rolebinding_after_removing_namespace_from_sourcenamespace | ArgoCD:default/example-argocd updated logger.go:42: 18:56:11 | 1-036_validate_role_rolebinding_for_source_namespace/6-validate_labels_role_rolebinding_after_removing_namespace_from_sourcenamespace | test step completed 6-validate_labels_role_rolebinding_after_removing_namespace_from_sourcenamespace logger.go:42: 18:56:11 | 1-036_validate_role_rolebinding_for_source_namespace | skipping kubernetes event logging logger.go:42: 18:56:11 | 1-036_validate_role_rolebinding_for_source_namespace | Deleting namespace: kuttl-test-sweeping-bug === CONT kuttl/harness/1-004_validate_argocd_installation logger.go:42: 18:56:18 | 1-004_validate_argocd_installation | Creating namespace: kuttl-test-trusted-cat logger.go:42: 18:56:18 | 1-004_validate_argocd_installation/1-check-prereqs | starting test step 1-check-prereqs I0723 18:56:19.437844 5681 request.go:682] Waited for 1.047384937s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/samples.operator.openshift.io/v1?timeout=32s logger.go:42: 18:56:20 | 1-004_validate_argocd_installation/1-check-prereqs | test step completed 1-check-prereqs logger.go:42: 18:56:20 | 1-004_validate_argocd_installation/2-change-operand | starting test step 2-change-operand logger.go:42: 18:56:23 | 1-004_validate_argocd_installation/2-change-operand | ArgoCD:openshift-gitops/openshift-gitops updated logger.go:42: 18:56:23 | 1-004_validate_argocd_installation/2-change-operand | test step completed 2-change-operand logger.go:42: 18:56:23 | 1-004_validate_argocd_installation/3-wait | starting test step 3-wait logger.go:42: 18:56:23 | 1-004_validate_argocd_installation/3-wait | running command: [sh -c sleep 5] I0723 18:56:29.482492 5681 request.go:682] Waited for 1.096162387s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/operators.coreos.com/v1alpha1?timeout=32s logger.go:42: 18:56:30 | 1-004_validate_argocd_installation/3-wait | test step completed 3-wait logger.go:42: 18:56:30 | 1-004_validate_argocd_installation/4- | starting test step 4- logger.go:42: 18:56:35 | 1-004_validate_argocd_installation/4- | test step completed 4- logger.go:42: 18:56:35 | 1-004_validate_argocd_installation | skipping kubernetes event logging logger.go:42: 18:56:35 | 1-004_validate_argocd_installation | Deleting namespace: kuttl-test-trusted-cat === CONT kuttl/harness/1-028_validate_run_on_infra logger.go:42: 18:56:41 | 1-028_validate_run_on_infra | Ignoring errors.yaml as it does not match file name regexp: ^(\d+)-(?:[^\.]+)(?:\.yaml)?$ logger.go:42: 18:56:41 | 1-028_validate_run_on_infra | Creating namespace: kuttl-test-champion-wolf logger.go:42: 18:56:41 | 1-028_validate_run_on_infra/1-install | starting test step 1-install logger.go:42: 18:56:41 | 1-028_validate_run_on_infra/1-install | running command: [sh -c oc patch gitopsservice cluster --type=merge --patch='{"spec": {"runOnInfra": true, "tolerations": [{"effect": "NoSchedule", "key": "infra", "value": "reserved"}]}}' ] logger.go:42: 18:56:41 | 1-028_validate_run_on_infra/1-install | gitopsservice.pipelines.openshift.io/cluster patched I0723 18:56:43.049014 5681 request.go:682] Waited for 1.045361076s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/quota.openshift.io/v1?timeout=32s logger.go:42: 18:56:44 | 1-028_validate_run_on_infra/1-install | test step completed 1-install logger.go:42: 18:56:44 | 1-028_validate_run_on_infra/2-custom-argocd | starting test step 2-custom-argocd logger.go:42: 18:56:46 | 1-028_validate_run_on_infra/2-custom-argocd | ArgoCD:kuttl-test-champion-wolf/argocd created logger.go:42: 18:56:58 | 1-028_validate_run_on_infra/2-custom-argocd | test step completed 2-custom-argocd logger.go:42: 18:56:58 | 1-028_validate_run_on_infra/3-change-back | starting test step 3-change-back logger.go:42: 18:56:58 | 1-028_validate_run_on_infra/3-change-back | running command: [sh -c oc patch gitopsservice cluster --type=json --patch='[{"op": "remove", "path": "/spec/tolerations"}, {"op": "remove", "path": "/spec/runOnInfra"}]' oc patch statefulset openshift-gitops-application-controller -n openshift-gitops -p '{"spec":{"replicas":0}}' ] logger.go:42: 18:56:59 | 1-028_validate_run_on_infra/3-change-back | gitopsservice.pipelines.openshift.io/cluster patched logger.go:42: 18:56:59 | 1-028_validate_run_on_infra/3-change-back | statefulset.apps/openshift-gitops-application-controller patched I0723 18:57:00.757991 5681 request.go:682] Waited for 1.03992905s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/monitoring.coreos.com/v1beta1?timeout=32s logger.go:42: 18:57:02 | 1-028_validate_run_on_infra/3-change-back | test step completed 3-change-back logger.go:42: 18:57:02 | 1-028_validate_run_on_infra/4-check | starting test step 4-check logger.go:42: 18:57:04 | 1-028_validate_run_on_infra/4-check | test step completed 4-check logger.go:42: 18:57:04 | 1-028_validate_run_on_infra | skipping kubernetes event logging logger.go:42: 18:57:04 | 1-028_validate_run_on_infra | Deleting namespace: kuttl-test-champion-wolf === CONT kuttl/harness/1-027_validate_operand_from_git logger.go:42: 18:57:11 | 1-027_validate_operand_from_git | Creating namespace: kuttl-test-enjoyed-shiner logger.go:42: 18:57:11 | 1-027_validate_operand_from_git/1- | starting test step 1- I0723 18:57:12.163654 5681 request.go:682] Waited for 1.039230803s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/policy/v1?timeout=32s logger.go:42: 18:57:13 | 1-027_validate_operand_from_git/1- | test step completed 1- logger.go:42: 18:57:13 | 1-027_validate_operand_from_git/2-install | starting test step 2-install logger.go:42: 18:57:16 | 1-027_validate_operand_from_git/2-install | Application:openshift-gitops/1-27-argocd created logger.go:42: 18:57:49 | 1-027_validate_operand_from_git/2-install | test step completed 2-install logger.go:42: 18:57:49 | 1-027_validate_operand_from_git/3-create-app | starting test step 3-create-app I0723 18:57:51.022672 5681 request.go:682] Waited for 1.047931811s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/snapshot.storage.k8s.io/v1?timeout=32s logger.go:42: 18:57:52 | 1-027_validate_operand_from_git/3-create-app | Application:test-1-27-custom/guestbook created logger.go:42: 18:57:59 | 1-027_validate_operand_from_git/3-create-app | test step completed 3-create-app logger.go:42: 18:57:59 | 1-027_validate_operand_from_git/99-delete | starting test step 99-delete I0723 18:58:14.398643 5681 request.go:682] Waited for 1.046043094s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/network.operator.openshift.io/v1?timeout=32s logger.go:42: 18:58:15 | 1-027_validate_operand_from_git/99-delete | test step completed 99-delete logger.go:42: 18:58:15 | 1-027_validate_operand_from_git | skipping kubernetes event logging logger.go:42: 18:58:15 | 1-027_validate_operand_from_git | Deleting namespace: kuttl-test-enjoyed-shiner === CONT kuttl/harness/1-005_validate_metrics logger.go:42: 18:58:21 | 1-005_validate_metrics | Creating namespace: kuttl-test-on-dogfish logger.go:42: 18:58:21 | 1-005_validate_metrics/1- | starting test step 1- I0723 18:58:24.412107 5681 request.go:682] Waited for 2.447776263s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/performance.openshift.io/v1alpha1?timeout=32s logger.go:42: 18:58:24 | 1-005_validate_metrics/1- | test step completed 1- logger.go:42: 18:58:24 | 1-005_validate_metrics | skipping kubernetes event logging logger.go:42: 18:58:24 | 1-005_validate_metrics | Deleting namespace: kuttl-test-on-dogfish === CONT kuttl/harness/1-031_validate_toolchain logger.go:42: 18:58:30 | 1-031_validate_toolchain | Creating namespace: kuttl-test-allowing-roughy logger.go:42: 18:58:30 | 1-031_validate_toolchain/1-check | starting test step 1-check logger.go:42: 18:58:30 | 1-031_validate_toolchain/1-check | running command: [sh -c set -e set -o pipefail # These variables need to be maintained according to the component matrix: https://spaces.redhat.com/display/GITOPS/GitOps+Component+Matrix expected_kustomizeVersion='v5.2.1' expected_helmVersion='v3.14.4' expected_argocdVersion='v2.11.6' expected_dexVersion='v2.37.0' expected_redisVersion='6.2.7' gitops_server_pod=$(oc get -n openshift-gitops pods \ -o custom-columns=NAME:.metadata.name --no-headers | \ grep openshift-gitops-server | \ head -1 ) dex_pod=$(oc get -n openshift-gitops pods \ -o custom-columns=NAME:.metadata.name --no-headers | \ grep openshift-gitops-dex-server | \ head -1 ) redis_pod=$(oc get -n openshift-gitops pods \ -o custom-columns=NAME:.metadata.name --no-headers | \ grep openshift-gitops-redis | \ head -1 ) route=$(oc get route -n openshift-gitops \ -o custom-columns=NAME:.spec.host --no-headers | \ grep openshift-gitops-server | \ head -1 ) kustomizeVersion=$(oc -n openshift-gitops exec $gitops_server_pod \ -- kustomize version ) helmVersion=$(oc -n openshift-gitops exec $gitops_server_pod \ -- helm version | \ sed -e 's/version.BuildInfo//' -e 's/\"//g' | \ awk -F':' '{ print $2 }' | \ awk -F',' '{ print $1 }' ) argocdVersion=$(oc -n openshift-gitops exec $gitops_server_pod \ -- argocd version --short --server $route --insecure | \ grep 'argocd-server' | \ awk -F' ' '{ print $2 }' | \ sed 's/\+.*//' ) dexVersion=$(oc -n openshift-gitops exec $dex_pod -- dex version 2>&1 | \ grep '^Dex Version' | \ awk -F': ' '{ print $2 }' ) redisVersion=$(oc -n openshift-gitops exec $redis_pod -- \ redis-server -v | awk -F '=' '{ print $2 }' | cut -d' ' -f 1 ) if test "${kustomizeVersion}" != "${expected_kustomizeVersion}"; then echo "Kustomize version mismatch. Should be ${expected_kustomizeVersion}, is ${kustomizeVersion}" exit 1 fi if test "${helmVersion}" != "${expected_helmVersion}"; then echo "Helm version mismatch. Should be ${expected_helmVersion}, is ${helmVersion}" exit 1 fi if test "${argocdVersion}" != "${expected_argocdVersion}"; then echo "ArgoCD version mismatch. Should be ${expected_argocdVersion}, is ${argocdVersion}" exit 1 fi if test "${dexVersion}" != "${expected_dexVersion}"; then echo "Dex version mismatch. Should be ${expected_dexVersion}, is ${dexVersion}" exit 1 fi if test "${redisVersion}" != "${expected_redisVersion}"; then echo "Redis version mismatch. Should be ${expected_redisVersion}, is ${redisVersion}" exit 1 fi ] logger.go:42: 18:58:35 | 1-031_validate_toolchain/1-check | time="2024-07-23T18:58:35Z" level=warning msg="Failed to invoke grpc call. Use flag --grpc-web in grpc calls. To avoid this warning message, use flag --grpc-web." I0723 18:58:37.558826 5681 request.go:682] Waited for 1.047747307s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/oauth.openshift.io/v1?timeout=32s logger.go:42: 18:58:38 | 1-031_validate_toolchain/1-check | test step completed 1-check logger.go:42: 18:58:38 | 1-031_validate_toolchain | skipping kubernetes event logging logger.go:42: 18:58:38 | 1-031_validate_toolchain | Deleting namespace: kuttl-test-allowing-roughy === CONT kuttl/harness/1-003_validate_cluster_config logger.go:42: 18:58:45 | 1-003_validate_cluster_config | Creating namespace: kuttl-test-steady-mosquito logger.go:42: 18:58:45 | 1-003_validate_cluster_config/0-cluster-scoped-argocd | starting test step 0-cluster-scoped-argocd I0723 18:58:47.604698 5681 request.go:682] Waited for 2.196396264s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/quota.openshift.io/v1?timeout=32s logger.go:42: 18:58:47 | 1-003_validate_cluster_config/0-cluster-scoped-argocd | Namespace:/argocd-e2e-cluster-config created logger.go:42: 18:58:47 | 1-003_validate_cluster_config/0-cluster-scoped-argocd | ArgoCD:argocd-e2e-cluster-config/example-argocd created logger.go:42: 18:58:47 | 1-003_validate_cluster_config/0-cluster-scoped-argocd | test step completed 0-cluster-scoped-argocd logger.go:42: 18:58:47 | 1-003_validate_cluster_config/1-cluster-scoped | starting test step 1-cluster-scoped logger.go:42: 18:58:47 | 1-003_validate_cluster_config/1-cluster-scoped | running command: [sh -c oc patch -n openshift-operators subscription openshift-gitops-operator \ --type merge --patch '{"spec": {"config": {"env": [{"name": "ARGOCD_CLUSTER_CONFIG_NAMESPACES", "value": "openshift-gitops, argocd-e2e-cluster-config"}]}}}' ] logger.go:42: 18:58:48 | 1-003_validate_cluster_config/1-cluster-scoped | subscription.operators.coreos.com/openshift-gitops-operator patched logger.go:42: 19:08:51 | 1-003_validate_cluster_config/1-cluster-scoped | test step failed 1-cluster-scoped case.go:364: failed in step 1-cluster-scoped case.go:366: clusterrolebindings.rbac.authorization.k8s.io "example-argocd-argocd-e2e-cluster-config-argocd-application-controller" not found case.go:366: clusterrolebindings.rbac.authorization.k8s.io "example-argocd-argocd-e2e-cluster-config-argocd-server" not found case.go:366: clusterroles.rbac.authorization.k8s.io "example-argocd-argocd-e2e-cluster-config-argocd-application-controller" not found case.go:366: clusterroles.rbac.authorization.k8s.io "example-argocd-argocd-e2e-cluster-config-argocd-server" not found logger.go:42: 19:08:51 | 1-003_validate_cluster_config | skipping kubernetes event logging logger.go:42: 19:08:51 | 1-003_validate_cluster_config | Deleting namespace: kuttl-test-steady-mosquito === CONT kuttl/harness/1-002_validate_backend_service logger.go:42: 19:08:58 | 1-002_validate_backend_service | Creating namespace: kuttl-test-correct-cobra logger.go:42: 19:08:58 | 1-002_validate_backend_service/1-check-backend-service | starting test step 1-check-backend-service I0723 19:08:59.152425 5681 request.go:682] Waited for 1.038590751s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/console.openshift.io/v1alpha1?timeout=32s logger.go:42: 19:09:00 | 1-002_validate_backend_service/1-check-backend-service | test step completed 1-check-backend-service logger.go:42: 19:09:00 | 1-002_validate_backend_service | skipping kubernetes event logging logger.go:42: 19:09:00 | 1-002_validate_backend_service | Deleting namespace: kuttl-test-correct-cobra === CONT kuttl/harness/1-035_validate_argocd_secret_repopulate logger.go:42: 19:09:06 | 1-035_validate_argocd_secret_repopulate | Creating namespace: kuttl-test-oriented-yak logger.go:42: 19:09:06 | 1-035_validate_argocd_secret_repopulate/1- | starting test step 1- I0723 19:09:09.160690 5681 request.go:682] Waited for 2.34305702s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/quota.openshift.io/v1?timeout=32s logger.go:42: 19:09:09 | 1-035_validate_argocd_secret_repopulate/1- | test step completed 1- logger.go:42: 19:09:09 | 1-035_validate_argocd_secret_repopulate/2-remove_secret_data | starting test step 2-remove_secret_data logger.go:42: 19:09:09 | 1-035_validate_argocd_secret_repopulate/2-remove_secret_data | running command: [sh -c oc patch secret/argocd-secret -p '{"data":null}' -n openshift-gitops ] logger.go:42: 19:09:09 | 1-035_validate_argocd_secret_repopulate/2-remove_secret_data | secret/argocd-secret patched logger.go:42: 19:09:12 | 1-035_validate_argocd_secret_repopulate/2-remove_secret_data | test step completed 2-remove_secret_data logger.go:42: 19:09:12 | 1-035_validate_argocd_secret_repopulate/3-check_secret_repopulate | starting test step 3-check_secret_repopulate logger.go:42: 19:09:12 | 1-035_validate_argocd_secret_repopulate/3-check_secret_repopulate | running command: [sh -c sleep 30 secret_data=$(oc get secret/argocd-secret -n openshift-gitops -o custom-columns=SECRET:.data --no-headers) if test "${secret_data}" = ""; then echo "argocd-secret was not repopulated" exit 1 fi exit 0 ] I0723 19:09:43.758914 5681 request.go:682] Waited for 1.046585864s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/quota.openshift.io/v1?timeout=32s logger.go:42: 19:09:45 | 1-035_validate_argocd_secret_repopulate/3-check_secret_repopulate | test step completed 3-check_secret_repopulate logger.go:42: 19:09:45 | 1-035_validate_argocd_secret_repopulate/4-check_controller_pod_status | starting test step 4-check_controller_pod_status logger.go:42: 19:09:45 | 1-035_validate_argocd_secret_repopulate/4-check_controller_pod_status | running command: [sh -c oc rollout status deployment openshift-gitops-operator-controller-manager -n openshift-operators --timeout=1m rollout_status=$? if [ "${rollout_status}" -ne 0 ]; then echo "openshift-gitops-operator-controller-manager deployment was not successfully rolled out" exit 1 fi exit 0 ] logger.go:42: 19:09:45 | 1-035_validate_argocd_secret_repopulate/4-check_controller_pod_status | deployment "openshift-gitops-operator-controller-manager" successfully rolled out logger.go:42: 19:09:48 | 1-035_validate_argocd_secret_repopulate/4-check_controller_pod_status | test step completed 4-check_controller_pod_status logger.go:42: 19:09:48 | 1-035_validate_argocd_secret_repopulate/5-argocd_login | starting test step 5-argocd_login logger.go:42: 19:09:48 | 1-035_validate_argocd_secret_repopulate/5-argocd_login | running command: [sh -c set -eo pipefail api_server=$(oc get routes -n openshift-gitops --field-selector metadata.name=openshift-gitops-server -o jsonpath="{.items[*]['spec.host']}") password=$(oc get secret openshift-gitops-cluster -n openshift-gitops -o jsonpath='{.data.admin\.password}' | base64 -d) output=$(argocd login $api_server --username admin --password $password --insecure) if ! [[ "${output}" =~ "'admin:login' logged in successfully" ]]; then if [[ "${output}" == *"rpc error: code = Unknown desc = server.secretkey is missing" ]]; then echo "server.secretkey is missing" exit 1 fi fi exit 0 ] logger.go:42: 19:09:49 | 1-035_validate_argocd_secret_repopulate/5-argocd_login | time="2024-07-23T19:09:49Z" level=warning msg="Failed to invoke grpc call. Use flag --grpc-web in grpc calls. To avoid this warning message, use flag --grpc-web." logger.go:42: 19:09:52 | 1-035_validate_argocd_secret_repopulate/5-argocd_login | test step completed 5-argocd_login logger.go:42: 19:09:52 | 1-035_validate_argocd_secret_repopulate | skipping kubernetes event logging logger.go:42: 19:09:52 | 1-035_validate_argocd_secret_repopulate | Deleting namespace: kuttl-test-oriented-yak === CONT kuttl/harness/1-034_validate_custom_roles logger.go:42: 19:09:58 | 1-034_validate_custom_roles | Creating namespace: kuttl-test-notable-haddock logger.go:42: 19:09:58 | 1-034_validate_custom_roles/1-install | starting test step 1-install I0723 19:09:59.689953 5681 request.go:682] Waited for 1.037673148s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/pipelines.openshift.io/v1alpha1?timeout=32s logger.go:42: 19:10:01 | 1-034_validate_custom_roles/1-install | Namespace:/test-1-034-custom created logger.go:42: 19:10:01 | 1-034_validate_custom_roles/1-install | ArgoCD:test-1-034-custom/argocd created logger.go:42: 19:10:01 | 1-034_validate_custom_roles/1-install | Namespace:/custom-role-namespace created logger.go:42: 19:10:01 | 1-034_validate_custom_roles/1-install | ClusterRole:/custom-argocd-role created logger.go:42: 19:10:13 | 1-034_validate_custom_roles/1-install | test step completed 1-install logger.go:42: 19:10:13 | 1-034_validate_custom_roles/2-patch-subscription | starting test step 2-patch-subscription logger.go:42: 19:10:13 | 1-034_validate_custom_roles/2-patch-subscription | running command: [sh -c oc patch -n openshift-operators subscription openshift-gitops-operator \ --type merge --patch '{"spec": {"config": {"env": [{"name": "CONTROLLER_CLUSTER_ROLE", "value": "custom-argocd-role"},{"name": "SERVER_CLUSTER_ROLE", "value": "custom-argocd-role"}]}}}' ] logger.go:42: 19:10:14 | 1-034_validate_custom_roles/2-patch-subscription | subscription.operators.coreos.com/openshift-gitops-operator patched logger.go:42: 19:10:14 | 1-034_validate_custom_roles/2-patch-subscription | running command: [sh -c sleep 10] I0723 19:10:25.561179 5681 request.go:682] Waited for 1.038062084s due to client-side throttling, not priority and fairness, request: GET:https://api.test-aws-412.ocp-gitops-qe.com:6443/apis/apps.openshift.io/v1?timeout=32s logger.go:42: 19:10:26 | 1-034_validate_custom_roles/2-patch-subscription | test step completed 2-patch-subscription logger.go:42: 19:10:26 | 1-034_validate_custom_roles/3- | starting test step 3- logger.go:42: 19:20:30 | 1-034_validate_custom_roles/3- | test step failed 3- case.go:364: failed in step 3- case.go:366: resource rbac.authorization.k8s.io/v1, Kind=Role argocd-argocd-server matched error assertion case.go:366: resource rbac.authorization.k8s.io/v1, Kind=Role argocd-argocd-application-controller matched error assertion case.go:366: resource rbac.authorization.k8s.io/v1, Kind=Role argocd-argocd-server matched error assertion case.go:366: resource rbac.authorization.k8s.io/v1, Kind=Role argocd-argocd-application-controller matched error assertion logger.go:42: 19:20:30 | 1-034_validate_custom_roles | skipping kubernetes event logging logger.go:42: 19:20:30 | 1-034_validate_custom_roles | Deleting namespace: kuttl-test-notable-haddock === CONT kuttl harness.go:405: run tests finished harness.go:513: cleaning up harness.go:570: removing temp folder: "" --- FAIL: kuttl (4089.23s) --- FAIL: kuttl/harness (0.00s) --- PASS: kuttl/harness/1-001_validate_kam_service (14.24s) --- PASS: kuttl/harness/1-041_validate_argocd_sync_alert (8.32s) --- PASS: kuttl/harness/1-110_validate_podsecurity_alerts (8.19s) --- PASS: kuttl/harness/1-109_validate_reencrypt_termination_policy (11.32s) --- PASS: kuttl/harness/1-083_validate_apps_in_any_namespace (84.08s) --- PASS: kuttl/harness/1-078_validate_default_argocd_consoleLink (73.89s) --- PASS: kuttl/harness/1-071_validate_node_selectors (33.96s) --- PASS: kuttl/harness/1-084_validate_prune_templates (42.31s) --- PASS: kuttl/harness/1-064_validate_tcp_reset_error (73.83s) --- PASS: kuttl/harness/1-056_validate_managed-by (82.45s) --- PASS: kuttl/harness/1-106_validate_argocd_metrics_controller (23.62s) --- PASS: kuttl/harness/1-050_validate_sso (179.45s) --- PASS: kuttl/harness/1-100_validate_rollouts_resources_creation (31.62s) --- PASS: kuttl/harness/1-042_validate_status_host (57.07s) --- PASS: kuttl/harness/1-086_validate_default_argocd_role (43.11s) --- PASS: kuttl/harness/1-085_validate_dynamic_plugin_installation (41.32s) --- PASS: kuttl/harness/1-101_validate_rollout_policyrules (8.55s) --- PASS: kuttl/harness/1-105_validate_label_selector (343.23s) --- PASS: kuttl/harness/1-104_validate_prometheus_alert (8.68s) --- PASS: kuttl/harness/1-006_validate_machine_config (29.12s) --- PASS: kuttl/harness/1-026-validate_backend_service_permissions (16.67s) --- PASS: kuttl/harness/1-020_validate_redis_ha_nonha (573.68s) --- PASS: kuttl/harness/1-040_validate_quoted_RBAC_group_names (16.22s) --- FAIL: kuttl/harness/1-018_validate_disable_default_instance (634.93s) --- PASS: kuttl/harness/1-010_validate-ootb-manage-other-namespace (48.22s) --- PASS: kuttl/harness/1-037_validate_applicationset_in_any_namespace (99.63s) --- PASS: kuttl/harness/1-036_validate_role_rolebinding_for_source_namespace (31.84s) --- PASS: kuttl/harness/1-004_validate_argocd_installation (23.18s) --- PASS: kuttl/harness/1-028_validate_run_on_infra (29.53s) --- PASS: kuttl/harness/1-027_validate_operand_from_git (70.86s) --- PASS: kuttl/harness/1-005_validate_metrics (8.71s) --- PASS: kuttl/harness/1-031_validate_toolchain (14.73s) --- FAIL: kuttl/harness/1-003_validate_cluster_config (612.70s) --- PASS: kuttl/harness/1-002_validate_backend_service (8.71s) --- PASS: kuttl/harness/1-035_validate_argocd_secret_repopulate (51.83s) --- FAIL: kuttl/harness/1-034_validate_custom_roles (638.04s) FAIL ========================== ==== E2E TESTS FAILED ==== ========================== Copying results Deleted temp working directory /kuttl-e2e/gitops-operator/tmp.f2TSCtxfEN make: *** [Makefile:4: e2e-tests] Error 1 make: Leaving directory '/kuttl-e2e/gitops-operator' File has been uploaded to CI/z-stream-test-pipeline-upgrade-412/240723-173013/sequential_2024.07.23-17.31.33.xml 🚀 File has been uploaded to CI/z-stream-test-pipeline-upgrade-412/240723-173013/sequential_2024.07.23-17.31.33.log 🚀 File has been uploaded to CI/z-stream-test-pipeline-upgrade-412/240723-173013/parallel_2024.07.23-17.31.33.xml 🚀 File has been uploaded to CI/z-stream-test-pipeline-upgrade-412/240723-173013/parallel_2024.07.23-17.31.33.log 🚀 Updating artifacts path to results http://uploader-devtools-gitops-services.apps.ocp-c1.prod.psi.redhat.com/CI/z-stream-test-pipeline-upgrade-412/240723-173013/